Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
d0d8b4e163 |
No files matched your search
@@ -86,7 +86,6 @@ MEMORY_VECTOR_STORE=chroma
|
||||
# MEMORY_CODING_DB_PATH= # coding_memory.db 路径(默认 {DATA_DIR}/data/coding_memory.db)
|
||||
# MEMORY_CODING_TOOL_RESULT_MAX_BYTES=2048 # 单条 tool_result 截断字节上限
|
||||
# MEMORY_CODING_VERBATIM_FALLBACK_ENABLED=false # 模型无结果时是否保存原文 fact;默认关,避免 tool trace 外泄
|
||||
# MEMORY_CODING_CHAIN_BLOCK_ENABLED=false # search 是否注入版本演进链块(diff);默认关,省 token
|
||||
# MEMORY_EXTRACT_HISTORY_TURNS=5 # 除本次 write 外,额外注入 extractor 的历史 user-turn 数
|
||||
|
||||
# ── 写入流程开关 ──
|
||||
|
||||
@@ -14,35 +14,6 @@ Hy Memory and MemHub are versioned and released independently.
|
||||
|
||||
## Unreleased
|
||||
|
||||
- Added SDK-authoritative lifecycle listing with explicit status filters,
|
||||
idempotent durable write outcomes, exact raw import/get/list APIs, and
|
||||
quiesced local data-plane snapshot/restore contracts for the MemHub
|
||||
authority migration.
|
||||
- Physical delete now reports incomplete VDB/Graph cleanup as a partial failure
|
||||
instead of claiming success after only one store was changed.
|
||||
- Coding search no longer injects the version-evolution diff block
|
||||
(`chain_block`) by default. Enable with
|
||||
`MEMORY_CODING_CHAIN_BLOCK_ENABLED=true` or
|
||||
`CodingConfig.chain_block_enabled=True` (write-side observe unchanged).
|
||||
- Fixed governed-memory revisions and repeated evidence links failing when the
|
||||
visibility dirty-marker trigger encounters the same memory twice. Startup
|
||||
now replaces the affected legacy trigger without changing schema versions
|
||||
or governed and visibility data.
|
||||
- Serialized each SDK SQLite store's shared connection lifecycle so concurrent
|
||||
history/cache operations cannot race with `close()` and crash the process.
|
||||
- Scoped Coding reconciliation targets to the current HY authority,
|
||||
conversation, and workspace, preventing near-duplicate sessions from
|
||||
mutating an experience or working fact owned by another evidence context.
|
||||
- Kept post-write authority verification failures available for read-only
|
||||
reconciliation while preventing automatic or generic confirmed retries from
|
||||
invoking the model again. Automatic reconciliation skips those sticky rows
|
||||
before applying its batch limit, and linked durable jobs exclusively own
|
||||
retry scheduling for their memory tasks.
|
||||
- Clarified that `add(metadata=...)` carries legacy SDK processing options,
|
||||
while persistent caller business context belongs in `meta`.
|
||||
- Added an opt-in `get(..., include_raw=True)` path for reading `L1_RAW`
|
||||
memories, including memories written in lite mode. The default remains
|
||||
hidden for backward compatibility.
|
||||
- Removed MemHub's task-scoped HyPermission store wrappers and global
|
||||
model-content consent switch. The trusted embedded Hy Memory SDK now uses
|
||||
its native local stores directly.
|
||||
|
||||
-87
@@ -1,87 +0,0 @@
|
||||
# HANDSOFF — MemHub / Hy Memory 架构改造
|
||||
|
||||
更新:2026-10-21
|
||||
权威细则:[`memhub/docs/ARCHITECTURE.md`](memhub/docs/ARCHITECTURE.md)(~~旧删线~~ + 新并列)
|
||||
配套:[`memhub/MEMORY.md`](memhub/MEMORY.md);其余 docs 已打「架构变更」条,冲突以 ARCHITECTURE Top Rules 为准。
|
||||
旧用户迁移:[`memhub/docs/contracts/sdk-data-plane-migration-v1.md`](memhub/docs/contracts/sdk-data-plane-migration-v1.md)。
|
||||
|
||||
> 根目录原先无有效 `HANDSOFF` 文件;本文件为清空后重写,只保留本次架构改造交接。
|
||||
|
||||
---
|
||||
|
||||
## IMPORTANT:测试执行规则
|
||||
|
||||
1. 先完整运行目标测试套件一次,收集**全部失败 case**。
|
||||
2. 按根因分组并批量修复,不得用 `-x` 反复重跑已经通过的前置测试。
|
||||
3. 修复后只重跑上次失败的 case。
|
||||
4. 失败 case 全部通过后,才再完整运行一次目标套件作为最终 gate。
|
||||
|
||||
## Top Rules(现行根本)
|
||||
|
||||
1. **所有对 memory 的读写(含存储 backend)都交给 SDK**,经 SDK config / env 配置。
|
||||
2. **MemHub 内对 memory 的原子操作也一律经 SDK**;缺接口 → **先补 SDK client**,禁止 Core 直改 VDB/Graph/SDK 私库或再维护正文权威。
|
||||
3. **MemHub 不存 memory 实体 / 引擎侧关系**(正文、supersede 权威、向量图副本等)。主责是管理面:harness、LLM model、异步写入队列、config、Core/Runtime 状态。
|
||||
- **可以且应当存 `memory_id`**(SDK 给外层的唯一身份)及组织元数据。
|
||||
- Matter 占位功能未发布,删除其表与入口;未来若重做,必须另立合同。
|
||||
|
||||
**推论:**
|
||||
|
||||
| 旧 | 新 |
|
||||
|---|---|
|
||||
| MemHub `evidence*` 原文权威 | 原文 → **SDK raw**;Evidence 权威拆除 |
|
||||
| `governed_*` 列表/版本/supersede 权威 | 拆除方向;UI/纠正/删除 → **授权 + SDK API**;组织层只挂 `memory_id` |
|
||||
| SDK success 后再镜像进 Governance 才算产品写入 | SDK 合同为准;MemHub 至多记 `memory_id` / 任务回执 |
|
||||
| 拦 CRUD / parse ops 双写 supersede | 禁止作为目标架构 |
|
||||
|
||||
---
|
||||
|
||||
## 已改文档(合同先于代码)
|
||||
|
||||
- 重写:`memhub/docs/ARCHITECTURE.md`、`memhub/MEMORY.md`
|
||||
- 条幅 + 关键段新旧对照:`contracts/*`(含 runtime-unification / unified-context-read / source-adapter 等)、`INSTALLATION.md`、`DEVELOPMENT_LOOP.md`、`MEMORY_EVENT_TIME_SCHEDULING_PLAN.md`、`PRODUCT_LOG_TROUBLESHOOTING.md`、`harness-support/README.md`
|
||||
- 各 harness 专页未逐段改写 → 冲突时以 ARCHITECTURE 为准
|
||||
|
||||
**代码尚未按此拆除。** 文档是目标架构,不是已完成迁移。
|
||||
|
||||
---
|
||||
|
||||
## 建议实现顺序
|
||||
|
||||
1. 冻结新功能写入 `governed_*` / MemHub `evidence*` 权威路径。
|
||||
2. 记忆库 UI / recall:授权 + SDK list/get/search/update/delete;组织层只挂 `memory_id`。
|
||||
3. 导入/同步:写入 SDK(含 raw);退役 Evidence 权威。
|
||||
4. 队列降级为「调度 SDK 调用」,完成态以 SDK 可验证结果为准。
|
||||
5. 清空/删除镜像台账;缺的能力开 **SDK PR**,不在 Core 补影子表。
|
||||
|
||||
旧用户的数据迁移不是独立的手工升级步骤:由 App 在可见的启动 loading
|
||||
阶段、Core 开始提供产品读写之前,完成 bundle、可续跑 backfill、reconciliation
|
||||
与最终 cutover。失败或缺少前置配置时 marker 保持未提交,不允许静默进入双写。
|
||||
|
||||
Fresh install 直接创建最新管理面 schema,禁止创建 `evidence*`、`governed_*`
|
||||
及旧 retrieval/Matter/session-shadow 表。Legacy upgrade 在 bundle 与 SDK
|
||||
迁移校验完成后物理 DROP 这些旧表;备份 bundle 才是审计/回滚载体,不在 live DB
|
||||
保留一套“兼容权威”。
|
||||
|
||||
---
|
||||
|
||||
## 管理面仍属 MemHub(勿误拆)
|
||||
|
||||
- harness:`source_grants`、sync desired-state(`config.json`)、适配器扫描(产出进 SDK,不是 Evidence 权威库)
|
||||
- model profiles / 密钥边界 / 模型内容许可
|
||||
- `memory_write_jobs` 单层异步队列、FIFO / event-time 调度与租约(始终属于 MemHub;调度,不是 memory 实体)。旧 `memory_tasks` 二级队列与 task receipt 删除。
|
||||
- pending job 尚未调用 SDK,因此没有 raw/memory ID;worker 对 SDK 只调用一次
|
||||
`add()`,SDK 内部完成 raw + extraction/reconciliation,成功后 MemHub 仅保存
|
||||
返回的 raw ID / memory IDs 与 outcome。禁止“先 import_raw 再排 extraction”。
|
||||
- Core / Runtime 收据与健康
|
||||
- Matter 尚未形成已发布产品合同;删除全部 `matter*` 占位表与运行时入口。未来若重新开发,
|
||||
以新合同重新设计,不为当前占位 schema 保留兼容层。
|
||||
- call_events 类审计元数据(不存记忆正文)
|
||||
- ~~session inbox / change_outbox 围绕 Evidence 外发~~ → 需按新规则重定义或标旧
|
||||
|
||||
---
|
||||
|
||||
## 接手时注意
|
||||
|
||||
- Pin 的公开 `hy-memory` 与「先补 SDK 接口」可能不同步:合源码 ≠ 用户包已带能力;用 `Refs` / pin PR 闭环。
|
||||
- Top Rule 与旧 `ARCHITECTURE`「SQLite 含 Governance 为业务权威」冲突处,**以本 HANDSOFF + 新 ARCHITECTURE 文首为准**。
|
||||
- 本地 `pr_principle.md` / `pr_review.md` 是 PR 流程台账,不替代本架构 HANDSOFF。
|
||||
@@ -32,9 +32,7 @@ English | [中文](README.zh.md)
|
||||
|
||||
## Latest News
|
||||
|
||||
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-10]**: 🚀 **Open-sourced** quantized **FP8** (**[Hy-Memory-30B-A3B-FP8](https://huggingface.co/tencent/Hy-Memory-30B-A3B-FP8)**) and **GGUF** (**[Hy-Memory-30B-A3B-GGUF](https://huggingface.co/tencent/Hy-Memory-30B-A3B-GGUF)**) versions of **Hy-Memory-30B-A3B** on Hugging Face!
|
||||
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-10]**: 📦 **Released [MemHub v1.0.4](https://github.com/Tencent-Hunyuan/Hy-Memory/releases/tag/v1.0.4)** and **[memhub-cli 1.0.5](https://www.npmjs.com/package/memhub-cli/v/1.0.5)**, with a signed macOS desktop app and verified macOS/Linux Runtimes!
|
||||
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-10]**: 🚀 **Released** **Hy Memory 2.0 (`hy-memory==2.0.4`)** on [PyPI](https://pypi.org/project/hy-memory/2.0.4/) and open-sourced the **Hy-Memory** monorepo together with **[MemHub](memhub/README.md)**!
|
||||
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-10]**: 🚀 **Released** **Hy Memory 2.0 (`hy-memory==2.0.3`)** on [PyPI](https://pypi.org/project/hy-memory/2.0.3/) and open-sourced the **Hy-Memory** monorepo together with **[MemHub](memhub/README.md)**!
|
||||
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-10]**: 🚀 **Open-sourced** **[Hy-Memory-30B-A3B](https://huggingface.co/tencent/Hy-Memory-30B-A3B)** on Hugging Face along with its [technical report](https://arxiv.org/abs/2610.22196) on attributed reinforcement learning for long-term memory writing!
|
||||
* *<mark>[new]</mark>* 🔥🔥 **[2026-09]**: 📊 **Released** **[CMB: A Coding Memory Benchmark for Long-Horizon and Interleaved Coding Agents](docs/papers/CMB_A_Coding_Memory_Benchmark_for_Long_Horizon_and_Interleaved_Coding_Agents.pdf)** (submitted to **ICLR 2027**).
|
||||
* *<mark>[new]</mark>* 🔥🔥 **[2026-08]**: 🎉 **Accepted** at **EMNLP 2026**: **[Memory Beyond Recall: A Dual-Process Cognitive Memory System for Self-Evolving LLM Agents](https://arxiv.org/abs/2606.09483)**!
|
||||
@@ -65,13 +63,13 @@ Welcome to the **Hy Memory Research Family**, covering cognitive chat memory, lo
|
||||
### 1. Installation
|
||||
|
||||
```bash
|
||||
pip install hy-memory==2.0.4 # core (includes ChromaDB and Kuzu)
|
||||
pip install "hy-memory[qdrant]==2.0.4" # + Qdrant
|
||||
pip install "hy-memory[faiss]==2.0.4" # + FAISS
|
||||
pip install "hy-memory[graph]==2.0.4" # + Neo4j
|
||||
pip install "hy-memory[mysql]==2.0.4" # + MySQL cache
|
||||
pip install "hy-memory[rerank]==2.0.4" # + local multilingual cross-encoder rerank
|
||||
pip install "hy-memory[all]==2.0.4" # everything
|
||||
pip install hy-memory==2.0.3 # core (includes ChromaDB and Kuzu)
|
||||
pip install "hy-memory[qdrant]==2.0.3" # + Qdrant
|
||||
pip install "hy-memory[faiss]==2.0.3" # + FAISS
|
||||
pip install "hy-memory[graph]==2.0.3" # + Neo4j
|
||||
pip install "hy-memory[mysql]==2.0.3" # + MySQL cache
|
||||
pip install "hy-memory[rerank]==2.0.3" # + local multilingual cross-encoder rerank
|
||||
pip install "hy-memory[all]==2.0.3" # everything
|
||||
```
|
||||
|
||||
### 2. Configuration
|
||||
@@ -153,7 +151,7 @@ result = client.search(
|
||||
|
||||
result["facts"] # fact layer
|
||||
result["experiences"] # experience layer
|
||||
result["chain_block"] # coding only: version-evolution context (empty unless MEMORY_CODING_CHAIN_BLOCK_ENABLED)
|
||||
result["chain_block"] # coding only: version-evolution context
|
||||
```
|
||||
|
||||
### 5. Routing, Source Isolation & Advanced APIs
|
||||
|
||||
+9
-11
@@ -32,9 +32,7 @@
|
||||
|
||||
## 最新动态
|
||||
|
||||
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-10]**: 🚀 **模型开源** 在 Hugging Face 发布 **Hy-Memory-30B-A3B** 的 **FP8**(**[Hy-Memory-30B-A3B-FP8](https://huggingface.co/tencent/Hy-Memory-30B-A3B-FP8)**)与 **GGUF**(**[Hy-Memory-30B-A3B-GGUF](https://huggingface.co/tencent/Hy-Memory-30B-A3B-GGUF)**)量化版本模型!
|
||||
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-10]**: 📦 **正式发布 [MemHub v1.0.4](https://github.com/Tencent-Hunyuan/Hy-Memory/releases/tag/v1.0.4)** 与 **[memhub-cli 1.0.5](https://www.npmjs.com/package/memhub-cli/v/1.0.5)**,提供已签名的 macOS 桌面应用及经过验证的 macOS/Linux Runtime!
|
||||
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-10]**: 🚀 **正式发布** **Hy Memory 2.0 (`hy-memory==2.0.4`)**(已上线 [PyPI](https://pypi.org/project/hy-memory/2.0.4/)),并开源 **Hy-Memory** monorepo 与 **[MemHub](memhub/README.md)**!
|
||||
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-10]**: 🚀 **正式发布** **Hy Memory 2.0 (`hy-memory==2.0.3`)**(已上线 [PyPI](https://pypi.org/project/hy-memory/2.0.3/)),并开源 **Hy-Memory** monorepo 与 **[MemHub](memhub/README.md)**!
|
||||
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-10]**: 🚀 **模型开源** 在 Hugging Face 发布 **[Hy-Memory-30B-A3B](https://huggingface.co/tencent/Hy-Memory-30B-A3B)** 长期记忆写入模型及[技术报告](https://arxiv.org/abs/2610.22196)!
|
||||
* *<mark>[new]</mark>* 🔥🔥 **[2026-09]**: 📊 **基准与方法发布** 推出长周期与交错式编程记忆评测基准 **[CMB: A Coding Memory Benchmark for Long-Horizon and Interleaved Coding Agents](docs/papers/CMB_A_Coding_Memory_Benchmark_for_Long_Horizon_and_Interleaved_Coding_Agents.pdf)**(已投稿 **ICLR 2027**)。
|
||||
* *<mark>[new]</mark>* 🔥🔥 **[2026-08]**: 🎉 **论文录用** 双系统认知记忆架构论文 **[Memory Beyond Recall: A Dual-Process Cognitive Memory System for Self-Evolving LLM Agents](https://arxiv.org/abs/2606.09483)** 被 **EMNLP 2026** 接收!
|
||||
@@ -65,13 +63,13 @@
|
||||
### 1. 安装
|
||||
|
||||
```bash
|
||||
pip install hy-memory==2.0.4 # 核心(含 ChromaDB 与 Kuzu)
|
||||
pip install "hy-memory[qdrant]==2.0.4" # + Qdrant
|
||||
pip install "hy-memory[faiss]==2.0.4" # + FAISS
|
||||
pip install "hy-memory[graph]==2.0.4" # + Neo4j
|
||||
pip install "hy-memory[mysql]==2.0.4" # + MySQL 缓存
|
||||
pip install "hy-memory[rerank]==2.0.4" # + 本地多语言 cross-encoder rerank
|
||||
pip install "hy-memory[all]==2.0.4" # 全部
|
||||
pip install hy-memory==2.0.3 # 核心(含 ChromaDB 与 Kuzu)
|
||||
pip install "hy-memory[qdrant]==2.0.3" # + Qdrant
|
||||
pip install "hy-memory[faiss]==2.0.3" # + FAISS
|
||||
pip install "hy-memory[graph]==2.0.3" # + Neo4j
|
||||
pip install "hy-memory[mysql]==2.0.3" # + MySQL 缓存
|
||||
pip install "hy-memory[rerank]==2.0.3" # + 本地多语言 cross-encoder rerank
|
||||
pip install "hy-memory[all]==2.0.3" # 全部
|
||||
```
|
||||
|
||||
### 2. 配置
|
||||
@@ -153,7 +151,7 @@ result = client.search(
|
||||
|
||||
result["facts"] # 事实层
|
||||
result["experiences"] # 经验层
|
||||
result["chain_block"] # 仅 coding:版本演进上下文(默认空;需 MEMORY_CODING_CHAIN_BLOCK_ENABLED)
|
||||
result["chain_block"] # 仅 coding:版本演进上下文
|
||||
```
|
||||
|
||||
### 5. 自动路由、来源过滤与进阶 API
|
||||
|
||||
@@ -197,7 +197,6 @@ SDK 不设置 Rerank Search 并发上限。开启后模型在 `HyMemoryClient`
|
||||
| `MEMORY_HISTORY_DB_PATH` | `{DATA_DIR}/data/history.db` | History SQLite 路径 |
|
||||
| `MEMORY_HISTORY_RECORD_SEARCHES` | `false` | 是否记录 Search 操作 |
|
||||
| `MEMORY_CODING_CHAIN_REPO_ROOT` | — | coding 版本链读取本地 Git diff 的仓库根目录 |
|
||||
| `MEMORY_CODING_CHAIN_BLOCK_ENABLED` | `false` | search(scene=coding) 是否注入版本演进链块(diff / `chain_block`);默认关 |
|
||||
|
||||
## HTTP Server
|
||||
|
||||
|
||||
@@ -78,8 +78,7 @@ default 多-query 超过 `MEMORY_QUERY_REWRITE_MAX_CHARS` 时不改写:Coding
|
||||
Coding 关闭则直接使用最后一条 query。
|
||||
|
||||
路由到 `working` / `coding` 后,Search 读取独立生产力记忆库,返回 `facts` /
|
||||
`experiences`,不使用 chat 三通道结构。`coding` 还返回 `chain_block` / `chain_used`
|
||||
(默认空;需 `MEMORY_CODING_CHAIN_BLOCK_ENABLED` 才注入 diff 块)。
|
||||
`experiences`,不使用 chat 三通道结构。`coding` 还返回 `chain_block` / `chain_used`。
|
||||
|
||||
## Chat 成功返回结构
|
||||
|
||||
@@ -153,7 +152,7 @@ client = HyMemoryClient.from_config({
|
||||
安装可选依赖:
|
||||
|
||||
```bash
|
||||
pip install 'hy-memory[rerank]==2.0.4'
|
||||
pip install 'hy-memory[rerank]==2.0.1'
|
||||
```
|
||||
|
||||
启用后,`HyMemoryClient(...)` 构造阶段会完成模型下载/读取、device 放置和一次 dummy
|
||||
|
||||
+3
-2
@@ -37,7 +37,8 @@ do not belong in the public source tree.
|
||||
- Root GitHub workflows run SDK and MemHub checks from their respective working
|
||||
directories. Native lifecycle workflows produce reviewable Actions artifacts;
|
||||
tag releases remain fail-closed on public identity, signing and notarization.
|
||||
- Hy Memory is published as `hy-memory==2.0.4`. MemHub pins that public
|
||||
distribution; its selected npm package is `memhub-cli` 1.0.5.
|
||||
- Hy Memory is published as `hy-memory==2.0.3`. MemHub pins that public
|
||||
distribution; its selected npm package is `memhub-cli`, which is not yet
|
||||
published.
|
||||
- Hy-Memory source uses Apache-2.0 at the repository root. Third-party
|
||||
components retain their own licenses and attribution notices.
|
||||
@@ -168,11 +168,9 @@ working/coding 搜索返回:
|
||||
|
||||
`limit` 分别限制 facts 与 experiences,因此两层最多各返回 `limit` 条。
|
||||
|
||||
`scene="coding"` 额外返回 `chain_block` 和 `chain_used` 字段(键始终存在;默认内容为空):
|
||||
`scene="coding"` 额外返回 `chain_block` 和 `chain_used`:
|
||||
|
||||
- 注入 diff / 版本演进块需显式开启 `MEMORY_CODING_CHAIN_BLOCK_ENABLED=true`
|
||||
(或 `CodingConfig.chain_block_enabled=True`)。默认关闭,以节约 prompt token。
|
||||
- 开启后,`chain_block` 是与已召回 fact 对齐的版本演进上下文;无法挂链时为 `""`。
|
||||
- `chain_block` 是与已召回 fact 对齐的版本演进上下文;无法挂链时为 `""`。
|
||||
- `chain_used` 是已经进入 chain block 的 `facts` 下标。
|
||||
- SDK 不会从 `facts` 删除这些条目。调用方若要拼成“chain block + 剩余 facts”,需自行按 `chain_used` 过滤。
|
||||
|
||||
@@ -224,12 +222,10 @@ working/coding 搜索返回:
|
||||
| `MEMORY_CODING_DB_PATH` | `{DATA_DIR}/data/coding_memory.db` | working 基础 SQLite 路径;coding/trace 路径由此派生 |
|
||||
| `MEMORY_CODING_TOOL_RESULT_MAX_BYTES` | `2048` | 单条 tool result 的字节上限 |
|
||||
| `MEMORY_CODING_VERBATIM_FALLBACK_ENABLED` | `false` | 模型无结果时是否保存原文 fact;默认关闭,避免 assistant/tool trace 直接展示 |
|
||||
| `MEMORY_CODING_CHAIN_BLOCK_ENABLED` | `false` | search 是否注入版本演进链块(`chain_block` / diff);默认关闭(opt-in) |
|
||||
| `MEMORY_EXTRACT_HISTORY_TURNS` | `5` | 回读的历史 user-turn 数;当前 prompt 最终最多使用 5 |
|
||||
| `MEMORY_CODING_CHAIN_REPO_ROOT` | 空 | coding 版本链读取本地 Git diff 的仓库路径 |
|
||||
|
||||
`HCM_*` 变量属于版本链的高级调优项,不是稳定的公共配置契约;普通接入不应依赖其默认值。
|
||||
其中 `HCM_BLOCK_OFF=1` 可在已开启 `MEMORY_CODING_CHAIN_BLOCK_ENABLED` 时做消融(完全不出块)。
|
||||
|
||||
## 8. 定向验证
|
||||
|
||||
|
||||
+7
-23
@@ -5,14 +5,14 @@
|
||||
## 安装与快速开始
|
||||
|
||||
```bash
|
||||
pip install hy-memory==2.0.4
|
||||
pip install hy-memory==2.0.3
|
||||
|
||||
# 可选后端
|
||||
pip install 'hy-memory[qdrant]==2.0.4'
|
||||
pip install 'hy-memory[faiss]==2.0.4'
|
||||
pip install 'hy-memory[graph]==2.0.4' # Neo4j;Kuzu 已在核心依赖中
|
||||
pip install 'hy-memory[mysql]==2.0.4'
|
||||
pip install 'hy-memory[rerank]==2.0.4' # 本地多语言 cross-encoder rerank
|
||||
pip install 'hy-memory[qdrant]==2.0.1'
|
||||
pip install 'hy-memory[faiss]==2.0.1'
|
||||
pip install 'hy-memory[graph]==2.0.1' # Neo4j;Kuzu 已在核心依赖中
|
||||
pip install 'hy-memory[mysql]==2.0.1'
|
||||
pip install 'hy-memory[rerank]==2.0.1' # 本地多语言 cross-encoder rerank
|
||||
```
|
||||
|
||||
默认 LLM 和 Embedding 均使用 OpenAI-compatible 接口。使用 OpenAI 时可只设置:
|
||||
@@ -106,15 +106,6 @@ result = client.add(
|
||||
)
|
||||
```
|
||||
|
||||
`metadata` 与 `meta` 不是同一字段:
|
||||
|
||||
- `metadata` 是兼容旧调用方保留的 SDK 处理选项,会进入内部
|
||||
`WriteRequest.extra`,不保证持久化或随查询结果返回。
|
||||
- `meta` 是调用方的业务上下文,会持久化并在 `search()`、`get()` 等结果中返回。
|
||||
同一记忆被多次合并时,各次 `meta` 会按写入顺序保存在列表中。
|
||||
|
||||
需要以后读回的数据必须放在 `meta`,不要放在 `metadata`。
|
||||
|
||||
`scene` 的公开取值:
|
||||
|
||||
- `None`:普通文本直接走 chat;messages 含 tool 时执行快速场景判断。
|
||||
@@ -210,8 +201,7 @@ result = client.search(
|
||||
```
|
||||
|
||||
`working` / `coding` 路径返回 `scene="productivity"`、`facts`、`experiences`,不使用
|
||||
chat 的三桶结构;`coding` 还会返回 `chain_block` 和 `chain_used` 字段(默认空;
|
||||
需 `MEMORY_CODING_CHAIN_BLOCK_ENABLED=true` 才注入 diff 块)。
|
||||
chat 的三桶结构;`coding` 还会返回 `chain_block` 和 `chain_used`。
|
||||
|
||||
#### `search_mode`
|
||||
|
||||
@@ -273,8 +263,6 @@ catalog 也可以在没有基础 L0 时形成只读的合成画像。
|
||||
|
||||
```python
|
||||
memory = client.get("memory_id")
|
||||
# lite 模式写入的是 L1_RAW;按 ID 读取时必须显式选择返回原文。
|
||||
raw_memory = client.get("memory_id", include_raw=True)
|
||||
result = client.update("memory_id", "更新后的内容")
|
||||
result = client.delete("memory_id")
|
||||
result = client.delete_all(user_id="user_001")
|
||||
@@ -294,10 +282,6 @@ result = client.clone_user("user_001", "user_002", agent_id=None)
|
||||
对应异步方法为 `async_get()`、`async_update()`、`async_delete()`、
|
||||
`async_delete_all()`、`async_list_memories()`、`async_clone_user()`。
|
||||
|
||||
`get()` / `async_get()` 默认继续隐藏 `L1_RAW`,因此不会改变现有调用方的原文访问
|
||||
边界。只有显式传入 `include_raw=True` 才返回该层;这也是 lite 模式按 ID 读取
|
||||
写入内容及其 `meta` 的方式。MemHub 的 HTTP 读取接口不开放此参数。
|
||||
|
||||
普通 `list_memories()` 返回 `vdb` 桶,并在 Graph 有结果时附带 `graph`;传入
|
||||
`workspace_id` 时返回生产力记忆的 `scene="productivity"` 和 `experiences`。
|
||||
|
||||
|
||||
@@ -55,13 +55,6 @@ except Exception:
|
||||
from .client import HyMemoryClient
|
||||
from .inspector import MemoryInspector
|
||||
from .runtime import SharedRuntime
|
||||
from .snapshot import (
|
||||
DataPlaneRestore,
|
||||
DataPlaneSnapshot,
|
||||
DataPlaneSnapshotRejected,
|
||||
restore_local_data_plane,
|
||||
snapshot_local_data_plane,
|
||||
)
|
||||
|
||||
# ====== 配置 ======
|
||||
from .config import MemoryConfig
|
||||
@@ -118,11 +111,6 @@ __all__ = [
|
||||
"HyMemoryClient",
|
||||
"MemoryInspector",
|
||||
"SharedRuntime",
|
||||
"DataPlaneRestore",
|
||||
"DataPlaneSnapshot",
|
||||
"DataPlaneSnapshotRejected",
|
||||
"restore_local_data_plane",
|
||||
"snapshot_local_data_plane",
|
||||
# 配置
|
||||
"MemoryConfig",
|
||||
# Embedding 端点覆盖
|
||||
|
||||
@@ -22,8 +22,7 @@ logger = logging.getLogger(__name__)
|
||||
|
||||
# ================================================================
|
||||
# 数据结构
|
||||
# ================================================================
|
||||
@dataclass
|
||||
# ================================================================@dataclass
|
||||
class SessionSummaryResult:
|
||||
"""Session 摘要结果 (V2)"""
|
||||
success: bool
|
||||
|
||||
@@ -39,7 +39,6 @@ import logging
|
||||
import hashlib
|
||||
import json
|
||||
import re
|
||||
from datetime import datetime
|
||||
from typing import Any, Dict, List, Optional
|
||||
import uuid
|
||||
|
||||
@@ -253,17 +252,6 @@ def _series_episode_id(
|
||||
return str(uuid.uuid5(uuid.NAMESPACE_URL, "hy-memory:series-episode:" + raw))
|
||||
|
||||
|
||||
def _series_memory_at(times: List[int]) -> Optional[datetime]:
|
||||
"""Earliest member timestamp as the same local wall clock it came from.
|
||||
|
||||
``_node_timestamp`` produces ``datetime.timestamp()`` values. Reading them
|
||||
back with ``utcfromtimestamp`` turns them into a naive *UTC* datetime, which
|
||||
the vector payload later re-interprets as local time and shifts by the
|
||||
machine's UTC offset.
|
||||
"""
|
||||
return datetime.fromtimestamp(min(times)) if times else None
|
||||
|
||||
|
||||
def _node_timestamp(node: Any) -> Optional[int]:
|
||||
value = getattr(node, "memory_at", None) or getattr(node, "gmt_created", None)
|
||||
if value is None:
|
||||
@@ -986,6 +974,8 @@ async def finalize_series_episode_containers(
|
||||
Stored text is only a title plus a structural intro. Exact L2 bodies remain in
|
||||
their canonical nodes and are referenced by ID from dated timeline segments.
|
||||
"""
|
||||
from datetime import datetime
|
||||
|
||||
from ..models.memory import MemoryLayer, MemoryNode, MemoryStatus, SourceType
|
||||
from ..pipelines._retrieval.session_episode import (
|
||||
add_parent_episode_membership,
|
||||
@@ -1071,7 +1061,7 @@ async def finalize_series_episode_containers(
|
||||
segment["memory_at"] for segment in plan["timeline"]
|
||||
if segment.get("memory_at") is not None
|
||||
]
|
||||
memory_at = _series_memory_at(times)
|
||||
memory_at = datetime.utcfromtimestamp(min(times)) if times else None
|
||||
content = f"{plan['title']}. {plan['intro']}".strip()
|
||||
series = MemoryNode(
|
||||
node_id=series_id, user_id=user_id,
|
||||
|
||||
@@ -1123,7 +1123,6 @@ class Extractor:
|
||||
max_tokens=self._llm_config.agent_max_tokens,
|
||||
tools=None,
|
||||
tool_choice=None,
|
||||
audit_stage="extract",
|
||||
)
|
||||
self._call_count += 1
|
||||
total_tokens += response.tokens_used
|
||||
@@ -1234,7 +1233,6 @@ class Extractor:
|
||||
max_tokens=self._llm_config.agent_max_tokens,
|
||||
tools=None,
|
||||
tool_choice=None,
|
||||
audit_stage="extract",
|
||||
)
|
||||
self._call_count += 1
|
||||
total_tokens += response.tokens_used
|
||||
|
||||
@@ -34,7 +34,6 @@ import os
|
||||
import weakref
|
||||
|
||||
from ..config import MemoryConfig
|
||||
from ..utils.model_call_context import model_call_stage
|
||||
from ..core.workload import llm_slot
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
@@ -349,7 +348,6 @@ class LLMProvider:
|
||||
stop: List[str] = None,
|
||||
tools: Optional[List[Dict[str, Any]]] = None,
|
||||
tool_choice: Optional[Any] = None,
|
||||
audit_stage: Optional[str] = None,
|
||||
**kwargs
|
||||
) -> LLMResponse:
|
||||
"""
|
||||
@@ -388,16 +386,15 @@ class LLMProvider:
|
||||
len(prompt), _content_fingerprint(prompt),
|
||||
)
|
||||
|
||||
with model_call_stage(audit_stage):
|
||||
response = await self._call_llm(
|
||||
prompt=prompt,
|
||||
max_tokens=max_tokens,
|
||||
temperature=temperature,
|
||||
stop=stop,
|
||||
tools=tools,
|
||||
tool_choice=tool_choice,
|
||||
**kwargs
|
||||
)
|
||||
response = await self._call_llm(
|
||||
prompt=prompt,
|
||||
max_tokens=max_tokens,
|
||||
temperature=temperature,
|
||||
stop=stop,
|
||||
tools=tools,
|
||||
tool_choice=tool_choice,
|
||||
**kwargs
|
||||
)
|
||||
|
||||
self._total_calls += 1
|
||||
self._total_tokens += response.tokens_used
|
||||
@@ -598,7 +595,6 @@ class LLMProvider:
|
||||
temperature: Optional[float] = None,
|
||||
tools: Optional[List[Dict[str, Any]]] = None,
|
||||
tool_choice: Optional[Any] = None,
|
||||
audit_stage: Optional[str] = None,
|
||||
) -> LLMResponse:
|
||||
"""
|
||||
多轮 messages 调用(支持 tool results 回传)。
|
||||
@@ -626,14 +622,13 @@ class LLMProvider:
|
||||
f"msgs={len(messages)} tools={len(tools) if tools else 0}"
|
||||
)
|
||||
|
||||
with model_call_stage(audit_stage):
|
||||
response = await self._chat_completions(
|
||||
messages,
|
||||
max_tokens=max_tokens,
|
||||
temperature=temperature,
|
||||
tools=tools,
|
||||
tool_choice=tool_choice,
|
||||
)
|
||||
response = await self._chat_completions(
|
||||
messages,
|
||||
max_tokens=max_tokens,
|
||||
temperature=temperature,
|
||||
tools=tools,
|
||||
tool_choice=tool_choice,
|
||||
)
|
||||
|
||||
self._total_calls += 1
|
||||
self._total_tokens += response.tokens_used
|
||||
|
||||
@@ -669,7 +669,6 @@ class MemoryReconciler:
|
||||
response = await llm.complete(
|
||||
prompt=prompt,
|
||||
max_tokens=500,
|
||||
audit_stage="reconcile",
|
||||
)
|
||||
text = response.content.strip()
|
||||
if "```json" in text:
|
||||
@@ -1384,7 +1383,6 @@ class MemoryReconciler:
|
||||
response = await llm.complete(
|
||||
prompt=prompt,
|
||||
max_tokens=dyn_max_tokens,
|
||||
audit_stage="reconcile",
|
||||
)
|
||||
except Exception as e:
|
||||
# LLM 彻底不可用(provider 已按 max_retries 重试过,网关 500 /
|
||||
|
||||
+118
-667
File diff suppressed because it is too large.
Load diff
@@ -647,9 +647,7 @@ class HyCodeMem:
|
||||
top-k,但它们的 diff 对当前工作区不可能有用,因此挂链前必须做同仓过滤。
|
||||
"""
|
||||
facts = facts or []
|
||||
# 高级消融开关: HCM_BLOCK_OFF=1 → 完全不出块, 也不从 bullet 删任何 fact。
|
||||
# 公共 opt-in 见 CodingConfig.chain_block_enabled /
|
||||
# MEMORY_CODING_CHAIN_BLOCK_ENABLED(默认关;客户端在调用 decorate 前拦截)。
|
||||
# 开关: HCM_BLOCK_OFF=1 → 完全不出块, 也不从 bullet 删任何 fact。
|
||||
if os.environ.get("HCM_BLOCK_OFF") == "1":
|
||||
return "", []
|
||||
picked, used = [], []
|
||||
|
||||
@@ -919,7 +919,6 @@ class CodingMemoryExtractor:
|
||||
resp = await self.llm_provider.complete(
|
||||
prompt=prompt,
|
||||
max_tokens=self.max_tokens,
|
||||
audit_stage="coding_extract",
|
||||
)
|
||||
except Exception as e:
|
||||
logger.warning(f"[coding-extract] LLM call failed: {e}")
|
||||
|
||||
@@ -466,13 +466,7 @@ class WorkingFactReconciler:
|
||||
return [], []
|
||||
|
||||
candidates = await self._retrieve_candidates(
|
||||
drafts,
|
||||
user_id=user_id,
|
||||
agent_id=agent_id,
|
||||
session_id=session_id,
|
||||
workspace_id=workspace_id,
|
||||
branch=branch,
|
||||
source=source,
|
||||
drafts, user_id=user_id, source=source,
|
||||
)
|
||||
|
||||
try:
|
||||
@@ -505,15 +499,7 @@ class WorkingFactReconciler:
|
||||
# ------------------------------------------------------------
|
||||
|
||||
async def _retrieve_candidates(
|
||||
self,
|
||||
drafts: List[WorkingFactDraft],
|
||||
*,
|
||||
user_id: str,
|
||||
agent_id: str,
|
||||
session_id: Optional[str],
|
||||
workspace_id: Optional[str],
|
||||
branch: Optional[str],
|
||||
source: str,
|
||||
self, drafts: List[WorkingFactDraft], *, user_id: str, source: str
|
||||
) -> List[List[WorkingFact]]:
|
||||
"""每条 draft 用 statement 召回同 user 的 top-N active 事实。
|
||||
|
||||
@@ -526,10 +512,6 @@ class WorkingFactReconciler:
|
||||
关掉时整份召回原样交给 LLM。适用于 subject 不是稳定实体名的数据集(自由文本、
|
||||
位置标签),那里精确字符串相等会把同实体判成不同主体,反而拦掉真实更新。
|
||||
"""
|
||||
# Empty/legacy session metadata remains readable, but cannot prove
|
||||
# conversation authority for a destructive SUPERSEDE.
|
||||
if not session_id:
|
||||
return [[] for _ in drafts]
|
||||
try:
|
||||
embeddings = await self.embed_service.embed_batch(
|
||||
[d.statement for d in drafts]
|
||||
@@ -543,10 +525,6 @@ class WorkingFactReconciler:
|
||||
try:
|
||||
hits = await self.store.search_facts(
|
||||
emb, user_id=user_id, top=self.candidates_per_draft,
|
||||
agent_id=agent_id,
|
||||
session_id=session_id,
|
||||
workspace_id=workspace_id,
|
||||
branch=branch,
|
||||
# 写入侧同样接词面通道:候选选错了 LLM 就没有机会判对。这里不传
|
||||
# min_score——写入侧要的是"有哪些可能的前身",让 LLM 看更宽的名单,
|
||||
# 阈值过滤是读取侧的事。
|
||||
@@ -593,7 +571,6 @@ class WorkingFactReconciler:
|
||||
resp = await self.llm_provider.complete(
|
||||
prompt=prompt,
|
||||
max_tokens=self.max_tokens,
|
||||
audit_stage="coding_fact_reconcile",
|
||||
)
|
||||
return self._parse_ops(resp.content)
|
||||
|
||||
@@ -660,7 +637,6 @@ class WorkingFactReconciler:
|
||||
resp = await self.llm_provider.complete(
|
||||
prompt=prompt,
|
||||
max_tokens=_CHAIN_SUMMARY_MAX_TOKENS,
|
||||
audit_stage="coding_fact_summary",
|
||||
)
|
||||
except Exception as e:
|
||||
logger.warning(f"[fact-reconcile] chain summary failed: {e}")
|
||||
@@ -816,15 +792,6 @@ class WorkingFactReconciler:
|
||||
old = await self.store.get_fact(old_id, user_id=user_id)
|
||||
if old is None:
|
||||
continue
|
||||
if (
|
||||
old.agent_id != agent_id
|
||||
or old.session_id != session_id
|
||||
or old.workspace_id != workspace_id
|
||||
):
|
||||
logger.warning(
|
||||
f"[fact-reconcile] refused cross-authority SUPERSEDE: {old.fact_id}"
|
||||
)
|
||||
continue
|
||||
# 闸门在候选阶段已经拦过一道,这里再拦一次:候选是按 statement 召回的,
|
||||
# 而 LLM 只能覆盖候选名单里的 id,理论上到不了这儿。真到了就说明前面
|
||||
# 哪一环漏了,宁可让这次覆盖不生效——多留一条旧事实是可恢复的,删掉
|
||||
|
||||
@@ -29,9 +29,6 @@ logger = logging.getLogger(__name__)
|
||||
|
||||
_JUDGE_MAX_TURNS = 20
|
||||
_JUDGE_HEAD_TURNS = 5
|
||||
_JUDGE_USER_CONTENT_LIMIT = 1_000
|
||||
_JUDGE_USER_CONTENT_HEAD = 200
|
||||
_JUDGE_USER_CONTENT_TAIL = 200
|
||||
_JUDGE_MAX_QUERY_CHARS = 300
|
||||
_JUDGE_MAX_QUERIES = 20
|
||||
|
||||
@@ -189,7 +186,6 @@ async def classify_messages_scene(
|
||||
prompt=prompt,
|
||||
max_tokens=max_tokens,
|
||||
temperature=temperature,
|
||||
audit_stage="coding_judge",
|
||||
)
|
||||
_elapsed = round((time.perf_counter() - _t0) * 1000, 2)
|
||||
result = _parse_scene_json(resp.content)
|
||||
@@ -635,13 +631,9 @@ def _format_turns(summary: List[Dict[str, Any]]) -> str:
|
||||
rows = []
|
||||
for t in summary:
|
||||
user = t.get("user", "") or ""
|
||||
if len(user) > _JUDGE_USER_CONTENT_LIMIT:
|
||||
omitted = len(user) - _JUDGE_USER_CONTENT_HEAD - _JUDGE_USER_CONTENT_TAIL
|
||||
user = (
|
||||
user[:_JUDGE_USER_CONTENT_HEAD]
|
||||
+ f"\n[...{omitted} chars omitted...]\n"
|
||||
+ user[-_JUDGE_USER_CONTENT_TAIL:]
|
||||
)
|
||||
# 单 turn 用户文本截到 200 字符
|
||||
if len(user) > 200:
|
||||
user = user[:200] + "..."
|
||||
tools = t.get("tools", []) or []
|
||||
rows.append(
|
||||
f"- turn {t.get('turn', '?')}: user={user!r}, tools={tools}"
|
||||
|
||||
@@ -240,7 +240,7 @@ class CodingMemoryReconciler:
|
||||
)
|
||||
|
||||
# 3. 校验 + 执行
|
||||
executed = await self._execute(ops, drafts, candidates_per_draft)
|
||||
executed = await self._execute(ops, drafts)
|
||||
logger.info(
|
||||
f"[coding-reconcile] drafts={len(drafts)} ops={len(executed)} "
|
||||
+ ", ".join(f"{a}={sum(1 for o in executed if o.action == a)}"
|
||||
@@ -263,12 +263,6 @@ class CodingMemoryReconciler:
|
||||
"""
|
||||
result: List[List[CodingMemory]] = []
|
||||
for d in drafts:
|
||||
# A coding mutation must be attributable to one conversation.
|
||||
# Empty/legacy session metadata stays readable through ordinary
|
||||
# search, but it cannot authorize UPDATE/DELETE.
|
||||
if not d.session_id:
|
||||
result.append([])
|
||||
continue
|
||||
try:
|
||||
# task 为主入口,search_keys 为附加入口(去重、去空)
|
||||
queries = [d.task] + [k for k in (d.search_keys or []) if k and k != d.task]
|
||||
@@ -280,8 +274,6 @@ class CodingMemoryReconciler:
|
||||
hits = await self.store.search_by_query_embedding(
|
||||
emb,
|
||||
user_id=d.user_id,
|
||||
agent_id=d.agent_id,
|
||||
session_id=d.session_id,
|
||||
workspace_id=d.workspace_id,
|
||||
branch=d.branch,
|
||||
top=self.candidates_per_draft * 4,
|
||||
@@ -318,7 +310,6 @@ class CodingMemoryReconciler:
|
||||
resp = await self.llm_provider.complete(
|
||||
prompt=prompt,
|
||||
max_tokens=self.max_tokens,
|
||||
audit_stage="coding_reconcile",
|
||||
)
|
||||
return self._parse_ops(resp.content), prompt, resp.content, resp
|
||||
|
||||
@@ -423,7 +414,6 @@ class CodingMemoryReconciler:
|
||||
self,
|
||||
ops: List[ReconcileOp],
|
||||
drafts: List[CodingMemoryDraft],
|
||||
candidates_per_draft: List[List[CodingMemory]],
|
||||
) -> List[ReconcileOp]:
|
||||
"""
|
||||
校验 + 执行 ops。返回成功执行的 ops(filter 掉无效 op)。
|
||||
@@ -463,11 +453,6 @@ class CodingMemoryReconciler:
|
||||
if not op.target_experience_id:
|
||||
logger.warning(f"[coding-reconcile] drop UPDATE missing target: {op.to_dict()}")
|
||||
continue
|
||||
if not self._target_was_offered(op, candidates_per_draft):
|
||||
logger.warning(
|
||||
f"[coding-reconcile] drop UPDATE outside authority scope: {op.to_dict()}"
|
||||
)
|
||||
continue
|
||||
if op.draft_idx in consumed:
|
||||
logger.warning(f"[coding-reconcile] drop UPDATE on consumed draft: {op.to_dict()}")
|
||||
continue
|
||||
@@ -490,11 +475,6 @@ class CodingMemoryReconciler:
|
||||
if not op.target_experience_id:
|
||||
logger.warning(f"[coding-reconcile] drop DELETE missing target: {op.to_dict()}")
|
||||
continue
|
||||
if not self._target_was_offered(op, candidates_per_draft):
|
||||
logger.warning(
|
||||
f"[coding-reconcile] drop DELETE outside authority scope: {op.to_dict()}"
|
||||
)
|
||||
continue
|
||||
# DELETE 需要知道 user_id;通过 draft_idx(如有)或第一个 draft 推断
|
||||
user_id = self._infer_user_id(op, drafts)
|
||||
if not user_id:
|
||||
@@ -531,19 +511,6 @@ class CodingMemoryReconciler:
|
||||
|
||||
return executed
|
||||
|
||||
@staticmethod
|
||||
def _target_was_offered(
|
||||
op: ReconcileOp,
|
||||
candidates_per_draft: List[List[CodingMemory]],
|
||||
) -> bool:
|
||||
"""Only mutate a target frozen into this draft's scoped candidate set."""
|
||||
if op.draft_idx is None or not (0 <= op.draft_idx < len(candidates_per_draft)):
|
||||
return False
|
||||
return any(
|
||||
candidate.experience_id == op.target_experience_id
|
||||
for candidate in candidates_per_draft[op.draft_idx]
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def _infer_user_id(
|
||||
op: ReconcileOp, drafts: List[CodingMemoryDraft]
|
||||
|
||||
+15
-98
@@ -209,14 +209,6 @@ def _scope_where(
|
||||
return sql, params
|
||||
|
||||
|
||||
def _in_clause(column: str, values: Optional[List[str]]) -> Tuple[str, List[Any]]:
|
||||
"""SQL ``column IN (...)`` + parameters; empty values mean no condition."""
|
||||
if not values:
|
||||
return "", []
|
||||
placeholders = ",".join("?" for _ in values)
|
||||
return f"{column} IN ({placeholders})", list(values)
|
||||
|
||||
|
||||
def _scope_matches(
|
||||
row_workspace_id: Optional[str],
|
||||
row_branch: Optional[str],
|
||||
@@ -537,20 +529,10 @@ class CodingMemoryStore:
|
||||
"""覆盖式更新一条 coding memory。"""
|
||||
# 拿旧 keys 长度,先删旧 VDB rows
|
||||
prev = await self.get_by_id(memory.experience_id, user_id=memory.user_id)
|
||||
if prev is None:
|
||||
raise ValueError("coding_memory_update_target_missing")
|
||||
if (
|
||||
not prev.session_id
|
||||
or not memory.session_id
|
||||
or prev.agent_id != memory.agent_id
|
||||
or prev.session_id != memory.session_id
|
||||
or prev.workspace_id != memory.workspace_id
|
||||
):
|
||||
raise ValueError("coding_memory_update_authority_mismatch")
|
||||
prev_key_count = 1 + len(prev.search_keys)
|
||||
prev_key_count = (1 + len(prev.search_keys)) if prev else 0
|
||||
|
||||
memory.updated_at = datetime.utcnow()
|
||||
if memory.created_at is None:
|
||||
if memory.created_at is None and prev is not None:
|
||||
memory.created_at = prev.created_at
|
||||
await _run_in_pool(self._upsert_sql_sync, memory)
|
||||
# 删旧 VDB
|
||||
@@ -689,10 +671,6 @@ class CodingMemoryStore:
|
||||
query_embedding: List[float],
|
||||
user_id: str,
|
||||
*,
|
||||
agent_id: Optional[str] = None,
|
||||
session_id: Optional[str] = None,
|
||||
agent_ids: Optional[List[str]] = None,
|
||||
session_ids: Optional[List[str]] = None,
|
||||
workspace_id: Optional[str] = None,
|
||||
branch: Optional[str] = None,
|
||||
top: int = 40,
|
||||
@@ -710,34 +688,21 @@ class CodingMemoryStore:
|
||||
},
|
||||
...
|
||||
]
|
||||
agent_id / session_id(或 agent_ids / session_ids)是写授权作用域,
|
||||
传入时精确匹配;workspace_id / branch 的规则见 _scope_where。
|
||||
不传任何作用域则维持纯语义召回;传了则只保留 SQLite 里落在作用域内的经验,
|
||||
并按选择率放大向量取数,避免"先取 top-N 再过滤"把召回吃掉。
|
||||
workspace_id / branch 是作用域过滤,规则见 _scope_where:不传则纯语义召回
|
||||
(行为与加过滤前完全一致);传了则只保留 SQLite 里落在作用域内的经验,并按
|
||||
作用域的选择率放大向量取数,避免"先取 top-N 再过滤"把召回吃掉。
|
||||
"""
|
||||
if self._vec is None:
|
||||
return []
|
||||
|
||||
effective_agent_ids = list(agent_ids) if agent_ids else (
|
||||
[agent_id] if agent_id else []
|
||||
)
|
||||
effective_session_ids = list(session_ids) if session_ids else (
|
||||
[session_id] if session_id else []
|
||||
)
|
||||
allowed: Optional[Set[str]] = None
|
||||
if effective_agent_ids or effective_session_ids or workspace_id:
|
||||
if workspace_id:
|
||||
allowed, total = await _run_in_pool(
|
||||
self._authority_scoped_experience_ids_sync,
|
||||
user_id,
|
||||
effective_agent_ids,
|
||||
effective_session_ids,
|
||||
workspace_id,
|
||||
branch,
|
||||
self._scoped_experience_ids_sync, user_id, workspace_id, branch
|
||||
)
|
||||
if not allowed:
|
||||
logger.info(
|
||||
f"[coding-store] no experience in scope: "
|
||||
f"agent_id={agent_id!r} session_id={session_id!r} "
|
||||
f"workspace_id={workspace_id!r} branch={branch!r} "
|
||||
f"(user has {total}); skip vdb search"
|
||||
)
|
||||
@@ -813,10 +778,6 @@ class CodingMemoryStore:
|
||||
query_embedding: List[float],
|
||||
user_id: str,
|
||||
*,
|
||||
agent_id: Optional[str] = None,
|
||||
session_id: Optional[str] = None,
|
||||
agent_ids: Optional[List[str]] = None,
|
||||
session_ids: Optional[List[str]] = None,
|
||||
workspace_id: Optional[str] = None,
|
||||
branch: Optional[str] = None,
|
||||
top: int = 5,
|
||||
@@ -830,9 +791,8 @@ class CodingMemoryStore:
|
||||
SUPERSEDE 时向量行已被删掉,理论上召不到 superseded;这里再按 SQLite 的
|
||||
status 过一道,避免两边不一致时把过期结论放出去。
|
||||
|
||||
agent_id / session_id(或 agent_ids / session_ids)是写授权作用域,
|
||||
传入时精确匹配;workspace_id / branch 的规则见 _scope_where。
|
||||
不传任何作用域则不过滤,传了按 SQLite 里的值筛,并按选择率放大向量取数。
|
||||
workspace_id / branch 是作用域过滤,规则见 _scope_where,语义与经验层一致:
|
||||
不传不过滤,传了按 SQLite 里的值筛,并按选择率放大向量取数。
|
||||
|
||||
向量库按 top 的 4 倍过采样:向量行和 SQLite 行不是一个事务里写的,中间挂掉
|
||||
就会留下"向量还在、SQLite 没了"的孤儿,软删时删向量失败也会留下 superseded
|
||||
@@ -848,27 +808,14 @@ class CodingMemoryStore:
|
||||
if self._fact_vec is None:
|
||||
return []
|
||||
|
||||
effective_agent_ids = list(agent_ids) if agent_ids else (
|
||||
[agent_id] if agent_id else []
|
||||
)
|
||||
effective_session_ids = list(session_ids) if session_ids else (
|
||||
[session_id] if session_id else []
|
||||
)
|
||||
fetch = top * 4
|
||||
if effective_agent_ids or effective_session_ids or workspace_id:
|
||||
if workspace_id:
|
||||
scoped, total = await _run_in_pool(
|
||||
self._count_active_facts_sync,
|
||||
user_id,
|
||||
effective_agent_ids,
|
||||
effective_session_ids,
|
||||
workspace_id,
|
||||
branch,
|
||||
self._count_active_facts_sync, user_id, workspace_id, branch
|
||||
)
|
||||
if not scoped:
|
||||
logger.info(
|
||||
f"[coding-store] no active fact in scope: "
|
||||
f"agent_ids={effective_agent_ids!r} "
|
||||
f"session_ids={effective_session_ids!r} "
|
||||
f"workspace_id={workspace_id!r} branch={branch!r} "
|
||||
f"(user has {total}); skip fact vdb search"
|
||||
)
|
||||
@@ -905,8 +852,6 @@ class CodingMemoryStore:
|
||||
alive = [
|
||||
(f, scores[f.fact_id]) for f in facts
|
||||
if f.status == "active"
|
||||
and (not effective_agent_ids or f.agent_id in effective_agent_ids)
|
||||
and (not effective_session_ids or f.session_id in effective_session_ids)
|
||||
and _scope_matches(f.workspace_id, f.branch, workspace_id, branch)
|
||||
]
|
||||
kept = [(f, s) for f, s in alive if s >= min_score]
|
||||
@@ -1105,28 +1050,14 @@ class CodingMemoryStore:
|
||||
by_id = {r["experience_id"]: _row_to_memory(r) for r in rows}
|
||||
return [by_id[eid] for eid in experience_ids if eid in by_id]
|
||||
|
||||
def _authority_scoped_experience_ids_sync(
|
||||
self,
|
||||
user_id: str,
|
||||
agent_ids: Optional[List[str]],
|
||||
session_ids: Optional[List[str]],
|
||||
workspace_id: Optional[str],
|
||||
branch: Optional[str],
|
||||
def _scoped_experience_ids_sync(
|
||||
self, user_id: str, workspace_id: str, branch: Optional[str]
|
||||
) -> Tuple[Set[str], int]:
|
||||
"""返回 (落在授权作用域内的 experience_id 集合, 该 user 的经验总数)。
|
||||
"""返回 (落在作用域内的 experience_id 集合, 该 user 的经验总数)。
|
||||
|
||||
一次查询同时拿到过滤结果和选择率分母:前者供向量命中做交集,后者定放大倍数。
|
||||
agent_ids / session_ids 为空表示该维度不筛;非空按 IN 精确匹配。
|
||||
"""
|
||||
scope_sql, scope_params = _scope_where(workspace_id, branch)
|
||||
agent_sql, agent_params = _in_clause("agent_id", agent_ids)
|
||||
if agent_sql:
|
||||
scope_sql = f"({agent_sql}) AND ({scope_sql})"
|
||||
scope_params = [*agent_params, *scope_params]
|
||||
session_sql, session_params = _in_clause("session_id", session_ids)
|
||||
if session_sql:
|
||||
scope_sql = f"({session_sql}) AND ({scope_sql})"
|
||||
scope_params = [*session_params, *scope_params]
|
||||
with self._lock:
|
||||
assert self._conn is not None
|
||||
cur = self._conn.execute(
|
||||
@@ -1138,27 +1069,13 @@ class CodingMemoryStore:
|
||||
return {r["experience_id"] for r in rows if r["in_scope"]}, len(rows)
|
||||
|
||||
def _count_active_facts_sync(
|
||||
self,
|
||||
user_id: str,
|
||||
agent_ids: Optional[List[str]],
|
||||
session_ids: Optional[List[str]],
|
||||
workspace_id: Optional[str],
|
||||
branch: Optional[str],
|
||||
self, user_id: str, workspace_id: str, branch: Optional[str]
|
||||
) -> Tuple[int, int]:
|
||||
"""返回 (作用域内 active 事实数, 该 user 的 active 事实总数)。
|
||||
|
||||
事实层的过滤在 get_facts 拿到对象之后用 _scope_matches 做,这里只要选择率。
|
||||
agent_ids / session_ids 为空表示该维度不筛;非空按 IN 精确匹配。
|
||||
"""
|
||||
scope_sql, scope_params = _scope_where(workspace_id, branch)
|
||||
agent_sql, agent_params = _in_clause("agent_id", agent_ids)
|
||||
if agent_sql:
|
||||
scope_sql = f"({agent_sql}) AND ({scope_sql})"
|
||||
scope_params = [*agent_params, *scope_params]
|
||||
session_sql, session_params = _in_clause("session_id", session_ids)
|
||||
if session_sql:
|
||||
scope_sql = f"({session_sql}) AND ({scope_sql})"
|
||||
scope_params = [*session_params, *scope_params]
|
||||
with self._lock:
|
||||
assert self._conn is not None
|
||||
row = self._conn.execute(
|
||||
|
||||
@@ -1136,9 +1136,6 @@ class CodingConfig:
|
||||
# LLM 失败或没有抽出 fact 时,是否把对话原文存成 working fact。
|
||||
# 默认关闭,避免 assistant/tool 轨迹以 statement 形式直接暴露给消费方。
|
||||
verbatim_fallback_enabled: bool = None
|
||||
# search(scene="coding") 是否注入版本演进链块(diff / chain_block)。
|
||||
# 默认关闭(opt-in):中重度 coding 用户可省下大量 prompt token;写入侧仍可 observe。
|
||||
chain_block_enabled: bool = None
|
||||
|
||||
def __post_init__(self):
|
||||
if self.enable is None:
|
||||
@@ -1156,10 +1153,6 @@ class CodingConfig:
|
||||
self.verbatim_fallback_enabled = _get_env_bool(
|
||||
"MEMORY_CODING_VERBATIM_FALLBACK_ENABLED", False
|
||||
)
|
||||
if self.chain_block_enabled is None:
|
||||
self.chain_block_enabled = _get_env_bool(
|
||||
"MEMORY_CODING_CHAIN_BLOCK_ENABLED", False
|
||||
)
|
||||
|
||||
|
||||
@dataclass
|
||||
|
||||
@@ -114,37 +114,6 @@ class CacheBase(ABC):
|
||||
"""查询 pipeline 中间结果日志"""
|
||||
...
|
||||
|
||||
# ================================================================
|
||||
# Durable write outcomes
|
||||
# ================================================================
|
||||
|
||||
@abstractmethod
|
||||
async def claim_write_outcome(
|
||||
self,
|
||||
request_id: str,
|
||||
user_id: str,
|
||||
agent_id: str,
|
||||
session_id: str,
|
||||
operation: str,
|
||||
) -> Optional[Dict[str, Any]]:
|
||||
"""Claim one idempotent write; return the existing row if already claimed."""
|
||||
...
|
||||
|
||||
@abstractmethod
|
||||
async def finish_write_outcome(
|
||||
self,
|
||||
request_id: str,
|
||||
status: str,
|
||||
result: Dict[str, Any],
|
||||
) -> bool:
|
||||
"""Persist the terminal/unknown result for a claimed write."""
|
||||
...
|
||||
|
||||
@abstractmethod
|
||||
async def get_write_outcome(self, request_id: str) -> Optional[Dict[str, Any]]:
|
||||
"""Read one durable write outcome by exact request id."""
|
||||
...
|
||||
|
||||
# ================================================================
|
||||
# Summary Buffer(rolling summary 的未消费 turn 暂存)
|
||||
#
|
||||
|
||||
+16
-130
@@ -81,21 +81,6 @@ _CREATE_TABLES_SQL = [
|
||||
)
|
||||
""",
|
||||
"""
|
||||
CREATE TABLE IF NOT EXISTS write_outcomes (
|
||||
request_id VARCHAR(256) NOT NULL,
|
||||
user_id VARCHAR(255) NOT NULL,
|
||||
agent_id VARCHAR(255) NOT NULL DEFAULT '',
|
||||
session_id VARCHAR(255) NOT NULL DEFAULT '',
|
||||
operation VARCHAR(32) NOT NULL,
|
||||
status VARCHAR(32) NOT NULL,
|
||||
result_json LONGTEXT NOT NULL,
|
||||
created_at VARCHAR(64) NOT NULL,
|
||||
updated_at VARCHAR(64) NOT NULL,
|
||||
PRIMARY KEY (request_id),
|
||||
KEY idx_write_outcomes_owner (user_id, agent_id, session_id)
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4
|
||||
""",
|
||||
"""
|
||||
CREATE TABLE IF NOT EXISTS system_metrics (
|
||||
minute_ts VARCHAR(32) NOT NULL,
|
||||
data LONGTEXT NOT NULL,
|
||||
@@ -474,128 +459,29 @@ class MysqlCache(CacheBase):
|
||||
logger.warning(f"get_pipeline_logs failed: {e}")
|
||||
return []
|
||||
|
||||
async def claim_write_outcome(
|
||||
self,
|
||||
request_id: str,
|
||||
user_id: str,
|
||||
agent_id: str,
|
||||
session_id: str,
|
||||
operation: str,
|
||||
) -> Optional[Dict[str, Any]]:
|
||||
now = datetime.now().isoformat()
|
||||
inserted = await self._execute(
|
||||
"""
|
||||
INSERT IGNORE INTO write_outcomes (
|
||||
request_id, user_id, agent_id, session_id, operation,
|
||||
status, result_json, created_at, updated_at
|
||||
) VALUES (%s, %s, %s, %s, %s, 'running', '{}', %s, %s)
|
||||
""",
|
||||
(request_id, user_id, agent_id, session_id, operation, now, now),
|
||||
)
|
||||
if inserted == 1:
|
||||
return None
|
||||
return await self.get_write_outcome(request_id)
|
||||
|
||||
async def finish_write_outcome(
|
||||
self,
|
||||
request_id: str,
|
||||
status: str,
|
||||
result: Dict[str, Any],
|
||||
) -> bool:
|
||||
updated = await self._execute(
|
||||
"""
|
||||
UPDATE write_outcomes
|
||||
SET status = %s, result_json = %s, updated_at = %s
|
||||
WHERE request_id = %s
|
||||
""",
|
||||
(
|
||||
status,
|
||||
json.dumps(result, ensure_ascii=False, default=str),
|
||||
datetime.now().isoformat(),
|
||||
request_id,
|
||||
),
|
||||
)
|
||||
return updated == 1
|
||||
|
||||
async def get_write_outcome(self, request_id: str) -> Optional[Dict[str, Any]]:
|
||||
row = await self._fetchone(
|
||||
"SELECT * FROM write_outcomes WHERE request_id = %s",
|
||||
(request_id,),
|
||||
)
|
||||
if row is None:
|
||||
return None
|
||||
try:
|
||||
row["result"] = json.loads(row.pop("result_json") or "{}")
|
||||
except (TypeError, json.JSONDecodeError):
|
||||
row["result"] = {}
|
||||
return row
|
||||
|
||||
# ================================================================
|
||||
# System Metrics(分钟粒度落盘)
|
||||
# ================================================================
|
||||
|
||||
async def store_metrics_minute(self, minute_ts: str, data: dict) -> None:
|
||||
import aiomysql
|
||||
|
||||
try:
|
||||
now = datetime.now().isoformat()
|
||||
# Read-merge-write must be one transaction: two instances flushing
|
||||
# the same minute used to overwrite each other's counters, and a
|
||||
# concurrent insert could lose the whole batch to a duplicate key.
|
||||
for attempt in range(2):
|
||||
try:
|
||||
async with self._pool.acquire() as conn:
|
||||
await conn.begin()
|
||||
try:
|
||||
async with conn.cursor(aiomysql.DictCursor) as cur:
|
||||
await cur.execute(
|
||||
"SELECT data FROM system_metrics "
|
||||
"WHERE minute_ts = %s FOR UPDATE",
|
||||
(minute_ts,),
|
||||
)
|
||||
row = await cur.fetchone()
|
||||
if row is not None:
|
||||
old_raw = (
|
||||
row["data"]
|
||||
if isinstance(row, dict)
|
||||
else row[0]
|
||||
)
|
||||
merged = self._merge_metric_buckets(
|
||||
json.loads(old_raw), data,
|
||||
)
|
||||
await cur.execute(
|
||||
"UPDATE system_metrics SET data = %s, "
|
||||
"created_at = %s WHERE minute_ts = %s",
|
||||
(
|
||||
json.dumps(merged, default=str),
|
||||
now,
|
||||
minute_ts,
|
||||
),
|
||||
)
|
||||
else:
|
||||
await cur.execute(
|
||||
"INSERT INTO system_metrics "
|
||||
"(minute_ts, data, created_at) "
|
||||
"VALUES (%s, %s, %s)",
|
||||
(
|
||||
minute_ts,
|
||||
json.dumps(data, default=str),
|
||||
now,
|
||||
),
|
||||
)
|
||||
await conn.commit()
|
||||
return
|
||||
except Exception:
|
||||
await conn.rollback()
|
||||
raise
|
||||
except (aiomysql.IntegrityError, aiomysql.OperationalError) as error:
|
||||
code = error.args[0] if error.args else None
|
||||
deadlock = code in (1205, 1213)
|
||||
if attempt or (isinstance(error, aiomysql.OperationalError) and not deadlock):
|
||||
raise
|
||||
# A competing writer inserted the row, or InnoDB chose us
|
||||
# as the deadlock victim. Retry so this batch merges.
|
||||
continue
|
||||
# 尝试 UPSERT
|
||||
existing = await self._fetchone(
|
||||
"SELECT data FROM system_metrics WHERE minute_ts = %s", (minute_ts,)
|
||||
)
|
||||
if existing:
|
||||
old_data = json.loads(existing["data"])
|
||||
merged = self._merge_metric_buckets(old_data, data)
|
||||
await self._execute(
|
||||
"UPDATE system_metrics SET data = %s, created_at = %s WHERE minute_ts = %s",
|
||||
(json.dumps(merged, default=str), now, minute_ts),
|
||||
)
|
||||
else:
|
||||
await self._execute(
|
||||
"INSERT INTO system_metrics (minute_ts, data, created_at) VALUES (%s, %s, %s)",
|
||||
(minute_ts, json.dumps(data, default=str), now),
|
||||
)
|
||||
except Exception as e:
|
||||
logger.warning(f"store_metrics_minute failed: {e}")
|
||||
|
||||
|
||||
+125
-232
@@ -7,7 +7,7 @@ Agent Memory V2 - SqliteCache
|
||||
设计模式跟随 history_store.py:
|
||||
- 标准库 sqlite3 + _run_in_sqlite_pool()
|
||||
- WAL 模式 + check_same_thread=False
|
||||
- 每实例 threading.RLock 保护共享连接的全部操作与关闭
|
||||
- threading.Lock 保护写操作
|
||||
|
||||
三张表:
|
||||
- memory_operations / pipeline_logs / system_metrics: 审计与观测
|
||||
@@ -87,20 +87,6 @@ CREATE TABLE IF NOT EXISTS pipeline_logs_v2 (
|
||||
CREATE INDEX IF NOT EXISTS idx_plog_request ON pipeline_logs_v2(request_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_plog_user ON pipeline_logs_v2(user_id);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS write_outcomes (
|
||||
request_id TEXT PRIMARY KEY,
|
||||
user_id TEXT NOT NULL,
|
||||
agent_id TEXT NOT NULL DEFAULT '',
|
||||
session_id TEXT NOT NULL DEFAULT '',
|
||||
operation TEXT NOT NULL,
|
||||
status TEXT NOT NULL,
|
||||
result_json TEXT NOT NULL DEFAULT '{}',
|
||||
created_at TEXT NOT NULL,
|
||||
updated_at TEXT NOT NULL
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_write_outcomes_owner
|
||||
ON write_outcomes(user_id, agent_id, session_id);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS system_metrics (
|
||||
minute_ts TEXT PRIMARY KEY,
|
||||
data TEXT NOT NULL,
|
||||
@@ -136,10 +122,7 @@ class SqliteCache(CacheBase):
|
||||
def __init__(self, config: MemoryConfig):
|
||||
self.config = config
|
||||
self._conn: Optional[sqlite3.Connection] = None
|
||||
# One connection is shared by the executor workers. Use the same lock
|
||||
# for reads, writes, initialization and close so close() cannot free
|
||||
# the connection while another worker is inside sqlite3.
|
||||
self._lock = threading.RLock()
|
||||
self._lock = threading.Lock()
|
||||
|
||||
# 读取 db_path
|
||||
cache_cfg = getattr(config, "cache", None)
|
||||
@@ -161,37 +144,35 @@ class SqliteCache(CacheBase):
|
||||
logger.debug(f"SqliteCache initialized: {self._db_path}")
|
||||
|
||||
def _init_sync(self) -> None:
|
||||
with self._lock:
|
||||
db_dir = os.path.dirname(self._db_path)
|
||||
if db_dir:
|
||||
os.makedirs(db_dir, exist_ok=True)
|
||||
self._conn = sqlite3.connect(self._db_path, check_same_thread=False)
|
||||
self._conn.row_factory = sqlite3.Row
|
||||
self._conn.execute("PRAGMA journal_mode=WAL;")
|
||||
self._conn.executescript(_CREATE_TABLES_SQL)
|
||||
self._migrate_pipeline_logs_columns()
|
||||
self._conn.commit()
|
||||
db_dir = os.path.dirname(self._db_path)
|
||||
if db_dir:
|
||||
os.makedirs(db_dir, exist_ok=True)
|
||||
self._conn = sqlite3.connect(self._db_path, check_same_thread=False)
|
||||
self._conn.row_factory = sqlite3.Row
|
||||
self._conn.execute("PRAGMA journal_mode=WAL;")
|
||||
self._conn.executescript(_CREATE_TABLES_SQL)
|
||||
self._migrate_pipeline_logs_columns()
|
||||
self._conn.commit()
|
||||
|
||||
def _migrate_pipeline_logs_columns(self) -> None:
|
||||
"""给旧库的 pipeline_logs_v2 补 namespace/session/event time 列。"""
|
||||
with self._lock:
|
||||
try:
|
||||
cols = {
|
||||
row["name"]
|
||||
for row in self._conn.execute("PRAGMA table_info(pipeline_logs_v2)")
|
||||
}
|
||||
_text_cols = ("namespace", "session_id")
|
||||
for col in _text_cols:
|
||||
if col not in cols:
|
||||
self._conn.execute(
|
||||
f"ALTER TABLE pipeline_logs_v2 ADD COLUMN {col} TEXT NOT NULL DEFAULT ''"
|
||||
)
|
||||
if "event_at_ms" not in cols:
|
||||
try:
|
||||
cols = {
|
||||
row["name"]
|
||||
for row in self._conn.execute("PRAGMA table_info(pipeline_logs_v2)")
|
||||
}
|
||||
_text_cols = ("namespace", "session_id")
|
||||
for col in _text_cols:
|
||||
if col not in cols:
|
||||
self._conn.execute(
|
||||
"ALTER TABLE pipeline_logs_v2 ADD COLUMN event_at_ms INTEGER NOT NULL DEFAULT 0"
|
||||
f"ALTER TABLE pipeline_logs_v2 ADD COLUMN {col} TEXT NOT NULL DEFAULT ''"
|
||||
)
|
||||
except Exception as e:
|
||||
logger.debug(f"[sqlite] pipeline_logs_v2 column migration skipped: {e}")
|
||||
if "event_at_ms" not in cols:
|
||||
self._conn.execute(
|
||||
"ALTER TABLE pipeline_logs_v2 ADD COLUMN event_at_ms INTEGER NOT NULL DEFAULT 0"
|
||||
)
|
||||
except Exception as e:
|
||||
logger.debug(f"[sqlite] pipeline_logs_v2 column migration skipped: {e}")
|
||||
|
||||
async def close(self) -> None:
|
||||
if self._conn is not None:
|
||||
@@ -199,10 +180,9 @@ class SqliteCache(CacheBase):
|
||||
logger.info("SqliteCache closed")
|
||||
|
||||
def _close_sync(self) -> None:
|
||||
with self._lock:
|
||||
if self._conn:
|
||||
self._conn.close()
|
||||
self._conn = None
|
||||
if self._conn:
|
||||
self._conn.close()
|
||||
self._conn = None
|
||||
|
||||
# ================================================================
|
||||
# 统计
|
||||
@@ -275,40 +255,39 @@ class SqliteCache(CacheBase):
|
||||
"""
|
||||
try:
|
||||
def _query():
|
||||
with self._lock:
|
||||
conditions = []
|
||||
params = []
|
||||
if request_id:
|
||||
conditions.append("request_id = ?")
|
||||
params.append(request_id)
|
||||
if memory_id:
|
||||
conditions.append("memory_id = ?")
|
||||
params.append(memory_id)
|
||||
if user_id:
|
||||
conditions.append("user_id = ?")
|
||||
params.append(user_id)
|
||||
conditions = []
|
||||
params = []
|
||||
if request_id:
|
||||
conditions.append("request_id = ?")
|
||||
params.append(request_id)
|
||||
if memory_id:
|
||||
conditions.append("memory_id = ?")
|
||||
params.append(memory_id)
|
||||
if user_id:
|
||||
conditions.append("user_id = ?")
|
||||
params.append(user_id)
|
||||
|
||||
if not conditions:
|
||||
return []
|
||||
if not conditions:
|
||||
return []
|
||||
|
||||
where = " AND ".join(conditions)
|
||||
rows = self._conn.execute(
|
||||
f"SELECT * FROM memory_operations_v2 WHERE {where} "
|
||||
f"ORDER BY id DESC LIMIT ?",
|
||||
params + [limit],
|
||||
).fetchall()
|
||||
result = []
|
||||
for r in rows:
|
||||
row = dict(r)
|
||||
# 反序列化 supersedes JSON 字符串 → list
|
||||
import json as _json
|
||||
raw_sup = row.get("supersedes", "")
|
||||
try:
|
||||
row["supersedes"] = _json.loads(raw_sup) if raw_sup else []
|
||||
except Exception:
|
||||
row["supersedes"] = []
|
||||
result.append(row)
|
||||
return result
|
||||
where = " AND ".join(conditions)
|
||||
rows = self._conn.execute(
|
||||
f"SELECT * FROM memory_operations_v2 WHERE {where} "
|
||||
f"ORDER BY id DESC LIMIT ?",
|
||||
params + [limit],
|
||||
).fetchall()
|
||||
result = []
|
||||
for r in rows:
|
||||
row = dict(r)
|
||||
# 反序列化 supersedes JSON 字符串 → list
|
||||
import json as _json
|
||||
raw_sup = row.get("supersedes", "")
|
||||
try:
|
||||
row["supersedes"] = _json.loads(raw_sup) if raw_sup else []
|
||||
except Exception:
|
||||
row["supersedes"] = []
|
||||
result.append(row)
|
||||
return result
|
||||
|
||||
return await _run_in_sqlite_pool(_query)
|
||||
except Exception as e:
|
||||
@@ -390,126 +369,43 @@ class SqliteCache(CacheBase):
|
||||
"""
|
||||
try:
|
||||
def _query():
|
||||
with self._lock:
|
||||
conditions = []
|
||||
params = []
|
||||
if request_id:
|
||||
conditions.append("request_id = ?")
|
||||
params.append(request_id)
|
||||
if user_id:
|
||||
conditions.append("user_id = ?")
|
||||
params.append(user_id)
|
||||
if step:
|
||||
conditions.append("step = ?")
|
||||
params.append(step)
|
||||
conditions = []
|
||||
params = []
|
||||
if request_id:
|
||||
conditions.append("request_id = ?")
|
||||
params.append(request_id)
|
||||
if user_id:
|
||||
conditions.append("user_id = ?")
|
||||
params.append(user_id)
|
||||
if step:
|
||||
conditions.append("step = ?")
|
||||
params.append(step)
|
||||
|
||||
if not conditions:
|
||||
return []
|
||||
if not conditions:
|
||||
return []
|
||||
|
||||
where = " AND ".join(conditions)
|
||||
rows = self._conn.execute(
|
||||
f"SELECT * FROM pipeline_logs_v2 WHERE {where} "
|
||||
f"ORDER BY id ASC LIMIT ?",
|
||||
params + [limit],
|
||||
).fetchall()
|
||||
out = [dict(r) for r in rows]
|
||||
# 派生可读的 event_at(ISO ms)供查询接口直接用;event_at_ms 为 0 时留空
|
||||
from datetime import datetime as _dt
|
||||
for _r in out:
|
||||
_ms = _r.get("event_at_ms") or 0
|
||||
_r["event_at"] = (
|
||||
_dt.fromtimestamp(_ms / 1000.0).isoformat(timespec="milliseconds")
|
||||
if _ms else ""
|
||||
)
|
||||
return out
|
||||
where = " AND ".join(conditions)
|
||||
rows = self._conn.execute(
|
||||
f"SELECT * FROM pipeline_logs_v2 WHERE {where} "
|
||||
f"ORDER BY id ASC LIMIT ?",
|
||||
params + [limit],
|
||||
).fetchall()
|
||||
out = [dict(r) for r in rows]
|
||||
# 派生可读的 event_at(ISO ms)供查询接口直接用;event_at_ms 为 0 时留空
|
||||
from datetime import datetime as _dt
|
||||
for _r in out:
|
||||
_ms = _r.get("event_at_ms") or 0
|
||||
_r["event_at"] = (
|
||||
_dt.fromtimestamp(_ms / 1000.0).isoformat(timespec="milliseconds")
|
||||
if _ms else ""
|
||||
)
|
||||
return out
|
||||
|
||||
return await _run_in_sqlite_pool(_query)
|
||||
except Exception as e:
|
||||
logger.warning(f"get_pipeline_logs failed: {e}")
|
||||
return []
|
||||
|
||||
# ================================================================
|
||||
# Durable write outcomes
|
||||
# ================================================================
|
||||
|
||||
async def claim_write_outcome(
|
||||
self,
|
||||
request_id: str,
|
||||
user_id: str,
|
||||
agent_id: str,
|
||||
session_id: str,
|
||||
operation: str,
|
||||
) -> Optional[Dict[str, Any]]:
|
||||
def _claim():
|
||||
with self._lock:
|
||||
now = datetime.now().isoformat()
|
||||
cursor = self._conn.execute(
|
||||
"""
|
||||
INSERT OR IGNORE INTO write_outcomes (
|
||||
request_id, user_id, agent_id, session_id, operation,
|
||||
status, result_json, created_at, updated_at
|
||||
) VALUES (?, ?, ?, ?, ?, 'running', '{}', ?, ?)
|
||||
""",
|
||||
(request_id, user_id, agent_id, session_id, operation, now, now),
|
||||
)
|
||||
self._conn.commit()
|
||||
if cursor.rowcount == 1:
|
||||
return None
|
||||
row = self._conn.execute(
|
||||
"SELECT * FROM write_outcomes WHERE request_id = ?",
|
||||
(request_id,),
|
||||
).fetchone()
|
||||
return dict(row) if row else None
|
||||
|
||||
existing = await _run_in_sqlite_pool(_claim)
|
||||
if existing is None:
|
||||
return None
|
||||
return await self.get_write_outcome(request_id)
|
||||
|
||||
async def finish_write_outcome(
|
||||
self,
|
||||
request_id: str,
|
||||
status: str,
|
||||
result: Dict[str, Any],
|
||||
) -> bool:
|
||||
def _finish():
|
||||
with self._lock:
|
||||
cursor = self._conn.execute(
|
||||
"""
|
||||
UPDATE write_outcomes
|
||||
SET status = ?, result_json = ?, updated_at = ?
|
||||
WHERE request_id = ?
|
||||
""",
|
||||
(
|
||||
status,
|
||||
json.dumps(result, ensure_ascii=False, default=str),
|
||||
datetime.now().isoformat(),
|
||||
request_id,
|
||||
),
|
||||
)
|
||||
self._conn.commit()
|
||||
return cursor.rowcount == 1
|
||||
|
||||
return await _run_in_sqlite_pool(_finish)
|
||||
|
||||
async def get_write_outcome(self, request_id: str) -> Optional[Dict[str, Any]]:
|
||||
def _get():
|
||||
with self._lock:
|
||||
row = self._conn.execute(
|
||||
"SELECT * FROM write_outcomes WHERE request_id = ?",
|
||||
(request_id,),
|
||||
).fetchone()
|
||||
return dict(row) if row else None
|
||||
|
||||
row = await _run_in_sqlite_pool(_get)
|
||||
if row is None:
|
||||
return None
|
||||
try:
|
||||
row["result"] = json.loads(row.pop("result_json") or "{}")
|
||||
except (TypeError, json.JSONDecodeError):
|
||||
row["result"] = {}
|
||||
return row
|
||||
|
||||
# ================================================================
|
||||
# System Metrics(分钟粒度落盘)
|
||||
# ================================================================
|
||||
@@ -518,27 +414,26 @@ class SqliteCache(CacheBase):
|
||||
"""存储一个分钟桶的增量指标数据(UPSERT: 同一分钟多次写入会合并)"""
|
||||
try:
|
||||
def _store():
|
||||
with self._lock:
|
||||
now = datetime.now().isoformat()
|
||||
# UPSERT: 如果同一分钟已存在,合并数据
|
||||
existing = self._conn.execute(
|
||||
"SELECT data FROM system_metrics WHERE minute_ts = ?",
|
||||
(minute_ts,),
|
||||
).fetchone()
|
||||
if existing:
|
||||
import json as _json
|
||||
old_data = _json.loads(existing["data"])
|
||||
merged = self._merge_metric_buckets(old_data, data)
|
||||
self._conn.execute(
|
||||
"UPDATE system_metrics SET data = ?, created_at = ? WHERE minute_ts = ?",
|
||||
(_json.dumps(merged, default=str), now, minute_ts),
|
||||
)
|
||||
else:
|
||||
self._conn.execute(
|
||||
"INSERT INTO system_metrics (minute_ts, data, created_at) VALUES (?, ?, ?)",
|
||||
(minute_ts, json.dumps(data, default=str), now),
|
||||
)
|
||||
self._conn.commit()
|
||||
now = datetime.now().isoformat()
|
||||
# UPSERT: 如果同一分钟已存在,合并数据
|
||||
existing = self._conn.execute(
|
||||
"SELECT data FROM system_metrics WHERE minute_ts = ?",
|
||||
(minute_ts,),
|
||||
).fetchone()
|
||||
if existing:
|
||||
import json as _json
|
||||
old_data = _json.loads(existing["data"])
|
||||
merged = self._merge_metric_buckets(old_data, data)
|
||||
self._conn.execute(
|
||||
"UPDATE system_metrics SET data = ?, created_at = ? WHERE minute_ts = ?",
|
||||
(_json.dumps(merged, default=str), now, minute_ts),
|
||||
)
|
||||
else:
|
||||
self._conn.execute(
|
||||
"INSERT INTO system_metrics (minute_ts, data, created_at) VALUES (?, ?, ?)",
|
||||
(minute_ts, json.dumps(data, default=str), now),
|
||||
)
|
||||
self._conn.commit()
|
||||
await _run_in_sqlite_pool(_store)
|
||||
except Exception as e:
|
||||
logger.warning(f"store_metrics_minute failed: {e}")
|
||||
@@ -547,18 +442,17 @@ class SqliteCache(CacheBase):
|
||||
"""读取指定时间范围内的分钟指标数据"""
|
||||
try:
|
||||
def _load():
|
||||
with self._lock:
|
||||
rows = self._conn.execute(
|
||||
"SELECT data FROM system_metrics WHERE minute_ts >= ? AND minute_ts <= ? ORDER BY minute_ts",
|
||||
(start_ts, end_ts),
|
||||
).fetchall()
|
||||
results = []
|
||||
for row in rows:
|
||||
try:
|
||||
results.append(json.loads(row["data"]))
|
||||
except (json.JSONDecodeError, TypeError):
|
||||
pass
|
||||
return results
|
||||
rows = self._conn.execute(
|
||||
"SELECT data FROM system_metrics WHERE minute_ts >= ? AND minute_ts <= ? ORDER BY minute_ts",
|
||||
(start_ts, end_ts),
|
||||
).fetchall()
|
||||
results = []
|
||||
for row in rows:
|
||||
try:
|
||||
results.append(json.loads(row["data"]))
|
||||
except (json.JSONDecodeError, TypeError):
|
||||
pass
|
||||
return results
|
||||
return await _run_in_sqlite_pool(_load)
|
||||
except Exception as e:
|
||||
logger.warning(f"load_metrics_range failed: {e}")
|
||||
@@ -568,12 +462,11 @@ class SqliteCache(CacheBase):
|
||||
"""删除指定时间之前的 metrics 数据"""
|
||||
try:
|
||||
def _cleanup():
|
||||
with self._lock:
|
||||
self._conn.execute(
|
||||
"DELETE FROM system_metrics WHERE minute_ts < ?",
|
||||
(before_ts,),
|
||||
)
|
||||
self._conn.commit()
|
||||
self._conn.execute(
|
||||
"DELETE FROM system_metrics WHERE minute_ts < ?",
|
||||
(before_ts,),
|
||||
)
|
||||
self._conn.commit()
|
||||
await _run_in_sqlite_pool(_cleanup)
|
||||
except Exception as e:
|
||||
logger.warning(f"cleanup_old_metrics failed: {e}")
|
||||
|
||||
+69
-113
@@ -16,7 +16,6 @@ import json
|
||||
import logging
|
||||
import os
|
||||
import sqlite3
|
||||
import threading
|
||||
from datetime import datetime, timezone
|
||||
from typing import Dict, List, Any, Optional
|
||||
|
||||
@@ -66,11 +65,6 @@ def _now_iso() -> str:
|
||||
return datetime.now(timezone.utc).isoformat()
|
||||
|
||||
|
||||
def _escape_like(value: str) -> str:
|
||||
"""Escape LIKE wildcards so a user id is matched literally."""
|
||||
return value.replace("\\", "\\\\").replace("%", "\\%").replace("_", "\\_")
|
||||
|
||||
|
||||
class HistoryStore:
|
||||
"""
|
||||
SQLite 历史存储 - 记录所有内存操作的审计日志。
|
||||
@@ -95,10 +89,6 @@ class HistoryStore:
|
||||
"""
|
||||
self._db_path = config.history.db_path
|
||||
self._conn: Optional[sqlite3.Connection] = None
|
||||
# sqlite3.Connection.close() must never race with an execute() on the
|
||||
# same connection. The lock also serializes operations on this shared
|
||||
# connection; separate HistoryStore instances remain independent.
|
||||
self._conn_lock = threading.RLock()
|
||||
|
||||
async def initialize(self) -> None:
|
||||
"""创建表和索引(如果不存在)"""
|
||||
@@ -107,21 +97,20 @@ class HistoryStore:
|
||||
|
||||
def _init_sync(self) -> None:
|
||||
"""同步初始化:创建目录、连接数据库、建表建索引"""
|
||||
with self._conn_lock:
|
||||
# 确保目录存在
|
||||
db_dir = os.path.dirname(self._db_path)
|
||||
if db_dir:
|
||||
os.makedirs(db_dir, exist_ok=True)
|
||||
# 确保目录存在
|
||||
db_dir = os.path.dirname(self._db_path)
|
||||
if db_dir:
|
||||
os.makedirs(db_dir, exist_ok=True)
|
||||
|
||||
self._conn = sqlite3.connect(self._db_path, check_same_thread=False)
|
||||
self._conn.row_factory = sqlite3.Row
|
||||
# 开启 WAL 模式,提高并发读写性能
|
||||
self._conn.execute("PRAGMA journal_mode=WAL;")
|
||||
self._conn = sqlite3.connect(self._db_path, check_same_thread=False)
|
||||
self._conn.row_factory = sqlite3.Row
|
||||
# 开启 WAL 模式,提高并发读写性能
|
||||
self._conn.execute("PRAGMA journal_mode=WAL;")
|
||||
|
||||
self._conn.execute(_CREATE_TABLE_SQL)
|
||||
for idx_sql in _CREATE_INDEXES_SQL:
|
||||
self._conn.execute(idx_sql)
|
||||
self._conn.commit()
|
||||
self._conn.execute(_CREATE_TABLE_SQL)
|
||||
for idx_sql in _CREATE_INDEXES_SQL:
|
||||
self._conn.execute(idx_sql)
|
||||
self._conn.commit()
|
||||
|
||||
# ================================================================
|
||||
# 记录操作
|
||||
@@ -229,14 +218,13 @@ class HistoryStore:
|
||||
return await _run_in_sqlite_pool(self._get_history_sync, memory_id)
|
||||
|
||||
def _get_history_sync(self, memory_id: str) -> List[Dict[str, Any]]:
|
||||
with self._conn_lock:
|
||||
if self._conn is None:
|
||||
return []
|
||||
cursor = self._conn.execute(
|
||||
"SELECT * FROM memory_history WHERE memory_id = ? ORDER BY created_at ASC",
|
||||
(memory_id,),
|
||||
)
|
||||
return [dict(row) for row in cursor.fetchall()]
|
||||
if self._conn is None:
|
||||
return []
|
||||
cursor = self._conn.execute(
|
||||
"SELECT * FROM memory_history WHERE memory_id = ? ORDER BY created_at ASC",
|
||||
(memory_id,),
|
||||
)
|
||||
return [dict(row) for row in cursor.fetchall()]
|
||||
|
||||
async def get_recent(
|
||||
self,
|
||||
@@ -256,43 +244,14 @@ class HistoryStore:
|
||||
return await _run_in_sqlite_pool(self._get_recent_sync, isolation_key, limit)
|
||||
|
||||
def _get_recent_sync(self, isolation_key: str, limit: int) -> List[Dict[str, Any]]:
|
||||
with self._conn_lock:
|
||||
if self._conn is None:
|
||||
return []
|
||||
cursor = self._conn.execute(
|
||||
"SELECT * FROM memory_history WHERE isolation_key = ? "
|
||||
"ORDER BY created_at DESC LIMIT ?",
|
||||
(isolation_key, limit),
|
||||
)
|
||||
return [dict(row) for row in cursor.fetchall()]
|
||||
|
||||
async def get_recent_for_user(
|
||||
self,
|
||||
user_id: str,
|
||||
limit: int = 50,
|
||||
) -> List[Dict[str, Any]]:
|
||||
"""获取某 user 下所有 agent 的最近操作。
|
||||
|
||||
isolation_key 形如 ``"{user_id}:{agent_id}"``。user_id 里的 LIKE
|
||||
通配符会被转义,保证只匹配这个 user,不会漏到 ``u1`` 之外。
|
||||
"""
|
||||
prefix = _escape_like(f"{user_id}:") + "%"
|
||||
return await _run_in_sqlite_pool(
|
||||
self._get_recent_prefix_sync, prefix, limit,
|
||||
if self._conn is None:
|
||||
return []
|
||||
cursor = self._conn.execute(
|
||||
"SELECT * FROM memory_history WHERE isolation_key = ? "
|
||||
"ORDER BY created_at DESC LIMIT ?",
|
||||
(isolation_key, limit),
|
||||
)
|
||||
|
||||
def _get_recent_prefix_sync(
|
||||
self, pattern: str, limit: int,
|
||||
) -> List[Dict[str, Any]]:
|
||||
with self._conn_lock:
|
||||
if self._conn is None:
|
||||
return []
|
||||
cursor = self._conn.execute(
|
||||
"SELECT * FROM memory_history WHERE isolation_key LIKE ? ESCAPE '\\' "
|
||||
"ORDER BY created_at DESC LIMIT ?",
|
||||
(pattern, limit),
|
||||
)
|
||||
return [dict(row) for row in cursor.fetchall()]
|
||||
return [dict(row) for row in cursor.fetchall()]
|
||||
|
||||
async def count(self, isolation_key: str = "") -> int:
|
||||
"""
|
||||
@@ -307,17 +266,16 @@ class HistoryStore:
|
||||
return await _run_in_sqlite_pool(self._count_sync, isolation_key)
|
||||
|
||||
def _count_sync(self, isolation_key: str) -> int:
|
||||
with self._conn_lock:
|
||||
if self._conn is None:
|
||||
return 0
|
||||
if isolation_key:
|
||||
cursor = self._conn.execute(
|
||||
"SELECT COUNT(*) FROM memory_history WHERE isolation_key = ?",
|
||||
(isolation_key,),
|
||||
)
|
||||
else:
|
||||
cursor = self._conn.execute("SELECT COUNT(*) FROM memory_history")
|
||||
return cursor.fetchone()[0]
|
||||
if self._conn is None:
|
||||
return 0
|
||||
if isolation_key:
|
||||
cursor = self._conn.execute(
|
||||
"SELECT COUNT(*) FROM memory_history WHERE isolation_key = ?",
|
||||
(isolation_key,),
|
||||
)
|
||||
else:
|
||||
cursor = self._conn.execute("SELECT COUNT(*) FROM memory_history")
|
||||
return cursor.fetchone()[0]
|
||||
|
||||
async def close(self) -> None:
|
||||
"""关闭数据库连接"""
|
||||
@@ -326,10 +284,9 @@ class HistoryStore:
|
||||
logger.info("[HistoryStore] closed")
|
||||
|
||||
def _close_sync(self) -> None:
|
||||
with self._conn_lock:
|
||||
if self._conn:
|
||||
self._conn.close()
|
||||
self._conn = None
|
||||
if self._conn:
|
||||
self._conn.close()
|
||||
self._conn = None
|
||||
|
||||
# ================================================================
|
||||
# 内部
|
||||
@@ -382,34 +339,33 @@ class HistoryStore:
|
||||
role: str = "",
|
||||
extra: Optional[Dict[str, Any]] = None,
|
||||
) -> int:
|
||||
with self._conn_lock:
|
||||
if self._conn is None:
|
||||
return -1
|
||||
now = _now_iso()
|
||||
extra_json = json.dumps(extra or {}, ensure_ascii=False)
|
||||
cursor = self._conn.execute(
|
||||
"""INSERT INTO memory_history
|
||||
(memory_id, isolation_key, event,
|
||||
old_memory, new_memory, old_status, new_status,
|
||||
change_reason, layer, actor_id, role, extra,
|
||||
created_at, updated_at)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)""",
|
||||
(
|
||||
memory_id,
|
||||
isolation_key,
|
||||
event,
|
||||
old_memory,
|
||||
new_memory,
|
||||
old_status,
|
||||
new_status,
|
||||
change_reason,
|
||||
layer,
|
||||
actor_id,
|
||||
role,
|
||||
extra_json,
|
||||
now,
|
||||
now,
|
||||
),
|
||||
)
|
||||
self._conn.commit()
|
||||
return cursor.lastrowid
|
||||
if self._conn is None:
|
||||
return -1
|
||||
now = _now_iso()
|
||||
extra_json = json.dumps(extra or {}, ensure_ascii=False)
|
||||
cursor = self._conn.execute(
|
||||
"""INSERT INTO memory_history
|
||||
(memory_id, isolation_key, event,
|
||||
old_memory, new_memory, old_status, new_status,
|
||||
change_reason, layer, actor_id, role, extra,
|
||||
created_at, updated_at)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)""",
|
||||
(
|
||||
memory_id,
|
||||
isolation_key,
|
||||
event,
|
||||
old_memory,
|
||||
new_memory,
|
||||
old_status,
|
||||
new_status,
|
||||
change_reason,
|
||||
layer,
|
||||
actor_id,
|
||||
role,
|
||||
extra_json,
|
||||
now,
|
||||
now,
|
||||
),
|
||||
)
|
||||
self._conn.commit()
|
||||
return cursor.lastrowid
|
||||
@@ -462,39 +462,14 @@ class FaissVectorStore(VectorStoreBase):
|
||||
return None
|
||||
|
||||
async def get_embeddings(self, node_ids: List[str]) -> Dict[str, List[float]]:
|
||||
"""批量取向量,供去重 / episode 摘要计算 cosine。
|
||||
|
||||
``_data`` 只保存 payload 与 FAISS 内部下标;普通 upsert 不额外存一份
|
||||
embedding,向量本体在 FAISS index 里。这里按当前有效下标 reconstruct,
|
||||
并用 ``_idx_to_id`` 校验,避免读到已被替换的旧下标。
|
||||
"""
|
||||
|
||||
def _get() -> Dict[str, List[float]]:
|
||||
out: Dict[str, List[float]] = {}
|
||||
with self._lock:
|
||||
for nid in node_ids:
|
||||
point_id = self._node_id_to_point_id(nid)
|
||||
entry = self._data.get(point_id)
|
||||
if entry is None:
|
||||
continue
|
||||
embedding = entry.get("embedding")
|
||||
if embedding:
|
||||
out[nid] = list(embedding)
|
||||
continue
|
||||
idx = entry.get("idx")
|
||||
if (
|
||||
idx is None
|
||||
or self._index is None
|
||||
or self._idx_to_id.get(int(idx)) != point_id
|
||||
):
|
||||
continue
|
||||
try:
|
||||
out[nid] = self._index.reconstruct(int(idx)).tolist()
|
||||
except Exception:
|
||||
continue
|
||||
return out
|
||||
|
||||
return await _run_in_vdb_pool(_get)
|
||||
"""批量取向量(in-memory _data 直接读 embedding),供去重算 cosine。"""
|
||||
out: Dict[str, List[float]] = {}
|
||||
for nid in node_ids:
|
||||
point_id = self._node_id_to_point_id(nid)
|
||||
entry = self._data.get(point_id)
|
||||
if entry and entry.get("embedding"):
|
||||
out[nid] = list(entry["embedding"])
|
||||
return out
|
||||
|
||||
# ================================================================
|
||||
# 删除
|
||||
|
||||
@@ -10,8 +10,8 @@ HY Memory - Intention (L7) 召回 + 惰性过期转换 + 复本判定
|
||||
节点的 layer 改写为 l2_fact(持久化 update_payload),并从意图结果里剔除——
|
||||
过期意图变成普通历史事实,下次走 normal 召回;
|
||||
- **判重(重点)**:对幸存意图按下述规则分组,同组只留一条本次返回:
|
||||
同 user_id + 同 agent_id + 同一天 memory_at + valid_until 相同
|
||||
(都为 None 或都相等)+ cosine 相似度 ≥ MEMORY_DEDUP_THRESHOLD (默认 0.95)
|
||||
同 user_id + 同一天 memory_at + valid_until 相同(都为 None 或都相等)
|
||||
+ cosine 相似度 ≥ MEMORY_DEDUP_THRESHOLD (默认 0.95)
|
||||
→ 视为同一个意图的复本(extractor 复读机 / 多 session 重复表达)。
|
||||
保留组里 `gmt_created` 最早的(老的先来先得,稳定 id);
|
||||
- **异步删库**:其余复本 fire-and-forget 从 VDB 删除,不影响本次调用 RT。
|
||||
@@ -112,19 +112,11 @@ def _partition_duplicates(
|
||||
if len(hits) < 2:
|
||||
return list(hits), []
|
||||
|
||||
# 1. 按 (user_id, agent_id, memory_at day, valid_until day) 分桶。
|
||||
# user_id / agent_id 必须进键:多用户检索时不同用户的同一天意图可能
|
||||
# 文本、日期都相同,但它们是两个独立的租户数据;只按日期分桶会把其中
|
||||
# 一个判成复本并硬删除。
|
||||
buckets: Dict[Tuple[Any, Any, Any, Any], List[int]] = {}
|
||||
# 1. 按 (memory_at day, valid_until day) 分桶
|
||||
buckets: Dict[Tuple[Any, Any], List[int]] = {}
|
||||
for idx, h in enumerate(hits):
|
||||
node = h.get("node")
|
||||
key = (
|
||||
getattr(node, "user_id", None) if node else None,
|
||||
getattr(node, "agent_id", None) if node else None,
|
||||
_memory_at_day(node),
|
||||
_valid_until_key(node),
|
||||
)
|
||||
key = (_memory_at_day(node), _valid_until_key(node))
|
||||
buckets.setdefault(key, []).append(idx)
|
||||
|
||||
def _gmt(h: Dict[str, Any]) -> float:
|
||||
@@ -237,8 +229,6 @@ async def recall_intentions(
|
||||
*,
|
||||
user_ids: Optional[List[str]] = None,
|
||||
agent_ids: Optional[List[str]] = None,
|
||||
isolation_key: str = "",
|
||||
isolation_keys: Optional[List[str]] = None,
|
||||
limit: int = 10,
|
||||
now: Optional[datetime] = None,
|
||||
stats_out: Optional[Dict[str, Any]] = None,
|
||||
@@ -253,9 +243,6 @@ async def recall_intentions(
|
||||
query_embedding: 查询向量
|
||||
user_ids: 用户隔离(单/多用户)
|
||||
agent_ids: 可选 agent 过滤
|
||||
isolation_key / isolation_keys: 精确的 (user, agent, session) 隔离键。
|
||||
与其它召回通道一致:调用方在按 session 过滤时只生成这两项,
|
||||
user_ids / agent_ids 为 None;不把它们透传下去就会退化成全库检索。
|
||||
limit: 召回上限(返回给上游的条数)
|
||||
now: 当前时间(默认 datetime.now(),测试可注入)
|
||||
stats_out: 可选 dict,本函数把分段耗时与数量**原地回填**进去,供调用方落
|
||||
@@ -296,8 +283,6 @@ async def recall_intentions(
|
||||
query_embedding=query_embedding,
|
||||
user_ids=user_ids or None,
|
||||
agent_ids=agent_ids,
|
||||
isolation_key=isolation_key,
|
||||
isolation_keys=isolation_keys,
|
||||
layers=[MemoryLayer.L7_INTENTION],
|
||||
limit=fetch_limit,
|
||||
score_threshold=0.0, # intention 不设语义门槛
|
||||
|
||||
@@ -1840,8 +1840,6 @@ class BaseReadPipeline(ReadPipeline):
|
||||
query_embedding,
|
||||
user_ids=isolation_params.get("user_ids"),
|
||||
agent_ids=isolation_params.get("agent_ids"),
|
||||
isolation_key=isolation_params.get("isolation_key") or "",
|
||||
isolation_keys=isolation_params.get("isolation_keys"),
|
||||
limit=request.intention_limit,
|
||||
stats_out=_intent_stats,
|
||||
include_sources=request.include_sources,
|
||||
|
||||
@@ -968,7 +968,7 @@ class MemoryWriter(WritePipeline):
|
||||
async def _run():
|
||||
try:
|
||||
nid = await summarize_closed_episode(
|
||||
vector_store=await self._get_vector_store(),
|
||||
vector_store=self.vector_store,
|
||||
embed_service=self.embed_service,
|
||||
llm_provider=self.llm_provider,
|
||||
episode_id=episode_id,
|
||||
|
||||
+6
-19
@@ -29,7 +29,7 @@ import logging
|
||||
import os
|
||||
import re
|
||||
import signal
|
||||
import threading
|
||||
import sys
|
||||
from datetime import datetime
|
||||
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
||||
from typing import Any, Dict, Optional
|
||||
@@ -495,16 +495,6 @@ class MemoryHTTPHandler(BaseHTTPRequestHandler):
|
||||
_json_response(self, 200, result)
|
||||
|
||||
|
||||
def _request_shutdown(server: ThreadingHTTPServer) -> None:
|
||||
"""Ask ``serve_forever`` to stop without blocking the calling thread.
|
||||
|
||||
``BaseServer.shutdown()`` waits for the ``serve_forever`` loop to exit. A
|
||||
signal handler runs on the very thread executing that loop, so calling it
|
||||
there deadlocks forever. Hand the call to a helper thread instead.
|
||||
"""
|
||||
threading.Thread(target=server.shutdown, daemon=True).start()
|
||||
|
||||
|
||||
def run_server(port: int = 19527, host: str = "127.0.0.1"):
|
||||
"""Start the HTTP server (blocking)."""
|
||||
# Configure logging — basicConfig for root; hy_memory level set after client init
|
||||
@@ -533,12 +523,13 @@ def run_server(port: int = 19527, host: str = "127.0.0.1"):
|
||||
logger.info(f"[server] Listening on http://{host}:{port}")
|
||||
logger.info(f"[server] Health check: http://{host}:{port}/healthz")
|
||||
|
||||
# Graceful shutdown. The signal handler only asks a helper thread to stop
|
||||
# the loop; cleanup runs after serve_forever() returns so it cannot race
|
||||
# the request threads or deadlock the serving thread.
|
||||
# Graceful shutdown
|
||||
def _shutdown(signum, frame):
|
||||
logger.info("[server] Shutting down...")
|
||||
_request_shutdown(server)
|
||||
server.shutdown()
|
||||
if _client is not None:
|
||||
_client.close()
|
||||
sys.exit(0)
|
||||
|
||||
signal.signal(signal.SIGINT, _shutdown)
|
||||
signal.signal(signal.SIGTERM, _shutdown)
|
||||
@@ -547,10 +538,6 @@ def run_server(port: int = 19527, host: str = "127.0.0.1"):
|
||||
server.serve_forever()
|
||||
except KeyboardInterrupt:
|
||||
_shutdown(None, None)
|
||||
finally:
|
||||
server.server_close()
|
||||
if _client is not None:
|
||||
_client.close()
|
||||
|
||||
|
||||
def main():
|
||||
|
||||
@@ -1,150 +0,0 @@
|
||||
"""Public, offline snapshot support for the SDK-owned local data plane."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import os
|
||||
import shutil
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
class DataPlaneSnapshotRejected(RuntimeError):
|
||||
def __init__(self, code: str) -> None:
|
||||
super().__init__(code)
|
||||
self.code = code
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class DataPlaneSnapshot:
|
||||
source_root: Path
|
||||
snapshot_root: Path
|
||||
files: tuple[dict[str, object], ...]
|
||||
total_bytes: int
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class DataPlaneRestore:
|
||||
restored_root: Path
|
||||
previous_root: Path | None
|
||||
files_restored: int
|
||||
total_bytes: int
|
||||
|
||||
|
||||
def _sha256(path: Path) -> str:
|
||||
digest = hashlib.sha256()
|
||||
with path.open("rb") as handle:
|
||||
for chunk in iter(lambda: handle.read(1024 * 1024), b""):
|
||||
digest.update(chunk)
|
||||
return digest.hexdigest()
|
||||
|
||||
|
||||
def snapshot_local_data_plane(
|
||||
*,
|
||||
source_root: Path,
|
||||
target_root: Path,
|
||||
writers_stopped: bool,
|
||||
) -> DataPlaneSnapshot:
|
||||
"""Copy one quiesced local SDK root and return a verified inventory.
|
||||
|
||||
Remote providers require their own export implementation and must not call
|
||||
this local-path contract.
|
||||
"""
|
||||
if not writers_stopped:
|
||||
raise DataPlaneSnapshotRejected("data_plane_snapshot_writers_running")
|
||||
source = source_root.resolve()
|
||||
target = target_root.resolve()
|
||||
if source.is_symlink() or not source.is_dir():
|
||||
raise DataPlaneSnapshotRejected("data_plane_snapshot_source_invalid")
|
||||
if target.exists():
|
||||
raise DataPlaneSnapshotRejected("data_plane_snapshot_target_exists")
|
||||
|
||||
try:
|
||||
target.mkdir(parents=True)
|
||||
for root, directories, files in os.walk(source, followlinks=False):
|
||||
root_path = Path(root)
|
||||
relative = root_path.relative_to(source)
|
||||
target_directory = target / relative
|
||||
target_directory.mkdir(parents=True, exist_ok=True)
|
||||
for name in directories:
|
||||
if (root_path / name).is_symlink():
|
||||
raise DataPlaneSnapshotRejected("data_plane_snapshot_symlink")
|
||||
for name in files:
|
||||
item = root_path / name
|
||||
if item.is_symlink():
|
||||
raise DataPlaneSnapshotRejected("data_plane_snapshot_symlink")
|
||||
shutil.copy2(item, target_directory / name)
|
||||
|
||||
inventory: list[dict[str, object]] = []
|
||||
for path in sorted(item for item in target.rglob("*") if item.is_file()):
|
||||
inventory.append(
|
||||
{
|
||||
"path": path.relative_to(target).as_posix(),
|
||||
"bytes": path.stat().st_size,
|
||||
"sha256": _sha256(path),
|
||||
}
|
||||
)
|
||||
for item in inventory:
|
||||
copied = target / str(item["path"])
|
||||
if copied.stat().st_size != item["bytes"] or _sha256(copied) != item["sha256"]:
|
||||
raise DataPlaneSnapshotRejected("data_plane_snapshot_verify_failed")
|
||||
except DataPlaneSnapshotRejected:
|
||||
shutil.rmtree(target, ignore_errors=True)
|
||||
raise
|
||||
except OSError as error:
|
||||
shutil.rmtree(target, ignore_errors=True)
|
||||
raise DataPlaneSnapshotRejected("data_plane_snapshot_failed") from error
|
||||
|
||||
return DataPlaneSnapshot(
|
||||
source_root=source,
|
||||
snapshot_root=target,
|
||||
files=tuple(inventory),
|
||||
total_bytes=sum(int(item["bytes"]) for item in inventory),
|
||||
)
|
||||
|
||||
|
||||
def restore_local_data_plane(
|
||||
*,
|
||||
snapshot_root: Path,
|
||||
target_root: Path,
|
||||
writers_stopped: bool,
|
||||
protective_root: Path | None = None,
|
||||
) -> DataPlaneRestore:
|
||||
"""Replace one local SDK root from a verified snapshot with rollback."""
|
||||
if not writers_stopped:
|
||||
raise DataPlaneSnapshotRejected("data_plane_restore_writers_running")
|
||||
source = snapshot_root.resolve()
|
||||
target = target_root.resolve()
|
||||
protective = (
|
||||
protective_root.resolve()
|
||||
if protective_root is not None
|
||||
else target.parent / f".{target.name}.pre-restore"
|
||||
)
|
||||
staging = target.parent / f".{target.name}.restore.tmp"
|
||||
if target.is_symlink() or protective.exists() or staging.exists():
|
||||
raise DataPlaneSnapshotRejected("data_plane_restore_target_unsafe")
|
||||
|
||||
copied = snapshot_local_data_plane(
|
||||
source_root=source,
|
||||
target_root=staging,
|
||||
writers_stopped=True,
|
||||
)
|
||||
previous: Path | None = None
|
||||
try:
|
||||
if target.exists():
|
||||
target.replace(protective)
|
||||
previous = protective
|
||||
staging.replace(target)
|
||||
except OSError as error:
|
||||
if previous is not None and previous.exists() and not target.exists():
|
||||
previous.replace(target)
|
||||
previous = None
|
||||
shutil.rmtree(staging, ignore_errors=True)
|
||||
raise DataPlaneSnapshotRejected("data_plane_restore_replace_failed") from error
|
||||
|
||||
return DataPlaneRestore(
|
||||
restored_root=target,
|
||||
previous_root=previous,
|
||||
files_restored=len(copied.files),
|
||||
total_bytes=copied.total_bytes,
|
||||
)
|
||||
@@ -1,24 +0,0 @@
|
||||
"""Context propagated to host-provided HTTP transports for cost accounting."""
|
||||
|
||||
from collections.abc import Iterator
|
||||
from contextlib import contextmanager
|
||||
from contextvars import ContextVar, Token
|
||||
|
||||
_model_call_stage: ContextVar[str] = ContextVar(
|
||||
"hy_memory_model_call_stage",
|
||||
default="other",
|
||||
)
|
||||
|
||||
|
||||
def current_model_call_stage() -> str:
|
||||
return _model_call_stage.get()
|
||||
|
||||
|
||||
@contextmanager
|
||||
def model_call_stage(stage: str | None) -> Iterator[None]:
|
||||
normalized = (stage or "other").strip().lower() or "other"
|
||||
token: Token[str] = _model_call_stage.set(normalized)
|
||||
try:
|
||||
yield
|
||||
finally:
|
||||
_model_call_stage.reset(token)
|
||||
@@ -18,32 +18,12 @@ from __future__ import annotations
|
||||
|
||||
import json
|
||||
import logging
|
||||
import re
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
from typing import Any, List, Optional
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
#: A namespace is a single path component, not a path. `\w` keeps Unicode
|
||||
#: names working while excluding separators, drive letters and NUL.
|
||||
_SAFE_SUBDIR = re.compile(r"[\w.-]{1,64}", re.UNICODE)
|
||||
|
||||
|
||||
def _safe_subdir(value: str) -> Optional[str]:
|
||||
"""Return ``value`` if it is safe as one directory name, else ``None``.
|
||||
|
||||
``user_id`` is caller-controlled. Without this check a value such as
|
||||
``"/tmp/evil__u1"`` makes ``Path(base) / "/tmp/evil"`` discard ``base``
|
||||
entirely; ``"../.."`` escapes it the same way.
|
||||
"""
|
||||
candidate = (value or "").strip()
|
||||
if not candidate or candidate in {".", ".."}:
|
||||
return None
|
||||
if not _SAFE_SUBDIR.fullmatch(candidate):
|
||||
return None
|
||||
return candidate
|
||||
|
||||
# 兼容旧 import
|
||||
TraceFileWriter = None # set after class definition
|
||||
|
||||
@@ -60,7 +40,7 @@ class PipelineLogWriter:
|
||||
|
||||
def __init__(self, log_dir: str, default_subdir: str = "default"):
|
||||
self._log_dir = log_dir
|
||||
self._default_subdir = _safe_subdir(default_subdir) or "default"
|
||||
self._default_subdir = default_subdir or "default"
|
||||
try:
|
||||
(Path(self._log_dir) / self._default_subdir).mkdir(parents=True, exist_ok=True)
|
||||
except Exception:
|
||||
@@ -69,7 +49,7 @@ class PipelineLogWriter:
|
||||
def _resolve_subdir(self, user_id: str) -> str:
|
||||
"""从 user_id 的 "{prefix}__{user_id}" 前缀取 prefix 作为子目录;无前缀则回落。"""
|
||||
if user_id and "__" in user_id:
|
||||
prefix = _safe_subdir(user_id.split("__", 1)[0])
|
||||
prefix = user_id.split("__", 1)[0].strip()
|
||||
if prefix:
|
||||
return prefix
|
||||
return self._default_subdir
|
||||
@@ -100,11 +80,7 @@ class PipelineLogWriter:
|
||||
split_namespace_user,
|
||||
)
|
||||
|
||||
# Providers define total_tokens as prompt+completion. Adding all
|
||||
# three fields doubled every cost estimate in this JSONL.
|
||||
tokens = (total_tokens or 0) or (
|
||||
(prompt_tokens or 0) + (completion_tokens or 0)
|
||||
)
|
||||
tokens = (prompt_tokens or 0) + (completion_tokens or 0) + (total_tokens or 0)
|
||||
namespace, uid = split_namespace_user(user_id or "")
|
||||
if not is_llm_content_logging_enabled():
|
||||
prompt = redact_content(prompt)
|
||||
@@ -135,14 +111,7 @@ class PipelineLogWriter:
|
||||
line = json.dumps(record, ensure_ascii=False, default=str)
|
||||
date_str = datetime.now().strftime("%Y-%m-%d")
|
||||
subdir = self._resolve_subdir(user_id)
|
||||
base_dir = Path(self._log_dir).resolve()
|
||||
log_dir = (base_dir / subdir).resolve()
|
||||
# Defence in depth: even if a future change lets a suspicious
|
||||
# component through, never write outside the configured root.
|
||||
try:
|
||||
log_dir.relative_to(base_dir)
|
||||
except ValueError:
|
||||
log_dir = base_dir / self._default_subdir
|
||||
log_dir = Path(self._log_dir) / subdir
|
||||
log_dir.mkdir(parents=True, exist_ok=True)
|
||||
log_file = log_dir / f"{date_str}.log"
|
||||
with open(log_file, "a", encoding="utf-8") as f:
|
||||
|
||||
+146
-112
@@ -1,60 +1,45 @@
|
||||
# Hy Memory 使用与配置
|
||||
|
||||
更新:2026-10-21
|
||||
更新:2026-10-12
|
||||
|
||||
**文档约定:** 以 [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md) **Top Rules** 为现行权威。
|
||||
~~删除线~~ = 历史产品合同(曾要求走 Evidence / Governance 台账),**不再作为目标**;SDK 用法示例仍可用于开发调试。
|
||||
本文说明 MemHub 当前使用的 Hy Memory SDK、配置边界,以及直接使用 SDK 时如何
|
||||
write、search,并把搜索结果格式化为可交给模型的 memory text chunk。
|
||||
|
||||
---
|
||||
|
||||
## Top Rules(与架构一致 · 新)
|
||||
|
||||
1. Memory 读写与存储 backend **只经 SDK**(config / env)。
|
||||
2. MemHub 对 memory 的原子操作 **只经 SDK**;缺接口先补 SDK。
|
||||
3. MemHub **不存 memory 实体**;**可以存 `memory_id`** 做组织/回执。原文进 **SDK raw**,不进 MemHub Evidence 权威库。
|
||||
|
||||
生产入口仍建议经 `HyMemoryBridge`(共享 Runtime、模型配置、密钥、harness/model 门禁),但 bridge **不得**再把 SDK 结果镜像成第二套 memory 正文权威。
|
||||
|
||||
---
|
||||
|
||||
## ~~旧开篇(已废弃)~~
|
||||
|
||||
~~本文说明 MemHub 当前使用的 Hy Memory SDK… 仓库内仍以 One Memory 命名的 Evidence、权限和 Governance 层属于 MemHub 的历史兼容模块;产品代码不应绕过这些层直接调用 SDK。~~
|
||||
|
||||
~~MemHub 在本地 Runtime 内嵌入 SDK,但不绕过自身的来源授权、模型授权、Evidence、Governance 和撤权边界。~~
|
||||
|
||||
**新:** 来源授权、模型授权、宿主召回授权仍是 MemHub 管理面门禁。~~Evidence / Governance 记忆台账~~ 按 Top Rules 拆除;memory 真相在 SDK。
|
||||
仓库内仍以 One Memory 命名的 Evidence、权限和 Governance 层属于 MemHub 的历史兼容
|
||||
模块;产品代码不应绕过这些层直接调用 SDK。本文中的
|
||||
原生 SDK 示例主要用于开发、调试和独立验证。
|
||||
|
||||
## 当前版本状态
|
||||
|
||||
当前源码固定使用(以仓库 pin / lockfile 为准):
|
||||
当前源码固定使用:
|
||||
|
||||
```text
|
||||
distribution: hy-memory
|
||||
version: 见 pyproject / lock(文档数字可能滞后)
|
||||
version: 2.0.3
|
||||
mode: 按运行时配置选择
|
||||
reader: hybrid
|
||||
```
|
||||
|
||||
仓库 pin、lockfile 和运行时版本守卫均固定到公开的 `hy-memory==2.0.3`。
|
||||
MemHub 在本地 Runtime 内嵌入 SDK,但不绕过自身的来源授权、模型授权、Evidence、
|
||||
Governance 和撤权边界。
|
||||
|
||||
## MemHub 运行时边界
|
||||
|
||||
### 新
|
||||
MemHub 对 SDK 使用以下公开、可复现的本地策略:
|
||||
|
||||
MemHub 对 SDK 使用公开、可复现的本地策略(经 bridge / `hy_runtime_config` 注入):
|
||||
- vector store 使用受管数据目录中的 Chroma;
|
||||
- graph store 使用受管数据目录中的 Kuzu;
|
||||
- cache 与 history 使用 SQLite;
|
||||
- reader 使用 `hybrid`;
|
||||
- reranker 默认关闭,不引入 Torch、Transformers 或额外模型;
|
||||
- 模型与远程 embedding 参数来自版本化 Model Profile;
|
||||
- API Key 进入系统安全存储,不写入仓库、安装包、日志或 HY metadata;
|
||||
- 服务端数据库、租户队列及宿主 shell 中的 ambient 凭据不会进入 SDK config。
|
||||
|
||||
- vector / graph / cache / history / coding 等 **一律 SDK config/env**;
|
||||
- 模型与 embedding 来自版本化 Model Profile;密钥进安全存储;
|
||||
- MemHub 管理库可存 harness/model/队列状态与 **`memory_id` 引用**。
|
||||
发布包固定应用以下 SDK 行为配置:
|
||||
|
||||
### ~~旧~~
|
||||
|
||||
~~- MemHub 不绕过 Evidence、Governance…~~
|
||||
~~- 生产路径不能用直接 `client.add()` 代替 Governance 检查。~~
|
||||
|
||||
发布包默认 SDK 行为环境变量仍集中在
|
||||
`apps/core/src/one_memory/hy_runtime_config.py`,并由下表锁定:
|
||||
|
||||
| 变量 | 值 |
|
||||
| 环境变量 | 值 |
|
||||
|---|---|
|
||||
| `MEMORY_MODE` | `ultra` |
|
||||
| `MEMORY_ENABLE_GRAPH` | `true` |
|
||||
@@ -86,27 +71,27 @@ MemHub 对 SDK 使用公开、可复现的本地策略(经 bridge / `hy_runtim
|
||||
| `RECONCILE_ENABLED` | `true` |
|
||||
| `HY_MEMORY_READER` | `hybrid` |
|
||||
|
||||
独立脚本直连 SDK 时须自配存储目录与模型端点。
|
||||
这些默认值集中在
|
||||
`apps/core/src/one_memory/hy_runtime_config.py`,并由发布测试和 lockfile 共同约束。
|
||||
独立脚本若绕过 MemHub 直接导入 SDK,应自行显式配置存储目录和模型端点。
|
||||
|
||||
## 原生 SDK 初始化
|
||||
|
||||
下面是独立调试示例。
|
||||
|
||||
**新:** 产品代码经 `HyMemoryBridge` 获取已配置的 client(管理面门禁),memory I/O 仍是 SDK。
|
||||
~~旧:必须额外提供 Evidence 绑定、写后核验进 Governance。~~
|
||||
下面是独立调试示例。产品代码应使用 One Memory 的 `HyMemoryBridge`,因为它额外提供来源
|
||||
授权、模型授权、Evidence 绑定、写后核验、Governance 和撤权保护。
|
||||
|
||||
```python
|
||||
from hy_memory import HyMemoryClient, MemoryConfig
|
||||
|
||||
config = MemoryConfig.from_env()
|
||||
|
||||
# 本地优先示例;实际目录应由调用方传入。
|
||||
# 本地优先示例;实际目录应由调用方传入,不能依赖 ~/.hy_memory 默认值。
|
||||
config.vector_store.provider = "chroma"
|
||||
config.vector_store.persist_directory = "/path/to/one-memory/hy-memory/vector"
|
||||
config.vector_store.persist_directory = "/path/to/one-memory/hy-memory/governed-v2/vector"
|
||||
config.graph_store.provider = "kuzu"
|
||||
config.graph_store.db_path = "/path/to/one-memory/hy-memory/graph"
|
||||
config.graph_store.db_path = "/path/to/one-memory/hy-memory/governed-v2/graph"
|
||||
config.cache.backend = "sqlite"
|
||||
config.cache.db_path = "/path/to/one-memory/hy-memory/cache.sqlite3"
|
||||
config.cache.db_path = "/path/to/one-memory/hy-memory/governed-v2/cache.sqlite3"
|
||||
|
||||
config.llm.provider = "openai"
|
||||
config.llm.model = "your-model"
|
||||
@@ -122,11 +107,12 @@ config.embedder.embedding_dims = 1024
|
||||
client = HyMemoryClient(config=config, mode="ultra")
|
||||
```
|
||||
|
||||
`embedding_dims` 必须与模型输出及已有向量/图存储一致。
|
||||
`embedding_dims` 必须与模型实际输出宽度以及已有向量/图存储一致。更换 embedding 模型、
|
||||
端点或维度不能直接复用旧 collection;One Memory 会检测图宽度不一致并拒绝静默重建。
|
||||
|
||||
## Write:写入对话
|
||||
|
||||
推荐把一段连续对话作为 OpenAI messages 写入(**含原文时由 SDK 接管 raw**,不再写入 MemHub Evidence 权威表):
|
||||
推荐把一段连续对话作为 OpenAI messages 写入:
|
||||
|
||||
```python
|
||||
from datetime import UTC, datetime
|
||||
@@ -145,55 +131,47 @@ result = client.add(
|
||||
request_id=str(uuid4()),
|
||||
metadata={
|
||||
"source_id": "cursor",
|
||||
# ~~evidence_ids 指向 MemHub Evidence 表~~ → 新:按 SDK raw / 元数据合同填写;勿依赖 MemHub evidence 权威
|
||||
"evidence_ids": ["evidence-001", "evidence-002"],
|
||||
"workspace_ids": ["workspace-abc"],
|
||||
"input_kind": "raw_conversation",
|
||||
},
|
||||
)
|
||||
```
|
||||
|
||||
参数规则:
|
||||
|
||||
| 参数 | 填法 | 约束 |
|
||||
| --- | --- | --- |
|
||||
| `data` | 字符串或 `[{role, content}]` | 对话优先 messages;不混入密钥与隐藏系统提示 |
|
||||
| `user_id` / `agent_id` / `session_id` | 隔离与会话键 | 产品侧授权决定合法 scope |
|
||||
| `memory_at` | 有时区 datetime | 历史导入用原始发生时间 |
|
||||
| `scene` | `chat` / `working` | 明确传入 |
|
||||
| `request_id` | 每次写入唯一 | 幂等与追踪 |
|
||||
| `metadata` / `meta` | 来源与透传 | 不放秘密 |
|
||||
| `data` | 字符串或 `[{role, content}]` | 对话优先用 messages;不混入 tool 参数、reasoning、密钥和隐藏系统提示 |
|
||||
| `user_id` | 稳定用户 ID | 一级隔离键;不能为空 |
|
||||
| `agent_id` | 记忆 scope | One Memory 中不是宿主 App 名的简单替代;跨 App scope 由授权和路由决定 |
|
||||
| `session_id` | 稳定会话 ID | 同一会话增量写入时保持一致 |
|
||||
| `memory_at` | 有时区的 `datetime` | 历史导入必须传原始发生时间,不能用导入时间替代 |
|
||||
| `scene` | `"chat"` 或 `"working"` | 普通对话明确传 `chat`;代码/tool 会话才用 `working` |
|
||||
| `request_id` | 每次写入唯一 ID | 用于幂等、追踪和写后核验 |
|
||||
| `metadata` | 来源和 Evidence 引用 | 不放秘密;One Memory 至少保存 source/evidence/workspace/input kind |
|
||||
| `meta` | 业务透传数据 | 不参与 embedding 和过滤;只放可安全返回的数据 |
|
||||
| `embed_override` | 临时 embedding 端点 | 只能换 endpoint/key/header/timeout;模型和维度不能改变 |
|
||||
|
||||
MemHub Runtime 对应用和 CLI 暴露等价的本机 HTTP 合同:
|
||||
|
||||
```text
|
||||
POST /api/v2/memory-writes
|
||||
GET /api/v2/user-profile
|
||||
|
||||
memhub write submit --messages-file messages.json --session <id>
|
||||
memhub profile
|
||||
```
|
||||
|
||||
`messages.json` 是 OpenAI 文本 messages 数组,当前接受 `user`、
|
||||
`assistant`、`tool`;系统/开发者隐藏提示不得作为记忆输入。Core 将输入冻结进
|
||||
`memory_write_jobs`,worker 对 SDK 执行一次 `add()`,成功后清除队列正文。
|
||||
`user-profile` / `memhub profile` 调用 SDK `get_user_profile()`,返回组装后的
|
||||
用户长期画像,不通过普通 Search 拼装。
|
||||
|
||||
返回值:
|
||||
返回值至少检查:
|
||||
|
||||
```python
|
||||
if result.get("success") is not True:
|
||||
raise RuntimeError("HY write failed")
|
||||
|
||||
memory_ids = list(result.get("memory_ids") or [])
|
||||
memory_ids = result.get("memory_ids") or []
|
||||
if result.get("memory_id"):
|
||||
memory_ids.append(result["memory_id"])
|
||||
# 新:需要组织/回执时,MemHub 只持久化这些 memory_id
|
||||
```
|
||||
|
||||
~~不要只相信 `success=True`。One Memory 还会…并在 Governance 层记录版本。生产路径不能用直接 `client.add()` 代替这些检查。~~
|
||||
**新:** 以 SDK 合同为准做成功判定与(若需要)`get` 回读;**不要**再写入 `governed_*`。缺核验能力 → **扩展 SDK API**。
|
||||
不要只相信 `success=True`。One Memory 还会按返回 ID `get()` 回读、核对 request/source/
|
||||
Evidence 支持关系,并在 Governance 层记录版本。生产路径不能用直接 `client.add()` 代替这些
|
||||
检查。
|
||||
|
||||
### 写入已抽取的记忆
|
||||
|
||||
只有输入本身就是已有记忆内容、且原始对话不可用时才使用:
|
||||
|
||||
```python
|
||||
result = client.add_extracted(
|
||||
["用户长期偏好简洁、直接的回答。"],
|
||||
@@ -204,16 +182,18 @@ result = client.add_extracted(
|
||||
request_id=str(uuid4()),
|
||||
metadata={
|
||||
"source_id": "memory-export",
|
||||
"evidence_ids": ["evidence-003"],
|
||||
"input_kind": "memory_content_without_raw",
|
||||
},
|
||||
)
|
||||
```
|
||||
|
||||
~~不能跳过 One Memory Evidence 绑定和 Governance。~~
|
||||
**新:** 仍走 SDK reconcile/持久化;MemHub 只做门禁与可选 `memory_id` 记录。
|
||||
它跳过 Extractor,但不能跳过 HY reconcile、持久化、One Memory Evidence 绑定和 Governance。
|
||||
|
||||
## Search:搜索记忆
|
||||
|
||||
普通 chat 搜索:
|
||||
|
||||
```python
|
||||
result = client.search(
|
||||
"用户偏好怎样的回答风格?",
|
||||
@@ -230,63 +210,117 @@ result = client.search(
|
||||
)
|
||||
```
|
||||
|
||||
参数表与返回结构同前(chat 三 channel / working 的 facts·experiences);调用方按 `scene` 解析,无法识别则报错。
|
||||
参数规则:
|
||||
|
||||
| 参数 | 填法 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| `query` | 当前问题 | 与 `queries` 二选一 |
|
||||
| `queries` | 多轮查询文本 | 最后一条是目标;只有需要 SDK 判断上下文时使用 |
|
||||
| `user_ids` | 获授权用户列表 | 必填且不能空;不要用空列表扩大范围 |
|
||||
| `agent_ids` | 获授权 scope 列表 | 空列表在 SDK 中表示所有 agent,产品代码必须先完成授权判断 |
|
||||
| `session_ids` | 可选会话范围 | 多 agent 与多 session 不能同时使用 |
|
||||
| `limit` | normal top-k | 默认 10;注入模型前还要受字符预算限制 |
|
||||
| `min_score` | normal 阈值 | One Memory 当前内部语义检索使用 0.25 |
|
||||
| `profile_min_score` | profile 阈值 | One Memory 当前使用 0.25 |
|
||||
| `profile_limit` | profile 数量 | 通常与 limit 一致 |
|
||||
| `intention_limit` | intention 数量 | 当前配置关闭 intention,传 0 |
|
||||
| `created_after` | Unix 秒 | 可选时间下界 |
|
||||
| `scene` | `chat` / `working` | 明确传入可避免额外场景判断模型调用 |
|
||||
| `reader` | `hybrid` | One Memory 固定使用该 reader |
|
||||
| `workspace_id` / `branch` | working scope | 仅 coding/working 路径使用 |
|
||||
| `layers` | 指定层级列表 | 在召回阶段过滤,避免无关层占用 limit |
|
||||
|
||||
chat 返回结构:
|
||||
|
||||
```python
|
||||
{
|
||||
"request_id": "...",
|
||||
"memories": {
|
||||
"profile": [...],
|
||||
"proactive": [...],
|
||||
"normal": [...],
|
||||
},
|
||||
"rewrite": {...},
|
||||
"elapsed_ms": 12.3,
|
||||
}
|
||||
```
|
||||
|
||||
working/coding 返回的是 `facts` 和 `experiences`,不能与 chat 三个 channel 静默合并。
|
||||
调用方必须根据 `scene` 选择对应结果;无法识别返回结构时应报错,而不是当作空结果。
|
||||
|
||||
## 使用 SDK 已格式化的 Search 内容
|
||||
|
||||
直接使用 SDK 返回的 `content`,不要第二套 prompt 拼装。
|
||||
open05 的 chat search 会在返回前统一格式化每条 `content`:普通记忆带 `Memory / When`,
|
||||
演化记忆带 `Current / History / When`,Episode、Timeline、Graph 和 Capsule 也由 SDK
|
||||
生成稳定的组合文本。`memory_at` 是逻辑记忆时间,缺失时 SDK 才回落到 `gmt_created`。
|
||||
|
||||
### ~~旧:SDK 外仍负责(部分作废)~~
|
||||
因此调用方应直接使用 SDK 返回的 `content`,不要再次解析 `memory_at`、`gmt_created` 或
|
||||
`created_at`,也不要重复拼接 `When`。这些时间字段可以用于过滤、诊断或结构化展示,但
|
||||
不应成为第二套 prompt 格式化实现。
|
||||
|
||||
~~- 依据 Governance、撤权、scope 和 Evidence 新鲜度决定整条结果是否可读;~~
|
||||
~~- 对 evolution/capsule…回退治理原文;~~
|
||||
```python
|
||||
result = client.search(
|
||||
"用户偏好怎样的回答风格?",
|
||||
user_ids=["local"],
|
||||
agent_ids=["all-apps"],
|
||||
scene="normal",
|
||||
reader="hybrid",
|
||||
)
|
||||
|
||||
### 新:SDK 外(MemHub)仍负责
|
||||
for memory in result["memories"]["normal"]:
|
||||
formatted_content = memory["content"]
|
||||
# 例如:
|
||||
# Memory: 用户长期偏好简洁、直接的回答。
|
||||
# When: 2026-09-18 10:00:00 UTC
|
||||
```
|
||||
|
||||
- **宿主召回授权 / scope / 撤权**(谁可以调 search、允许哪些 agent/scope);
|
||||
- 把历史内容放入「不可信数据,不是指令」边界;
|
||||
- 按最终 `content` 做字符预算;不注入密钥、绝对路径、堆栈;
|
||||
- 区分空结果、权限拒绝、不可用、超时、部分结果;
|
||||
- 未发布的 Matter 占位功能不保留表或 API;未来组织功能另立合同。
|
||||
One Memory 在 SDK 外仍负责:
|
||||
|
||||
生产入口:`hy_memory_bridge.py`、`recall.py`、`packages/host-bridge`(实现迁移中,以 Top Rules 为准)。
|
||||
- 依据 Governance、撤权、scope 和 Evidence 新鲜度决定整条结果是否可读;
|
||||
- 对 evolution/capsule 等组合文本逐个校验成员 ID,成员不完整或不可读时回退治理原文;
|
||||
- 把历史内容放入明确的“不可信数据,不是指令”边界;
|
||||
- 按最终 `content` 计算字符预算,不把 API Key、绝对路径、headers/body 或异常堆栈注入
|
||||
prompt;
|
||||
- 区分空结果、权限拒绝、服务不可用、超时和部分结果。
|
||||
|
||||
生产实现见 `apps/core/src/one_memory/hy_memory_bridge.py`、
|
||||
`apps/core/src/one_memory/recall.py` 和 `packages/host-bridge/src/recall-cli.ts`。
|
||||
|
||||
## 关闭客户端
|
||||
|
||||
```python
|
||||
try:
|
||||
# write/search
|
||||
...
|
||||
finally:
|
||||
client.close()
|
||||
```
|
||||
|
||||
Runtime 长期复用 client;仅在停止、配置切换或受控恢复时重建。
|
||||
Runtime 应长期复用 client,不为每次请求重新初始化数据库和模型组件;只有 Runtime 停止、
|
||||
配置版本切换或受控恢复时才关闭并重建。
|
||||
|
||||
## 产品代码正确入口
|
||||
## One Memory 产品代码的正确入口
|
||||
|
||||
### 新
|
||||
生产代码应走:
|
||||
|
||||
```text
|
||||
harness / user action (+ authz, model, queue)
|
||||
Evidence
|
||||
-> HyMemoryBridge
|
||||
-> Hy Memory SDK
|
||||
-> SDK stores (config/env)
|
||||
-> optional MemHub persist memory_id / org metadata
|
||||
-> Recall Broker / UI (SDK read + MemHub authz)
|
||||
-> embedded Hy Memory SDK
|
||||
-> native Vector / Coding / Graph stores
|
||||
-> write result verification
|
||||
-> Governance
|
||||
-> Recall Broker / UI
|
||||
```
|
||||
|
||||
不要在业务模块里私建绕过 **管理面门禁** 的 client。
|
||||
~~不要绕过 Evidence / Governance 台账。~~(台账已非目标)
|
||||
不要在新业务模块里直接创建 `HyMemoryClient`,否则会绕过:
|
||||
|
||||
### ~~旧入口(废弃)~~
|
||||
- 来源读取授权;
|
||||
- 共享 Runtime、模型配置、API Key 安全存储与请求预算;
|
||||
- Evidence 引用与作者归属;
|
||||
- 写后回读验证;
|
||||
- Governance 状态、版本、更正和删除级联。
|
||||
|
||||
~~```text
|
||||
Evidence -> HyMemoryBridge -> SDK -> write verification -> Governance -> UI
|
||||
```~~
|
||||
|
||||
~~绕过会丢掉:Evidence 引用、Governance 版本/更正/删除级联。~~
|
||||
**新:** 版本/更正/删除级联以 **SDK** 为准;MemHub 组织层只引用 `memory_id`。
|
||||
|
||||
Hy Memory SDK 是 MemHub 内置可信引擎,管理本机 Hy Memory 数据。
|
||||
来源授权决定哪些宿主数据可进入写入路径;宿主 Recall Grant 决定外部应用可读范围。
|
||||
这两类边界 **不** 意味着 MemHub 再存一套 memory 实体。
|
||||
Hy Memory SDK 是 MemHub 的内置可信组件,可自由访问和管理本机的全部
|
||||
Hy Memory 数据。来源授权仍决定哪些宿主数据可以进入本机;宿主 Recall Grant
|
||||
仍决定哪些外部 AI 应用可以读取治理后的记忆。这两类边界不限制内置 SDK 的本地 CRUD。
|
||||
+11
-25
@@ -1,30 +1,18 @@
|
||||
# MemHub
|
||||
# MemHub / One Memory
|
||||
|
||||
MemHub是一个本地优先的共享记忆产品。它把用户明确授权的AI对话和编码记录汇总到一份本地记忆中,再由用户分别授权给支持的AI工具调用。你可以继续使用熟悉的客户端,同时保留跨工具延续的上下文。Hy Memory是MemHub使用的Python SDK和记忆引擎。
|
||||
|
||||
# 💡❤️ MemHub 刚刚开源,欢迎一起优化共建! ❤️💡
|
||||
|
||||
**# 💭 MemHub处于preview阶段,当前公开版本可以下载和体验,部分平台制品、安装路径与宿主体验仍在持续优化验证。安装前请先查看Release里的实际制品和版本说明。**
|
||||
|
||||
> 🙋 如果你遇到问题/有建议:欢迎提交Issue,并附上MemHub版本、操作系统与芯片架构、复现步骤、预期结果、实际结果和经过脱敏的日志。请不要上传APIKey、真实对话、个人路径或其他敏感信息。
|
||||
|
||||
> 我们会公开跟进可复现的问题,持续发布修复,并根据真实反馈完善安装、同步和召回体验。
|
||||
|
||||
**# 感谢每一条反馈,期待你和我们一起打磨MemHub的体验 :)**
|
||||
|
||||
> 💗 MemHub has just been open-sourced and is under active development. The current public release is available for download andevaluation. Some platform packages, installation paths, and host integrations are still being completed and verified. Check the assets
|
||||
and release notes before installing.
|
||||
If you run into a problem, please open an Issue with the version, platform,reproduction steps, and sanitized logs, We wilreviewreports, publish fixes, and keep the support matrix up to date.
|
||||
MemHub 是一个本地优先的共享记忆产品:把用户授权的多个 AI 产品中的历史对话和新增内容
|
||||
汇总到同一份记忆,并让这些 AI 在独立授权后复用它。Hy Memory 是 MemHub 使用的 Python
|
||||
SDK 和记忆引擎。One Memory / `one-memory` 是历史模块、数据目录、服务和兼容命令名称。
|
||||
|
||||
## 发行与平台合同
|
||||
|
||||
当前重构验证固定使用开发预览 `hy-memory==2.0.5.dev3`。MemHub 源码版本为 `1.0.4`
|
||||
(build 13),GitHub 仓库为
|
||||
公共 Python SDK 为 `hy-memory==2.0.3`。MemHub 源码版本为 `1.0.3`
|
||||
(build 7),GitHub 仓库为
|
||||
[`Tencent-Hunyuan/Hy-Memory`](https://github.com/Tencent-Hunyuan/Hy-Memory),npm 包名为
|
||||
`memhub-cli`。当前代码架构见
|
||||
[`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md)。
|
||||
|
||||
`v1.0.4` 的发行目标是 Linux x64/arm64 Runtime 与 macOS arm64/x64 App 和 Runtime。
|
||||
`v1.0.3` 的发行目标是 Linux x64/arm64 Runtime 与 macOS arm64/x64 App 和 Runtime。
|
||||
Linux 制品由 GitHub Actions 在原生架构 runner 上构建并完成生命周期验收;macOS
|
||||
制品由发行流水线完成 Developer ID 签名、公证和验收后手动上传。Windows 不支持。
|
||||
只有同时出现在 GitHub Release 与 `release-manifest.json` 中、且通过公开校验的制品才属于
|
||||
@@ -37,16 +25,14 @@ Linux 制品由 GitHub Actions 在原生架构 runner 上构建并完成生命
|
||||
## 核心原则
|
||||
|
||||
- 一次安装、一个 Runtime、一份数据、多种客户端。
|
||||
- Hy Memory SDK 是 memory/raw/vector/graph/coding 数据面的唯一权威;Python Core
|
||||
管理授权、模型、异步队列、Runtime 状态与 Matters 引用。
|
||||
- Python Core 是唯一记忆数据面和唯一业务数据库写入者。
|
||||
- 来源读取、模型处理和宿主读取是相互独立的授权边界。
|
||||
- 正式写链路为 `memory_write_jobs -> SDK add() -> raw/memory ID receipt`;
|
||||
Matter/Wiki 只保存并解析 SDK ID 引用。
|
||||
- 正式记忆链路保持 `Evidence -> Hy Memory -> Governance -> Matters/Wiki`。
|
||||
- 产品不包含屏幕、Accessibility / AXTree、OCR、键鼠、剪贴板或后台录音采集。
|
||||
|
||||
## 仓库结构
|
||||
|
||||
- `apps/core`:Python / FastAPI Kernel,负责授权、模型、队列、SDK bridge、Matters 与 API。
|
||||
- `apps/core`:Python / FastAPI Kernel,负责 Evidence、Hy Memory、治理、检索与 API。
|
||||
- `apps/cli`:安装、控制和诊断共享 Runtime 的 `one-memory` CLI。
|
||||
- `apps/web`:React / TypeScript 管理界面。
|
||||
- `apps/desktop`:Electron 桌面壳与图形安装入口。
|
||||
@@ -115,7 +101,7 @@ MemHub 的 SQLite 数据默认位于 `~/.one-memory`,不在 npm 包或 Runtime
|
||||
|
||||
## Hy Memory
|
||||
|
||||
当前开发依赖固定为公开 PyPI 开发预览 `hy-memory==2.0.5.dev3`。SDK 配置、权限边界及 search
|
||||
当前开发依赖固定为公开 PyPI 包 `hy-memory==2.0.3`。SDK 配置、权限边界及 search
|
||||
返回格式见 [`MEMORY.md`](MEMORY.md)。HY search 返回的 `content` 已包含 SDK 统一生成的
|
||||
逻辑时间和演化上下文;调用方不应再次转换时间戳或重复拼接 `When`。
|
||||
|
||||
|
||||
@@ -3,9 +3,9 @@
|
||||
"dataLicense": "CC0-1.0",
|
||||
"SPDXID": "SPDXRef-DOCUMENT",
|
||||
"name": "one-memory",
|
||||
"documentNamespace": "https://one-memory.local/spdx/2026-10-17T13:52:48Z",
|
||||
"documentNamespace": "https://one-memory.local/spdx/2026-10-17T11:07:57Z",
|
||||
"creationInfo": {
|
||||
"created": "2026-10-17T13:52:48Z",
|
||||
"created": "2026-10-17T11:07:57Z",
|
||||
"creators": [
|
||||
"Tool: one-memory-generate-sbom"
|
||||
]
|
||||
@@ -2335,7 +2335,7 @@
|
||||
{
|
||||
"SPDXID": "SPDXRef-Package-hy-memory",
|
||||
"name": "hy-memory",
|
||||
"versionInfo": "2.0.5.dev3",
|
||||
"versionInfo": "2.0.3",
|
||||
"downloadLocation": "NOASSERTION",
|
||||
"filesAnalyzed": false,
|
||||
"licenseConcluded": "Apache-2.0",
|
||||
|
||||
@@ -2,14 +2,14 @@
|
||||
|
||||
MemHub itself is licensed under Apache-2.0 (see `LICENSE`).
|
||||
It bundles the runtime dependencies listed below.
|
||||
Generated from the installed dependency tree on 2026-10-17T13:52:48Z
|
||||
Generated from the installed dependency tree on 2026-10-17T11:07:57Z
|
||||
by `python3 scripts/generate_sbom.py`. Do not edit by hand.
|
||||
|
||||
Per-package license texts are not reproduced here yet; the license
|
||||
column is filled in as part of release review, and the SBOM records
|
||||
`NOASSERTION` until then rather than guessing.
|
||||
|
||||
## hy-memory 2.0.5.dev3
|
||||
## hy-memory 2.0.3
|
||||
|
||||
License: Apache-2.0
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "@one-memory/cli",
|
||||
"version": "1.0.4",
|
||||
"version": "1.0.3",
|
||||
"description": "MemHub bootstrap CLI for installing and managing the local-first shared memory runtime",
|
||||
"license": "Apache-2.0",
|
||||
"repository": {
|
||||
|
||||
@@ -12,6 +12,7 @@
|
||||
* It is generated rather than hand-written so the two cannot drift: this reads
|
||||
* the real package.json and only strips what is provably not needed at runtime.
|
||||
*/
|
||||
import { execFileSync } from "node:child_process";
|
||||
import { mkdirSync, readFileSync, writeFileSync } from "node:fs";
|
||||
import { createRequire } from "node:module";
|
||||
import { dirname, resolve } from "node:path";
|
||||
@@ -19,14 +20,12 @@ import { fileURLToPath } from "node:url";
|
||||
|
||||
const here = dirname(fileURLToPath(import.meta.url));
|
||||
const root = resolve(here, "..");
|
||||
const repositoryRoot = resolve(root, "../..");
|
||||
const require = createRequire(import.meta.url);
|
||||
const { assertSemver } = require("../../../scripts/release-version.cjs");
|
||||
const { resolveBuildVersion } = require("../../../scripts/release-version.cjs");
|
||||
const releaseVersion = resolveBuildVersion(process.env, repositoryRoot);
|
||||
const pkgPath = resolve(root, "package.json");
|
||||
const pkg = JSON.parse(readFileSync(pkgPath, "utf8"));
|
||||
const publishVersion = assertSemver(
|
||||
process.env.MEMHUB_NPM_VERSION?.trim() || pkg.version,
|
||||
"MEMHUB_NPM_VERSION",
|
||||
);
|
||||
const publishName = process.env.MEMHUB_NPM_PACKAGE?.trim();
|
||||
if (!publishName || !/^(?:@[a-z0-9._-]+\/)?[a-z0-9._-]+$/.test(publishName)) {
|
||||
throw new Error(
|
||||
@@ -54,7 +53,7 @@ mkdirSync(outDir, { recursive: true });
|
||||
|
||||
const published = { ...pkg };
|
||||
published.name = publishName;
|
||||
published.version = publishVersion;
|
||||
published.version = releaseVersion;
|
||||
for (const section of ["dependencies", "devDependencies", "peerDependencies"]) {
|
||||
if (!published[section]) continue;
|
||||
published[section] = Object.fromEntries(
|
||||
|
||||
@@ -29,9 +29,10 @@ const REQUIRED_COMMAND_SURFACE = [
|
||||
"sources",
|
||||
"sync",
|
||||
"memories",
|
||||
"profile",
|
||||
"calls",
|
||||
"import-file",
|
||||
"notes",
|
||||
"extraction",
|
||||
"graph",
|
||||
"models",
|
||||
"host",
|
||||
@@ -54,6 +55,7 @@ const REQUIRED_COMMAND_SURFACE = [
|
||||
"scan",
|
||||
"progress",
|
||||
"jobs",
|
||||
"processing",
|
||||
"path",
|
||||
"adapter",
|
||||
"write",
|
||||
@@ -80,15 +82,16 @@ const REQUIRED_MARKERS = [
|
||||
*/
|
||||
const REQUIRED_SUBCOMMANDS = [
|
||||
{ command: "calls", marker: "memhub calls clear" },
|
||||
{ command: "import-file", marker: "memhub import-file upload" },
|
||||
{ command: "import-file", marker: "memhub import-file commit" },
|
||||
{ command: "import-file", marker: "memhub import-file discard" },
|
||||
{ command: "notes", marker: "memhub notes add" },
|
||||
{ command: "notes", marker: "memhub notes process" },
|
||||
{ command: "memories", marker: "memhub memories update" },
|
||||
{ command: "memories", marker: "memhub memories delete" },
|
||||
{ command: "write", marker: "memhub write submit" },
|
||||
{ command: "write", marker: "memhub write status" },
|
||||
{ command: "extraction", marker: "memhub extraction set" },
|
||||
{ command: "graph", marker: "memhub graph width" },
|
||||
{ command: "models", marker: "memhub models edit" },
|
||||
{ command: "models", marker: "memhub models activate" },
|
||||
{ command: "processing", marker: "memhub processing extract" },
|
||||
{ command: "jobs", marker: "memhub jobs start" },
|
||||
{ command: "jobs", marker: "memhub jobs <pause|resume|cancel>" },
|
||||
];
|
||||
@@ -119,6 +122,24 @@ const STOPPED_DESTRUCTIVE_PROBES = [
|
||||
name: "calls_clear_all_apply",
|
||||
args: ["calls", "clear", "--all", "--confirm"],
|
||||
},
|
||||
{
|
||||
name: "import_file_commit_apply",
|
||||
args: [
|
||||
"import-file",
|
||||
"commit",
|
||||
"22222222-2222-4222-8222-222222222222",
|
||||
"--confirm",
|
||||
],
|
||||
},
|
||||
{
|
||||
name: "import_file_discard_apply",
|
||||
args: [
|
||||
"import-file",
|
||||
"discard",
|
||||
"22222222-2222-4222-8222-222222222222",
|
||||
"--confirm",
|
||||
],
|
||||
},
|
||||
{
|
||||
name: "notes_process_apply",
|
||||
args: [
|
||||
@@ -128,10 +149,18 @@ const STOPPED_DESTRUCTIVE_PROBES = [
|
||||
"--confirm",
|
||||
],
|
||||
},
|
||||
{
|
||||
name: "extraction_set_apply",
|
||||
args: ["extraction", "set", "--window", "6m", "--confirm"],
|
||||
},
|
||||
{
|
||||
name: "graph_width_apply",
|
||||
args: ["graph", "width", "--decision", "keep_existing", "--confirm"],
|
||||
},
|
||||
{
|
||||
name: "processing_extract_apply",
|
||||
args: ["processing", "extract", "--confirm"],
|
||||
},
|
||||
{
|
||||
name: "models_activate_apply",
|
||||
args: [
|
||||
|
||||
@@ -1785,11 +1785,14 @@ describe("stopped Runtime is exit 3, not a usage error", () => {
|
||||
["task", "--id", "nope", "--json"],
|
||||
[
|
||||
"write",
|
||||
"submit",
|
||||
"--session",
|
||||
"s",
|
||||
"--messages-file",
|
||||
"/tmp/messages.json",
|
||||
"--from",
|
||||
"1",
|
||||
"--to",
|
||||
"1",
|
||||
"--evidence",
|
||||
"e",
|
||||
"--json",
|
||||
],
|
||||
["backup", "create", "--target", "/tmp/x", "--yes", "--json"],
|
||||
|
||||
+48
-218
@@ -4,11 +4,7 @@ import { createReadStream } from "node:fs";
|
||||
import { access } from "node:fs/promises";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
import {
|
||||
type HealthResponse,
|
||||
type ImportJob,
|
||||
isHostId,
|
||||
} from "@one-memory/contracts";
|
||||
import type { HealthResponse, ImportJob } from "@one-memory/contracts";
|
||||
import {
|
||||
type AcquiredRuntimeDistribution,
|
||||
acquireRuntimeDistribution,
|
||||
@@ -80,7 +76,6 @@ import {
|
||||
createCoreReader,
|
||||
createUiSessionRequest,
|
||||
deleteModelProfile,
|
||||
parseOpenAIMessages,
|
||||
probeAdapter,
|
||||
readAdapters,
|
||||
readMemoryWrite,
|
||||
@@ -100,6 +95,7 @@ import {
|
||||
createCoreOperations,
|
||||
} from "./core-operations.js";
|
||||
import {
|
||||
MEMORY_STATUSES,
|
||||
type SourceConnectionCandidate,
|
||||
validateImportJob,
|
||||
validateSourceConnectionCandidates,
|
||||
@@ -335,7 +331,7 @@ import {
|
||||
renderVersion,
|
||||
renderVersionDocument,
|
||||
} from "./usage.js";
|
||||
import { CLI_VERSION, PRODUCT, RUNTIME_RELEASE_VERSION } from "./version.js";
|
||||
import { CLI_VERSION, PRODUCT } from "./version.js";
|
||||
import { runWatchLoop, type WatchEvent, type WatchRead } from "./watch-loop.js";
|
||||
|
||||
export interface RunControls {
|
||||
@@ -421,8 +417,7 @@ async function selectInstallSource(
|
||||
throw new Error("release_manifest_url_unavailable");
|
||||
}
|
||||
const manifestUrl =
|
||||
explicitManifestUrl ??
|
||||
template.replaceAll("{version}", RUNTIME_RELEASE_VERSION);
|
||||
explicitManifestUrl ?? template.replaceAll("{version}", CLI_VERSION);
|
||||
const allowedHosts = parseAllowedReleaseHosts(
|
||||
environment.MEMHUB_RELEASE_ALLOWED_HOSTS,
|
||||
);
|
||||
@@ -467,7 +462,7 @@ async function resolveRemoteInstallPreview(
|
||||
): Promise<ResolvedRuntimeRelease> {
|
||||
return (controls.resolveRuntimeRelease ?? resolveRuntimeRelease)({
|
||||
manifestUrl: selection.manifestUrl,
|
||||
expectedVersion: RUNTIME_RELEASE_VERSION,
|
||||
expectedVersion: CLI_VERSION,
|
||||
channel: selection.channel,
|
||||
platform: process.platform,
|
||||
arch: process.arch,
|
||||
@@ -481,7 +476,7 @@ async function acquireRemoteInstallSource(
|
||||
): Promise<AcquiredRuntimeDistribution> {
|
||||
return (controls.acquireRuntimeDistribution ?? acquireRuntimeDistribution)({
|
||||
manifestUrl: selection.manifestUrl,
|
||||
expectedVersion: RUNTIME_RELEASE_VERSION,
|
||||
expectedVersion: CLI_VERSION,
|
||||
channel: selection.channel,
|
||||
platform: process.platform,
|
||||
arch: process.arch,
|
||||
@@ -922,36 +917,7 @@ async function runCommand(
|
||||
if (attachment === null) {
|
||||
return runtimeUnavailable(descriptor, asJson);
|
||||
}
|
||||
if (parsed.kind === "status") {
|
||||
const receipt = await readMemoryWrite(attachment, parsed.id);
|
||||
process.stdout.write(
|
||||
asJson
|
||||
? `${JSON.stringify({ schema_version: 1, write: receipt })}\n`
|
||||
: receipt === null
|
||||
? "未找到该 request id。\n"
|
||||
: `request_id: ${receipt.request_id}\nstate: ${receipt.state}\n`,
|
||||
);
|
||||
return EXIT_OK;
|
||||
}
|
||||
const document = await readBoundedJson(parsed.messages_file);
|
||||
const messages = parseOpenAIMessages(document);
|
||||
if (messages === null) {
|
||||
return reject(
|
||||
descriptor,
|
||||
asJson,
|
||||
"messages 文件必须是含 user/assistant/tool 文本消息的 OpenAI messages 数组。",
|
||||
);
|
||||
}
|
||||
const receipt = await submitMemoryWrite(attachment, {
|
||||
session: parsed.session,
|
||||
source: parsed.source,
|
||||
request_id: parsed.request_id ?? randomUUID(),
|
||||
messages,
|
||||
...(parsed.memory_at === undefined
|
||||
? {}
|
||||
: { memory_at: parsed.memory_at }),
|
||||
...(parsed.priority === undefined ? {} : { priority: parsed.priority }),
|
||||
});
|
||||
const receipt = await submitMemoryWrite(attachment, parsed);
|
||||
process.stdout.write(
|
||||
asJson
|
||||
? `${JSON.stringify({ schema_version: 1, write: receipt })}\n`
|
||||
@@ -2323,75 +2289,6 @@ async function runCommand(
|
||||
emitReadDocument(named, document, asJson);
|
||||
return sourceProgressExitCode(document);
|
||||
}
|
||||
if (named === "profile") {
|
||||
if (arguments_.slice(1).some((value) => value !== "--json")) return usage();
|
||||
const config = resolveRuntimeCliConfig(environment, process.argv[1]);
|
||||
const attached = await attachClients(config.dataDir, {
|
||||
readerTimeoutMs: 15_000,
|
||||
});
|
||||
if (!attached.reader) {
|
||||
const document = {
|
||||
schema_version: 2,
|
||||
product: PRODUCT,
|
||||
state: "unavailable",
|
||||
code: attached.failure ?? "runtime_attach_failed",
|
||||
};
|
||||
emit(
|
||||
asJson
|
||||
? `${JSON.stringify(document)}\n`
|
||||
: `用户画像读取失败:${document.code}\n`,
|
||||
);
|
||||
return EXIT_RUNTIME_STOPPED;
|
||||
}
|
||||
const outcome = await attached.reader<unknown>("/api/v2/user-profile");
|
||||
if (outcome.state !== "ok") {
|
||||
const document = {
|
||||
schema_version: 2,
|
||||
product: PRODUCT,
|
||||
state: outcome.state,
|
||||
code: outcome.code,
|
||||
};
|
||||
emit(
|
||||
asJson
|
||||
? `${JSON.stringify(document)}\n`
|
||||
: `用户画像读取失败:${outcome.code}\n`,
|
||||
);
|
||||
return EXIT_FAILED;
|
||||
}
|
||||
const value = outcome.data as {
|
||||
request_id?: unknown;
|
||||
profile?: unknown;
|
||||
elapsed_ms?: unknown;
|
||||
};
|
||||
if (
|
||||
typeof value.request_id !== "string" ||
|
||||
!(
|
||||
value.profile === null ||
|
||||
(typeof value.profile === "object" && !Array.isArray(value.profile))
|
||||
) ||
|
||||
typeof value.elapsed_ms !== "number"
|
||||
) {
|
||||
return reject(descriptor, asJson, "用户画像响应格式无效。");
|
||||
}
|
||||
const document = {
|
||||
schema_version: 2,
|
||||
product: PRODUCT,
|
||||
state: "ok",
|
||||
...value,
|
||||
};
|
||||
const content =
|
||||
value.profile && typeof value.profile === "object"
|
||||
? (value.profile as { content?: unknown }).content
|
||||
: null;
|
||||
emit(
|
||||
asJson
|
||||
? `${JSON.stringify(document)}\n`
|
||||
: value.profile === null
|
||||
? "尚未形成用户长期画像。\n"
|
||||
: `${typeof content === "string" ? content : JSON.stringify(value.profile)}\n`,
|
||||
);
|
||||
return EXIT_OK;
|
||||
}
|
||||
if (named === "memories") {
|
||||
const parsed = await parseMemoryCommand(arguments_.slice(1));
|
||||
if (!parsed) return usage();
|
||||
@@ -2424,8 +2321,6 @@ async function runCommand(
|
||||
}
|
||||
if (blocked || !reader) {
|
||||
if (
|
||||
parsed.kind === "update" ||
|
||||
parsed.kind === "delete" ||
|
||||
parsed.kind === "plan-add" ||
|
||||
parsed.kind === "plan-existing" ||
|
||||
parsed.kind === "apply"
|
||||
@@ -2474,62 +2369,6 @@ async function runCommand(
|
||||
);
|
||||
return document.memory.state === "ok" ? EXIT_OK : EXIT_FAILED;
|
||||
}
|
||||
if (parsed.kind === "update" || parsed.kind === "delete") {
|
||||
const content =
|
||||
parsed.kind === "update"
|
||||
? await readMemoryContentFile(parsed.contentFile)
|
||||
: null;
|
||||
if (parsed.kind === "update" && content === null) return usage();
|
||||
const plan = {
|
||||
schema_version: 2,
|
||||
product: PRODUCT,
|
||||
state: "preview",
|
||||
operation: parsed.kind,
|
||||
memory_id: parsed.memoryId,
|
||||
...(content === null
|
||||
? {}
|
||||
: {
|
||||
content_sha256: createHash("sha256")
|
||||
.update(content)
|
||||
.digest("hex"),
|
||||
}),
|
||||
};
|
||||
if (!parsed.confirmed) {
|
||||
emit(
|
||||
asJson
|
||||
? `${JSON.stringify(plan)}\n`
|
||||
: `计划:${parsed.kind} ${parsed.memoryId}\n使用 --confirm 执行。\n`,
|
||||
);
|
||||
return EXIT_OK;
|
||||
}
|
||||
if (!operations) return EXIT_FAILED;
|
||||
const writable = operations as CoreOperations &
|
||||
CorePutOperations &
|
||||
CoreDeleteOperations;
|
||||
const path = `/api/v2/memories/${encodeURIComponent(parsed.memoryId)}`;
|
||||
const outcome =
|
||||
parsed.kind === "update"
|
||||
? await writable.put<unknown>(path, { content })
|
||||
: await writable.delete<unknown>(path);
|
||||
const document = {
|
||||
schema_version: 2,
|
||||
product: PRODUCT,
|
||||
state: outcome.state,
|
||||
operation: parsed.kind,
|
||||
memory_id: parsed.memoryId,
|
||||
...(outcome.state === "committed"
|
||||
? { result: outcome.data }
|
||||
: { code: outcome.code }),
|
||||
};
|
||||
emit(
|
||||
asJson
|
||||
? `${JSON.stringify(document)}\n`
|
||||
: outcome.state === "committed"
|
||||
? `已${parsed.kind === "update" ? "更新" : "删除"}记忆 ${parsed.memoryId}\n`
|
||||
: `操作失败:${outcome.code}\n`,
|
||||
);
|
||||
return outcome.state === "committed" ? EXIT_OK : EXIT_FAILED;
|
||||
}
|
||||
if (parsed.kind === "plan-add") {
|
||||
const content = await readMemoryContentFile(parsed.contentFile);
|
||||
if (content === null) return usage();
|
||||
@@ -2553,6 +2392,7 @@ async function runCommand(
|
||||
operation: parsed.operation,
|
||||
memoryId: parsed.memoryId,
|
||||
...(content === undefined ? {} : { content }),
|
||||
...(parsed.scopes.length === 0 ? {} : { scopes: parsed.scopes }),
|
||||
});
|
||||
emitMemoryFlow(document, asJson);
|
||||
return memoryFlowExitCode(document);
|
||||
@@ -5315,10 +5155,10 @@ type HostCommand =
|
||||
args: unknown[];
|
||||
};
|
||||
|
||||
// Host ids are read with the same grammar as the permission layer and the
|
||||
// grant store (`@one-memory/contracts`): `deepseek-harness:web` is a real id —
|
||||
// that host is configured per profile — and `host list` prints it, so rejecting
|
||||
// it here would mean listing a host the CLI then refuses to act on.
|
||||
// `deepseek-harness:web` is a real id — that host is configured per profile —
|
||||
// and `host list` prints it. Rejecting it here would mean listing a host the
|
||||
// CLI then refuses to act on.
|
||||
const HOST_ID = /^[a-z0-9][a-z0-9-]{0,31}(:[a-z0-9][a-z0-9-]{0,31})?$/;
|
||||
const FINGERPRINT = /^[0-9a-f]{64}$/;
|
||||
|
||||
/** Host subcommands.
|
||||
@@ -5341,7 +5181,7 @@ function parseHostCommand(rest: readonly string[]): HostCommand | null {
|
||||
const token = tokens[index];
|
||||
if (token === "--host") {
|
||||
const value = tokens[index + 1];
|
||||
if (!value || !isHostId(value)) return null;
|
||||
if (!value || !HOST_ID.test(value)) return null;
|
||||
if (hostId !== undefined) return null;
|
||||
hostId = value;
|
||||
index += 1;
|
||||
@@ -5374,7 +5214,7 @@ function parseHostCommand(rest: readonly string[]): HostCommand | null {
|
||||
qualifier !== undefined &&
|
||||
(verb === "status" || verb === "install" || verb === "uninstall")
|
||||
) {
|
||||
if (!isHostId(qualifier)) return null;
|
||||
if (!HOST_ID.test(qualifier)) return null;
|
||||
hostId = qualifier;
|
||||
// Drop the consumed host positional so the verb/qualifier checks below see
|
||||
// the same shape as the flag form.
|
||||
@@ -5390,7 +5230,7 @@ function parseHostCommand(rest: readonly string[]): HostCommand | null {
|
||||
positional[2] !== undefined
|
||||
) {
|
||||
const trailing = positional[2];
|
||||
if (!isHostId(trailing)) return null;
|
||||
if (!HOST_ID.test(trailing)) return null;
|
||||
hostId = trailing;
|
||||
positional.splice(2, 1);
|
||||
verb = positional[0];
|
||||
@@ -5939,13 +5779,6 @@ function emitReadDocument(
|
||||
type MemoryCommand =
|
||||
| { kind: "list"; query: MemoryQuery }
|
||||
| { kind: "show"; memoryId: string }
|
||||
| {
|
||||
kind: "update";
|
||||
memoryId: string;
|
||||
contentFile: string;
|
||||
confirmed: boolean;
|
||||
}
|
||||
| { kind: "delete"; memoryId: string; confirmed: boolean }
|
||||
| {
|
||||
kind: "plan-add";
|
||||
contentFile: string;
|
||||
@@ -5955,9 +5788,10 @@ type MemoryCommand =
|
||||
}
|
||||
| {
|
||||
kind: "plan-existing";
|
||||
operation: "correct" | "delete";
|
||||
operation: "correct" | "delete" | "scope";
|
||||
memoryId: string;
|
||||
contentFile?: string;
|
||||
scopes: string[];
|
||||
}
|
||||
| { kind: "apply"; planFile: string; contentFile?: string };
|
||||
|
||||
@@ -5973,33 +5807,6 @@ async function parseMemoryCommand(
|
||||
? { kind: "show", memoryId }
|
||||
: null;
|
||||
}
|
||||
if (action === "update" || action === "delete") {
|
||||
const memoryId = tokens[1];
|
||||
if (!memoryId || !isMemoryId(memoryId)) return null;
|
||||
let contentFile: string | undefined;
|
||||
let confirmed = false;
|
||||
for (let index = 2; index < tokens.length; index += 1) {
|
||||
const flag = tokens[index];
|
||||
if (flag === "--confirm") {
|
||||
if (confirmed) return null;
|
||||
confirmed = true;
|
||||
continue;
|
||||
}
|
||||
const value = tokens[index + 1];
|
||||
if (value === undefined || value.startsWith("--")) return null;
|
||||
if (flag !== "--content-file" || contentFile !== undefined) return null;
|
||||
contentFile = value;
|
||||
index += 1;
|
||||
}
|
||||
if (action === "update")
|
||||
return contentFile
|
||||
? { kind: "update", memoryId, contentFile, confirmed }
|
||||
: null;
|
||||
return contentFile === undefined
|
||||
? { kind: "delete", memoryId, confirmed }
|
||||
: null;
|
||||
}
|
||||
if (action === "plan" || action === "apply") return null;
|
||||
if (action === "plan") return parseMemoryPlanCommand(tokens.slice(1));
|
||||
if (action === "apply") {
|
||||
let planFile: string | undefined;
|
||||
@@ -6025,7 +5832,10 @@ async function parseMemoryCommand(
|
||||
: null;
|
||||
}
|
||||
if (action !== "list") return null;
|
||||
let query: string | undefined;
|
||||
let queryFile: string | undefined;
|
||||
let status: string | undefined;
|
||||
let source: string | undefined;
|
||||
let offset = 0;
|
||||
let limit = 100;
|
||||
let offsetSet = false;
|
||||
@@ -6034,7 +5844,11 @@ async function parseMemoryCommand(
|
||||
const flag = tokens[index];
|
||||
const value = tokens[index + 1];
|
||||
if (value === undefined || value.startsWith("--")) return null;
|
||||
if (flag === "--status" && status === undefined) status = value;
|
||||
if (flag === "--query" && query === undefined) query = value;
|
||||
else if (flag === "--query-file" && queryFile === undefined)
|
||||
queryFile = value;
|
||||
else if (flag === "--status" && status === undefined) status = value;
|
||||
else if (flag === "--source" && source === undefined) source = value;
|
||||
else if (flag === "--limit" && !limitSet) {
|
||||
limit = Number(value);
|
||||
limitSet = true;
|
||||
@@ -6043,11 +5857,24 @@ async function parseMemoryCommand(
|
||||
offsetSet = true;
|
||||
} else return null;
|
||||
}
|
||||
if (status !== undefined && (status.length === 0 || status.length > 64))
|
||||
if (query !== undefined && queryFile !== undefined) return null;
|
||||
if (queryFile !== undefined) {
|
||||
query = (await readQueryFile(queryFile)) ?? undefined;
|
||||
if (query === undefined) return null;
|
||||
}
|
||||
if (
|
||||
query !== undefined &&
|
||||
(query.length === 0 || query.length > QUERY_MAX_CHARACTERS)
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
if (status !== undefined && !MEMORY_STATUSES.includes(status as never)) {
|
||||
return null;
|
||||
}
|
||||
if (source !== undefined && source.length === 0) return null;
|
||||
if (!Number.isSafeInteger(limit) || limit < 1 || limit > 500) return null;
|
||||
if (!Number.isSafeInteger(offset) || offset < 0) return null;
|
||||
return { kind: "list", query: { status, offset, limit } };
|
||||
return { kind: "list", query: { query, status, source, offset, limit } };
|
||||
}
|
||||
|
||||
function parseMemoryPlanCommand(
|
||||
@@ -6055,7 +5882,7 @@ function parseMemoryPlanCommand(
|
||||
): MemoryCommand | null {
|
||||
const operation = tokens[0];
|
||||
const add = operation === "add";
|
||||
if (!add && !["correct", "delete"].includes(operation)) {
|
||||
if (!add && !["correct", "delete", "scope"].includes(operation)) {
|
||||
return null;
|
||||
}
|
||||
const memoryId = add ? undefined : tokens[1];
|
||||
@@ -6074,7 +5901,7 @@ function parseMemoryPlanCommand(
|
||||
else if (flag === "--kind" && add && !kindSet) {
|
||||
memoryKind = value;
|
||||
kindSet = true;
|
||||
} else if (flag === "--scope" && add && !scopes.includes(value))
|
||||
} else if (flag === "--scope" && !scopes.includes(value))
|
||||
scopes.push(value);
|
||||
else if (flag === "--evidence" && add && !evidenceIds.includes(value))
|
||||
evidenceIds.push(value);
|
||||
@@ -6094,15 +5921,18 @@ function parseMemoryPlanCommand(
|
||||
}
|
||||
if (
|
||||
(operation === "correct" && !contentFile) ||
|
||||
(operation === "delete" && (contentFile !== undefined || scopes.length > 0))
|
||||
(operation === "scope" && scopes.length === 0) ||
|
||||
(!["correct", "scope"].includes(operation) &&
|
||||
(contentFile !== undefined || scopes.length > 0))
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
return {
|
||||
kind: "plan-existing",
|
||||
operation: operation as "correct" | "delete",
|
||||
operation: operation as "correct" | "delete" | "scope",
|
||||
memoryId: memoryId as string,
|
||||
...(contentFile ? { contentFile } : {}),
|
||||
scopes,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -6690,7 +6520,7 @@ function parseInstallOptions(
|
||||
...(releaseManifestUrl ? { releaseManifestUrl } : {}),
|
||||
releaseChannel:
|
||||
releaseChannel ??
|
||||
(RUNTIME_RELEASE_VERSION.includes("-") ? "prerelease" : "stable"),
|
||||
(CLI_VERSION.includes("-") ? "prerelease" : "stable"),
|
||||
installRoot: resolvedInstallRoot,
|
||||
home: resolvedHome,
|
||||
uid: uid ?? process.getuid?.(),
|
||||
|
||||
@@ -347,6 +347,22 @@ export const COMMANDS: readonly CommandDescriptor[] = [
|
||||
],
|
||||
exit_codes: [EXIT_OK, EXIT_FAILED, EXIT_USAGE, EXIT_RUNTIME_STOPPED],
|
||||
},
|
||||
{
|
||||
name: "processing",
|
||||
summary: "手动推进待处理证据;默认只预览,确认后执行。",
|
||||
usage:
|
||||
"memhub processing extract [--evidence <id> ...] [--confirm] [--json]",
|
||||
options: [
|
||||
{
|
||||
flag: "--evidence <id>",
|
||||
summary:
|
||||
"processing extract 要提前处理的证据 ID,可重复;不给则推进下一批。",
|
||||
},
|
||||
DIRECT_CONFIRM_OPTION,
|
||||
JSON_OPTION,
|
||||
],
|
||||
exit_codes: [EXIT_OK, EXIT_FAILED, EXIT_USAGE, EXIT_RUNTIME_STOPPED],
|
||||
},
|
||||
{
|
||||
name: "models",
|
||||
summary:
|
||||
@@ -371,14 +387,25 @@ export const COMMANDS: readonly CommandDescriptor[] = [
|
||||
},
|
||||
{
|
||||
name: "memories",
|
||||
summary: "通过 SDK 数据面列出、查看、修改或物理删除记忆。",
|
||||
summary:
|
||||
"列出/查看记忆,或生成并显式确认带版本与 request identity 的治理操作计划。",
|
||||
usage:
|
||||
"memhub memories list [--status <state>] [--offset <n>] [--limit <n>] [--json]\n memhub memories show <memory-id> [--json]\n memhub memories update <memory-id> --content-file <path> [--confirm] [--json]\n memhub memories delete <memory-id> [--confirm] [--json]",
|
||||
"memhub memories list [--query <text>] [--query-file <path>] [--status <state>] [--source <id>] [--offset <n>] [--limit <n>] [--json]\n memhub memories show <memory-id> [--json]\n memhub memories plan add --content-file <path> --scope <scope> --evidence <id> [--kind <kind>] [--json]\n memhub memories plan <correct|delete|scope> <memory-id> [--content-file <path>] [--scope <scope>] [--json]\n memhub memories apply --plan-file <path> --confirm [--content-file <path>] [--json]",
|
||||
options: [
|
||||
{
|
||||
flag: "--query <text>",
|
||||
summary:
|
||||
"按内容过滤;会出现在 shell 历史里,敏感词请改用 --query-file。",
|
||||
},
|
||||
{
|
||||
flag: "--query-file <path>",
|
||||
summary: "从文件读取过滤词,避免查询内容进入命令行历史。",
|
||||
},
|
||||
{
|
||||
flag: "--status <state>",
|
||||
summary: `记忆状态,取值 ${MEMORY_STATUSES.join("、")}。`,
|
||||
},
|
||||
{ flag: "--source <id>", summary: "只看某个来源产生的记忆。" },
|
||||
{ flag: "--offset <n>", summary: "跳过前 n 条,默认 0。" },
|
||||
{
|
||||
flag: "--limit <n>",
|
||||
@@ -389,18 +416,24 @@ export const COMMANDS: readonly CommandDescriptor[] = [
|
||||
summary:
|
||||
"从有界普通文件读取新增/纠正正文;计划只保存摘要,执行时再次校验。",
|
||||
},
|
||||
DIRECT_CONFIRM_OPTION,
|
||||
{
|
||||
flag: "--kind <chat|working|coding|schema|intention>",
|
||||
summary: "新增记忆的类型,默认 chat。",
|
||||
},
|
||||
{
|
||||
flag: "--scope <scope>",
|
||||
summary: "记忆使用范围,可重复;private 必须独占。",
|
||||
},
|
||||
{
|
||||
flag: "--evidence <id>",
|
||||
summary: "新增记忆引用的 Evidence ID,可重复。",
|
||||
},
|
||||
PLAN_FILE_OPTION,
|
||||
EXPLICIT_CONFIRM_OPTION,
|
||||
JSON_OPTION,
|
||||
],
|
||||
exit_codes: [EXIT_OK, EXIT_FAILED, EXIT_USAGE, EXIT_RUNTIME_STOPPED],
|
||||
},
|
||||
{
|
||||
name: "profile",
|
||||
summary: "读取 SDK 组装后的用户长期画像(L0 + 已启用的长期画像补充)。",
|
||||
usage: "memhub profile [--json]",
|
||||
options: [DATA_DIR_OPTION, JSON_OPTION],
|
||||
exit_codes: [EXIT_OK, EXIT_FAILED, EXIT_USAGE, EXIT_RUNTIME_STOPPED],
|
||||
},
|
||||
{
|
||||
name: "calls",
|
||||
summary:
|
||||
@@ -421,6 +454,28 @@ export const COMMANDS: readonly CommandDescriptor[] = [
|
||||
],
|
||||
exit_codes: [EXIT_OK, EXIT_FAILED, EXIT_USAGE, EXIT_RUNTIME_STOPPED],
|
||||
},
|
||||
{
|
||||
name: "import-file",
|
||||
summary:
|
||||
"上传、提交或丢弃一份历史导入文件;提交与丢弃默认只出计划,需 --confirm。",
|
||||
usage:
|
||||
"memhub import-file upload --file <path> [--kind raw_conversation|memory_content] [--confirm] [--json]\n memhub import-file commit <upload-id> [--confirm] [--json]\n memhub import-file discard <upload-id> [--confirm] [--json]",
|
||||
options: [
|
||||
{
|
||||
flag: "--file <path>",
|
||||
summary:
|
||||
"要上传的普通文件(非符号链接,<=128 MiB,.zip/.json/.md/.txt)。",
|
||||
},
|
||||
{
|
||||
flag: "--kind <kind>",
|
||||
summary:
|
||||
"输入类型,默认 raw_conversation;memory_content 用于记忆正文。",
|
||||
},
|
||||
DIRECT_CONFIRM_OPTION,
|
||||
JSON_OPTION,
|
||||
],
|
||||
exit_codes: [EXIT_OK, EXIT_FAILED, EXIT_USAGE, EXIT_RUNTIME_STOPPED],
|
||||
},
|
||||
{
|
||||
name: "notes",
|
||||
summary:
|
||||
@@ -438,6 +493,20 @@ export const COMMANDS: readonly CommandDescriptor[] = [
|
||||
],
|
||||
exit_codes: [EXIT_OK, EXIT_FAILED, EXIT_USAGE, EXIT_RUNTIME_STOPPED],
|
||||
},
|
||||
{
|
||||
name: "extraction",
|
||||
summary: "设置提取窗口;默认只出计划,--confirm 后才写入。",
|
||||
usage: "memhub extraction set --window <3m|6m|1y|all> [--confirm] [--json]",
|
||||
options: [
|
||||
{
|
||||
flag: "--window <preset>",
|
||||
summary: "提取窗口预设:3m、6m、1y 或 all。",
|
||||
},
|
||||
DIRECT_CONFIRM_OPTION,
|
||||
JSON_OPTION,
|
||||
],
|
||||
exit_codes: [EXIT_OK, EXIT_FAILED, EXIT_USAGE, EXIT_RUNTIME_STOPPED],
|
||||
},
|
||||
{
|
||||
name: "graph",
|
||||
summary: "图谱宽度不匹配时的用户决定;默认只出计划,--confirm 后才执行。",
|
||||
@@ -653,16 +722,16 @@ export const COMMANDS: readonly CommandDescriptor[] = [
|
||||
},
|
||||
{
|
||||
name: "write",
|
||||
summary: "提交 OpenAI messages 到 SDK 写入队列,或按 request id 查询状态。",
|
||||
summary:
|
||||
"提交一次记忆写入,返回可查询的 request id;写入异步完成,不可重复提交。",
|
||||
usage:
|
||||
"memhub write submit --messages-file <path> --session <id> [--source <id>] [--request-id <id>] [--memory-at <RFC3339>] [--live] [--json]\n memhub write status --id <request-id> [--json]",
|
||||
"memhub write --session <id> --from <round> --to <round> --evidence <id> [--evidence <id>...] [--live] [--json]",
|
||||
options: [DATA_DIR_OPTION, JSON_OPTION],
|
||||
exit_codes: [EXIT_OK, EXIT_USAGE, EXIT_FAILED],
|
||||
},
|
||||
{
|
||||
name: "task",
|
||||
summary:
|
||||
"兼容别名:按 request id 查询写入状态;新脚本请使用 write status。",
|
||||
summary: "查询写入任务状态;unknown 表示结果待定,不可据此重发。",
|
||||
usage: "memhub task --id <request-id> [--json]",
|
||||
options: [DATA_DIR_OPTION, JSON_OPTION],
|
||||
exit_codes: [EXIT_OK, EXIT_USAGE, EXIT_FAILED],
|
||||
|
||||
@@ -74,12 +74,14 @@ const COMMANDS = [
|
||||
"sync",
|
||||
"progress",
|
||||
"jobs",
|
||||
"processing",
|
||||
"models",
|
||||
"memories",
|
||||
"profile",
|
||||
// C6 App BLOCK (25f4ecf): UI write operations that had no CLI equivalent.
|
||||
"calls",
|
||||
"import-file",
|
||||
"notes",
|
||||
"extraction",
|
||||
"graph",
|
||||
"host",
|
||||
"recall",
|
||||
@@ -560,7 +562,7 @@ describe("documented commands", () => {
|
||||
// `sync <pause|resume>` is documented as one alternation, so it is
|
||||
// checked as an entry rather than as two literal subcommands.
|
||||
sync: ["status", "<pause|resume>"],
|
||||
memories: ["list", "show", "update", "delete"],
|
||||
memories: ["list", "show", "plan", "apply"],
|
||||
host: ["list", "status", "install", "uninstall", "sync"],
|
||||
};
|
||||
|
||||
|
||||
@@ -384,101 +384,6 @@ describe("unified context search", () => {
|
||||
expect(streams.stdout()).not.toContain("must-not-print");
|
||||
});
|
||||
|
||||
it("counts characters as code points so an emoji hit stays grounded", async () => {
|
||||
// Core reports used_characters with Python's len(), which counts code
|
||||
// points: this text is 10 characters there and 12 UTF-16 units here.
|
||||
const text = "消费保 📍 扩展 📡";
|
||||
expect(text.length).toBe(12);
|
||||
expect(Array.from(text).length).toBe(10);
|
||||
transport.post.mockResolvedValue({
|
||||
status: 200,
|
||||
data: {
|
||||
...RESPONSE,
|
||||
paths: {
|
||||
...RESPONSE.paths,
|
||||
memory: {
|
||||
...RESPONSE.paths.memory,
|
||||
status: "empty",
|
||||
returned_count: 0,
|
||||
},
|
||||
},
|
||||
items: [
|
||||
{
|
||||
...(RESPONSE.items[0] as Record<string, unknown>),
|
||||
messages: [
|
||||
{
|
||||
...(
|
||||
RESPONSE.items[0] as {
|
||||
messages: Array<Record<string, unknown>>;
|
||||
}
|
||||
).messages[0],
|
||||
text,
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
used_characters: 10,
|
||||
},
|
||||
});
|
||||
const streams = captureStreams();
|
||||
try {
|
||||
expect(await run(["recall", "test", "--json"])).toBe(0);
|
||||
} finally {
|
||||
streams.restore();
|
||||
}
|
||||
expect(JSON.parse(streams.stdout())).toMatchObject({
|
||||
state: "grounded",
|
||||
used_characters: 10,
|
||||
items: [{ kind: "raw", messages: [{ text }] }],
|
||||
});
|
||||
});
|
||||
|
||||
it("still rejects a response whose character total does not match", async () => {
|
||||
// The same text counted as UTF-16 units is exactly what Core does not mean,
|
||||
// so the cross-check must keep failing instead of accepting both totals.
|
||||
const text = "消费保 📍 扩展 📡";
|
||||
transport.post.mockResolvedValue({
|
||||
status: 200,
|
||||
data: {
|
||||
...RESPONSE,
|
||||
paths: {
|
||||
...RESPONSE.paths,
|
||||
memory: {
|
||||
...RESPONSE.paths.memory,
|
||||
status: "empty",
|
||||
returned_count: 0,
|
||||
},
|
||||
},
|
||||
items: [
|
||||
{
|
||||
...(RESPONSE.items[0] as Record<string, unknown>),
|
||||
messages: [
|
||||
{
|
||||
...(
|
||||
RESPONSE.items[0] as {
|
||||
messages: Array<Record<string, unknown>>;
|
||||
}
|
||||
).messages[0],
|
||||
text,
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
used_characters: 12,
|
||||
},
|
||||
});
|
||||
const streams = captureStreams();
|
||||
try {
|
||||
expect(await run(["recall", "test", "--json"])).toBe(1);
|
||||
} finally {
|
||||
streams.restore();
|
||||
}
|
||||
expect(JSON.parse(streams.stdout())).toMatchObject({
|
||||
state: "invalid",
|
||||
code: "response_invalid",
|
||||
});
|
||||
});
|
||||
|
||||
it("reports endpoint denial without exposing the response body", async () => {
|
||||
transport.post.mockResolvedValue({
|
||||
status: 403,
|
||||
|
||||
@@ -181,9 +181,9 @@ function validateContextResponse(
|
||||
items.push(item);
|
||||
characters +=
|
||||
item.kind === "memory"
|
||||
? characterCount(item.content)
|
||||
? String(item.content).length
|
||||
: (item.messages as Array<Record<string, unknown>>).reduce(
|
||||
(sum, message) => sum + characterCount(message.text),
|
||||
(sum, message) => sum + String(message.text).length,
|
||||
0,
|
||||
);
|
||||
}
|
||||
@@ -330,17 +330,6 @@ function record(value: unknown): Record<string, unknown> | null {
|
||||
: null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Count characters the way Core does. Python's len() counts Unicode code
|
||||
* points; String.prototype.length counts UTF-16 code units, so an emoji outside
|
||||
* the BMP is one character to Core and two here. Comparing the two totals made
|
||||
* every response containing an emoji fail the used_characters cross-check and
|
||||
* surface as response_invalid.
|
||||
*/
|
||||
function characterCount(value: unknown): number {
|
||||
return Array.from(String(value)).length;
|
||||
}
|
||||
|
||||
function boundedText(
|
||||
value: unknown,
|
||||
maximum: number,
|
||||
|
||||
@@ -16,10 +16,7 @@ const READ_ONLY_PATHS = [
|
||||
"/api/v1/memory-tasks",
|
||||
"/api/v1/model-profiles",
|
||||
"/api/v1/sources",
|
||||
"/api/v2/memory-library",
|
||||
"/api/v2/user-profile",
|
||||
] as const;
|
||||
const SDK_MEMORY_DETAIL = /^\/api\/v2\/memories\/[A-Za-z0-9][\w.-]{0,127}$/;
|
||||
|
||||
/** Contract (C0, H25): the one-time loopback code exchange for `memhub ui`.
|
||||
* It is a POST because it mints a short-lived code; the CLI sends the bearer
|
||||
@@ -198,7 +195,6 @@ export function isReadOnlyPath(path: string): boolean {
|
||||
if (IMPORT_JOB.test(path)) return true;
|
||||
if (OPERATION_RECEIPT.test(path)) return true;
|
||||
if (MEMORY_MUTATION_RECEIPT.test(path)) return true;
|
||||
if (SDK_MEMORY_DETAIL.test(path)) return true;
|
||||
if (!MEMORY_DETAIL.test(path)) return false;
|
||||
const segment = path.slice("/api/v1/memories/".length);
|
||||
return !(RESERVED_MEMORY_SEGMENTS as readonly string[]).includes(segment);
|
||||
@@ -215,43 +211,6 @@ export interface MemoryWriteReceipt {
|
||||
created_at: string;
|
||||
updated_at: string;
|
||||
detail?: string;
|
||||
raw_ids?: string[];
|
||||
memory_ids?: string[];
|
||||
}
|
||||
|
||||
export interface OpenAIMessage {
|
||||
role: "user" | "assistant" | "tool";
|
||||
content: string;
|
||||
}
|
||||
|
||||
function isRecord(value: unknown): value is Record<string, unknown> {
|
||||
return value !== null && typeof value === "object" && !Array.isArray(value);
|
||||
}
|
||||
|
||||
export function parseOpenAIMessages(value: unknown): OpenAIMessage[] | null {
|
||||
const candidate = Array.isArray(value)
|
||||
? value
|
||||
: isRecord(value) && Array.isArray(value.messages)
|
||||
? value.messages
|
||||
: null;
|
||||
if (candidate === null || candidate.length < 1 || candidate.length > 200)
|
||||
return null;
|
||||
const messages: OpenAIMessage[] = [];
|
||||
for (const item of candidate) {
|
||||
if (!isRecord(item)) return null;
|
||||
if (!["user", "assistant", "tool"].includes(String(item.role))) return null;
|
||||
if (
|
||||
typeof item.content !== "string" ||
|
||||
item.content.length < 1 ||
|
||||
item.content.length > 12_000
|
||||
)
|
||||
return null;
|
||||
messages.push({
|
||||
role: item.role as OpenAIMessage["role"],
|
||||
content: item.content,
|
||||
});
|
||||
}
|
||||
return messages.some((message) => message.role === "user") ? messages : null;
|
||||
}
|
||||
|
||||
/** A6: submit a write through the durable entry every entry shares.
|
||||
@@ -278,23 +237,19 @@ export async function submitMemoryWrite(
|
||||
attachment: RuntimeAttachmentLike,
|
||||
request: {
|
||||
session: string;
|
||||
source: string;
|
||||
request_id: string;
|
||||
messages: readonly OpenAIMessage[];
|
||||
memory_at?: string;
|
||||
first_round: number;
|
||||
last_round: number;
|
||||
evidence_ids: readonly string[];
|
||||
priority?: "live" | "normal";
|
||||
},
|
||||
): Promise<MemoryWriteReceipt> {
|
||||
const response = await axios.post<MemoryWriteReceipt>(
|
||||
`${baseUrlOf(attachment)}/api/v2/memory-writes`,
|
||||
`${baseUrlOf(attachment)}/api/v1/memory-writes`,
|
||||
{
|
||||
request_id: request.request_id,
|
||||
session_id: request.session,
|
||||
source_id: request.source,
|
||||
messages: [...request.messages],
|
||||
...(request.memory_at === undefined
|
||||
? {}
|
||||
: { memory_at: request.memory_at }),
|
||||
internal_session_id: request.session,
|
||||
first_round: request.first_round,
|
||||
last_round: request.last_round,
|
||||
evidence_ids: [...request.evidence_ids],
|
||||
...(request.priority === undefined ? {} : { priority: request.priority }),
|
||||
},
|
||||
{
|
||||
@@ -425,7 +380,7 @@ export async function readMemoryWrite(
|
||||
): Promise<MemoryWriteReceipt | null> {
|
||||
try {
|
||||
const response = await axios.get<MemoryWriteReceipt>(
|
||||
`${baseUrlOf(attachment)}/api/v2/memory-writes/${encodeURIComponent(requestId)}`,
|
||||
`${baseUrlOf(attachment)}/api/v1/memory-writes/${encodeURIComponent(requestId)}`,
|
||||
{
|
||||
headers: { Authorization: `Bearer ${attachment.apiToken}` },
|
||||
timeout: 10_000,
|
||||
|
||||
@@ -6,7 +6,6 @@ const WRITE_PATHS = [
|
||||
] as const;
|
||||
const MEMORY_WRITE_PATH =
|
||||
/^\/api\/v1\/memories(?:\/[A-Za-z0-9][\w.-]{0,127}\/(?:correct|scopes|actions\/delete))?$/;
|
||||
const SDK_MEMORY_PATH = /^\/api\/v2\/memories\/[A-Za-z0-9][\w.-]{0,127}$/;
|
||||
const MODEL_PROBE_PATH =
|
||||
/^\/api\/v1\/model-profiles\/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}\/probe$/;
|
||||
const MODEL_PROFILE_PATH =
|
||||
@@ -171,7 +170,7 @@ export function createCoreOperations(attachment: {
|
||||
body: Record<string, unknown>,
|
||||
signal?: AbortSignal,
|
||||
) {
|
||||
if (path !== EXTRACTION_SETTINGS_PATH && !SDK_MEMORY_PATH.test(path)) {
|
||||
if (path !== EXTRACTION_SETTINGS_PATH) {
|
||||
throw new Error("core_operation_path_not_allowed");
|
||||
}
|
||||
let response: { status: number; data: unknown };
|
||||
@@ -233,7 +232,7 @@ export function createCoreOperations(attachment: {
|
||||
};
|
||||
},
|
||||
async delete<T>(path: string, signal?: AbortSignal) {
|
||||
if (!IMPORT_FILE.test(path) && !SDK_MEMORY_PATH.test(path)) {
|
||||
if (!IMPORT_FILE.test(path)) {
|
||||
throw new Error("core_operation_path_not_allowed");
|
||||
}
|
||||
let response: { status: number; data: unknown };
|
||||
|
||||
@@ -0,0 +1,718 @@
|
||||
import { mkdtemp, rm, writeFile } from "node:fs/promises";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
import { beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import { run } from "./cli.js";
|
||||
import type { CoreReader } from "./core-client.js";
|
||||
import type { CoreOperations } from "./core-operations.js";
|
||||
import {
|
||||
executeMemoryMutation,
|
||||
inspectMemoryPlan,
|
||||
memoryPlanAcceptsContent,
|
||||
planExistingMemoryMutation,
|
||||
planMemoryAdd,
|
||||
readMemoryMutationReceipt,
|
||||
} from "./memory-flows.js";
|
||||
import { readMemoryDetail } from "./memory-report.js";
|
||||
|
||||
const transport = vi.hoisted(() => ({
|
||||
get: vi.fn(),
|
||||
post: vi.fn(),
|
||||
put: vi.fn(),
|
||||
patch: vi.fn(),
|
||||
delete: vi.fn(),
|
||||
}));
|
||||
|
||||
vi.mock("axios", () => ({
|
||||
default: { create: vi.fn(() => transport), get: vi.fn() },
|
||||
}));
|
||||
|
||||
const runtime = vi.hoisted(() => ({
|
||||
inspectRuntimeStatus: vi.fn(),
|
||||
discoverRuntimeAttachment: vi.fn(),
|
||||
stopRuntimeProcess: vi.fn(),
|
||||
}));
|
||||
|
||||
vi.mock("@one-memory/runtime-control", async (importOriginal) => ({
|
||||
...(await importOriginal<typeof import("@one-memory/runtime-control")>()),
|
||||
...runtime,
|
||||
}));
|
||||
|
||||
const MEMORY = {
|
||||
memory_id: "memory-1",
|
||||
user_id: "local",
|
||||
kind: "working",
|
||||
status: "confirmed",
|
||||
content: "Existing synthetic memory.",
|
||||
scopes: ["all-apps"],
|
||||
evidence_ids: ["evidence-1"],
|
||||
origin: "extracted",
|
||||
tags: ["synthetic"],
|
||||
revision: 3,
|
||||
created_at: "2026-09-19T00:00:00Z",
|
||||
updated_at: "2026-09-19T00:00:01Z",
|
||||
deleted_at: null,
|
||||
};
|
||||
|
||||
const DETAIL = {
|
||||
memory: MEMORY,
|
||||
evidence: [
|
||||
{
|
||||
evidence_id: "evidence-1",
|
||||
available: true,
|
||||
source_id: "codex",
|
||||
written_by: "person",
|
||||
created_at: "2026-09-19T00:00:00Z",
|
||||
verbatim: "Synthetic source text.",
|
||||
},
|
||||
],
|
||||
history: [
|
||||
{
|
||||
revision: 3,
|
||||
action: "create",
|
||||
status: "confirmed",
|
||||
content: "Existing synthetic memory.",
|
||||
scopes: ["all-apps"],
|
||||
created_at: "2026-09-19T00:00:01Z",
|
||||
},
|
||||
],
|
||||
index_sync: null,
|
||||
};
|
||||
|
||||
const ATTACHED = {
|
||||
instanceId: "instance-1",
|
||||
apiToken: "synthetic-runtime-token",
|
||||
host: "127.0.0.1" as const,
|
||||
port: 8765,
|
||||
servicePid: 4242,
|
||||
health: {
|
||||
status: "ok",
|
||||
service: "one-memory-core",
|
||||
version: "1.2.3",
|
||||
instance_id: "instance-1",
|
||||
},
|
||||
};
|
||||
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks();
|
||||
runtime.inspectRuntimeStatus.mockResolvedValue({
|
||||
state: "running",
|
||||
code: "runtime_verified",
|
||||
instanceId: "instance-1",
|
||||
host: "127.0.0.1",
|
||||
port: 8765,
|
||||
version: "1.2.3",
|
||||
});
|
||||
runtime.discoverRuntimeAttachment.mockResolvedValue(ATTACHED);
|
||||
transport.get.mockImplementation(async (url: string) =>
|
||||
url === "/api/v1/memories/memory-1"
|
||||
? { status: 200, data: DETAIL }
|
||||
: { status: 404, data: {} },
|
||||
);
|
||||
});
|
||||
|
||||
describe("memory governance plans", () => {
|
||||
it("plans a manual memory without putting private content in the plan", () => {
|
||||
const result = planMemoryAdd({
|
||||
content: "Private synthetic addition.",
|
||||
memoryKind: "working",
|
||||
scopes: ["all-apps"],
|
||||
evidenceIds: ["evidence-1"],
|
||||
});
|
||||
expect(result.state).toBe("preview");
|
||||
expect(JSON.stringify(result)).not.toContain("Private synthetic addition.");
|
||||
if (result.state !== "preview") throw new Error("preview expected");
|
||||
expect(result.plan).toMatchObject({
|
||||
operation: "add",
|
||||
memory_kind: "working",
|
||||
scopes: ["all-apps"],
|
||||
evidence_ids: ["evidence-1"],
|
||||
content_characters: 27,
|
||||
});
|
||||
expect(inspectMemoryPlan(result.plan)).toMatchObject({
|
||||
operationId: result.plan.request_id,
|
||||
needsContent: true,
|
||||
});
|
||||
expect(
|
||||
memoryPlanAcceptsContent(result.plan, "Private synthetic addition."),
|
||||
).toBe(true);
|
||||
expect(memoryPlanAcceptsContent(result.plan, "Different content.")).toBe(
|
||||
false,
|
||||
);
|
||||
});
|
||||
|
||||
it("binds correction to the current version, source, author and scopes", async () => {
|
||||
const result = await planExistingMemoryMutation(detailReader(), {
|
||||
operation: "correct",
|
||||
memoryId: "memory-1",
|
||||
content: "Corrected synthetic memory.",
|
||||
scopes: ["codex"],
|
||||
});
|
||||
expect(result.state).toBe("preview");
|
||||
if (result.state !== "preview") throw new Error("preview expected");
|
||||
expect(result.plan).toMatchObject({
|
||||
memory_id: "memory-1",
|
||||
expected_revision: 3,
|
||||
expected_status: "confirmed",
|
||||
expected_origin: "extracted",
|
||||
scopes: ["codex"],
|
||||
current: {
|
||||
revision: 3,
|
||||
evidence: [
|
||||
{
|
||||
evidence_id: "evidence-1",
|
||||
source_id: "codex",
|
||||
written_by: "person",
|
||||
},
|
||||
],
|
||||
},
|
||||
});
|
||||
expect(JSON.stringify(result)).not.toContain("Corrected synthetic memory.");
|
||||
});
|
||||
|
||||
it("posts a confirmed plan once and validates the bound receipt", async () => {
|
||||
const preview = planMemoryAdd({
|
||||
content: "Private synthetic addition.",
|
||||
memoryKind: "working",
|
||||
scopes: ["all-apps"],
|
||||
evidenceIds: ["evidence-1"],
|
||||
});
|
||||
if (preview.state !== "preview") throw new Error("preview expected");
|
||||
const post = vi.fn().mockResolvedValue({
|
||||
state: "committed",
|
||||
data: {
|
||||
...MEMORY,
|
||||
memory_id: "created-1",
|
||||
content: "Private synthetic addition.",
|
||||
status: "confirmed",
|
||||
origin: "manual",
|
||||
revision: 1,
|
||||
},
|
||||
});
|
||||
const result = await executeMemoryMutation(
|
||||
preview.plan,
|
||||
"Private synthetic addition.",
|
||||
{ post } as CoreOperations,
|
||||
);
|
||||
expect(result.state).toBe("committed");
|
||||
expect(post).toHaveBeenCalledWith(
|
||||
"/api/v1/memories",
|
||||
expect.objectContaining({
|
||||
request_id: preview.plan.request_id,
|
||||
draft: expect.objectContaining({
|
||||
origin: "manual",
|
||||
content: "Private synthetic addition.",
|
||||
}),
|
||||
}),
|
||||
);
|
||||
expect(JSON.stringify(result)).not.toContain("Private synthetic addition.");
|
||||
});
|
||||
|
||||
it("does not take a created memory that is not in effect as the add", async () => {
|
||||
const preview = planMemoryAdd({
|
||||
content: "Private synthetic addition.",
|
||||
memoryKind: "working",
|
||||
scopes: ["all-apps"],
|
||||
evidenceIds: ["evidence-1"],
|
||||
});
|
||||
if (preview.state !== "preview") throw new Error("preview expected");
|
||||
const result = await executeMemoryMutation(
|
||||
preview.plan,
|
||||
"Private synthetic addition.",
|
||||
{
|
||||
post: vi.fn().mockResolvedValue({
|
||||
state: "committed",
|
||||
data: {
|
||||
...MEMORY,
|
||||
memory_id: "created-1",
|
||||
content: "Private synthetic addition.",
|
||||
status: "deleted",
|
||||
origin: "manual",
|
||||
revision: 1,
|
||||
},
|
||||
}),
|
||||
} as CoreOperations,
|
||||
);
|
||||
expect(result).toMatchObject({
|
||||
state: "unknown",
|
||||
code: "memory_receipt_invalid",
|
||||
});
|
||||
});
|
||||
|
||||
it.each([["delete", "/actions/delete", "deleted"]] as const)(
|
||||
"executes %s against the revision-bound action route",
|
||||
async (operation, suffix, status) => {
|
||||
const preview = await planExistingMemoryMutation(detailReader(), {
|
||||
operation,
|
||||
memoryId: "memory-1",
|
||||
});
|
||||
if (preview.state !== "preview") throw new Error("preview expected");
|
||||
const post = vi.fn().mockResolvedValue({
|
||||
state: "committed",
|
||||
data: { ...MEMORY, status, revision: 4 },
|
||||
});
|
||||
const result = await executeMemoryMutation(preview.plan, undefined, {
|
||||
post,
|
||||
} as CoreOperations);
|
||||
expect(result.state).toBe("committed");
|
||||
expect(post).toHaveBeenCalledWith(`/api/v1/memories/memory-1${suffix}`, {
|
||||
request_id: preview.plan.request_id,
|
||||
expected_revision: 3,
|
||||
});
|
||||
},
|
||||
);
|
||||
|
||||
it("executes correction and scope changes without changing origin", async () => {
|
||||
for (const operation of ["correct", "scope"] as const) {
|
||||
const content =
|
||||
operation === "correct" ? "Corrected synthetic memory." : undefined;
|
||||
const preview = await planExistingMemoryMutation(detailReader(), {
|
||||
operation,
|
||||
memoryId: "memory-1",
|
||||
...(content ? { content } : {}),
|
||||
scopes: ["codex"],
|
||||
});
|
||||
if (preview.state !== "preview") throw new Error("preview expected");
|
||||
const post = vi.fn().mockResolvedValue({
|
||||
state: "committed",
|
||||
data: {
|
||||
...MEMORY,
|
||||
content: content ?? MEMORY.content,
|
||||
scopes: ["codex"],
|
||||
revision: 4,
|
||||
},
|
||||
});
|
||||
const result = await executeMemoryMutation(preview.plan, content, {
|
||||
post,
|
||||
} as CoreOperations);
|
||||
expect(result.state).toBe("committed");
|
||||
expect(post.mock.calls[0][0]).toBe(
|
||||
`/api/v1/memories/memory-1/${operation === "scope" ? "scopes" : "correct"}`,
|
||||
);
|
||||
expect(post.mock.calls[0][1]).toMatchObject({
|
||||
request_id: preview.plan.request_id,
|
||||
expected_revision: 3,
|
||||
scopes: ["codex"],
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
it("treats a misbound HTTP 200 receipt as unknown", async () => {
|
||||
const preview = await planExistingMemoryMutation(detailReader(), {
|
||||
operation: "delete",
|
||||
memoryId: "memory-1",
|
||||
});
|
||||
if (preview.state !== "preview") throw new Error("preview expected");
|
||||
const result = await executeMemoryMutation(preview.plan, undefined, {
|
||||
post: vi.fn().mockResolvedValue({
|
||||
state: "committed",
|
||||
data: {
|
||||
...MEMORY,
|
||||
memory_id: "another-memory",
|
||||
status: "deleted",
|
||||
revision: 4,
|
||||
},
|
||||
}),
|
||||
});
|
||||
expect(result).toMatchObject({
|
||||
state: "unknown",
|
||||
code: "memory_receipt_invalid",
|
||||
operation_id: preview.plan.request_id,
|
||||
});
|
||||
});
|
||||
|
||||
it("recovers only an exactly bound governance transaction receipt", async () => {
|
||||
const preview = await planExistingMemoryMutation(detailReader(), {
|
||||
operation: "delete",
|
||||
memoryId: "memory-1",
|
||||
});
|
||||
if (preview.state !== "preview") throw new Error("preview expected");
|
||||
const result = await readMemoryMutationReceipt(
|
||||
receiptReader({
|
||||
schema_version: 2,
|
||||
request_id: preview.plan.request_id,
|
||||
state: "committed",
|
||||
operation: "delete",
|
||||
memory_id: "memory-1",
|
||||
result_revision: 4,
|
||||
result_status: "deleted",
|
||||
memory_state: "deleted",
|
||||
migrated_from: null,
|
||||
}),
|
||||
preview.plan,
|
||||
);
|
||||
expect(result).toMatchObject({
|
||||
state: "recovered",
|
||||
operation_id: preview.plan.request_id,
|
||||
receipt: {
|
||||
operation: "delete",
|
||||
memory_id: "memory-1",
|
||||
result_revision: 4,
|
||||
result_status: "deleted",
|
||||
},
|
||||
});
|
||||
expect(JSON.stringify(result)).not.toContain("content_sha256");
|
||||
});
|
||||
|
||||
it.each([
|
||||
["operation", "correct"],
|
||||
["memory_id", "memory-2"],
|
||||
["result_revision", 5],
|
||||
["result_status", "confirmed"],
|
||||
["migrated_from", "reject"],
|
||||
] as const)("refuses a receipt with wrong %s", async (field, wrong) => {
|
||||
const preview = await planExistingMemoryMutation(detailReader(), {
|
||||
operation: "delete",
|
||||
memoryId: "memory-1",
|
||||
});
|
||||
if (preview.state !== "preview") throw new Error("preview expected");
|
||||
const result = await readMemoryMutationReceipt(
|
||||
receiptReader({
|
||||
schema_version: 2,
|
||||
request_id: preview.plan.request_id,
|
||||
state: "committed",
|
||||
operation: "delete",
|
||||
memory_id: "memory-1",
|
||||
result_revision: 4,
|
||||
result_status: "deleted",
|
||||
memory_state: "deleted",
|
||||
migrated_from: null,
|
||||
[field]: wrong,
|
||||
}),
|
||||
preview.plan,
|
||||
);
|
||||
expect(result).toMatchObject({
|
||||
state: "unknown",
|
||||
code: "memory_mutation_receipt_invalid",
|
||||
});
|
||||
});
|
||||
|
||||
it("keeps not-found, denied and unavailable lookups unresolved", async () => {
|
||||
const preview = await planExistingMemoryMutation(detailReader(), {
|
||||
operation: "delete",
|
||||
memoryId: "memory-1",
|
||||
});
|
||||
if (preview.state !== "preview") throw new Error("preview expected");
|
||||
const notFound = await readMemoryMutationReceipt(
|
||||
receiptReader({
|
||||
schema_version: 2,
|
||||
request_id: preview.plan.request_id,
|
||||
state: "not_found",
|
||||
operation: null,
|
||||
memory_id: null,
|
||||
result_revision: null,
|
||||
result_status: null,
|
||||
memory_state: null,
|
||||
migrated_from: null,
|
||||
}),
|
||||
preview.plan,
|
||||
);
|
||||
expect(notFound).toMatchObject({
|
||||
state: "unknown",
|
||||
code: "memory_mutation_receipt_not_found_after_attempt",
|
||||
});
|
||||
for (const state of ["denied", "unavailable"] as const) {
|
||||
const result = await readMemoryMutationReceipt(
|
||||
async () => ({ state, code: `synthetic_${state}` }),
|
||||
preview.plan,
|
||||
);
|
||||
expect(result).toMatchObject({
|
||||
state: "unknown",
|
||||
code: `memory_mutation_receipt_synthetic_${state}`,
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
it("maps an add plan only to the Core create receipt", async () => {
|
||||
const preview = planMemoryAdd({
|
||||
content: "Private synthetic addition.",
|
||||
memoryKind: "working",
|
||||
scopes: ["all-apps"],
|
||||
evidenceIds: ["evidence-1"],
|
||||
});
|
||||
if (preview.state !== "preview") throw new Error("preview expected");
|
||||
const result = await readMemoryMutationReceipt(
|
||||
receiptReader({
|
||||
schema_version: 2,
|
||||
request_id: preview.plan.request_id,
|
||||
state: "committed",
|
||||
operation: "create",
|
||||
memory_id: "created-1",
|
||||
result_revision: 1,
|
||||
result_status: "confirmed",
|
||||
memory_state: "present",
|
||||
migrated_from: null,
|
||||
}),
|
||||
preview.plan,
|
||||
);
|
||||
expect(result).toMatchObject({
|
||||
state: "recovered",
|
||||
receipt: {
|
||||
operation: "create",
|
||||
memory_id: "created-1",
|
||||
result_revision: 1,
|
||||
result_status: "confirmed",
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
it("shows bounded history fields including scopes and status", async () => {
|
||||
const result = await readMemoryDetail(detailReader(), "memory-1");
|
||||
expect(result.history[0]).toMatchObject({
|
||||
revision: 3,
|
||||
action: "create",
|
||||
status: "confirmed",
|
||||
scopes: ["all-apps"],
|
||||
content: "Existing synthetic memory.",
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe("memory pending fence", () => {
|
||||
it("rejects missing confirmation and invalid plans before any write", async () => {
|
||||
for (const argv of [
|
||||
["memories", "apply", "--plan-file", "/synthetic/plan", "--json"],
|
||||
["memories", "plan", "delete", "graph", "--json"],
|
||||
["memories", "plan", "delete", "memory-1", "--scope", "codex", "--json"],
|
||||
["memories", "plan", "confirm", "memory-1", "--json"],
|
||||
["memories", "plan", "reject", "memory-1", "--json"],
|
||||
[
|
||||
"memories",
|
||||
"plan",
|
||||
"add",
|
||||
"--content-file",
|
||||
"/synthetic/content",
|
||||
"--scope",
|
||||
"all-apps",
|
||||
"--json",
|
||||
],
|
||||
]) {
|
||||
const streams = captureStreams();
|
||||
try {
|
||||
expect(await run(argv)).toBe(2);
|
||||
} finally {
|
||||
streams.restore();
|
||||
}
|
||||
}
|
||||
expect(transport.post).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("generates a bounded correction preview without writing", async () => {
|
||||
const directory = await mkdtemp(
|
||||
path.join(os.tmpdir(), "om-memory-preview-"),
|
||||
);
|
||||
const contentFile = path.join(directory, "content.txt");
|
||||
await writeFile(contentFile, "Corrected synthetic memory.");
|
||||
const streams = captureStreams();
|
||||
try {
|
||||
expect(
|
||||
await run([
|
||||
"memories",
|
||||
"plan",
|
||||
"correct",
|
||||
"memory-1",
|
||||
"--content-file",
|
||||
contentFile,
|
||||
"--scope",
|
||||
"codex",
|
||||
"--json",
|
||||
]),
|
||||
).toBe(0);
|
||||
} finally {
|
||||
streams.restore();
|
||||
await rm(directory, { recursive: true, force: true });
|
||||
}
|
||||
const output = JSON.parse(streams.stdout());
|
||||
expect(output).toMatchObject({
|
||||
state: "preview",
|
||||
plan: {
|
||||
operation: "correct",
|
||||
memory_id: "memory-1",
|
||||
expected_revision: 3,
|
||||
scopes: ["codex"],
|
||||
},
|
||||
});
|
||||
expect(streams.stdout()).not.toContain("Corrected synthetic memory.");
|
||||
expect(transport.post).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("checks content identity before claiming the durable fence", async () => {
|
||||
const preview = planMemoryAdd({
|
||||
content: "Private synthetic addition.",
|
||||
memoryKind: "working",
|
||||
scopes: ["all-apps"],
|
||||
evidenceIds: ["evidence-1"],
|
||||
});
|
||||
if (preview.state !== "preview") throw new Error("preview expected");
|
||||
const directory = await mkdtemp(
|
||||
path.join(os.tmpdir(), "om-memory-content-"),
|
||||
);
|
||||
const planFile = path.join(directory, "plan.json");
|
||||
const wrongFile = path.join(directory, "wrong.txt");
|
||||
const correctFile = path.join(directory, "correct.txt");
|
||||
await writeFile(planFile, JSON.stringify(preview));
|
||||
await writeFile(wrongFile, "Different content.");
|
||||
await writeFile(correctFile, "Private synthetic addition.");
|
||||
const wrong = captureStreams();
|
||||
try {
|
||||
expect(
|
||||
await run([
|
||||
"memories",
|
||||
"apply",
|
||||
"--plan-file",
|
||||
planFile,
|
||||
"--content-file",
|
||||
wrongFile,
|
||||
"--confirm",
|
||||
"--json",
|
||||
]),
|
||||
).toBe(2);
|
||||
} finally {
|
||||
wrong.restore();
|
||||
}
|
||||
transport.post.mockResolvedValue({
|
||||
status: 200,
|
||||
data: {
|
||||
...MEMORY,
|
||||
memory_id: "created-1",
|
||||
content: "Private synthetic addition.",
|
||||
status: "confirmed",
|
||||
origin: "manual",
|
||||
revision: 1,
|
||||
},
|
||||
});
|
||||
const correct = captureStreams();
|
||||
try {
|
||||
expect(
|
||||
await run([
|
||||
"memories",
|
||||
"apply",
|
||||
"--plan-file",
|
||||
planFile,
|
||||
"--content-file",
|
||||
correctFile,
|
||||
"--confirm",
|
||||
"--json",
|
||||
]),
|
||||
).toBe(0);
|
||||
} finally {
|
||||
correct.restore();
|
||||
await rm(directory, { recursive: true, force: true });
|
||||
}
|
||||
expect(transport.post).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it("recovers an unknown mutation after restart without reposting", async () => {
|
||||
const preview = await planExistingMemoryMutation(detailReader(), {
|
||||
operation: "delete",
|
||||
memoryId: "memory-1",
|
||||
});
|
||||
if (preview.state !== "preview") throw new Error("preview expected");
|
||||
const directory = await mkdtemp(path.join(os.tmpdir(), "om-memory-plan-"));
|
||||
const planFile = path.join(directory, "plan.json");
|
||||
await writeFile(planFile, JSON.stringify(preview));
|
||||
transport.post.mockRejectedValue(new Error("synthetic disconnect"));
|
||||
let receiptLookups = 0;
|
||||
transport.get.mockImplementation(async (url: string) => {
|
||||
if (url === "/api/v1/memories/memory-1") {
|
||||
return { status: 200, data: DETAIL };
|
||||
}
|
||||
if (url === `/api/v1/memory-mutations/${preview.plan.request_id}`) {
|
||||
receiptLookups += 1;
|
||||
if (receiptLookups === 1) throw new Error("synthetic lookup outage");
|
||||
return {
|
||||
status: 200,
|
||||
data: {
|
||||
schema_version: 2,
|
||||
request_id: preview.plan.request_id,
|
||||
state: "committed",
|
||||
operation: "delete",
|
||||
memory_id: "memory-1",
|
||||
result_revision: 4,
|
||||
result_status: "deleted",
|
||||
memory_state: "deleted",
|
||||
migrated_from: null,
|
||||
},
|
||||
};
|
||||
}
|
||||
return { status: 404, data: {} };
|
||||
});
|
||||
const first = captureStreams();
|
||||
try {
|
||||
expect(
|
||||
await run([
|
||||
"memories",
|
||||
"apply",
|
||||
"--plan-file",
|
||||
planFile,
|
||||
"--confirm",
|
||||
"--json",
|
||||
]),
|
||||
).toBe(1);
|
||||
} finally {
|
||||
first.restore();
|
||||
}
|
||||
expect(JSON.parse(first.stdout())).toMatchObject({
|
||||
state: "unknown",
|
||||
code: "memory_mutation_receipt_transport_failed",
|
||||
});
|
||||
const second = captureStreams();
|
||||
try {
|
||||
expect(
|
||||
await run([
|
||||
"memories",
|
||||
"apply",
|
||||
"--plan-file",
|
||||
planFile,
|
||||
"--confirm",
|
||||
"--json",
|
||||
]),
|
||||
).toBe(0);
|
||||
} finally {
|
||||
second.restore();
|
||||
await rm(directory, { recursive: true, force: true });
|
||||
}
|
||||
expect(JSON.parse(second.stdout())).toMatchObject({
|
||||
state: "recovered",
|
||||
receipt: {
|
||||
operation: "delete",
|
||||
memory_id: "memory-1",
|
||||
result_revision: 4,
|
||||
result_status: "deleted",
|
||||
},
|
||||
});
|
||||
expect(transport.post).toHaveBeenCalledTimes(1);
|
||||
expect(receiptLookups).toBe(2);
|
||||
});
|
||||
});
|
||||
|
||||
function captureStreams() {
|
||||
let stdout = "";
|
||||
let stderr = "";
|
||||
const out = vi.spyOn(process.stdout, "write").mockImplementation((chunk) => {
|
||||
stdout += String(chunk);
|
||||
return true;
|
||||
});
|
||||
const error = vi
|
||||
.spyOn(process.stderr, "write")
|
||||
.mockImplementation((chunk) => {
|
||||
stderr += String(chunk);
|
||||
return true;
|
||||
});
|
||||
return {
|
||||
stdout: () => stdout,
|
||||
stderr: () => stderr,
|
||||
restore: () => {
|
||||
out.mockRestore();
|
||||
error.mockRestore();
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
function detailReader(): CoreReader {
|
||||
return async <T>() => ({ state: "ok", data: DETAIL as T });
|
||||
}
|
||||
|
||||
function receiptReader(receipt: unknown): CoreReader {
|
||||
return async <T>() => ({ state: "ok", data: receipt as T });
|
||||
}
|
||||
@@ -4,11 +4,15 @@ import {
|
||||
buildMemoryActionRequest,
|
||||
buildMemoryCorrectRequest,
|
||||
buildMemoryCreateRequest,
|
||||
buildMemoryScopesRequest,
|
||||
parseMemoryReceipt,
|
||||
} from "@one-memory/operations";
|
||||
import type { CoreReader } from "./core-client.js";
|
||||
import type { CoreOperations } from "./core-operations.js";
|
||||
import { validateMemoryDetail } from "./core-responses.js";
|
||||
import {
|
||||
validateGovernedMemory,
|
||||
validateMemoryDetail,
|
||||
} from "./core-responses.js";
|
||||
import { isMemoryId } from "./memory-report.js";
|
||||
import { PRODUCT } from "./version.js";
|
||||
|
||||
@@ -19,7 +23,7 @@ const MEMORY_KINDS = [
|
||||
"schema",
|
||||
"intention",
|
||||
] as const;
|
||||
const OPERATIONS = ["add", "correct", "delete"] as const;
|
||||
const OPERATIONS = ["add", "correct", "delete", "scope"] as const;
|
||||
const MEMORY_ORIGINS = ["extracted", "answered", "agent", "manual"] as const;
|
||||
const MUTATION_RECEIPT_KEYS = new Set([
|
||||
"schema_version",
|
||||
@@ -141,12 +145,15 @@ export async function planExistingMemoryMutation(
|
||||
operation: Exclude<(typeof OPERATIONS)[number], "add">;
|
||||
memoryId: string;
|
||||
content?: string;
|
||||
scopes?: string[];
|
||||
},
|
||||
): Promise<MemoryFlowDocument> {
|
||||
if (
|
||||
!isMemoryId(input.memoryId) ||
|
||||
(input.operation === "correct" && input.content === undefined) ||
|
||||
(input.content !== undefined && !validContent(input.content))
|
||||
(input.content !== undefined && !validContent(input.content)) ||
|
||||
((input.operation === "scope" || input.scopes !== undefined) &&
|
||||
!validScopes(input.scopes ?? []))
|
||||
) {
|
||||
return invalid("memory_plan_input_invalid");
|
||||
}
|
||||
@@ -169,6 +176,7 @@ export async function planExistingMemoryMutation(
|
||||
expected_revision: memory.revision,
|
||||
expected_status: memory.status,
|
||||
expected_origin: memory.origin,
|
||||
...(input.scopes ? { scopes: input.scopes } : {}),
|
||||
...(input.content ? contentIdentity(input.content) : {}),
|
||||
current: {
|
||||
status: memory.status,
|
||||
@@ -399,7 +407,15 @@ function operationRequest(
|
||||
plan.memory_id as string,
|
||||
plan.expected_revision as number,
|
||||
content as string,
|
||||
undefined,
|
||||
plan.scopes,
|
||||
plan.request_id,
|
||||
);
|
||||
}
|
||||
if (plan.operation === "scope") {
|
||||
return buildMemoryScopesRequest(
|
||||
plan.memory_id as string,
|
||||
plan.expected_revision as number,
|
||||
plan.scopes ?? [],
|
||||
plan.request_id,
|
||||
);
|
||||
}
|
||||
@@ -435,7 +451,10 @@ function matchesReceipt(
|
||||
return false;
|
||||
}
|
||||
if (plan.operation === "delete" && memory.status !== "deleted") return false;
|
||||
if (plan.operation === "correct" && memory.status !== plan.expected_status) {
|
||||
if (
|
||||
(plan.operation === "correct" || plan.operation === "scope") &&
|
||||
memory.status !== plan.expected_status
|
||||
) {
|
||||
return false;
|
||||
}
|
||||
if (plan.operation === "correct" && memory.content !== content) return false;
|
||||
@@ -471,7 +490,11 @@ function validateMutationReceipt(
|
||||
: null;
|
||||
}
|
||||
const expectedOperation =
|
||||
plan.operation === "add" ? "create" : plan.operation;
|
||||
plan.operation === "add"
|
||||
? "create"
|
||||
: plan.operation === "scope"
|
||||
? "set_scopes"
|
||||
: plan.operation;
|
||||
const expectedRevision =
|
||||
plan.operation === "add" ? 1 : (plan.expected_revision as number) + 1;
|
||||
const expectedStatus = plan.operation === "delete" ? "deleted" : "confirmed";
|
||||
@@ -538,7 +561,9 @@ function validatePlan(value: unknown): MemoryMutationPlan | null {
|
||||
plan.expected_status !== "confirmed" ||
|
||||
!oneOf(plan.expected_origin, MEMORY_ORIGINS) ||
|
||||
!validCurrent(plan) ||
|
||||
(plan.operation === "correct" && !contentFields(plan))
|
||||
(plan.operation === "correct" && !contentFields(plan)) ||
|
||||
((plan.operation === "scope" || plan.scopes !== undefined) &&
|
||||
!validScopes(plan.scopes))
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
|
||||
@@ -17,7 +17,7 @@ const FIXTURE = JSON.parse(
|
||||
),
|
||||
"utf8",
|
||||
),
|
||||
) as { receipts: Record<"add" | "correct" | "delete", Receipt> };
|
||||
) as { receipts: Record<"add" | "correct" | "scope" | "delete", Receipt> };
|
||||
|
||||
interface Receipt {
|
||||
schema_version: number;
|
||||
@@ -103,8 +103,11 @@ describe("memory mutation receipt v2 from Core", () => {
|
||||
});
|
||||
});
|
||||
|
||||
it.each<["correct" | "delete", { content?: string }]>([
|
||||
it.each<
|
||||
["correct" | "scope" | "delete", { content?: string; scopes?: string[] }]
|
||||
>([
|
||||
["correct", { content: "Corrected synthetic memory." }],
|
||||
["scope", { scopes: ["codex"] }],
|
||||
["delete", {}],
|
||||
])(
|
||||
"recovers %s from the receipt Core returns for it",
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
import type { SdkMemoryPage, SdkMemoryRecord } from "@one-memory/contracts";
|
||||
import type { GovernedMemory } from "@one-memory/contracts";
|
||||
import { type CoreReader, RESERVED_MEMORY_SEGMENTS } from "./core-client.js";
|
||||
import { validateMemoryDetail, validateMemoryPage } from "./core-responses.js";
|
||||
import { PRODUCT } from "./version.js";
|
||||
|
||||
/** Output budget: a long memory is shortened visibly, never dropped silently. */
|
||||
@@ -46,8 +47,10 @@ export async function readMemoryPage(
|
||||
offset: query.offset,
|
||||
limit: query.limit,
|
||||
};
|
||||
if (query.status !== undefined) parameters.statuses = query.status;
|
||||
const outcome = await reader<unknown>("/api/v2/memory-library", parameters);
|
||||
if (query.query !== undefined) parameters.query = query.query;
|
||||
if (query.status !== undefined) parameters.status = query.status;
|
||||
if (query.source !== undefined) parameters.source = query.source;
|
||||
const outcome = await reader<unknown>("/api/v1/memory-library", parameters);
|
||||
if (outcome.state !== "ok") {
|
||||
return {
|
||||
schema_version: 1,
|
||||
@@ -56,7 +59,7 @@ export async function readMemoryPage(
|
||||
memories: [],
|
||||
};
|
||||
}
|
||||
const page = sdkMemoryPage(outcome.data);
|
||||
const page = validateMemoryPage(outcome.data, query);
|
||||
if (page === null) {
|
||||
return {
|
||||
schema_version: 1,
|
||||
@@ -65,7 +68,7 @@ export async function readMemoryPage(
|
||||
memories: [],
|
||||
};
|
||||
}
|
||||
const items = page.memories;
|
||||
const items = page.items;
|
||||
return {
|
||||
schema_version: 1,
|
||||
product: PRODUCT,
|
||||
@@ -85,7 +88,7 @@ export async function readMemoryDetail(
|
||||
reader: CoreReader,
|
||||
memoryId: string,
|
||||
): Promise<MemoryDetailDocument> {
|
||||
const outcome = await reader<unknown>(`/api/v2/memories/${memoryId}`);
|
||||
const outcome = await reader<unknown>(`/api/v1/memories/${memoryId}`);
|
||||
if (outcome.state !== "ok") {
|
||||
return {
|
||||
schema_version: 1,
|
||||
@@ -95,8 +98,8 @@ export async function readMemoryDetail(
|
||||
history: [],
|
||||
};
|
||||
}
|
||||
const memory = sdkMemory(outcome.data);
|
||||
if (memory === null || memory.memory_id !== memoryId) {
|
||||
const detail = validateMemoryDetail(outcome.data, memoryId);
|
||||
if (detail === null) {
|
||||
return {
|
||||
schema_version: 1,
|
||||
product: PRODUCT,
|
||||
@@ -108,9 +111,33 @@ export async function readMemoryDetail(
|
||||
return {
|
||||
schema_version: 1,
|
||||
product: PRODUCT,
|
||||
memory: { state: "ok", ...describeMemory(memory) },
|
||||
evidence: [],
|
||||
history: [],
|
||||
memory: { state: "ok", ...describeMemory(detail.memory) },
|
||||
evidence: (detail.evidence ?? []).map((item) => {
|
||||
const budgeted = budget(item.verbatim ?? "");
|
||||
return {
|
||||
evidence_id: item.evidence_id,
|
||||
available: item.available,
|
||||
source_id: item.source_id,
|
||||
written_by: item.written_by,
|
||||
created_at: item.created_at,
|
||||
excerpt: budgeted.text,
|
||||
excerpt_truncated: budgeted.truncated,
|
||||
excerpt_characters: budgeted.characters,
|
||||
};
|
||||
}),
|
||||
history: (detail.history ?? []).map((item) => {
|
||||
const content = budget(item.content);
|
||||
return {
|
||||
revision: item.revision,
|
||||
action: item.action,
|
||||
status: item.status,
|
||||
scopes: item.scopes,
|
||||
created_at: item.created_at,
|
||||
content: content.text,
|
||||
content_truncated: content.truncated,
|
||||
content_characters: content.characters,
|
||||
};
|
||||
}),
|
||||
};
|
||||
}
|
||||
|
||||
@@ -129,7 +156,9 @@ export function renderMemoryPageText(document: MemoryPageDocument): string {
|
||||
];
|
||||
for (const memory of document.memories) {
|
||||
lines.push(
|
||||
`- ${memory.memory_id} ${memory.status}/${memory.layer} source=${memory.source ?? "未知"}`,
|
||||
`- ${memory.memory_id} ${memory.status}/${memory.kind} rev ${memory.revision} scopes=${
|
||||
(memory.scopes as string[]).join(",") || "无"
|
||||
}`,
|
||||
` ${memory.content}${memory.content_truncated ? ` …(共 ${memory.content_characters} 字)` : ""}`,
|
||||
);
|
||||
}
|
||||
@@ -148,91 +177,54 @@ export function renderMemoryDetailText(document: MemoryDetailDocument): string {
|
||||
}
|
||||
const lines = [
|
||||
`记忆 ${memory.memory_id}`,
|
||||
`状态:${memory.status} 层:${memory.layer} 来源:${memory.source ?? "未知"}`,
|
||||
`owner:${memory.owner ?? "未知"} latest:${memory.is_latest}`,
|
||||
`raw IDs:${(memory.source_raw_ids as string[]).join(",") || "无"}`,
|
||||
`状态:${memory.status} 类型:${memory.kind} 来源方式:${memory.origin} 版本:${memory.revision}`,
|
||||
`授权范围:${(memory.scopes as string[]).join(",") || "无"}`,
|
||||
`证据:${(memory.evidence_ids as string[]).join(",") || "无"}`,
|
||||
`更新时间:${memory.updated_at}`,
|
||||
"",
|
||||
String(memory.content),
|
||||
memory.content_truncated ? `…(共 ${memory.content_characters} 字)` : "",
|
||||
"",
|
||||
"SDK memory 数据不在 MemHub 中保留 Evidence/版本副本。",
|
||||
"原文摘录:",
|
||||
];
|
||||
for (const item of document.evidence) {
|
||||
lines.push(
|
||||
`- ${item.evidence_id} 来源=${item.source_id ?? "未知"} 作者=${item.written_by ?? "未知"} 可读=${item.available}`,
|
||||
` ${item.excerpt}${item.excerpt_truncated ? ` …(共 ${item.excerpt_characters} 字)` : ""}`,
|
||||
);
|
||||
}
|
||||
if (document.evidence.length === 0) lines.push("(无可读原文)");
|
||||
lines.push("", "版本历史:");
|
||||
for (const item of document.history) {
|
||||
lines.push(
|
||||
`- rev ${item.revision} ${item.action} ${item.status} scopes=${(item.scopes as string[]).join(",")}`,
|
||||
` ${item.content}${item.content_truncated ? ` …(共 ${item.content_characters} 字)` : ""}`,
|
||||
);
|
||||
}
|
||||
if (document.history.length === 0) lines.push("(无)");
|
||||
lines.push("");
|
||||
return lines.join("\n");
|
||||
}
|
||||
|
||||
function describeMemory(memory: SdkMemoryRecord): Record<string, unknown> {
|
||||
function describeMemory(memory: GovernedMemory): Record<string, unknown> {
|
||||
const budgeted = budget(memory.content ?? "");
|
||||
return {
|
||||
memory_id: memory.memory_id,
|
||||
status: memory.status,
|
||||
layer: memory.layer,
|
||||
is_latest: memory.is_latest,
|
||||
owner: memory.owner,
|
||||
source: memory.source,
|
||||
source_raw_ids: memory.source_raw_ids,
|
||||
supersedes: memory.supersedes,
|
||||
superseded_by: memory.superseded_by,
|
||||
kind: memory.kind,
|
||||
origin: memory.origin,
|
||||
revision: memory.revision,
|
||||
scopes: memory.scopes ?? [],
|
||||
evidence_ids: memory.evidence_ids ?? [],
|
||||
tags: memory.tags ?? [],
|
||||
memory_at: memory.memory_at,
|
||||
gmt_created: memory.gmt_created,
|
||||
created_at: memory.created_at,
|
||||
updated_at: memory.updated_at,
|
||||
content: budgeted.text,
|
||||
content_truncated: budgeted.truncated,
|
||||
content_characters: budgeted.characters,
|
||||
};
|
||||
}
|
||||
|
||||
function sdkMemoryPage(value: unknown): SdkMemoryPage | null {
|
||||
if (!record(value) || !Array.isArray(value.memories)) return null;
|
||||
const memories = value.memories.map(sdkMemory);
|
||||
if (memories.some((item) => item === null)) return null;
|
||||
if (!integer(value.total) || !integer(value.limit) || !integer(value.offset))
|
||||
return null;
|
||||
return {
|
||||
memories: memories as SdkMemoryRecord[],
|
||||
total: value.total,
|
||||
limit: value.limit,
|
||||
offset: value.offset,
|
||||
};
|
||||
}
|
||||
|
||||
function sdkMemory(value: unknown): SdkMemoryRecord | null {
|
||||
if (
|
||||
!record(value) ||
|
||||
!text(value.memory_id) ||
|
||||
typeof value.content !== "string" ||
|
||||
!text(value.layer) ||
|
||||
!text(value.status) ||
|
||||
typeof value.is_latest !== "boolean" ||
|
||||
!strings(value.supersedes) ||
|
||||
!strings(value.superseded_by) ||
|
||||
!strings(value.tags) ||
|
||||
!strings(value.source_raw_ids) ||
|
||||
!text(value.user_id) ||
|
||||
!text(value.agent_id)
|
||||
)
|
||||
return null;
|
||||
return value as unknown as SdkMemoryRecord;
|
||||
}
|
||||
|
||||
function record(value: unknown): value is Record<string, unknown> {
|
||||
return value !== null && typeof value === "object" && !Array.isArray(value);
|
||||
}
|
||||
|
||||
function text(value: unknown): value is string {
|
||||
return typeof value === "string" && value.length > 0;
|
||||
}
|
||||
|
||||
function strings(value: unknown): value is string[] {
|
||||
return (
|
||||
Array.isArray(value) && value.every((item) => typeof item === "string")
|
||||
);
|
||||
}
|
||||
|
||||
function integer(value: unknown): value is number {
|
||||
return Number.isInteger(value) && Number(value) >= 0;
|
||||
}
|
||||
|
||||
function budget(value: string): {
|
||||
text: string;
|
||||
truncated: boolean;
|
||||
|
||||
@@ -122,9 +122,7 @@ describe("package matrix harness", () => {
|
||||
process.execPath,
|
||||
[script, runtimeDist, "--stopped-commands", "--json"],
|
||||
{ env: isolatedEnv },
|
||||
).catch((error: { stdout?: string; stderr?: string }) => {
|
||||
throw new Error(`${error.stdout ?? ""}\n${error.stderr ?? ""}`);
|
||||
});
|
||||
);
|
||||
const document = JSON.parse(positive.stdout) as {
|
||||
state: string;
|
||||
commands: {
|
||||
@@ -139,8 +137,8 @@ describe("package matrix harness", () => {
|
||||
};
|
||||
expect(document.state).toBe("verified");
|
||||
const results = document.commands.stopped_destructive.results;
|
||||
// Every destructive command fails closed while Runtime is stopped.
|
||||
expect(Object.keys(results)).toHaveLength(7);
|
||||
// 11 destructive commands, each unknown/runtime_not_running with exit 3.
|
||||
expect(Object.keys(results)).toHaveLength(11);
|
||||
for (const [name, result] of Object.entries(results)) {
|
||||
expect(result, `${name} did not fail closed`).toEqual({
|
||||
exit: 3,
|
||||
@@ -164,7 +162,7 @@ describe("package matrix harness", () => {
|
||||
const shim = [
|
||||
"#!/usr/bin/env node",
|
||||
"const a=process.argv.slice(2);",
|
||||
"if(['calls','notes','graph','models','memories'].includes(a[0])){",
|
||||
"if(['calls','import-file','notes','extraction','graph','models'].includes(a[0])||(a[0]==='processing'&&a[1]==='extract')){",
|
||||
' process.stdout.write(\'{"schema_version":1,"product":"memhub","state":"committed","operation_id":"x","receipt":{}}\\n\');',
|
||||
" process.exit(0);",
|
||||
"}",
|
||||
@@ -190,7 +188,7 @@ describe("package matrix harness", () => {
|
||||
expect(failure?.state).toBe("failed");
|
||||
expect(
|
||||
failure?.failures.filter((item) => item.includes("did not fail closed")),
|
||||
).toHaveLength(7);
|
||||
).toHaveLength(11);
|
||||
});
|
||||
|
||||
it("can require an expected bundle SHA and reports bundle metadata", {
|
||||
@@ -290,9 +288,10 @@ describe("package matrix harness", () => {
|
||||
"sources",
|
||||
"sync",
|
||||
"memories",
|
||||
"profile",
|
||||
"calls",
|
||||
"import-file",
|
||||
"notes",
|
||||
"extraction",
|
||||
"graph",
|
||||
"models",
|
||||
"host",
|
||||
@@ -315,6 +314,7 @@ describe("package matrix harness", () => {
|
||||
"scan",
|
||||
"progress",
|
||||
"jobs",
|
||||
"processing",
|
||||
"path",
|
||||
"adapter",
|
||||
"write",
|
||||
@@ -344,7 +344,7 @@ describe("package matrix harness", () => {
|
||||
'command "models" is missing usage "memhub models activate"',
|
||||
);
|
||||
expect(failures).toContain(
|
||||
'command "memories" is missing usage "memhub memories update"',
|
||||
'command "import-file" is missing usage "memhub import-file discard"',
|
||||
);
|
||||
expect(failures).toContain(
|
||||
'command "jobs" is missing usage "memhub jobs start"',
|
||||
|
||||
@@ -146,16 +146,17 @@ describe("memhub short-command contract", () => {
|
||||
).join("\n");
|
||||
for (const shortForm of [
|
||||
"memhub calls clear",
|
||||
"memhub import-file upload",
|
||||
"memhub import-file commit",
|
||||
"memhub import-file discard",
|
||||
"memhub notes add",
|
||||
"memhub notes process",
|
||||
"memhub models setup",
|
||||
"memhub models activate",
|
||||
"memhub models edit",
|
||||
"memhub extraction set",
|
||||
"memhub graph width",
|
||||
"memhub write submit",
|
||||
"memhub write status",
|
||||
"memhub memories update",
|
||||
"memhub memories delete",
|
||||
"memhub processing extract",
|
||||
]) {
|
||||
expect(flattened).toContain(shortForm);
|
||||
}
|
||||
|
||||
@@ -6,7 +6,6 @@
|
||||
|
||||
import { describe, expect, it } from "vitest";
|
||||
|
||||
import { parseOpenAIMessages } from "./core-client.js";
|
||||
import {
|
||||
asPlanDocument,
|
||||
describe as describePlan,
|
||||
@@ -260,92 +259,93 @@ describe("unknown flags are rejected", () => {
|
||||
|
||||
describe("write", () => {
|
||||
const base = [
|
||||
"submit",
|
||||
"--session",
|
||||
"s1",
|
||||
"--messages-file",
|
||||
"/tmp/messages.json",
|
||||
"--from",
|
||||
"1",
|
||||
"--to",
|
||||
"5",
|
||||
"--evidence",
|
||||
"e1",
|
||||
];
|
||||
|
||||
it("accepts a well-formed write", () => {
|
||||
const parsed = parseWriteCommand(base);
|
||||
expect(parsed).toEqual({
|
||||
kind: "submit",
|
||||
session: "s1",
|
||||
source: "memhub-cli",
|
||||
messages_file: "/tmp/messages.json",
|
||||
first_round: 1,
|
||||
last_round: 5,
|
||||
evidence_ids: ["e1"],
|
||||
});
|
||||
});
|
||||
|
||||
it("rejects a write with no messages file", () => {
|
||||
expect(parseWriteCommand(["submit", "--session", "s1"])).toBeNull();
|
||||
});
|
||||
|
||||
it("accepts source, request identity, and RFC3339 time", () => {
|
||||
it("rejects a write with no evidence", () => {
|
||||
expect(
|
||||
parseWriteCommand([
|
||||
...base,
|
||||
"--source",
|
||||
"cursor",
|
||||
"--request-id",
|
||||
"request-1",
|
||||
"--memory-at",
|
||||
"2026-10-22T00:00:00+08:00",
|
||||
]),
|
||||
).toMatchObject({
|
||||
source: "cursor",
|
||||
request_id: "request-1",
|
||||
memory_at: "2026-10-22T00:00:00+08:00",
|
||||
});
|
||||
});
|
||||
|
||||
it("rejects a timestamp without timezone", () => {
|
||||
expect(
|
||||
parseWriteCommand([...base, "--memory-at", "2026-10-22T00:00:00"]),
|
||||
parseWriteCommand(["--session", "s1", "--from", "1", "--to", "5"]),
|
||||
).toBeNull();
|
||||
});
|
||||
|
||||
it("parses status under write", () => {
|
||||
expect(parseWriteCommand(["status", "--id", "request-1"])).toEqual({
|
||||
kind: "status",
|
||||
id: "request-1",
|
||||
});
|
||||
it("rejects a descending round range", () => {
|
||||
expect(
|
||||
parseWriteCommand([
|
||||
"--session",
|
||||
"s1",
|
||||
"--from",
|
||||
"5",
|
||||
"--to",
|
||||
"1",
|
||||
"--evidence",
|
||||
"e1",
|
||||
]),
|
||||
).toBeNull();
|
||||
});
|
||||
|
||||
it("rejects a round starting below one", () => {
|
||||
expect(
|
||||
parseWriteCommand([
|
||||
"--session",
|
||||
"s1",
|
||||
"--from",
|
||||
"0",
|
||||
"--to",
|
||||
"5",
|
||||
"--evidence",
|
||||
"e1",
|
||||
]),
|
||||
).toBeNull();
|
||||
});
|
||||
|
||||
it("rejects a non-integer round", () => {
|
||||
expect(
|
||||
parseWriteCommand([
|
||||
"--session",
|
||||
"s1",
|
||||
"--from",
|
||||
"x",
|
||||
"--to",
|
||||
"5",
|
||||
"--evidence",
|
||||
"e1",
|
||||
]),
|
||||
).toBeNull();
|
||||
});
|
||||
|
||||
it("accepts several evidence ids", () => {
|
||||
const parsed = parseWriteCommand([...base, "--evidence", "e2"]);
|
||||
expect(parsed?.evidence_ids).toEqual(["e1", "e2"]);
|
||||
});
|
||||
|
||||
it("marks --live as priority live", () => {
|
||||
expect(parseWriteCommand([...base, "--live"])).toMatchObject({
|
||||
kind: "submit",
|
||||
priority: "live",
|
||||
});
|
||||
expect(parseWriteCommand([...base, "--live"])?.priority).toBe("live");
|
||||
});
|
||||
|
||||
it("leaves priority unset by default", () => {
|
||||
expect(parseWriteCommand(base)).not.toHaveProperty("priority");
|
||||
expect(parseWriteCommand(base)?.priority).toBeUndefined();
|
||||
});
|
||||
|
||||
it("rejects a repeated --json", () => {
|
||||
expect(parseWriteCommand([...base, "--json", "--json"])).toBeNull();
|
||||
});
|
||||
|
||||
it("validates OpenAI text messages and requires a user turn", () => {
|
||||
expect(
|
||||
parseOpenAIMessages({
|
||||
messages: [
|
||||
{ role: "user", content: "Remember Tuesday." },
|
||||
{ role: "assistant", content: "Understood.", ignored: true },
|
||||
],
|
||||
}),
|
||||
).toEqual([
|
||||
{ role: "user", content: "Remember Tuesday." },
|
||||
{ role: "assistant", content: "Understood." },
|
||||
]);
|
||||
expect(
|
||||
parseOpenAIMessages([{ role: "assistant", content: "No user turn." }]),
|
||||
).toBeNull();
|
||||
expect(
|
||||
parseOpenAIMessages([{ role: "system", content: "Hidden prompt." }]),
|
||||
).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
describe("task", () => {
|
||||
|
||||
@@ -255,73 +255,52 @@ export function restoreConfirmationMatches(
|
||||
return confirmPath === source;
|
||||
}
|
||||
|
||||
export interface WriteSubmitCommand {
|
||||
readonly kind: "submit";
|
||||
export interface WriteCommand {
|
||||
readonly session: string;
|
||||
readonly source: string;
|
||||
readonly messages_file: string;
|
||||
readonly request_id?: string;
|
||||
readonly memory_at?: string;
|
||||
readonly first_round: number;
|
||||
readonly last_round: number;
|
||||
readonly evidence_ids: readonly string[];
|
||||
readonly priority?: "live" | "normal";
|
||||
}
|
||||
|
||||
export interface WriteStatusCommand {
|
||||
readonly kind: "status";
|
||||
readonly id: string;
|
||||
}
|
||||
|
||||
export type WriteCommand = WriteSubmitCommand | WriteStatusCommand;
|
||||
|
||||
/** Parse SDK-native `write submit|status`. */
|
||||
/** Parse `write`. Evidence is required: no write without something to cite. */
|
||||
export function parseWriteCommand(
|
||||
args: readonly string[],
|
||||
): WriteCommand | null {
|
||||
if (args.filter((item) => item === "--json").length > 1) return null;
|
||||
const meaningful = args.filter((item) => item !== "--json");
|
||||
if (meaningful[0] === "status") {
|
||||
const status = parseTaskCommand(meaningful.slice(1));
|
||||
return status === null ? null : { kind: "status", id: status.id };
|
||||
}
|
||||
const input = meaningful[0] === "submit" ? meaningful.slice(1) : meaningful;
|
||||
let session: string | undefined;
|
||||
let source = "memhub-cli";
|
||||
let messagesFile: string | undefined;
|
||||
let requestId: string | undefined;
|
||||
let memoryAt: string | undefined;
|
||||
let first: number | undefined;
|
||||
let last: number | undefined;
|
||||
let priority: "live" | "normal" | undefined;
|
||||
const evidence: string[] = [];
|
||||
|
||||
for (let index = 0; index < input.length; index += 1) {
|
||||
const arg = input[index];
|
||||
for (let index = 0; index < args.length; index += 1) {
|
||||
const arg = args[index];
|
||||
if (arg === "--json") continue;
|
||||
if (arg === "--live") {
|
||||
priority = "live";
|
||||
continue;
|
||||
}
|
||||
const value = input[index + 1];
|
||||
const value = args[index + 1];
|
||||
if (value === undefined || value.startsWith("--")) return null;
|
||||
index += 1;
|
||||
if (arg === "--session") session = value;
|
||||
else if (arg === "--source") source = value;
|
||||
else if (arg === "--messages-file") messagesFile = value;
|
||||
else if (arg === "--request-id") requestId = value;
|
||||
else if (arg === "--memory-at") memoryAt = value;
|
||||
else if (arg === "--from") first = Number(value);
|
||||
else if (arg === "--to") last = Number(value);
|
||||
else if (arg === "--evidence") evidence.push(value);
|
||||
else return null;
|
||||
}
|
||||
|
||||
if (session === undefined || messagesFile === undefined || !source)
|
||||
if (session === undefined || first === undefined || last === undefined)
|
||||
return null;
|
||||
if (
|
||||
memoryAt !== undefined &&
|
||||
(!Number.isFinite(Date.parse(memoryAt)) ||
|
||||
!/(?:Z|[+-]\d\d:\d\d)$/.test(memoryAt))
|
||||
)
|
||||
return null;
|
||||
const command: WriteSubmitCommand = {
|
||||
kind: "submit",
|
||||
if (!Number.isInteger(first) || first < 1) return null;
|
||||
if (!Number.isInteger(last) || last < first) return null;
|
||||
if (evidence.length === 0) return null;
|
||||
const command: WriteCommand = {
|
||||
session,
|
||||
source,
|
||||
messages_file: messagesFile,
|
||||
...(requestId === undefined ? {} : { request_id: requestId }),
|
||||
...(memoryAt === undefined ? {} : { memory_at: memoryAt }),
|
||||
first_round: first,
|
||||
last_round: last,
|
||||
evidence_ids: evidence,
|
||||
};
|
||||
return priority === undefined ? command : { ...command, priority };
|
||||
}
|
||||
|
||||
@@ -48,11 +48,7 @@ describe("publish manifest is production-safe", () => {
|
||||
execFileSync("node", ["scripts/prepare-publish-manifest.mjs"], {
|
||||
cwd: root,
|
||||
stdio: "pipe",
|
||||
env: {
|
||||
...process.env,
|
||||
MEMHUB_NPM_PACKAGE: "@example/memhub-cli",
|
||||
MEMHUB_NPM_VERSION: "1.0.5",
|
||||
},
|
||||
env: { ...process.env, MEMHUB_NPM_PACKAGE: "@example/memhub-cli" },
|
||||
});
|
||||
const published = JSON.parse(
|
||||
readFileSync(path.join(root, "dist-publish", "package.json"), "utf8"),
|
||||
@@ -60,7 +56,6 @@ describe("publish manifest is production-safe", () => {
|
||||
expect(published.dependencies).toBeUndefined();
|
||||
expect(published.private).toBe(false);
|
||||
expect(published.name).toBe("@example/memhub-cli");
|
||||
expect(published.version).toBe("1.0.5");
|
||||
expect(published.bin).toMatchObject({
|
||||
memhub: "dist/one-memory.cjs",
|
||||
"memhub-cli": "dist/one-memory.cjs",
|
||||
|
||||
@@ -197,9 +197,6 @@ const READ_ONLY_PATHS = [
|
||||
"/api/v1/home-summary",
|
||||
"/api/v1/memory-library",
|
||||
"/api/v1/memories/",
|
||||
"/api/v2/memory-library",
|
||||
"/api/v2/memories/",
|
||||
"/api/v2/user-profile",
|
||||
];
|
||||
|
||||
const BACKGROUND_HEALTHY = {
|
||||
@@ -565,43 +562,6 @@ describe("CLI-20 status summary", () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe("SDK user profile", () => {
|
||||
it("prints the assembled long-term profile from the v2 endpoint", async () => {
|
||||
transport.get.mockImplementation(async (url: string) =>
|
||||
url === "/api/v2/user-profile"
|
||||
? {
|
||||
status: 200,
|
||||
data: {
|
||||
request_id: "profile-request",
|
||||
profile: {
|
||||
memory_id: "l0-head",
|
||||
content: "name is Alice",
|
||||
layer: "L0_BASIC_INFO",
|
||||
basic_info: { name: "Alice" },
|
||||
},
|
||||
elapsed_ms: 1.2,
|
||||
},
|
||||
}
|
||||
: { status: 404, data: {} },
|
||||
);
|
||||
const streams = captureStreams();
|
||||
try {
|
||||
expect(await run(["profile", "--json"], {})).toBe(0);
|
||||
} finally {
|
||||
streams.restore();
|
||||
}
|
||||
expect(transport.get).toHaveBeenCalledWith(
|
||||
"/api/v2/user-profile",
|
||||
undefined,
|
||||
);
|
||||
expect(JSON.parse(streams.stdout())).toMatchObject({
|
||||
schema_version: 2,
|
||||
state: "ok",
|
||||
profile: { memory_id: "l0-head", content: "name is Alice" },
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe("CLI-21 memory reads", () => {
|
||||
it("keeps memory reads open long enough for a busy Core and explains failure", async () => {
|
||||
const axios = (await import("axios")).default as unknown as {
|
||||
@@ -650,14 +610,14 @@ describe("CLI-21 memory reads", () => {
|
||||
|
||||
it("lists a page and states truthfully that more remains", async () => {
|
||||
transport.get.mockImplementation(async (url: string) => {
|
||||
if (url !== "/api/v2/memory-library") return { status: 404, data: {} };
|
||||
if (url !== "/api/v1/memory-library") return { status: 404, data: {} };
|
||||
return {
|
||||
status: 200,
|
||||
data: {
|
||||
total: 12,
|
||||
offset: 0,
|
||||
limit: 2,
|
||||
memories: [memory("m1", "第一条记忆"), memory("m2", "x".repeat(400))],
|
||||
items: [memory("m1", "第一条记忆"), memory("m2", "x".repeat(400))],
|
||||
},
|
||||
};
|
||||
});
|
||||
@@ -680,8 +640,10 @@ describe("CLI-21 memory reads", () => {
|
||||
});
|
||||
expect(report.memories[0]).toMatchObject({
|
||||
memory_id: "m1",
|
||||
status: "active",
|
||||
layer: "L2_FACT",
|
||||
status: "confirmed",
|
||||
revision: 3,
|
||||
scopes: ["all-apps"],
|
||||
evidence_ids: ["e1"],
|
||||
content_truncated: false,
|
||||
});
|
||||
expect(report.memories[1]).toMatchObject({
|
||||
@@ -696,8 +658,12 @@ describe("CLI-21 memory reads", () => {
|
||||
[
|
||||
"memories",
|
||||
"list",
|
||||
"--query",
|
||||
"部署",
|
||||
"--status",
|
||||
"active",
|
||||
"confirmed",
|
||||
"--source",
|
||||
"codex",
|
||||
"--offset",
|
||||
"10",
|
||||
"--limit",
|
||||
@@ -706,23 +672,30 @@ describe("CLI-21 memory reads", () => {
|
||||
],
|
||||
{},
|
||||
).catch(() => undefined);
|
||||
expect(transport.get).toHaveBeenCalledWith("/api/v2/memory-library", {
|
||||
expect(transport.get).toHaveBeenCalledWith("/api/v1/memory-library", {
|
||||
params: {
|
||||
statuses: "active",
|
||||
query: "部署",
|
||||
status: "confirmed",
|
||||
source: "codex",
|
||||
offset: 10,
|
||||
limit: 5,
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
it("rejects retired memory query flags before calling Core", async () => {
|
||||
it("reads a sensitive query from a file so it stays out of shell history", async () => {
|
||||
const directory = await mkdtemp(path.join(tmpdir(), "om-cli-query-"));
|
||||
const file = path.join(directory, "query.txt");
|
||||
await writeFile(file, "机密查询\n", { mode: 0o600 });
|
||||
expect(
|
||||
await run(["memories", "list", "--query-file", file, "--json"], {}),
|
||||
).toBe(2);
|
||||
expect(transport.get).not.toHaveBeenCalled();
|
||||
await run(["memories", "list", "--query-file", file, "--json"], {}).catch(
|
||||
() => undefined,
|
||||
);
|
||||
expect(transport.get).toHaveBeenCalledWith(
|
||||
"/api/v1/memory-library",
|
||||
expect.objectContaining({
|
||||
params: expect.objectContaining({ query: "机密查询" }),
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
||||
it("rejects contradictory or unreadable query input before calling Core", async () => {
|
||||
@@ -732,7 +705,7 @@ describe("CLI-21 memory reads", () => {
|
||||
["memories", "list", "--limit", "0"],
|
||||
["memories", "list", "--limit", "501"],
|
||||
["memories", "list", "--offset", "-1"],
|
||||
["memories", "list", "--status", ""],
|
||||
["memories", "list", "--status", "unknown-status"],
|
||||
["memories", "show"],
|
||||
["memories"],
|
||||
]) {
|
||||
@@ -750,7 +723,7 @@ describe("CLI-21 memory reads", () => {
|
||||
it("distinguishes an empty page from a denied or missing one", async () => {
|
||||
transport.get.mockResolvedValue({
|
||||
status: 200,
|
||||
data: { total: 0, offset: 0, limit: 100, memories: [] },
|
||||
data: { total: 0, offset: 0, limit: 100, items: [] },
|
||||
});
|
||||
const empty = captureStreams();
|
||||
try {
|
||||
@@ -790,10 +763,33 @@ describe("CLI-21 memory reads", () => {
|
||||
});
|
||||
});
|
||||
|
||||
it("shows one SDK memory without fabricated Evidence or versions", async () => {
|
||||
it("shows one memory with its source, version and evidence", async () => {
|
||||
transport.get.mockResolvedValue({
|
||||
status: 200,
|
||||
data: memory("m1", "记忆正文"),
|
||||
data: {
|
||||
memory: memory("m1", "记忆正文"),
|
||||
evidence: [
|
||||
{
|
||||
evidence_id: "e1",
|
||||
available: true,
|
||||
source_id: "codex",
|
||||
written_by: "user",
|
||||
created_at: "T",
|
||||
verbatim: "y".repeat(500),
|
||||
},
|
||||
],
|
||||
history: [
|
||||
{
|
||||
memory_id: "m1",
|
||||
revision: 3,
|
||||
action: "correct",
|
||||
status: "confirmed",
|
||||
content: "记忆正文",
|
||||
scopes: ["all-apps"],
|
||||
created_at: "T",
|
||||
},
|
||||
],
|
||||
},
|
||||
});
|
||||
const streams = captureStreams();
|
||||
try {
|
||||
@@ -803,17 +799,28 @@ describe("CLI-21 memory reads", () => {
|
||||
}
|
||||
const report = JSON.parse(streams.stdout());
|
||||
expect(transport.get).toHaveBeenCalledWith(
|
||||
"/api/v2/memories/m1",
|
||||
"/api/v1/memories/m1",
|
||||
undefined,
|
||||
);
|
||||
expect(report.memory).toMatchObject({
|
||||
state: "ok",
|
||||
memory_id: "m1",
|
||||
layer: "L2_FACT",
|
||||
status: "active",
|
||||
revision: 3,
|
||||
scopes: ["all-apps"],
|
||||
});
|
||||
expect(report.evidence[0]).toMatchObject({
|
||||
evidence_id: "e1",
|
||||
source_id: "codex",
|
||||
excerpt_truncated: true,
|
||||
excerpt_characters: 500,
|
||||
});
|
||||
expect(report.history[0]).toMatchObject({
|
||||
revision: 3,
|
||||
action: "correct",
|
||||
status: "confirmed",
|
||||
scopes: ["all-apps"],
|
||||
content: "记忆正文",
|
||||
});
|
||||
expect(report.evidence).toEqual([]);
|
||||
expect(report.history).toEqual([]);
|
||||
});
|
||||
|
||||
// Core registers /api/v1/memories/graph before /{memory_id}, and that static
|
||||
@@ -833,8 +840,6 @@ describe("CLI-21 memory reads", () => {
|
||||
expect(isReadOnlyPath("/api/v1/memories/graph")).toBe(false);
|
||||
expect(isReadOnlyPath("/api/v1/memories/ask")).toBe(false);
|
||||
expect(isReadOnlyPath("/api/v1/memories/m1")).toBe(true);
|
||||
expect(isReadOnlyPath("/api/v2/memory-library")).toBe(true);
|
||||
expect(isReadOnlyPath("/api/v2/memories/m1")).toBe(true);
|
||||
expect(isReadOnlyPath("/api/v1/model-content-permission")).toBe(false);
|
||||
expect(
|
||||
isReadOnlyPath(
|
||||
@@ -1132,20 +1137,17 @@ function memory(id: string, content: string) {
|
||||
return {
|
||||
memory_id: id,
|
||||
user_id: "local",
|
||||
agent_id: "one-memory",
|
||||
layer: "L2_FACT",
|
||||
status: "active",
|
||||
is_latest: true,
|
||||
supersedes: [],
|
||||
superseded_by: [],
|
||||
kind: "chat",
|
||||
status: "confirmed",
|
||||
content,
|
||||
scopes: ["all-apps"],
|
||||
evidence_ids: ["e1"],
|
||||
origin: "extracted",
|
||||
tags: [],
|
||||
owner: "user",
|
||||
source_raw_ids: ["raw-1"],
|
||||
session_id: "s1",
|
||||
source: "codex",
|
||||
memory_at: null,
|
||||
gmt_created: null,
|
||||
revision: 3,
|
||||
created_at: "T",
|
||||
updated_at: "T",
|
||||
deleted_at: null,
|
||||
};
|
||||
}
|
||||
|
||||
|
||||
@@ -160,7 +160,7 @@ describe("calls clear", () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe.skip("retired import-file", () => {
|
||||
describe("import-file", () => {
|
||||
it("never uploads without --confirm", async () => {
|
||||
const directory = await mkdtemp(path.join(tmpdir(), "om-import-"));
|
||||
const file = path.join(directory, "export.json");
|
||||
@@ -356,7 +356,7 @@ describe("notes / extraction / graph / manual extract", () => {
|
||||
expect(JSON.parse(applied.stdout).state).toBe("committed");
|
||||
});
|
||||
|
||||
it.skip("plans then applies the extraction window", async () => {
|
||||
it("plans then applies the extraction window", async () => {
|
||||
const preview = await capturedRun([
|
||||
"extraction",
|
||||
"set",
|
||||
@@ -417,7 +417,7 @@ describe("notes / extraction / graph / manual extract", () => {
|
||||
);
|
||||
});
|
||||
|
||||
it.skip("plans manual extraction and applies it only on confirm", async () => {
|
||||
it("plans manual extraction and applies it only on confirm", async () => {
|
||||
const preview = await capturedRun([
|
||||
"processing",
|
||||
"extract",
|
||||
|
||||
@@ -1,9 +1,5 @@
|
||||
declare const __ONE_MEMORY_VERSION__: string;
|
||||
declare const __ONE_MEMORY_BUILD_NUMBER__: string;
|
||||
declare const __MEMHUB_DEFAULT_RUNTIME_VERSION__: string;
|
||||
|
||||
const SEMVER =
|
||||
/^(0|[1-9]\d*)\.(0|[1-9]\d*)\.(0|[1-9]\d*)(?:-[0-9A-Za-z-]+(?:\.[0-9A-Za-z-]+)*)?(?:\+[0-9A-Za-z-]+(?:\.[0-9A-Za-z-]+)*)?$/;
|
||||
|
||||
const embeddedVersion =
|
||||
typeof __ONE_MEMORY_VERSION__ === "string"
|
||||
@@ -13,18 +9,13 @@ const embeddedBuildNumber =
|
||||
typeof __ONE_MEMORY_BUILD_NUMBER__ === "string"
|
||||
? __ONE_MEMORY_BUILD_NUMBER__.trim()
|
||||
: "";
|
||||
const embeddedRuntimeVersion =
|
||||
typeof __MEMHUB_DEFAULT_RUNTIME_VERSION__ === "string"
|
||||
? __MEMHUB_DEFAULT_RUNTIME_VERSION__.trim()
|
||||
: "";
|
||||
|
||||
export const CLI_VERSION = SEMVER.test(embeddedVersion)
|
||||
? embeddedVersion
|
||||
: "development";
|
||||
|
||||
export const RUNTIME_RELEASE_VERSION = SEMVER.test(embeddedRuntimeVersion)
|
||||
? embeddedRuntimeVersion
|
||||
: CLI_VERSION;
|
||||
export const CLI_VERSION =
|
||||
/^(0|[1-9]\d*)\.(0|[1-9]\d*)\.(0|[1-9]\d*)(?:-[0-9A-Za-z-]+(?:\.[0-9A-Za-z-]+)*)?(?:\+[0-9A-Za-z-]+(?:\.[0-9A-Za-z-]+)*)?$/.test(
|
||||
embeddedVersion,
|
||||
)
|
||||
? embeddedVersion
|
||||
: "development";
|
||||
|
||||
export const CLI_BUILD_NUMBER = /^\d+$/.test(embeddedBuildNumber)
|
||||
? Number(embeddedBuildNumber)
|
||||
|
||||
@@ -2,22 +2,11 @@ import { createRequire } from "node:module";
|
||||
import { defineConfig } from "tsup";
|
||||
|
||||
const require = createRequire(import.meta.url);
|
||||
const { assertSemver, resolveBuildVersion } =
|
||||
const { resolveBuildVersion } =
|
||||
require("../../scripts/release-version.cjs") as {
|
||||
assertSemver(value: string, label?: string): string;
|
||||
resolveBuildVersion(environment?: NodeJS.ProcessEnv, root?: string): string;
|
||||
};
|
||||
const releaseVersion = resolveBuildVersion(process.env);
|
||||
const packageVersion = (require("./package.json") as { version: string })
|
||||
.version;
|
||||
const cliVersion = assertSemver(
|
||||
process.env.MEMHUB_NPM_VERSION?.trim() || packageVersion,
|
||||
"MEMHUB_NPM_VERSION",
|
||||
);
|
||||
const runtimeReleaseVersion = assertSemver(
|
||||
process.env.MEMHUB_DEFAULT_RUNTIME_VERSION?.trim() || releaseVersion,
|
||||
"MEMHUB_DEFAULT_RUNTIME_VERSION",
|
||||
);
|
||||
|
||||
export default defineConfig({
|
||||
entry: { "one-memory": "src/entry.ts" },
|
||||
@@ -26,7 +15,7 @@ export default defineConfig({
|
||||
outDir: "dist",
|
||||
clean: true,
|
||||
define: {
|
||||
__ONE_MEMORY_VERSION__: JSON.stringify(cliVersion),
|
||||
__ONE_MEMORY_VERSION__: JSON.stringify(releaseVersion),
|
||||
__ONE_MEMORY_BUILD_NUMBER__: JSON.stringify(
|
||||
process.env.ONE_MEMORY_BUILD_NUMBER ?? "unknown",
|
||||
),
|
||||
@@ -36,15 +25,11 @@ export default defineConfig({
|
||||
__MEMHUB_DEFAULT_RELEASE_MANIFEST_URL_TEMPLATE__: JSON.stringify(
|
||||
process.env.MEMHUB_DEFAULT_RELEASE_MANIFEST_URL_TEMPLATE ?? "",
|
||||
),
|
||||
__MEMHUB_DEFAULT_RUNTIME_VERSION__: JSON.stringify(runtimeReleaseVersion),
|
||||
},
|
||||
// The shared operations package ships TypeScript sources, so it must be
|
||||
// inlined into the CJS bundle the same way runtime-control is; otherwise the
|
||||
// emitted bundle tries to `require` a `.js` file that only exists as `.ts`.
|
||||
// `@one-memory/contracts` is the same kind of package and is imported for a
|
||||
// value (the host-id grammar), so it has to be inlined too.
|
||||
noExternal: [
|
||||
"@one-memory/contracts",
|
||||
"@one-memory/operations",
|
||||
"@one-memory/runtime-control",
|
||||
"axios",
|
||||
|
||||
@@ -11,7 +11,6 @@ export default defineConfig({
|
||||
__ONE_MEMORY_BUILD_NUMBER__: JSON.stringify("unknown"),
|
||||
__ONE_MEMORY_COMMIT__: JSON.stringify("unknown"),
|
||||
__MEMHUB_DEFAULT_RELEASE_MANIFEST_URL_TEMPLATE__: JSON.stringify(""),
|
||||
__MEMHUB_DEFAULT_RUNTIME_VERSION__: JSON.stringify(version),
|
||||
},
|
||||
test: {
|
||||
testTimeout: 120_000,
|
||||
|
||||
@@ -4,7 +4,7 @@ build-backend = "uv_build"
|
||||
|
||||
[project]
|
||||
name = "one-memory"
|
||||
version = "1.0.4"
|
||||
version = "1.0.3"
|
||||
description = "MemHub local-first memory core"
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.12"
|
||||
@@ -12,7 +12,7 @@ dependencies = [
|
||||
"fastapi>=0.141.1",
|
||||
"filelock>=3.32.4",
|
||||
# Public SDK version validated against MemHub's permission and recall contracts.
|
||||
"hy-memory==2.0.5.dev3",
|
||||
"hy-memory==2.0.3",
|
||||
# Retrieval's derived vector index directly consumes these locked versions.
|
||||
"chromadb==1.5.9",
|
||||
"numpy==2.5.2",
|
||||
|
||||
File diff suppressed because it is too large.
Load diff
@@ -1,5 +1,4 @@
|
||||
import hmac
|
||||
import ipaddress
|
||||
from collections.abc import Callable
|
||||
from urllib.parse import urlsplit
|
||||
|
||||
@@ -36,15 +35,7 @@ def is_loopback_origin(origin: str) -> bool:
|
||||
host = parsed.hostname
|
||||
if not host:
|
||||
return False
|
||||
if host in LOOPBACK_HOSTNAMES:
|
||||
return True
|
||||
# A prefix check is not enough: "127.attacker.example" is a perfectly valid
|
||||
# DNS name that resolves wherever its owner wants (including 127.0.0.1 via
|
||||
# DNS rebinding). Parse the address and ask the standard library.
|
||||
try:
|
||||
return ipaddress.ip_address(host).is_loopback
|
||||
except ValueError:
|
||||
return False
|
||||
return host in LOOPBACK_HOSTNAMES or host.startswith("127.")
|
||||
|
||||
|
||||
def local_api_guard(
|
||||
|
||||
@@ -175,22 +175,6 @@ async def in_interactive_read[T](
|
||||
return await in_read_pool(interactive_read_pool(), work, *args, **kwargs)
|
||||
|
||||
|
||||
def _retrieve[T](future: "asyncio.Future[T]") -> None:
|
||||
"""Mark a shielded read's outcome as seen.
|
||||
|
||||
`in_read_pool` shields the wrapped worker, so a caller that gives up and
|
||||
cancels leaves the read running to completion. When that read then fails,
|
||||
nothing awaits the future any more: asyncio reports the orphan at GC as
|
||||
"Future exception was never retrieved", the loop handler turns it into a
|
||||
`core.loop.task_exception` error diagnostic, and a lane that degraded on
|
||||
its own budget -- correctly, and already reported -- looks like a crashed
|
||||
task on top of it. Retrieving here keeps the diagnostics honest. Awaiting
|
||||
still raises for callers that are still there, so no failure is hidden.
|
||||
"""
|
||||
if not future.cancelled():
|
||||
future.exception()
|
||||
|
||||
|
||||
async def in_read_pool[T](
|
||||
pool: ThreadPoolExecutor,
|
||||
work: Callable[..., T],
|
||||
@@ -209,9 +193,7 @@ async def in_read_pool[T](
|
||||
_interactive_reads -= 1
|
||||
|
||||
worker.add_done_callback(released)
|
||||
pending = asyncio.wrap_future(worker)
|
||||
pending.add_done_callback(_retrieve)
|
||||
return await asyncio.shield(pending)
|
||||
return await asyncio.shield(asyncio.wrap_future(worker))
|
||||
|
||||
|
||||
async def wait_for_interactive_reads() -> None:
|
||||
|
||||
@@ -76,7 +76,8 @@ def _memory(db: sqlite3.Connection, reference: CallReference) -> SourceRecordRef
|
||||
if row["status"] == "deleted" or row["deleted_at"] is not None:
|
||||
return _hidden(reference)
|
||||
|
||||
# Same local revocation/ownership predicate as SourceReadAccess.memories.
|
||||
# Same local revocation/ownership predicate as SourceReadAccess.memories,
|
||||
# evaluated BEFORE freshness (which can read support text), in this snapshot.
|
||||
# Its public methods open their own connections, so cannot be used here.
|
||||
access = db.execute(
|
||||
"SELECT EXISTS (SELECT 1 FROM governed_memory_evidence "
|
||||
@@ -120,8 +121,8 @@ def resolve_call_references(database_path: Path, event: CallEvent) -> list[Sourc
|
||||
|
||||
Local-owner only: event host attribution is NOT authorization. Explicit local
|
||||
source revocations deny access; absent grants retain legacy local semantics.
|
||||
A memory needs at least one accessible active citation. Evidence freshness
|
||||
proofs are retired and do not affect availability.
|
||||
A memory needs at least one accessible active citation. No proof preserves
|
||||
shared freshness compatibility, not a claim of verified historical freshness.
|
||||
|
||||
Any read/parse/budget failure raises a fixed whole-request error, never a
|
||||
partial result or per-item deletion. The route owner must map this separately
|
||||
|
||||
@@ -1,5 +1,3 @@
|
||||
import asyncio
|
||||
import hashlib
|
||||
import os
|
||||
import signal
|
||||
import socket
|
||||
@@ -77,16 +75,6 @@ def run(arguments: Sequence[str], *, output: TextIO | None = None) -> int:
|
||||
command, *options = arguments
|
||||
if command == "restore":
|
||||
return restore(options, stream)
|
||||
if command == "migration-bundle":
|
||||
return migration_bundle(options, stream)
|
||||
if command == "sdk-migrate":
|
||||
return sdk_migrate(options, stream)
|
||||
if command == "sdk-cutover":
|
||||
return sdk_cutover(options, stream)
|
||||
if command == "sdk-migration-status":
|
||||
return sdk_migration_status(options, stream)
|
||||
if command == "sdk-cleanup":
|
||||
return sdk_cleanup(options, stream)
|
||||
if command == "boundary" and not options:
|
||||
from one_memory.workbuddy_boundary import write_hook
|
||||
|
||||
@@ -107,11 +95,6 @@ def run(arguments: Sequence[str], *, output: TextIO | None = None) -> int:
|
||||
stream.write(
|
||||
"Usage: one-memory [serve [--maintenance|--reader]|status [--json]|doctor [--json]"
|
||||
"|restore --source <managed-backup> --yes [--json]"
|
||||
"|migration-bundle --bundle-id <id> --runtime-build <build> --yes [--json]"
|
||||
"|sdk-migrate --migration-id <id> --bundle-id <id> --yes [--json]"
|
||||
"|sdk-cutover --migration-id <id> --bundle-id <id> --yes [--json]"
|
||||
"|sdk-migration-status [--json]"
|
||||
"|sdk-cleanup --yes [--json]"
|
||||
"|boundary [--peek|--ack TOKEN]|capture SOURCE]\n"
|
||||
)
|
||||
return 2
|
||||
@@ -156,6 +139,8 @@ def restore(options: Sequence[str], stream: TextIO) -> int:
|
||||
return 2
|
||||
|
||||
settings = load_settings()
|
||||
from one_memory.managed_restore import RestoreRejected, restore_managed_backup
|
||||
|
||||
status = inspect_runtime(settings.data_dir)
|
||||
if status.state != "stopped":
|
||||
document = {
|
||||
@@ -175,48 +160,8 @@ def restore(options: Sequence[str], stream: TextIO) -> int:
|
||||
}
|
||||
_write_restore(stream, as_json, document, "预览通过;加 --yes 才会覆盖当前数据库。")
|
||||
return 4
|
||||
from one_memory.managed_restore import RestoreRejected, restore_managed_backup
|
||||
from one_memory.sdk_migration_bundle import (
|
||||
MigrationBundleRejected,
|
||||
restore_sdk_migration_bundle,
|
||||
)
|
||||
|
||||
is_sdk_bundle = source.endswith(".sdk-bundle")
|
||||
if is_sdk_bundle:
|
||||
try:
|
||||
sdk_result = restore_sdk_migration_bundle(
|
||||
data_dir=settings.data_dir,
|
||||
source=source,
|
||||
runtime_stopped=True,
|
||||
)
|
||||
except MigrationBundleRejected as error:
|
||||
document = {
|
||||
"schema_version": 1,
|
||||
"state": "failed",
|
||||
"code": error.code,
|
||||
}
|
||||
_write_restore(stream, as_json, document, f"恢复失败:{error.code}")
|
||||
return 1
|
||||
document = {
|
||||
"schema_version": 1,
|
||||
"state": "restored",
|
||||
"bundle_id": sdk_result.bundle_id,
|
||||
"restored_files": sdk_result.restored_files,
|
||||
"restored_bytes": sdk_result.restored_bytes,
|
||||
"previous_database_path": str(sdk_result.previous_database_path),
|
||||
"previous_sdk_root": (
|
||||
str(sdk_result.previous_sdk_root) if sdk_result.previous_sdk_root else None
|
||||
),
|
||||
}
|
||||
_write_restore(
|
||||
stream,
|
||||
as_json,
|
||||
document,
|
||||
f"已恢复完整迁移备份:{sdk_result.bundle_id}。",
|
||||
)
|
||||
return 0
|
||||
try:
|
||||
managed_result = restore_managed_backup(data_dir=settings.data_dir, source=source)
|
||||
result = restore_managed_backup(data_dir=settings.data_dir, source=source)
|
||||
except RestoreRejected as error:
|
||||
document = {
|
||||
"schema_version": 1,
|
||||
@@ -228,15 +173,15 @@ def restore(options: Sequence[str], stream: TextIO) -> int:
|
||||
document = {
|
||||
"schema_version": 1,
|
||||
"state": "restored",
|
||||
"restored_bytes": managed_result.restored_bytes,
|
||||
"previous_backup_path": str(managed_result.previous_backup_path),
|
||||
"source_schema_version": managed_result.source_schema_version,
|
||||
"restored_bytes": result.restored_bytes,
|
||||
"previous_backup_path": str(result.previous_backup_path),
|
||||
"source_schema_version": result.source_schema_version,
|
||||
}
|
||||
_write_restore(
|
||||
stream,
|
||||
as_json,
|
||||
document,
|
||||
f"已恢复:{managed_result.previous_backup_path} 为保护性备份。",
|
||||
f"已恢复:{result.previous_backup_path} 为保护性备份。",
|
||||
)
|
||||
return 0
|
||||
|
||||
@@ -253,506 +198,6 @@ def _write_restore(
|
||||
stream.write(f"{text}\n")
|
||||
|
||||
|
||||
def migration_bundle(options: Sequence[str], stream: TextIO) -> int:
|
||||
"""Create a complete management + SDK bundle while Runtime is stopped."""
|
||||
bundle_id: str | None = None
|
||||
runtime_build: str | None = None
|
||||
confirmed = False
|
||||
as_json = False
|
||||
index = 0
|
||||
while index < len(options):
|
||||
option = options[index]
|
||||
if option == "--bundle-id" and index + 1 < len(options):
|
||||
bundle_id = options[index + 1]
|
||||
index += 2
|
||||
continue
|
||||
if option == "--runtime-build" and index + 1 < len(options):
|
||||
runtime_build = options[index + 1]
|
||||
index += 2
|
||||
continue
|
||||
if option == "--yes":
|
||||
confirmed = True
|
||||
index += 1
|
||||
continue
|
||||
if option == "--json":
|
||||
as_json = True
|
||||
index += 1
|
||||
continue
|
||||
stream.write(f"one-memory: 参数无效:{option}\n")
|
||||
return 2
|
||||
if not bundle_id or not runtime_build:
|
||||
stream.write("one-memory: migration-bundle 需要 --bundle-id <id> --runtime-build <build>\n")
|
||||
return 2
|
||||
|
||||
settings = load_settings()
|
||||
status = inspect_runtime(settings.data_dir)
|
||||
if status.state != "stopped":
|
||||
document = {
|
||||
"schema_version": 1,
|
||||
"state": "blocked",
|
||||
"code": "sdk_migration_bundle_runtime_running",
|
||||
"runtime_state": status.state,
|
||||
}
|
||||
_write_restore(stream, as_json, document, "Runtime 仍在运行;不能创建迁移备份。")
|
||||
return 4
|
||||
if not confirmed:
|
||||
document = {
|
||||
"schema_version": 1,
|
||||
"state": "preview",
|
||||
"bundle_id": bundle_id,
|
||||
"requires_confirmation": True,
|
||||
}
|
||||
_write_restore(stream, as_json, document, "预览通过;加 --yes 后创建完整迁移备份。")
|
||||
return 4
|
||||
|
||||
from one_memory.sdk_migration_bundle import (
|
||||
MigrationBundleRejected,
|
||||
create_sdk_migration_bundle,
|
||||
inspect_sdk_migration_bundle,
|
||||
)
|
||||
|
||||
try:
|
||||
result = create_sdk_migration_bundle(
|
||||
data_dir=settings.data_dir,
|
||||
bundle_id=bundle_id,
|
||||
runtime_build=runtime_build,
|
||||
runtime_stopped=True,
|
||||
)
|
||||
except MigrationBundleRejected as error:
|
||||
if error.code == "sdk_migration_bundle_exists":
|
||||
try:
|
||||
manifest = inspect_sdk_migration_bundle(
|
||||
data_dir=settings.data_dir,
|
||||
bundle_id=bundle_id,
|
||||
)
|
||||
except MigrationBundleRejected:
|
||||
pass
|
||||
else:
|
||||
document = {
|
||||
"schema_version": 1,
|
||||
"state": "reused",
|
||||
"bundle_id": bundle_id,
|
||||
"file_count": manifest.get("file_count"),
|
||||
"total_bytes": manifest.get("total_bytes"),
|
||||
}
|
||||
_write_restore(
|
||||
stream,
|
||||
as_json,
|
||||
document,
|
||||
f"已验证并复用迁移备份:{bundle_id}",
|
||||
)
|
||||
return 0
|
||||
document = {
|
||||
"schema_version": 1,
|
||||
"state": "failed",
|
||||
"code": error.code,
|
||||
}
|
||||
_write_restore(stream, as_json, document, f"迁移备份失败:{error.code}")
|
||||
return 1
|
||||
document = {
|
||||
"schema_version": 1,
|
||||
"state": "created",
|
||||
"bundle_id": result.bundle_id,
|
||||
"bundle_path": str(result.bundle_path),
|
||||
"file_count": result.file_count,
|
||||
"total_bytes": result.total_bytes,
|
||||
}
|
||||
_write_restore(stream, as_json, document, f"完整迁移备份已创建:{result.bundle_path}")
|
||||
return 0
|
||||
|
||||
|
||||
def sdk_migrate(options: Sequence[str], stream: TextIO) -> int:
|
||||
"""Backfill legacy memory/raw into SDK while keeping legacy authority."""
|
||||
migration_id: str | None = None
|
||||
bundle_id: str | None = None
|
||||
confirmed = False
|
||||
as_json = False
|
||||
index = 0
|
||||
while index < len(options):
|
||||
option = options[index]
|
||||
if option == "--migration-id" and index + 1 < len(options):
|
||||
migration_id = options[index + 1]
|
||||
index += 2
|
||||
continue
|
||||
if option == "--bundle-id" and index + 1 < len(options):
|
||||
bundle_id = options[index + 1]
|
||||
index += 2
|
||||
continue
|
||||
if option == "--yes":
|
||||
confirmed = True
|
||||
index += 1
|
||||
continue
|
||||
if option == "--json":
|
||||
as_json = True
|
||||
index += 1
|
||||
continue
|
||||
stream.write(f"one-memory: 参数无效:{option}\n")
|
||||
return 2
|
||||
if not migration_id or not bundle_id:
|
||||
stream.write("one-memory: sdk-migrate 需要 --migration-id <id> --bundle-id <id>\n")
|
||||
return 2
|
||||
|
||||
settings = load_settings()
|
||||
status = inspect_runtime(settings.data_dir)
|
||||
if status.state != "stopped":
|
||||
document = {
|
||||
"schema_version": 1,
|
||||
"state": "blocked",
|
||||
"code": "sdk_migration_runtime_running",
|
||||
"runtime_state": status.state,
|
||||
}
|
||||
_write_restore(stream, as_json, document, "Runtime 仍在运行;不能迁移。")
|
||||
return 4
|
||||
if not confirmed:
|
||||
document = {
|
||||
"schema_version": 1,
|
||||
"state": "preview",
|
||||
"migration_id": migration_id,
|
||||
"bundle_id": bundle_id,
|
||||
"requires_confirmation": True,
|
||||
"commits_cutover": False,
|
||||
}
|
||||
_write_restore(
|
||||
stream,
|
||||
as_json,
|
||||
document,
|
||||
"预览通过;加 --yes 后执行可续跑 backfill,但不会切换 authority。",
|
||||
)
|
||||
return 4
|
||||
|
||||
from one_memory.database_access import close_main_databases
|
||||
from one_memory.hy_memory_bridge import HyMemoryBridge
|
||||
from one_memory.hy_runtime_config import HY_MEMORY_NAMESPACE, HY_MEMORY_VERSION
|
||||
from one_memory.memory_server.migration_runner import SqliteMigrationStore, migrate
|
||||
from one_memory.memory_server.migrations import MIGRATIONS
|
||||
from one_memory.model_profiles import ModelProfileStore
|
||||
from one_memory.sdk_migration_bundle import (
|
||||
MigrationBundleRejected,
|
||||
inspect_sdk_migration_bundle,
|
||||
)
|
||||
from one_memory.sdk_migration_runner import BackfillResult, run_sdk_data_plane_backfill
|
||||
from one_memory.secrets import KeyringSecretStore, SecretStoreUnavailableError
|
||||
|
||||
database = settings.data_dir / "one-memory.sqlite3"
|
||||
try:
|
||||
manifest = inspect_sdk_migration_bundle(
|
||||
data_dir=settings.data_dir,
|
||||
bundle_id=bundle_id,
|
||||
)
|
||||
asyncio.run(migrate(SqliteMigrationStore(database), MIGRATIONS))
|
||||
close_main_databases()
|
||||
profiles = ModelProfileStore(database)
|
||||
identity = profiles.active_identity()
|
||||
if identity is None:
|
||||
raise RuntimeError("model_profile_not_active")
|
||||
profile = profiles.snapshot(*identity)
|
||||
secrets = KeyringSecretStore()
|
||||
secret = profiles.resolve_snapshot_secret(*identity, secrets)
|
||||
if profile.has_api_key and secret is None:
|
||||
raise RuntimeError("model_secret_unavailable")
|
||||
except MigrationBundleRejected as error:
|
||||
code = error.code
|
||||
except SecretStoreUnavailableError:
|
||||
code = "secure_storage_unavailable"
|
||||
except Exception as error:
|
||||
code = (
|
||||
str(error)
|
||||
if str(error)
|
||||
in {
|
||||
"model_profile_not_active",
|
||||
"model_secret_unavailable",
|
||||
}
|
||||
else "sdk_migration_preflight_failed"
|
||||
)
|
||||
else:
|
||||
namespace_fingerprint = hashlib.sha256(HY_MEMORY_NAMESPACE.encode()).hexdigest()
|
||||
backend_material = json.dumps(
|
||||
{
|
||||
"namespace": HY_MEMORY_NAMESPACE,
|
||||
"embedding_provider": str(profile.embedding_provider_kind or ""),
|
||||
"embedding_model": profile.embedding_model,
|
||||
"embedding_dimensions": profile.embedding_dimensions,
|
||||
"vector": "chroma",
|
||||
"cache": "sqlite",
|
||||
"history": "sqlite",
|
||||
"coding": "sqlite",
|
||||
},
|
||||
sort_keys=True,
|
||||
separators=(",", ":"),
|
||||
)
|
||||
backend_fingerprint = hashlib.sha256(backend_material.encode()).hexdigest()
|
||||
|
||||
async def run_backfill() -> BackfillResult:
|
||||
bridge = HyMemoryBridge(
|
||||
profile=profile,
|
||||
api_key=secret,
|
||||
data_dir=settings.data_dir,
|
||||
)
|
||||
try:
|
||||
return await run_sdk_data_plane_backfill(
|
||||
database_path=database,
|
||||
bridge=bridge,
|
||||
migration_id=migration_id,
|
||||
bundle_id=bundle_id,
|
||||
source_runtime_build=str(manifest.get("runtime_build") or ""),
|
||||
target_sdk_version=HY_MEMORY_VERSION,
|
||||
source_schema_version=MIGRATIONS[-1].version,
|
||||
namespace_fingerprint=namespace_fingerprint,
|
||||
backend_fingerprint=backend_fingerprint,
|
||||
)
|
||||
finally:
|
||||
await bridge.close()
|
||||
|
||||
try:
|
||||
result = asyncio.run(run_backfill())
|
||||
except Exception:
|
||||
code = "sdk_migration_backfill_failed"
|
||||
else:
|
||||
document = {
|
||||
"schema_version": 1,
|
||||
"state": result.state,
|
||||
"phase": result.phase,
|
||||
"migration_id": result.migration_id,
|
||||
"counts": result.counts,
|
||||
"memory_items": result.memory_items,
|
||||
"raw_items": result.raw_items,
|
||||
"commits_cutover": False,
|
||||
}
|
||||
_write_restore(
|
||||
stream,
|
||||
as_json,
|
||||
document,
|
||||
"SDK backfill 已完成;authority 尚未切换。",
|
||||
)
|
||||
return 0 if result.state == "ready" else 4
|
||||
|
||||
document = {"schema_version": 1, "state": "failed", "code": code}
|
||||
_write_restore(stream, as_json, document, f"SDK backfill 失败:{code}")
|
||||
return 1
|
||||
|
||||
|
||||
def sdk_cutover(options: Sequence[str], stream: TextIO) -> int:
|
||||
"""Commit SDK authority only after offline reconciliation and work drain."""
|
||||
migration_id: str | None = None
|
||||
bundle_id: str | None = None
|
||||
confirmed = False
|
||||
as_json = False
|
||||
index = 0
|
||||
while index < len(options):
|
||||
option = options[index]
|
||||
if option == "--migration-id" and index + 1 < len(options):
|
||||
migration_id = options[index + 1]
|
||||
index += 2
|
||||
continue
|
||||
if option == "--bundle-id" and index + 1 < len(options):
|
||||
bundle_id = options[index + 1]
|
||||
index += 2
|
||||
continue
|
||||
if option == "--yes":
|
||||
confirmed = True
|
||||
index += 1
|
||||
continue
|
||||
if option == "--json":
|
||||
as_json = True
|
||||
index += 1
|
||||
continue
|
||||
stream.write(f"one-memory: 参数无效:{option}\n")
|
||||
return 2
|
||||
if not migration_id or not bundle_id:
|
||||
stream.write("one-memory: sdk-cutover 需要 --migration-id <id> --bundle-id <id>\n")
|
||||
return 2
|
||||
settings = load_settings()
|
||||
status = inspect_runtime(settings.data_dir)
|
||||
if status.state != "stopped":
|
||||
document = {
|
||||
"schema_version": 1,
|
||||
"state": "blocked",
|
||||
"code": "sdk_cutover_runtime_running",
|
||||
"runtime_state": status.state,
|
||||
}
|
||||
_write_restore(stream, as_json, document, "Runtime 仍在运行;不能切换 authority。")
|
||||
return 4
|
||||
if not confirmed:
|
||||
document = {
|
||||
"schema_version": 1,
|
||||
"state": "preview",
|
||||
"migration_id": migration_id,
|
||||
"bundle_id": bundle_id,
|
||||
"requires_confirmation": True,
|
||||
}
|
||||
_write_restore(stream, as_json, document, "预览通过;加 --yes 后提交 SDK authority。")
|
||||
return 4
|
||||
|
||||
from one_memory.hy_runtime_config import HY_MEMORY_VERSION
|
||||
from one_memory.sdk_data_plane_migration import DataPlaneMigrationError
|
||||
from one_memory.sdk_migration_bundle import MigrationBundleRejected
|
||||
from one_memory.sdk_migration_runner import commit_reconciled_cutover
|
||||
|
||||
try:
|
||||
marker = commit_reconciled_cutover(
|
||||
data_dir=settings.data_dir,
|
||||
database_path=settings.data_dir / "one-memory.sqlite3",
|
||||
migration_id=migration_id,
|
||||
bundle_id=bundle_id,
|
||||
sdk_version=HY_MEMORY_VERSION,
|
||||
)
|
||||
except (DataPlaneMigrationError, MigrationBundleRejected) as error:
|
||||
code = getattr(error, "code", str(error))
|
||||
document = {
|
||||
"schema_version": 1,
|
||||
"state": "blocked",
|
||||
"code": code,
|
||||
"migration_id": migration_id,
|
||||
"bundle_id": bundle_id,
|
||||
}
|
||||
_write_restore(stream, as_json, document, f"SDK authority 未切换:{code}")
|
||||
return 4
|
||||
document = {
|
||||
"schema_version": 1,
|
||||
"state": "cutover",
|
||||
"migration_id": marker.migration_id,
|
||||
"bundle_id": marker.backup_bundle_id,
|
||||
"sdk_version": marker.sdk_version,
|
||||
"completed_at": marker.completed_at,
|
||||
}
|
||||
_write_restore(stream, as_json, document, "SDK authority 已切换。")
|
||||
return 0
|
||||
|
||||
|
||||
def sdk_migration_status(options: Sequence[str], stream: TextIO) -> int:
|
||||
"""Read-only startup probe used before taking the Runtime maintenance fence."""
|
||||
if list(options) not in ([], ["--json"]):
|
||||
stream.write("one-memory: sdk-migration-status 只接受 --json\n")
|
||||
return 2
|
||||
as_json = "--json" in options
|
||||
settings = load_settings()
|
||||
database_path = settings.data_dir / "one-memory.sqlite3"
|
||||
if not database_path.is_file():
|
||||
document = {
|
||||
"schema_version": 1,
|
||||
"authority": "legacy",
|
||||
"state": "empty",
|
||||
"migration_id": None,
|
||||
}
|
||||
else:
|
||||
import sqlite3
|
||||
|
||||
from one_memory.database_access import main_database
|
||||
from one_memory.sdk_data_plane_migration import (
|
||||
MemoryAuthority,
|
||||
SdkDataPlaneMigrationStore,
|
||||
)
|
||||
from one_memory.target_schema import legacy_tables_present
|
||||
|
||||
migration_store = SdkDataPlaneMigrationStore(database_path)
|
||||
marker = migration_store.cutover()
|
||||
authority = migration_store.authority()
|
||||
stat = database_path.stat()
|
||||
generation = hashlib.sha256(f"{stat.st_size}:{stat.st_mtime_ns}".encode()).hexdigest()[:16]
|
||||
if authority is MemoryAuthority.SDK:
|
||||
legacy_tables = legacy_tables_present(database_path)
|
||||
document = {
|
||||
"schema_version": 1,
|
||||
"authority": "sdk",
|
||||
"state": "cleanup_required" if legacy_tables else "complete",
|
||||
"migration_id": marker.migration_id if marker else None,
|
||||
"generation": generation,
|
||||
"legacy_tables": list(legacy_tables),
|
||||
}
|
||||
else:
|
||||
with main_database(database_path).read(row_factory=sqlite3.Row) as database:
|
||||
tables = {
|
||||
str(row["name"])
|
||||
for row in database.execute("SELECT name FROM sqlite_master WHERE type='table'")
|
||||
}
|
||||
counts = {
|
||||
"memory": (
|
||||
int(
|
||||
database.execute("SELECT COUNT(*) FROM governed_memories").fetchone()[0]
|
||||
)
|
||||
if "governed_memories" in tables
|
||||
else 0
|
||||
),
|
||||
"raw": (
|
||||
int(database.execute("SELECT COUNT(*) FROM evidence").fetchone()[0])
|
||||
if "evidence" in tables
|
||||
else 0
|
||||
),
|
||||
}
|
||||
document = {
|
||||
"schema_version": 1,
|
||||
"authority": "legacy",
|
||||
"state": "required" if any(counts.values()) else "empty",
|
||||
"migration_id": None,
|
||||
"counts": counts,
|
||||
"generation": generation,
|
||||
}
|
||||
if as_json:
|
||||
stream.write(f"{json.dumps(document, ensure_ascii=False)}\n")
|
||||
else:
|
||||
stream.write(f"SDK migration: {document['state']} (authority={document['authority']})\n")
|
||||
return 0
|
||||
|
||||
|
||||
def sdk_cleanup(options: Sequence[str], stream: TextIO) -> int:
|
||||
if set(options) - {"--yes", "--json"} or options.count("--yes") > 1:
|
||||
stream.write("one-memory: sdk-cleanup 只接受 --yes [--json]\n")
|
||||
return 2
|
||||
confirmed = "--yes" in options
|
||||
as_json = "--json" in options
|
||||
settings = load_settings()
|
||||
status = inspect_runtime(settings.data_dir)
|
||||
if status.state != "stopped":
|
||||
document = {
|
||||
"schema_version": 1,
|
||||
"state": "blocked",
|
||||
"code": "sdk_cleanup_runtime_running",
|
||||
}
|
||||
_write_restore(stream, as_json, document, "Runtime 仍在运行;不能清理旧表。")
|
||||
return 4
|
||||
if not confirmed:
|
||||
_write_restore(
|
||||
stream,
|
||||
as_json,
|
||||
{
|
||||
"schema_version": 1,
|
||||
"state": "preview",
|
||||
"requires_confirmation": True,
|
||||
},
|
||||
"加 --yes 后删除已迁移旧表。",
|
||||
)
|
||||
return 4
|
||||
from one_memory.sdk_data_plane_migration import DataPlaneMigrationError
|
||||
from one_memory.sdk_migration_bundle import MigrationBundleRejected
|
||||
from one_memory.sdk_migration_runner import resume_legacy_cleanup
|
||||
|
||||
try:
|
||||
dropped = resume_legacy_cleanup(
|
||||
data_dir=settings.data_dir,
|
||||
database_path=settings.data_dir / "one-memory.sqlite3",
|
||||
)
|
||||
except (DataPlaneMigrationError, MigrationBundleRejected) as error:
|
||||
code = getattr(error, "code", str(error))
|
||||
_write_restore(
|
||||
stream,
|
||||
as_json,
|
||||
{"schema_version": 1, "state": "blocked", "code": code},
|
||||
f"旧表清理未完成:{code}",
|
||||
)
|
||||
return 4
|
||||
_write_restore(
|
||||
stream,
|
||||
as_json,
|
||||
{
|
||||
"schema_version": 1,
|
||||
"state": "clean",
|
||||
"dropped_tables": list(dropped),
|
||||
},
|
||||
"旧表已删除。",
|
||||
)
|
||||
return 0
|
||||
|
||||
|
||||
def _status_text(command: str, status: RuntimeStatus) -> str:
|
||||
if status.state == "running":
|
||||
location = f"{status.host}:{status.port}"
|
||||
@@ -821,13 +266,7 @@ def serve(*, maintenance: bool = False, reader: bool = False) -> None:
|
||||
)
|
||||
application = create_reader_app(reader_settings)
|
||||
print(f"reader_port:{port}", file=sys.stderr, flush=True)
|
||||
uvicorn.run(
|
||||
application,
|
||||
host=settings.host,
|
||||
port=port,
|
||||
reload=False,
|
||||
access_log=False,
|
||||
)
|
||||
uvicorn.run(application, host=settings.host, port=port, reload=False)
|
||||
return
|
||||
|
||||
from one_memory.api import create_app
|
||||
@@ -858,21 +297,9 @@ def serve(*, maintenance: bool = False, reader: bool = False) -> None:
|
||||
)
|
||||
if maintenance:
|
||||
with CoreRuntime(settings, publish_receipt=False):
|
||||
uvicorn.run(
|
||||
application,
|
||||
host=settings.host,
|
||||
port=settings.port,
|
||||
reload=False,
|
||||
access_log=False,
|
||||
)
|
||||
uvicorn.run(application, host=settings.host, port=settings.port, reload=False)
|
||||
else:
|
||||
uvicorn.run(
|
||||
application,
|
||||
host=settings.host,
|
||||
port=settings.port,
|
||||
reload=False,
|
||||
access_log=False,
|
||||
)
|
||||
uvicorn.run(application, host=settings.host, port=settings.port, reload=False)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
||||
@@ -89,51 +89,6 @@ class MemoryContextItem(BaseModel):
|
||||
ContextItem = Annotated[RawContextItem | MemoryContextItem, Field(discriminator="kind")]
|
||||
|
||||
|
||||
class SdkRawContextItem(BaseModel):
|
||||
model_config = ConfigDict(frozen=True, extra="forbid")
|
||||
|
||||
kind: Literal["raw"] = "raw"
|
||||
raw_id: str = Field(min_length=1, max_length=512)
|
||||
content: str = Field(max_length=20000)
|
||||
source: str = Field(min_length=1, max_length=128)
|
||||
session_id: str = Field(min_length=1, max_length=512)
|
||||
memory_at: datetime | None = None
|
||||
score: float = Field(ge=0, le=1)
|
||||
truncated: bool = False
|
||||
|
||||
|
||||
class SdkMemoryContextItem(BaseModel):
|
||||
model_config = ConfigDict(frozen=True, extra="forbid")
|
||||
|
||||
kind: Literal["memory"] = "memory"
|
||||
memory_id: str = Field(min_length=1, max_length=512)
|
||||
content: str = Field(max_length=20000)
|
||||
layer: str = Field(max_length=64)
|
||||
status: str = Field(max_length=32)
|
||||
source: str | None = Field(default=None, max_length=128)
|
||||
updated_at: datetime | None = None
|
||||
score: float = Field(ge=0, le=1)
|
||||
truncated: bool = False
|
||||
|
||||
|
||||
SdkContextItem = Annotated[
|
||||
SdkRawContextItem | SdkMemoryContextItem,
|
||||
Field(discriminator="kind"),
|
||||
]
|
||||
|
||||
|
||||
class SdkContextSearchResponse(BaseModel):
|
||||
model_config = ConfigDict(frozen=True, extra="forbid")
|
||||
|
||||
schema_version: Literal[2] = 2
|
||||
request_id: str = Field(min_length=1, max_length=64)
|
||||
state: Literal["grounded", "empty", "partial", "unavailable"]
|
||||
items: tuple[SdkContextItem, ...] = Field(max_length=20)
|
||||
used_characters: int = Field(ge=0, le=20000)
|
||||
truncated: bool
|
||||
total_ms: float = Field(ge=0)
|
||||
|
||||
|
||||
class ContextLaneResult(BaseModel):
|
||||
"""Internal bounded adapter result; the aggregator applies shared budgets."""
|
||||
|
||||
|
||||
@@ -196,7 +196,7 @@ def read_raw_context(
|
||||
if _prefers_declared_facts(request.query)
|
||||
else "1"
|
||||
)
|
||||
person_authored = "CASE WHEN e.role='user' THEN 1 ELSE 0 END"
|
||||
person_authored = "CASE WHEN e.written_by='person' THEN 1 ELSE 0 END"
|
||||
query_echo = (
|
||||
"CASE WHEN trim(e.verbatim)=? OR trim(e.verbatim)=? THEN 0 ELSE 1 END"
|
||||
if _prefers_declared_facts(request.query)
|
||||
@@ -470,8 +470,9 @@ def revalidate_raw_context(
|
||||
)
|
||||
except (ValueError, TypeError):
|
||||
break
|
||||
if (row["role"], created) != (
|
||||
if (row["role"], row["written_by"], created) != (
|
||||
message.role,
|
||||
message.written_by,
|
||||
message.created_at,
|
||||
):
|
||||
break
|
||||
|
||||
@@ -66,12 +66,12 @@ def _field(text: str) -> ResumeField | None:
|
||||
return None
|
||||
|
||||
|
||||
def _raw_class(role: str, evidence_id: str, accepted: frozenset[str]) -> EvidenceClass:
|
||||
if role == "user":
|
||||
def _raw_class(written_by: str, evidence_id: str, accepted: frozenset[str]) -> EvidenceClass:
|
||||
if written_by == "person":
|
||||
return "person_statement"
|
||||
if role == "assistant":
|
||||
if written_by == "agent":
|
||||
return "agent_report"
|
||||
if role == "tool" and evidence_id in accepted:
|
||||
if written_by == "tool" and evidence_id in accepted:
|
||||
return "system_acceptance"
|
||||
return "tool_output"
|
||||
|
||||
@@ -118,7 +118,7 @@ def assemble_resume_packet(
|
||||
field=_field(message.text) or "unclassified",
|
||||
text=message.text,
|
||||
evidence_class=_raw_class(
|
||||
message.role,
|
||||
message.written_by,
|
||||
message.evidence_id,
|
||||
verified_system_evidence_ids,
|
||||
),
|
||||
|
||||
@@ -4,9 +4,7 @@ import asyncio
|
||||
import sqlite3
|
||||
import time
|
||||
from collections.abc import Sequence
|
||||
from datetime import UTC, datetime
|
||||
from typing import cast
|
||||
from uuid import uuid4
|
||||
|
||||
from fastapi import APIRouter, HTTPException, Request, Response
|
||||
from fastapi.params import Depends
|
||||
@@ -33,10 +31,6 @@ from one_memory.context_models import (
|
||||
ContextSemanticStatus,
|
||||
MemoryContextItem,
|
||||
RawContextItem,
|
||||
SdkContextItem,
|
||||
SdkContextSearchResponse,
|
||||
SdkMemoryContextItem,
|
||||
SdkRawContextItem,
|
||||
)
|
||||
from one_memory.context_raw import revalidate_raw_context
|
||||
from one_memory.context_search import search_context
|
||||
@@ -426,154 +420,3 @@ def context_router(dependencies: Sequence[Depends]) -> APIRouter:
|
||||
return final
|
||||
|
||||
return router
|
||||
|
||||
|
||||
def sdk_context_router(dependencies: Sequence[Depends]) -> APIRouter:
|
||||
"""SDK-native context: no Evidence/Governance ids or fabricated revisions."""
|
||||
router = APIRouter(prefix="/api/v2/context", dependencies=dependencies)
|
||||
|
||||
@router.post("/search", response_model=SdkContextSearchResponse)
|
||||
async def search(
|
||||
value: ContextSearchRequest,
|
||||
request: Request,
|
||||
response: Response,
|
||||
) -> SdkContextSearchResponse:
|
||||
response.headers["Cache-Control"] = "no-store"
|
||||
started = time.monotonic()
|
||||
access = external_access(request.headers)
|
||||
database = cast(EvidenceStore, request.app.state.evidence_store).database_path
|
||||
try:
|
||||
authority = await asyncio.to_thread(
|
||||
resolve_context_authority,
|
||||
value,
|
||||
access,
|
||||
database,
|
||||
deadline=started + value.timeout_ms / 1000,
|
||||
)
|
||||
except HTTPException:
|
||||
raise
|
||||
except Exception:
|
||||
raise HTTPException(503, "context_authority_unavailable") from None
|
||||
|
||||
bridge = getattr(request.app.state, "sdk_library_bridge", None)
|
||||
if bridge is None:
|
||||
raise HTTPException(503, "sdk_context_unavailable")
|
||||
|
||||
async def raw_lane() -> tuple[SdkRawContextItem, ...]:
|
||||
if "raw" not in value.kinds or not authority.raw_allowed:
|
||||
return ()
|
||||
hits = await bridge.search_sdk_raw(
|
||||
value.query,
|
||||
user_id="local",
|
||||
source_ids=tuple(authority.raw_sources or ()),
|
||||
limit=value.limit,
|
||||
)
|
||||
return tuple(
|
||||
SdkRawContextItem(
|
||||
raw_id=hit.raw_id,
|
||||
content=hit.content,
|
||||
source=hit.source,
|
||||
session_id=hit.session_id,
|
||||
memory_at=(
|
||||
datetime.fromtimestamp(hit.memory_at, tz=UTC)
|
||||
if hit.memory_at is not None
|
||||
else None
|
||||
),
|
||||
score=hit.score,
|
||||
)
|
||||
for hit in hits
|
||||
)
|
||||
|
||||
async def memory_lane() -> tuple[SdkMemoryContextItem, ...]:
|
||||
if "memory" not in value.kinds or not authority.memory_allowed:
|
||||
return ()
|
||||
hits = await bridge.search_memories(
|
||||
value.query,
|
||||
user_id="local",
|
||||
agent_id="*",
|
||||
limit=value.limit,
|
||||
)
|
||||
items: list[SdkMemoryContextItem] = []
|
||||
for hit in hits:
|
||||
memory = await bridge.get_sdk_memory(hit.memory_id)
|
||||
if memory is None or memory.status != "active":
|
||||
continue
|
||||
items.append(
|
||||
SdkMemoryContextItem(
|
||||
memory_id=memory.memory_id,
|
||||
content=(
|
||||
hit.formatted_content
|
||||
if hit.sdk_formatted
|
||||
and hit.formatting_complete
|
||||
and hit.formatted_content
|
||||
else memory.content
|
||||
),
|
||||
layer=memory.layer,
|
||||
status=memory.status,
|
||||
source=memory.source,
|
||||
updated_at=(
|
||||
datetime.fromtimestamp(memory.gmt_created, tz=UTC)
|
||||
if memory.gmt_created is not None
|
||||
else None
|
||||
),
|
||||
score=hit.score,
|
||||
)
|
||||
)
|
||||
return tuple(items)
|
||||
|
||||
lanes = (raw_lane(), memory_lane())
|
||||
try:
|
||||
raw_items, memory_items = await asyncio.wait_for(
|
||||
asyncio.gather(*lanes, return_exceptions=True),
|
||||
timeout=value.timeout_ms / 1000,
|
||||
)
|
||||
except TimeoutError:
|
||||
return SdkContextSearchResponse(
|
||||
request_id=str(uuid4()),
|
||||
state="unavailable",
|
||||
items=(),
|
||||
used_characters=0,
|
||||
truncated=True,
|
||||
total_ms=(time.monotonic() - started) * 1000,
|
||||
)
|
||||
partial = isinstance(raw_items, Exception) or isinstance(memory_items, Exception)
|
||||
raw_candidates = (
|
||||
() if isinstance(raw_items, Exception) else cast(tuple[SdkContextItem, ...], raw_items)
|
||||
)
|
||||
memory_candidates = (
|
||||
()
|
||||
if isinstance(memory_items, Exception)
|
||||
else cast(tuple[SdkContextItem, ...], memory_items)
|
||||
)
|
||||
candidates: list[SdkContextItem] = [
|
||||
*raw_candidates,
|
||||
*memory_candidates,
|
||||
]
|
||||
candidates.sort(key=lambda item: item.score, reverse=True)
|
||||
selected: list[SdkContextItem] = []
|
||||
used = 0
|
||||
truncated = False
|
||||
for item in candidates:
|
||||
if len(selected) >= value.limit:
|
||||
truncated = True
|
||||
break
|
||||
content = item.content
|
||||
remaining = value.character_budget - used
|
||||
if remaining <= 0:
|
||||
truncated = True
|
||||
break
|
||||
if len(content) > remaining:
|
||||
item = item.model_copy(update={"content": content[:remaining], "truncated": True})
|
||||
truncated = True
|
||||
selected.append(item)
|
||||
used += len(item.content)
|
||||
return SdkContextSearchResponse(
|
||||
request_id=str(uuid4()),
|
||||
state=("partial" if partial else "grounded" if selected else "empty"),
|
||||
items=tuple(selected),
|
||||
used_characters=used,
|
||||
truncated=truncated,
|
||||
total_ms=(time.monotonic() - started) * 1000,
|
||||
)
|
||||
|
||||
return router
|
||||
@@ -345,7 +345,7 @@ def _lexical_raw_rows(
|
||||
if context_raw._prefers_declared_facts(query.query)
|
||||
else "1"
|
||||
)
|
||||
person_authored = "CASE WHEN e.role='user' THEN 1 ELSE 0 END"
|
||||
person_authored = "CASE WHEN e.written_by='person' THEN 1 ELSE 0 END"
|
||||
query_echo = (
|
||||
"CASE WHEN trim(e.verbatim)=? OR trim(e.verbatim)=? THEN 0 ELSE 1 END"
|
||||
if context_raw._prefers_declared_facts(query.query)
|
||||
@@ -507,7 +507,7 @@ def _raw_item(row: sqlite3.Row, start1: int, matched_by: tuple[str, ...]) -> Raw
|
||||
"source_id": row["source_id"],
|
||||
"conversation_id": row["conversation_id"],
|
||||
"role": row["role"],
|
||||
"written_by": "person" if row["role"] == "user" else "agent",
|
||||
"written_by": "person" if row["written_by"] == "person" else "agent",
|
||||
"event_time": event_time.isoformat() if event_time else None,
|
||||
"spans": [{"start_cp": start, "end_cp": start + len(text), "text": text}],
|
||||
"matched_by": list(matched_by),
|
||||
|
||||
@@ -1039,7 +1039,7 @@ def close_main_databases() -> None:
|
||||
def log_main_database_stats(moment: str) -> None:
|
||||
"""One line in core.log: how many connections are open and who waited."""
|
||||
stats = main_database_stats()
|
||||
logger.debug(
|
||||
logger.info(
|
||||
"main_database_connections moment=%s %s",
|
||||
moment,
|
||||
" ".join(f"{key}={value}" for key, value in stats.items()),
|
||||
|
||||
@@ -19,7 +19,6 @@ from __future__ import annotations
|
||||
|
||||
import atexit
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
@@ -140,10 +139,6 @@ _DETAIL_KEYS = frozenset(
|
||||
"accounted_ms",
|
||||
"unaccounted_ms",
|
||||
"conversation_hash",
|
||||
# The sedimentation switch: a bounded boolean, the same shape as
|
||||
# `spawned` and `index_complete`. Left out of this list it did not
|
||||
# drop a log line, it turned the switch's own request into a 500.
|
||||
"enabled",
|
||||
}
|
||||
)
|
||||
_trace_id: ContextVar[str | None] = ContextVar("one_memory_trace_id", default=None)
|
||||
@@ -654,22 +649,12 @@ def diagnostic_event(**values: object) -> None:
|
||||
active = _active
|
||||
if active is None:
|
||||
return
|
||||
if values.get("level") == "debug" and not logging.getLogger().isEnabledFor(logging.DEBUG):
|
||||
return
|
||||
try:
|
||||
active.write(**values) # type: ignore[arg-type]
|
||||
except OSError:
|
||||
# Diagnostics must not become a new product outage when a disk is full
|
||||
# or the log directory is temporarily unavailable.
|
||||
return
|
||||
except ValueError:
|
||||
# Same promise, same reason, for a rejected payload instead of a failed
|
||||
# write. An unlisted `details` key once turned every external recall
|
||||
# into a 500 and then killed the process, and the sedimentation switch
|
||||
# into a permanent failure. The payload itself is still wrong and is
|
||||
# still caught in CI by tests/test_diagnostic_payload_keys.py; what
|
||||
# must not happen is the caller paying for it.
|
||||
return
|
||||
|
||||
|
||||
def current_session_id() -> str | None:
|
||||
|
||||
@@ -10,8 +10,11 @@ from uuid import UUID
|
||||
from fastapi import HTTPException, Request
|
||||
|
||||
from one_memory.database_access import main_database
|
||||
from one_memory.matters.models import Matter, MatterContext
|
||||
from one_memory.matters.store import MatterStore
|
||||
from one_memory.memory_governance import MemoryGovernanceStore
|
||||
|
||||
TOOLS = frozenset({"om_recall", "om_ask", "om_conversations", "om_diagnostic_trace"})
|
||||
TOOLS = frozenset({"om_recall", "om_ask", "om_conversations", "om_matters", "om_diagnostic_trace"})
|
||||
_ENVELOPE_HEADERS = frozenset(
|
||||
{
|
||||
"x-one-memory-read-context",
|
||||
@@ -24,9 +27,8 @@ _ENVELOPE_HEADERS = frozenset(
|
||||
_BROKER_ROUTES = {
|
||||
("POST", "/api/v1/recall"): "om_recall",
|
||||
("POST", "/api/v1/memories/ask"): "om_ask",
|
||||
("POST", "/api/v2/memories/ask"): "om_ask",
|
||||
("POST", "/api/v2/memories/ask/stream"): "om_ask",
|
||||
("GET", "/api/v1/conversations/search"): "om_conversations",
|
||||
("POST", "/api/v1/matters/context/search"): "om_matters",
|
||||
("POST", "/api/v1/diagnostics/trace"): "om_diagnostic_trace",
|
||||
("POST", "/api/v1/boundary/peek"): "om_recall",
|
||||
("POST", "/api/v1/boundary/ack"): "om_recall",
|
||||
@@ -66,6 +68,50 @@ class ExternalMemoryAccess:
|
||||
}
|
||||
return sources - revoked
|
||||
|
||||
def allows_matter(self, matter: Matter, governance: MemoryGovernanceStore) -> bool:
|
||||
if (
|
||||
not matter.agent_access
|
||||
or matter.availability != "ready"
|
||||
or matter.source_engine != "hy-memory"
|
||||
):
|
||||
return False
|
||||
support_ids: set[str] = set()
|
||||
for claim in matter.claims:
|
||||
if not claim.memory_refs:
|
||||
return False
|
||||
for reference in claim.memory_refs:
|
||||
memory = governance.get(reference.memory_id, user_id="local")
|
||||
if (
|
||||
memory is None
|
||||
or memory.revision != reference.revision
|
||||
or memory.status != reference.status
|
||||
or self.scopes.isdisjoint(memory.scopes)
|
||||
):
|
||||
return False
|
||||
support_ids.update(memory.evidence_ids)
|
||||
# Titles, boundaries and uncertainties can depend on uncited anchors.
|
||||
# Referenced memories can also have support not quoted by this matter.
|
||||
with main_database(governance.database_path).read() as db:
|
||||
row = db.execute(
|
||||
"SELECT dependencies FROM matter_versions WHERE matter_id=? AND version=?",
|
||||
(matter.matter_id, matter.version),
|
||||
).fetchone()
|
||||
if row is None:
|
||||
return False
|
||||
dependencies = json.loads(row[0])
|
||||
if not dependencies:
|
||||
return False
|
||||
for evidence_id in set(dependencies) | support_ids:
|
||||
source = db.execute(
|
||||
"SELECT e.source_id FROM evidence e JOIN source_grants g "
|
||||
"USING(user_id,source_id) WHERE e.user_id='local' AND e.evidence_id=? "
|
||||
"AND g.revoked_at IS NULL",
|
||||
(evidence_id,),
|
||||
).fetchone()
|
||||
if source is None or f"source:{source[0]}" not in self.scopes:
|
||||
return False
|
||||
return bool(matter.claims)
|
||||
|
||||
|
||||
def external_access(headers: Mapping[str, str]) -> ExternalMemoryAccess | None:
|
||||
# Starlette Headers.items() preserves duplicate HTTP fields; .get() does not.
|
||||
@@ -128,10 +174,7 @@ def require_external_endpoint(request: Request) -> None:
|
||||
access = external_access(request.headers)
|
||||
if access is None:
|
||||
return
|
||||
if request.method == "POST" and request.url.path in {
|
||||
"/api/v1/context/search",
|
||||
"/api/v2/context/search",
|
||||
}:
|
||||
if (request.method, request.url.path) == ("POST", "/api/v1/context/search"):
|
||||
# The route resolves each requested lane against its original grant.
|
||||
# Passing this envelope gate does not authorize either lane by itself.
|
||||
if access.tools.isdisjoint({"om_recall", "om_conversations"}):
|
||||
@@ -146,7 +189,46 @@ def require_external_endpoint(request: Request) -> None:
|
||||
raise HTTPException(403, "external_endpoint_denied")
|
||||
if tool not in access.tools:
|
||||
raise HTTPException(403, "external_recall_scope_denied")
|
||||
if tool == "om_conversations":
|
||||
if tool in {"om_conversations", "om_matters"}:
|
||||
access.require(tool, frozenset({"all-apps"}))
|
||||
if not any(scope.startswith("source:") for scope in access.scopes):
|
||||
raise HTTPException(403, "external_source_permission_required")
|
||||
|
||||
|
||||
def search_external_matter_contexts(
|
||||
access: ExternalMemoryAccess,
|
||||
query: str,
|
||||
*,
|
||||
matters: MatterStore,
|
||||
governance: MemoryGovernanceStore,
|
||||
) -> list[MatterContext]:
|
||||
"""Apply installation bounds before rendering quotes or consuming the result limit."""
|
||||
access.require("om_matters", frozenset({"all-apps"}))
|
||||
access.readable_source_ids(governance.database_path)
|
||||
terms = query.casefold().split()
|
||||
found: list[MatterContext] = []
|
||||
for matter in matters.list():
|
||||
if not access.allows_matter(matter, governance):
|
||||
continue
|
||||
searchable = f"{matter.title} {matter.purpose} {' '.join(matter.tags)}".casefold()
|
||||
if not all(term in searchable for term in terms):
|
||||
continue
|
||||
context = matters.context(matter.matter_id, character_budget=20_000)
|
||||
current = matters.get(matter.matter_id)
|
||||
if (
|
||||
context.availability != "ready"
|
||||
or current.version != matter.version
|
||||
or context.version != matter.version
|
||||
or not access.allows_matter(current, governance)
|
||||
):
|
||||
continue
|
||||
found.append(context)
|
||||
if len(found) == 3:
|
||||
break
|
||||
# Rendering a later item may race a revocation affecting an earlier one.
|
||||
released = []
|
||||
for context in found:
|
||||
current = matters.get(context.matter_id)
|
||||
if current.version == context.version and access.allows_matter(current, governance):
|
||||
released.append(context)
|
||||
return released
|
||||
@@ -1,15 +1,75 @@
|
||||
"""Persisted extraction-window choice stored in Core-owned config.json."""
|
||||
"""Persisted extraction-window choice.
|
||||
|
||||
Kept apart from onboarding: the window can change long after the first run,
|
||||
and it has to survive a restart so a widened window does not silently snap
|
||||
back to ninety days.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import sqlite3
|
||||
from collections.abc import Iterator
|
||||
from contextlib import contextmanager
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from pydantic import BaseModel, ConfigDict
|
||||
|
||||
from one_memory.database import (
|
||||
take_the_write_lock_first,
|
||||
)
|
||||
from one_memory.database_access import main_database
|
||||
from one_memory.memory_queue import ExtractionWindowPreset
|
||||
from one_memory.memory_server.config_state import ConfigStore
|
||||
|
||||
#: Schema owned by `memory_server.migrations` (migration 17). No local copy.
|
||||
from one_memory.memory_server.migrations import extraction_settings_ddl
|
||||
from one_memory.memory_server.store_binding import store_for
|
||||
|
||||
|
||||
def _is_mysql(store: Any) -> bool:
|
||||
"""Whether the bound store is the MySQL one.
|
||||
|
||||
Asked of the store rather than guessed from a global, because two engines
|
||||
can exist in one process during the migration and a global would make the
|
||||
answer depend on whichever was resolved last.
|
||||
"""
|
||||
from one_memory.memory_server.sync_facade import SyncStoreFacade
|
||||
|
||||
return isinstance(store, SyncStoreFacade)
|
||||
|
||||
|
||||
def _value(row: object, index: int) -> object:
|
||||
"""Read a column from either a sqlite3.Row or a plain tuple.
|
||||
|
||||
`sqlite3.Row` supports indexing by name and by position; a MySQL tuple only
|
||||
by position. Reading by position works on both, which is why the SELECT
|
||||
names its column explicitly.
|
||||
"""
|
||||
try:
|
||||
return row[index] # type: ignore[index]
|
||||
except (TypeError, KeyError):
|
||||
return row[index] # type: ignore[index]
|
||||
|
||||
|
||||
SCHEMA = extraction_settings_ddl("sqlite")
|
||||
|
||||
#: The upsert, per engine.
|
||||
#:
|
||||
#: SQLite has `ON CONFLICT ... DO UPDATE` and MySQL does not; MySQL spells the
|
||||
#: same thing `ON DUPLICATE KEY UPDATE`. They are not interchangeable, and
|
||||
#: getting this wrong does not raise — it either fails to update the existing
|
||||
#: row or refuses to insert, so the setting silently keeps its old value.
|
||||
UPSERT_SQLITE = """
|
||||
INSERT INTO extraction_settings (singleton, window_preset)
|
||||
VALUES (1, ?)
|
||||
ON CONFLICT(singleton) DO UPDATE SET window_preset = excluded.window_preset
|
||||
"""
|
||||
UPSERT_MYSQL = """
|
||||
INSERT INTO extraction_settings (singleton, window_preset)
|
||||
VALUES (1, %s)
|
||||
ON DUPLICATE KEY UPDATE window_preset = VALUES(window_preset)
|
||||
"""
|
||||
|
||||
|
||||
class ExtractionSettings(BaseModel):
|
||||
@@ -21,34 +81,111 @@ class ExtractionSettings(BaseModel):
|
||||
window_preset: ExtractionWindowPreset = ExtractionWindowPreset.THREE_MONTHS
|
||||
|
||||
|
||||
class _SyncCursor:
|
||||
"""A synchronous `.execute()` over an async MySQL connection.
|
||||
|
||||
Exists so the calling code does not have to know which engine it is on:
|
||||
both engines expose `execute(sql, params)` returning something with
|
||||
`fetchone()`/`fetchall()`. Without it every call site would need two
|
||||
branches, which is the duplication this migration is trying to remove.
|
||||
|
||||
Statements run on the facade's loop via `run_coroutine_threadsafe`, so the
|
||||
caller stays synchronous while the MySQL I/O happens off the main loop.
|
||||
"""
|
||||
|
||||
def __init__(self, connection: Any, loop: asyncio.AbstractEventLoop) -> None:
|
||||
self._connection = connection
|
||||
self._loop = loop
|
||||
|
||||
def execute(self, sql: str, params: tuple[Any, ...] = ()) -> _SyncResult:
|
||||
async def _run() -> Any:
|
||||
async with self._connection.cursor() as cursor:
|
||||
await cursor.execute(sql, params)
|
||||
try:
|
||||
return await cursor.fetchall()
|
||||
except Exception:
|
||||
return None
|
||||
return None
|
||||
|
||||
rows = asyncio.run_coroutine_threadsafe(_run(), self._loop).result(timeout=30)
|
||||
return _SyncResult(rows)
|
||||
|
||||
|
||||
class _SyncResult:
|
||||
"""The small surface the call sites use: `fetchone` / `fetchall`."""
|
||||
|
||||
def __init__(self, rows: Any) -> None:
|
||||
self._rows = rows or []
|
||||
|
||||
def fetchone(self) -> Any:
|
||||
return self._rows[0] if self._rows else None
|
||||
|
||||
def fetchall(self) -> Any:
|
||||
return self._rows
|
||||
|
||||
|
||||
class ExtractionSettingsStore:
|
||||
def __init__(self, database_path: Path) -> None:
|
||||
self.database_path = database_path
|
||||
self._config = ConfigStore(database_path.parent)
|
||||
self._adopt_legacy_value()
|
||||
self._store: Any = store_for(database_path)
|
||||
self._initialize()
|
||||
|
||||
def _initialize(self) -> None:
|
||||
"""Create the table through the store, not a raw `executescript`.
|
||||
|
||||
`executescript` is a sqlite3-only convenience; the MySQL path has no
|
||||
equivalent, so the schema goes through the ordinary checkout instead.
|
||||
"""
|
||||
with self._connect(initialize=True) as connection:
|
||||
connection.execute(SCHEMA)
|
||||
|
||||
def _upsert_sql(self) -> str:
|
||||
return UPSERT_MYSQL if _is_mysql(self._store) else UPSERT_SQLITE
|
||||
|
||||
def get(self) -> ExtractionSettings:
|
||||
return ExtractionSettings(window_preset=self._config.extraction_window_preset())
|
||||
with self._connect(initialize=True) as connection:
|
||||
row = connection.execute(
|
||||
"SELECT window_preset FROM extraction_settings WHERE singleton = 1"
|
||||
).fetchone()
|
||||
if row is None:
|
||||
return ExtractionSettings()
|
||||
raw = _value(row, 0)
|
||||
return ExtractionSettings(window_preset=ExtractionWindowPreset(str(raw)))
|
||||
|
||||
def set_window_preset(self, preset: ExtractionWindowPreset) -> ExtractionSettings:
|
||||
self._config.set_extraction_window_preset(preset)
|
||||
with self._connect(writing=True) as connection:
|
||||
connection.execute(self._upsert_sql(), (preset.value,))
|
||||
return self.get()
|
||||
|
||||
def _adopt_legacy_value(self) -> None:
|
||||
"""Move the old singleton into config without creating its table."""
|
||||
if not self.database_path.exists():
|
||||
@contextmanager
|
||||
def _connect(
|
||||
self,
|
||||
*,
|
||||
initialize: bool = False,
|
||||
writing: bool = False,
|
||||
) -> Iterator[Any]:
|
||||
"""A checkout that yields something with `.execute()` on both engines.
|
||||
|
||||
On SQLite this is the existing behaviour, including the write lock and
|
||||
`sqlite3.Row`. On MySQL the facade hands back a connection whose
|
||||
`execute` is async, so statements are routed to the facade's loop and
|
||||
the caller still sees a synchronous `.execute(...)`.
|
||||
"""
|
||||
if _is_mysql(self._store):
|
||||
checkout = self._store.write() if (initialize or writing) else self._store.read()
|
||||
with checkout as connection:
|
||||
yield _SyncCursor(connection, self._store._loop)
|
||||
return
|
||||
try:
|
||||
with main_database(self.database_path).read() as connection:
|
||||
row = connection.execute(
|
||||
"SELECT window_preset FROM extraction_settings WHERE singleton=1"
|
||||
).fetchone()
|
||||
except sqlite3.DatabaseError:
|
||||
return
|
||||
if row is None:
|
||||
return
|
||||
try:
|
||||
preset = ExtractionWindowPreset(str(row[0]))
|
||||
except ValueError:
|
||||
return
|
||||
self._config.migrate_extraction_window_preset(preset)
|
||||
|
||||
database = main_database(self.database_path)
|
||||
checkout = (
|
||||
database.write(autocommit=True, row_factory=sqlite3.Row)
|
||||
if initialize or writing
|
||||
else database.read(row_factory=sqlite3.Row)
|
||||
)
|
||||
with checkout as connection, connection:
|
||||
if writing:
|
||||
take_the_write_lock_first(connection)
|
||||
else:
|
||||
connection.autocommit = False
|
||||
yield connection
|
||||
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,72 @@
|
||||
"""Whether HY still holds a memory, read so that a failed read is never taken for an absence.
|
||||
|
||||
The pinned HY SDK turns read failures into "not found" at every level a
|
||||
caller would normally use: `get` drops a failed graph query, Chroma's
|
||||
`get_by_id` drops any error, and Kuzu's `get_node` answers None while the
|
||||
graph is unavailable. A deletion confirmed through them can be confirmed by a
|
||||
store that was never read. So each store is asked here one level lower, where
|
||||
a failure still raises, without changing the SDK.
|
||||
"""
|
||||
|
||||
from enum import StrEnum
|
||||
from importlib import import_module
|
||||
from typing import Any
|
||||
|
||||
|
||||
class HyPresence(StrEnum):
|
||||
PRESENT = "present"
|
||||
ABSENT = "absent"
|
||||
UNKNOWN = "unknown"
|
||||
|
||||
|
||||
def confirm_deleted(client: object, memory_id: str) -> HyPresence:
|
||||
"""PRESENT if either store still has it; ABSENT only if both were read and have none."""
|
||||
native: Any = client
|
||||
answers = (_in_vectors(native, memory_id), _in_graph(native, memory_id))
|
||||
if HyPresence.PRESENT in answers:
|
||||
return HyPresence.PRESENT
|
||||
if HyPresence.UNKNOWN in answers:
|
||||
return HyPresence.UNKNOWN
|
||||
return HyPresence.ABSENT
|
||||
|
||||
|
||||
def _in_vectors(native: Any, memory_id: str) -> HyPresence:
|
||||
chroma = import_module("hy_memory.data.vector_store_chroma")
|
||||
store = native._vector_store
|
||||
if not isinstance(store, chroma.ChromaVectorStore) or store._collection is None:
|
||||
return HyPresence.UNKNOWN
|
||||
point_id = store._node_id_to_point_id(memory_id)
|
||||
|
||||
async def read() -> Any:
|
||||
return await chroma._run_in_vdb_pool(store._collection.get, ids=[point_id], include=[])
|
||||
|
||||
try:
|
||||
found = native._loop_thread.run(read())
|
||||
except Exception:
|
||||
return HyPresence.UNKNOWN
|
||||
return HyPresence.PRESENT if found["ids"] else HyPresence.ABSENT
|
||||
|
||||
|
||||
def _in_graph(native: Any, memory_id: str) -> HyPresence:
|
||||
store = native._graph_store
|
||||
if store is None:
|
||||
# HY was set up without a graph: there is nowhere else for it to hold one.
|
||||
return HyPresence.ABSENT
|
||||
kuzu = import_module("hy_memory.data.graph_store_kuzu")
|
||||
if not isinstance(store, kuzu.KuzuGraphStore) or not store._available or store._conn is None:
|
||||
return HyPresence.UNKNOWN
|
||||
|
||||
def count() -> int:
|
||||
result = store._conn.execute(
|
||||
"MATCH (m:Memory {node_id: $nid}) RETURN count(m);", {"nid": memory_id}
|
||||
)
|
||||
return int(result.get_next()[0])
|
||||
|
||||
async def read() -> int:
|
||||
return int(await kuzu._run_in_kuzu_pool(count))
|
||||
|
||||
try:
|
||||
held = native._loop_thread.run(read())
|
||||
except Exception:
|
||||
return HyPresence.UNKNOWN
|
||||
return HyPresence.PRESENT if held else HyPresence.ABSENT
|
||||
@@ -200,16 +200,6 @@ class HyReconcileGuard:
|
||||
details={"result_count": len(result.ops)},
|
||||
)
|
||||
return result
|
||||
try:
|
||||
await self._check_targets(result, kwargs)
|
||||
except ValueError as error:
|
||||
code = str(error)
|
||||
self.failures.append(code)
|
||||
raise
|
||||
except Exception as error:
|
||||
code = "hy_memory_write_verification_failed"
|
||||
self.failures.append(code)
|
||||
raise ValueError(code) from error
|
||||
owners: dict[str, set[str]] = {}
|
||||
for item in kwargs.get("new_memories_with_meta") or []:
|
||||
if item.get("owner") in {"user", "agent"}:
|
||||
|
||||
@@ -14,7 +14,7 @@ from pathlib import Path
|
||||
from typing import Literal, cast
|
||||
|
||||
HY_MEMORY_DISTRIBUTION = "hy-memory"
|
||||
HY_MEMORY_VERSION = "2.0.5.dev3"
|
||||
HY_MEMORY_VERSION = "2.0.3"
|
||||
HY_MEMORY_NAMESPACE = "governed-v2"
|
||||
DiagnosticLogLevel = Literal["error", "warning", "info", "debug"]
|
||||
|
||||
@@ -31,12 +31,11 @@ _COLLECTION_LOGGERS = (
|
||||
"httpx",
|
||||
"httpcore",
|
||||
)
|
||||
_NOISY_LIBRARY_LOGGERS = ("httpx", "httpcore", "watchfiles", "jieba")
|
||||
|
||||
# One Memory deliberately stays on ultra because its existing, user-visible
|
||||
# profile/graph flow calls digest(), which the pinned public SDK exposes only
|
||||
# for ultra clients. Reader and non-reranker behavior remain pinned for
|
||||
# compatibility with the SDK authority-migration development preview.
|
||||
# compatibility with public SDK 2.0.3.
|
||||
HY_MEMORY_MODE = "ultra"
|
||||
# The SDK only recognises `legacy` and `hybrid` (see
|
||||
# hy_memory/pipelines/_retrieval/config.py: `ALL_READERS = (READER_LEGACY,
|
||||
@@ -140,15 +139,6 @@ def apply_diagnostic_log_level(level: DiagnosticLogLevel) -> None:
|
||||
for name in tuple(logging.Logger.manager.loggerDict):
|
||||
if any(name == root or name.startswith(f"{root}.") for root in _COLLECTION_LOGGERS):
|
||||
logging.getLogger(name).setLevel(selected)
|
||||
# Successful HTTP exchanges, file-watch notifications and tokenizer
|
||||
# initialization are useful only while actively debugging. At INFO they
|
||||
# dominated core.log and the in-app panel without describing product work.
|
||||
noisy_level = selected if selected == logging.DEBUG else max(selected, logging.WARNING)
|
||||
for root in _NOISY_LIBRARY_LOGGERS:
|
||||
logging.getLogger(root).setLevel(noisy_level)
|
||||
for name in tuple(logging.Logger.manager.loggerDict):
|
||||
if any(name == root or name.startswith(f"{root}.") for root in _NOISY_LIBRARY_LOGGERS):
|
||||
logging.getLogger(name).setLevel(noisy_level)
|
||||
|
||||
os.environ["MEMORY_LOG_LEVEL"] = logging.getLevelName(selected)
|
||||
os.environ["MEMORY_PIPELINE_TRACE_ENABLED"] = "true" if level == "debug" else "false"
|
||||
|
||||
@@ -7,7 +7,7 @@ from typing import Any, Literal
|
||||
|
||||
from pydantic import BaseModel, ConfigDict
|
||||
|
||||
from one_memory.imports.models import EvidenceContentKind, EvidenceRecord
|
||||
from one_memory.imports.models import EvidenceContentKind, EvidenceRecord, WrittenBy
|
||||
from one_memory.input_sanitization import sanitize_model_input
|
||||
|
||||
|
||||
@@ -61,7 +61,17 @@ class HyWriteBindings:
|
||||
seed.workspace,
|
||||
)
|
||||
or record.content_kind is not EvidenceContentKind.RAW_CONVERSATION
|
||||
or record.role not in {"user", "assistant", "tool"}
|
||||
or (record.role, record.written_by)
|
||||
not in {
|
||||
# D98: the same change as the turn assembly. A sub agent's
|
||||
# question is a `user` message marked written by the
|
||||
# agent; excluding it here would make the source context
|
||||
# disagree with what was actually sent.
|
||||
("user", WrittenBy.PERSON),
|
||||
("user", WrittenBy.AGENT),
|
||||
("assistant", WrittenBy.AGENT),
|
||||
("tool", WrittenBy.TOOL),
|
||||
}
|
||||
):
|
||||
raise ValueError("hy_memory_write_verification_context_scope")
|
||||
self.all_sources: list[dict[str, Any]] = [
|
||||
|
||||
@@ -152,7 +152,7 @@ class HyStepTimer:
|
||||
if value is None:
|
||||
continue
|
||||
diagnostic_event(
|
||||
level="debug",
|
||||
level="info",
|
||||
component=STEP_COMPONENTS[name],
|
||||
event="hy.step.completed",
|
||||
outcome="succeeded",
|
||||
@@ -168,7 +168,7 @@ class HyStepTimer:
|
||||
)
|
||||
model = measured.get("model")
|
||||
diagnostic_event(
|
||||
level="debug",
|
||||
level="info",
|
||||
component="hy.bridge",
|
||||
event="hy.steps.summary",
|
||||
outcome="succeeded",
|
||||
|
||||
@@ -0,0 +1,41 @@
|
||||
"""Process-local coordination for HY's native stores."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import weakref
|
||||
from collections.abc import AsyncIterator
|
||||
from contextlib import asynccontextmanager
|
||||
|
||||
_WRITE_LOCKS: weakref.WeakKeyDictionary[asyncio.AbstractEventLoop, asyncio.Lock] = (
|
||||
weakref.WeakKeyDictionary()
|
||||
)
|
||||
|
||||
|
||||
def native_store_write_lock() -> asyncio.Lock:
|
||||
"""Return the write lock for the current event loop."""
|
||||
loop = asyncio.get_running_loop()
|
||||
lock = _WRITE_LOCKS.get(loop)
|
||||
if lock is None:
|
||||
lock = asyncio.Lock()
|
||||
_WRITE_LOCKS[loop] = lock
|
||||
return lock
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
async def native_store_read_slot() -> AsyncIterator[bool]:
|
||||
"""Admit a native HY read only while no write is active.
|
||||
|
||||
Chroma's native reader may retain the GIL after an asyncio timeout. Reads
|
||||
therefore share the physical-store lock, but never wait behind a writer:
|
||||
host recall can immediately fall back to the governance ledger.
|
||||
"""
|
||||
lock = native_store_write_lock()
|
||||
if lock.locked():
|
||||
yield False
|
||||
return
|
||||
await lock.acquire()
|
||||
try:
|
||||
yield True
|
||||
finally:
|
||||
lock.release()
|
||||
@@ -449,11 +449,8 @@ def safe_project_label(value: str | None) -> str | None:
|
||||
def opaque_workspace_id(source_id: str, workspace: str | None) -> str | None:
|
||||
if not workspace:
|
||||
return None
|
||||
if len(workspace) > 4_096 or any(ord(character) < 32 for character in workspace):
|
||||
return None
|
||||
# Workspace is an opaque identity, not a filesystem target. Hash the
|
||||
# caller's exact value without expansion, resolution, stat, open, or join.
|
||||
digest = hashlib.sha256(f"{source_id}\0{workspace}".encode()).hexdigest()
|
||||
normalized = str(Path(workspace).expanduser())
|
||||
digest = hashlib.sha256(f"{source_id}\0{normalized}".encode()).hexdigest()
|
||||
return f"workspace:v1:{digest}"
|
||||
|
||||
|
||||
|
||||
@@ -34,11 +34,10 @@ class ScanIssueCode(StrEnum):
|
||||
SOURCE_BUSY = "source_busy"
|
||||
UNSUPPORTED_SCHEMA = "unsupported_schema"
|
||||
BUDGET_EXCEEDED = "budget_exceeded"
|
||||
#: The file is bigger than a per-file ceiling: either the import's own
|
||||
#: ceiling or the access gate's per-root `max_file_bytes`. Both are skipped
|
||||
#: with this reason rather than read, because one oversized history file
|
||||
#: can hold the import open long enough to take the process with it, and
|
||||
#: the user-facing cause is the same size limit.
|
||||
#: The file is bigger than the import's own per-file ceiling. Skipped with
|
||||
#: a reason rather than read, because one oversized history file can hold
|
||||
#: the import open long enough to take the process with it. Distinct from
|
||||
#: the access layer's `FILE_TOO_LARGE`, which is a per-root limit.
|
||||
FILE_TOO_LARGE = "file_too_large"
|
||||
HISTORY_WINDOW_EXCLUDED = "history_window_excluded"
|
||||
|
||||
|
||||
@@ -105,7 +105,7 @@ async def iterate_in_worker[T](
|
||||
# was unwinding with, and a diagnostic is not worth an import.
|
||||
try:
|
||||
diagnostic_event(
|
||||
level="debug",
|
||||
level="info",
|
||||
component="import.async_bridge",
|
||||
event="import.source_read_finished",
|
||||
outcome="succeeded",
|
||||
|
||||
@@ -1,7 +1,6 @@
|
||||
import sqlite3
|
||||
from contextlib import AbstractContextManager
|
||||
from datetime import UTC, datetime
|
||||
from enum import StrEnum
|
||||
from pathlib import Path
|
||||
|
||||
from pydantic import BaseModel, ConfigDict
|
||||
@@ -21,23 +20,6 @@ CURSOR_MAX_SOURCE_FILE_BYTES = 64 * 1024 * 1024 * 1024
|
||||
GRANT_SCHEMA = source_grants_ddl("sqlite")
|
||||
|
||||
|
||||
class SourceAuthorizationState(StrEnum):
|
||||
"""Why a source cannot be read right now, or `ok` when it can.
|
||||
|
||||
The API used to answer `authorized=false` for all of these at once, which
|
||||
is how a revoked grant, a rebuilt history directory and a moved candidate
|
||||
all looked like the same bug and all got the same advice. The values carry
|
||||
no paths and no user content, because they are written to diagnostics.
|
||||
"""
|
||||
|
||||
OK = "ok"
|
||||
MISSING_GRANT = "missing_grant"
|
||||
REVOKED_GRANT = "revoked_grant"
|
||||
SOURCE_IDENTITY_CHANGED = "source_identity_changed"
|
||||
SOURCE_ROOT_UNAVAILABLE = "source_root_unavailable"
|
||||
CANDIDATE_CHANGED = "candidate_changed"
|
||||
|
||||
|
||||
class SourceGrantReceipt(BaseModel):
|
||||
model_config = ConfigDict(frozen=True, extra="forbid")
|
||||
|
||||
@@ -164,46 +146,6 @@ class SourceGrantStore:
|
||||
raise SourceAccessError(SourceAccessErrorCode.SOURCE_IDENTITY_CHANGED)
|
||||
return authorized
|
||||
|
||||
def authorization_state(
|
||||
self,
|
||||
*,
|
||||
user_id: str,
|
||||
source_id: str,
|
||||
candidate: Path,
|
||||
) -> SourceAuthorizationState:
|
||||
"""Why reading this source would be refused.
|
||||
|
||||
Read-only and fails the same way `authorized_root` does: this names a
|
||||
state for the UI and for diagnostics, it does not widen access. The
|
||||
candidate is compared by path, so a grant that still points somewhere
|
||||
else reads as a moved candidate rather than as a rebuilt directory —
|
||||
the two need different fixes.
|
||||
"""
|
||||
with self._connect() as connection:
|
||||
row = connection.execute(
|
||||
"SELECT * FROM source_grants WHERE user_id=? AND source_id=?",
|
||||
(user_id, source_id),
|
||||
).fetchone()
|
||||
if row is None:
|
||||
return SourceAuthorizationState.MISSING_GRANT
|
||||
if row["revoked_at"] is not None or row["revision"] <= 0:
|
||||
return SourceAuthorizationState.REVOKED_GRANT
|
||||
granted = Path(row["requested_path"])
|
||||
if not granted.exists():
|
||||
return SourceAuthorizationState.SOURCE_ROOT_UNAVAILABLE
|
||||
try:
|
||||
if not root_identity_matches(granted, row["root_identity"]):
|
||||
return SourceAuthorizationState.SOURCE_IDENTITY_CHANGED
|
||||
except SourceAccessError:
|
||||
# Unreadable or gone between the two calls: there is no grant to
|
||||
# repair, only a directory to make available again.
|
||||
return SourceAuthorizationState.SOURCE_ROOT_UNAVAILABLE
|
||||
# Same directory, but the app now looks somewhere else. Re-authorizing
|
||||
# the old grant would not help; the candidate has to be granted.
|
||||
if granted != candidate:
|
||||
return SourceAuthorizationState.CANDIDATE_CHANGED
|
||||
return SourceAuthorizationState.OK
|
||||
|
||||
def revoke(self, *, user_id: str, source_id: str) -> SourceGrantReceipt:
|
||||
now = datetime.now(UTC).isoformat()
|
||||
with self._connect(writing=True) as connection:
|
||||
|
||||
@@ -106,7 +106,7 @@ class PollingLiveSyncSupervisor:
|
||||
level=(
|
||||
"warn"
|
||||
if value.state in {LiveSyncState.DEGRADED, LiveSyncState.UNAVAILABLE}
|
||||
else "debug"
|
||||
else "info"
|
||||
),
|
||||
component="import.live_sync",
|
||||
event="live_sync.state_changed",
|
||||
@@ -282,24 +282,11 @@ class PollingLiveSyncSupervisor:
|
||||
and job.status
|
||||
in {ImportStatus.PENDING, ImportStatus.RUNNING, ImportStatus.INTERRUPTED}
|
||||
]
|
||||
bounded_history_incomplete = any(
|
||||
job.source_id == source_id
|
||||
and job.operation == "history"
|
||||
and job.history_after is not None
|
||||
and job.status is not ImportStatus.COMPLETED
|
||||
for job in source_jobs
|
||||
)
|
||||
# Establish the realtime EOF independently even while history owns
|
||||
# this source. Otherwise the P1 job starts from the P2 checkpoint
|
||||
# (or from zero) and becomes a second full-history scan.
|
||||
adapter = self.import_supervisor.adapters[source_id]
|
||||
live_cursors = await self._live_cursors(source_id, adapter)
|
||||
# A bounded onboarding history is the authority for everything up
|
||||
# to its EOF. An adapter without a native live baseline has no safe
|
||||
# realtime cursor until that history finishes; starting a sync from
|
||||
# zero here would bypass `history_after` and import the old corpus.
|
||||
if bounded_history_incomplete and not live_cursors:
|
||||
return None
|
||||
await self._live_cursors(source_id, adapter)
|
||||
# Only one scan of a source may be in flight: they share the
|
||||
# cursor, and two readers means every file read twice. With a
|
||||
# catch-up still holding the source there is no free moment to ask
|
||||
|
||||
@@ -156,7 +156,7 @@ class FileWatchLiveSync:
|
||||
if source is not None
|
||||
):
|
||||
diagnostic_event(
|
||||
level="debug",
|
||||
level="info",
|
||||
component="import.live_watch",
|
||||
event="live_sync.hook_received",
|
||||
outcome="queued",
|
||||
@@ -190,7 +190,7 @@ class FileWatchLiveSync:
|
||||
try:
|
||||
await self._trigger_past(source_id, observed)
|
||||
diagnostic_event(
|
||||
level="debug",
|
||||
level="info",
|
||||
component="import.live_watch",
|
||||
event="live_sync.hook_completed",
|
||||
outcome="written",
|
||||
|
||||
@@ -2,7 +2,7 @@ import hashlib
|
||||
import json
|
||||
import re
|
||||
import unicodedata
|
||||
from datetime import UTC, datetime
|
||||
from datetime import datetime
|
||||
from enum import StrEnum
|
||||
from typing import Annotated
|
||||
from uuid import UUID, uuid5
|
||||
@@ -79,8 +79,6 @@ class EvidenceInput(BaseModel):
|
||||
source_ordinal: str | None = Field(default=None, min_length=1, max_length=256)
|
||||
content_kind: EvidenceContentKind = EvidenceContentKind.RAW_CONVERSATION
|
||||
role: str = Field(min_length=1, max_length=128)
|
||||
# Legacy import/storage metadata only. Protocol `role` is authoritative;
|
||||
# this value must not affect identity, admission, scheduling or ownership.
|
||||
written_by: WrittenBy
|
||||
created_at: datetime | None = None
|
||||
verbatim: str = Field(min_length=1, max_length=MAX_VERBATIM_CHARACTERS)
|
||||
@@ -151,22 +149,17 @@ class EvidenceInput(BaseModel):
|
||||
)
|
||||
)
|
||||
else:
|
||||
# `role` is the protocol-level speaker identity. `written_by` was a
|
||||
# parser guess and differed between two imports of the same host
|
||||
# message (notably sub-agent/user and unknown/user), minting two
|
||||
# Evidence rows. Keep it out of every new identity.
|
||||
#
|
||||
# No workspace and no source_ordinal in the derived key: the hook
|
||||
# lane has neither (it reports no workspace and a constant ordinal),
|
||||
# so including either gave the same message a different key per lane
|
||||
# and stored it twice. Both are record metadata here, not identity.
|
||||
identity = "\0".join(
|
||||
(
|
||||
"derived-v4",
|
||||
"derived-v3",
|
||||
self.user_id,
|
||||
self.source_id,
|
||||
self.conversation_id,
|
||||
self.role,
|
||||
self.written_by,
|
||||
self.content_hash,
|
||||
)
|
||||
)
|
||||
@@ -175,54 +168,6 @@ class EvidenceInput(BaseModel):
|
||||
# two rows. The capture time is record metadata, not identity.
|
||||
return hashlib.sha256(identity.encode()).hexdigest()
|
||||
|
||||
@property
|
||||
def written_by_derived_v3_idempotency_key(self) -> str:
|
||||
"""The last identity that treated the parser's author guess as canonical.
|
||||
|
||||
New builds key by protocol role. During an upgrade this key lets a
|
||||
rescan find and reuse the row already stored by derived-v3 instead of
|
||||
inserting the same host message under a second Evidence id.
|
||||
"""
|
||||
identity = "\0".join(
|
||||
(
|
||||
"derived-v3",
|
||||
self.user_id,
|
||||
self.source_id,
|
||||
self.conversation_id,
|
||||
self.written_by,
|
||||
self.content_hash,
|
||||
)
|
||||
)
|
||||
return hashlib.sha256(identity.encode()).hexdigest()
|
||||
|
||||
@property
|
||||
def written_by_derived_v3_idempotency_keys(self) -> tuple[str, ...]:
|
||||
"""Every legacy author guess this protocol role may have carried."""
|
||||
guesses = {
|
||||
"user": (
|
||||
WrittenBy.PERSON,
|
||||
WrittenBy.AGENT,
|
||||
WrittenBy.UNKNOWN,
|
||||
),
|
||||
"assistant": (WrittenBy.AGENT, WrittenBy.UNKNOWN, WrittenBy.PERSON),
|
||||
"tool": (WrittenBy.TOOL, WrittenBy.UNKNOWN),
|
||||
}.get(self.role, tuple(WrittenBy))
|
||||
ordered = (self.written_by, *guesses)
|
||||
keys: list[str] = []
|
||||
for written_by in dict.fromkeys(ordered):
|
||||
identity = "\0".join(
|
||||
(
|
||||
"derived-v3",
|
||||
self.user_id,
|
||||
self.source_id,
|
||||
self.conversation_id,
|
||||
written_by,
|
||||
self.content_hash,
|
||||
)
|
||||
)
|
||||
keys.append(hashlib.sha256(identity.encode()).hexdigest())
|
||||
return tuple(keys)
|
||||
|
||||
@property
|
||||
def legacy_message_idempotency_key(self) -> str | None:
|
||||
"""The pre-workspace key for a message-shaped input, when one exists.
|
||||
@@ -267,13 +212,7 @@ class EvidenceInput(BaseModel):
|
||||
"""
|
||||
return self.derived_v2_idempotency_key("derived-v2", workspace_agnostic=False)
|
||||
|
||||
def derived_v2_idempotency_key(
|
||||
self,
|
||||
label: str,
|
||||
*,
|
||||
workspace_agnostic: bool,
|
||||
written_by: WrittenBy | None = None,
|
||||
) -> str:
|
||||
def derived_v2_idempotency_key(self, label: str, *, workspace_agnostic: bool) -> str:
|
||||
identity = "\0".join(
|
||||
(
|
||||
label,
|
||||
@@ -281,7 +220,7 @@ class EvidenceInput(BaseModel):
|
||||
self.source_id,
|
||||
*([] if workspace_agnostic else [self.workspace or "\uffffnull"]),
|
||||
self.conversation_id,
|
||||
written_by or self.written_by,
|
||||
self.written_by,
|
||||
self.created_at.isoformat() if self.created_at else "",
|
||||
self.source_ordinal or "",
|
||||
# v2 hashed the cleaned body; v1 predates raw_verbatim entirely,
|
||||
@@ -291,26 +230,6 @@ class EvidenceInput(BaseModel):
|
||||
)
|
||||
return hashlib.sha256(identity.encode()).hexdigest()
|
||||
|
||||
@property
|
||||
def older_written_by_idempotency_keys(self) -> tuple[str, ...]:
|
||||
keys: list[str] = []
|
||||
for written_by in WrittenBy:
|
||||
keys.append(
|
||||
self.derived_v2_idempotency_key(
|
||||
"derived-v1",
|
||||
workspace_agnostic=True,
|
||||
written_by=written_by,
|
||||
)
|
||||
)
|
||||
keys.append(
|
||||
self.derived_v2_idempotency_key(
|
||||
"derived-v2",
|
||||
workspace_agnostic=False,
|
||||
written_by=written_by,
|
||||
)
|
||||
)
|
||||
return tuple(keys)
|
||||
|
||||
@property
|
||||
def evidence_id(self) -> str:
|
||||
return str(uuid5(EVIDENCE_NAMESPACE, self.idempotency_key))
|
||||
@@ -330,7 +249,6 @@ class EvidenceRecord(BaseModel):
|
||||
source_ordinal: str | None
|
||||
content_kind: EvidenceContentKind
|
||||
role: str
|
||||
# Retained while old databases and export formats still carry the column.
|
||||
written_by: WrittenBy
|
||||
created_at: datetime | None
|
||||
verbatim: str
|
||||
@@ -343,33 +261,3 @@ class EvidenceRecord(BaseModel):
|
||||
@property
|
||||
def origins(self) -> frozenset[EvidenceOrigin]:
|
||||
return frozenset(self.origin_meta)
|
||||
|
||||
|
||||
def materialize_write_input(
|
||||
value: EvidenceInput,
|
||||
*,
|
||||
observed_at: datetime | None = None,
|
||||
) -> EvidenceRecord:
|
||||
"""Freeze source input directly into a write-job payload, without Evidence SQL."""
|
||||
now = observed_at or datetime.now(UTC)
|
||||
return EvidenceRecord(
|
||||
evidence_id=value.evidence_id,
|
||||
user_id=value.user_id,
|
||||
source_id=value.source_id,
|
||||
workspace=value.workspace,
|
||||
project=value.project,
|
||||
conversation_id=value.conversation_id,
|
||||
message_id=value.message_id,
|
||||
correlation_id=value.correlation_id,
|
||||
source_ordinal=value.source_ordinal,
|
||||
content_kind=value.content_kind,
|
||||
role=value.role,
|
||||
written_by=value.written_by,
|
||||
created_at=value.created_at,
|
||||
verbatim=value.verbatim,
|
||||
origin_meta=value.origin_meta,
|
||||
content_hash=value.content_hash,
|
||||
revision=1,
|
||||
imported_at=now,
|
||||
updated_at=now,
|
||||
)
|
||||
@@ -37,9 +37,8 @@ from one_memory.imports.jobs import (
|
||||
ImportStatus,
|
||||
InvalidJobTransitionError,
|
||||
)
|
||||
from one_memory.imports.models import EvidenceRecord, materialize_write_input
|
||||
from one_memory.imports.models import EvidenceRecord
|
||||
from one_memory.imports.storage import EvidenceStore, PendingMemoryDispatch
|
||||
from one_memory.memory_server.durable_queue import DurableWriteQueue
|
||||
from one_memory.service_levels import ServiceLevel, service_level_context
|
||||
|
||||
_SKIP_BATCH_SECONDS = 0.1
|
||||
@@ -167,8 +166,6 @@ class ImportRunner:
|
||||
on_evidence: Callable[[EvidenceRecord], object] | None = None,
|
||||
on_stable_evidence: Callable[[EvidenceRecord], object] | None = None,
|
||||
heartbeat_interval_seconds: float = 10.0,
|
||||
sdk_queue: DurableWriteQueue | None = None,
|
||||
sdk_authority: Callable[[], bool] | None = None,
|
||||
) -> None:
|
||||
if evidence_store.database_path.resolve() != job_store.database_path.resolve():
|
||||
raise ValueError("Evidence and import jobs must share one database")
|
||||
@@ -178,8 +175,6 @@ class ImportRunner:
|
||||
self.on_evidence = on_evidence
|
||||
self.on_stable_evidence = on_stable_evidence
|
||||
self._live_import = False
|
||||
self.sdk_queue = sdk_queue
|
||||
self.sdk_authority = sdk_authority or (lambda: False)
|
||||
|
||||
async def run(
|
||||
self,
|
||||
@@ -204,7 +199,7 @@ class ImportRunner:
|
||||
raise ValueError("Import job source does not match adapter source")
|
||||
job, runner_token = await in_the_background(self.job_store.claim, job_id)
|
||||
diagnostic_event(
|
||||
level="debug",
|
||||
level="info",
|
||||
component="import.parser",
|
||||
event="import.parse.started",
|
||||
outcome="started",
|
||||
@@ -315,7 +310,7 @@ class ImportRunner:
|
||||
if event.kind == "progress" and str(event.phase) != last_phase:
|
||||
phase = str(event.phase)
|
||||
diagnostic_event(
|
||||
level="debug",
|
||||
level="info",
|
||||
component="import.parser",
|
||||
event="import.phase_changed",
|
||||
outcome="changed",
|
||||
@@ -521,7 +516,7 @@ class ImportRunner:
|
||||
async def _emit_parse_result(self, job: ImportJob, started: float) -> None:
|
||||
succeeded = job.status is ImportStatus.COMPLETED
|
||||
diagnostic_event(
|
||||
level="debug" if succeeded else "warn",
|
||||
level="info" if succeeded else "warn",
|
||||
component="import.parser",
|
||||
event="import.parse.completed",
|
||||
outcome="succeeded" if succeeded else str(job.status),
|
||||
@@ -657,18 +652,6 @@ class ImportRunner:
|
||||
count_skips()
|
||||
if isinstance(event, ScanCursorTouch):
|
||||
self.job_store._upsert_cursor(connection, row, event.cursor, now)
|
||||
if (
|
||||
self.sdk_authority()
|
||||
and self.sdk_queue is not None
|
||||
and event.cursor.read_to_end
|
||||
):
|
||||
self.sdk_queue.finalize_sdk_file(
|
||||
user_id=current.user_id,
|
||||
source_id=current.source_id,
|
||||
file_key=event.cursor.file_key,
|
||||
connection=connection,
|
||||
)
|
||||
continue
|
||||
if self.on_stable_evidence is not None and event.cursor.read_to_end:
|
||||
# Read without consuming: the durable submit runs after
|
||||
# this transaction closes, so the pending marker must
|
||||
@@ -686,23 +669,6 @@ class ImportRunner:
|
||||
(current.user_id, current.source_id, event.cursor.file_key)
|
||||
)
|
||||
continue
|
||||
if self.sdk_authority() and self.sdk_queue is not None:
|
||||
record = materialize_write_input(event.evidence)
|
||||
self.sdk_queue.stage_sdk_input(
|
||||
record,
|
||||
file_key=event.cursor.file_key,
|
||||
connection=connection,
|
||||
priority="live" if self._live_import else "normal",
|
||||
)
|
||||
self.job_store.record_evidence(
|
||||
job_id,
|
||||
runner_token=runner_token,
|
||||
cursor=event.cursor,
|
||||
was_created=True,
|
||||
was_revised=False,
|
||||
connection=connection,
|
||||
)
|
||||
continue
|
||||
result = self.evidence_store.put(event.evidence, connection=connection)
|
||||
self.job_store.record_evidence(
|
||||
job_id,
|
||||
|
||||
@@ -141,18 +141,11 @@ def parse_codex_record(
|
||||
if role == "user":
|
||||
written_by = _user_author(record, payload, session)
|
||||
# Machine inputs may quote a whole transcript, including user_query tags.
|
||||
# Protocol role is authoritative; host author metadata does not change
|
||||
# how a user message is cleaned.
|
||||
if not (
|
||||
content.startswith(
|
||||
"The following is the Codex agent history whose request action you are assessing."
|
||||
)
|
||||
and ">>> TRANSCRIPT START" in content
|
||||
and ">>> TRANSCRIPT END" in content
|
||||
):
|
||||
# Session provenance is not proof of who wrote any particular future turn.
|
||||
if written_by is WrittenBy.PERSON:
|
||||
content = strip_host_wrappers(content)
|
||||
if not content:
|
||||
return ParsedSkip(ScanIssueCode.NO_TEXT)
|
||||
if not content:
|
||||
return ParsedSkip(ScanIssueCode.NO_TEXT)
|
||||
return ParsedMessage(
|
||||
conversation_id=conversation_from_relative_path(context.relative_path),
|
||||
message_id=first_string(payload.get("id"), record.get("id")),
|
||||
|
||||
@@ -44,7 +44,6 @@ from one_memory.imports.discovery import (
|
||||
discovered_off_loop,
|
||||
)
|
||||
from one_memory.imports.models import (
|
||||
MAX_VERBATIM_CHARACTERS,
|
||||
EvidenceContentKind,
|
||||
EvidenceInput,
|
||||
EvidenceOrigin,
|
||||
@@ -66,21 +65,6 @@ from one_memory.imports.sources.parsing import parse_timestamp
|
||||
|
||||
_log = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def _clamp_verbatim(text: str) -> str:
|
||||
"""Keep one pathological record from violating EvidenceInput's limit.
|
||||
|
||||
A single pasted dump larger than MAX_VERBATIM_CHARACTERS used to fail
|
||||
validation, retry, and then wedge the whole history import on the same
|
||||
record. Keep the head and make the truncation explicit and actionable.
|
||||
"""
|
||||
if len(text) <= MAX_VERBATIM_CHARACTERS:
|
||||
return text
|
||||
marker = f"\n…[truncated: original {len(text)} chars]"
|
||||
keep = max(0, MAX_VERBATIM_CHARACTERS - len(marker))
|
||||
return text[:keep] + marker
|
||||
|
||||
|
||||
#: At most one read heartbeat per second. Faster would be more writes than the
|
||||
#: counts are worth; slower leaves the job's visible state stale.
|
||||
_READ_PROGRESS_INTERVAL_SECONDS = 1.0
|
||||
@@ -371,7 +355,7 @@ class JsonlSourceAdapter:
|
||||
source_file, checkpoint, header = await asyncio.to_thread(
|
||||
self._prepare, listed, options
|
||||
)
|
||||
_log.debug(
|
||||
_log.info(
|
||||
"import_file_read source=%s file=%d/%d file_bytes=%d planned_bytes=%d",
|
||||
self.descriptor.source_id,
|
||||
index,
|
||||
@@ -392,7 +376,7 @@ class JsonlSourceAdapter:
|
||||
else DEFAULT_IMPORT_MAX_FILE_BYTES
|
||||
)
|
||||
if source_file.size > budget:
|
||||
_log.debug(
|
||||
_log.info(
|
||||
"import_file_budgeted source=%s file=%d/%d file_bytes=%d budget=%d path=%s",
|
||||
self.descriptor.source_id,
|
||||
index,
|
||||
@@ -1092,10 +1076,8 @@ def to_evidence(
|
||||
role=parsed.role,
|
||||
written_by=parsed.written_by,
|
||||
created_at=parsed.created_at,
|
||||
verbatim=_clamp_verbatim(parsed.verbatim),
|
||||
raw_verbatim=(
|
||||
None if parsed.raw_verbatim is None else _clamp_verbatim(parsed.raw_verbatim)
|
||||
),
|
||||
verbatim=parsed.verbatim,
|
||||
raw_verbatim=parsed.raw_verbatim,
|
||||
origin_meta={
|
||||
EvidenceOrigin.FILE: OriginMetadata(
|
||||
source_version=origin.source_version or source_version,
|
||||
@@ -1120,11 +1102,6 @@ def discovery_failure_code(
|
||||
) -> ScanIssueCode:
|
||||
if failure.reason_code is SourceAccessErrorCode.BUDGET_EXCEEDED:
|
||||
return ScanIssueCode.BUDGET_EXCEEDED
|
||||
if failure.reason_code is SourceAccessErrorCode.FILE_TOO_LARGE:
|
||||
# The file is readable and its format is fine; only its size is over
|
||||
# the access gate. Reporting that as `unsupported_schema` sends the
|
||||
# user looking for a parser problem that does not exist.
|
||||
return ScanIssueCode.FILE_TOO_LARGE
|
||||
return ScanIssueCode.UNSUPPORTED_SCHEMA
|
||||
|
||||
|
||||
@@ -1132,8 +1109,6 @@ def _scan_failure_code(code: SourceAccessErrorCode) -> ScanIssueCode:
|
||||
"""Map a per-file open failure onto a machine code, never onto free text."""
|
||||
if code is SourceAccessErrorCode.BUDGET_EXCEEDED:
|
||||
return ScanIssueCode.BUDGET_EXCEEDED
|
||||
if code is SourceAccessErrorCode.FILE_TOO_LARGE:
|
||||
return ScanIssueCode.FILE_TOO_LARGE
|
||||
if code is SourceAccessErrorCode.SYMLINK_NOT_ALLOWED:
|
||||
return ScanIssueCode.UNSUPPORTED_SCHEMA
|
||||
if code in {
|
||||
|
||||
@@ -7,7 +7,7 @@ import sqlite3
|
||||
import stat
|
||||
from collections.abc import AsyncIterator, Iterator
|
||||
from dataclasses import dataclass, replace
|
||||
from datetime import UTC, datetime, timedelta, timezone
|
||||
from datetime import UTC, datetime, timedelta
|
||||
from functools import partial
|
||||
|
||||
from pydantic import JsonValue
|
||||
@@ -82,11 +82,6 @@ _CURSOR_HOST_CONTEXT_START = re.compile(
|
||||
)
|
||||
_CURSOR_USER_QUERY = re.compile(r"<user_query\b")
|
||||
_CURSOR_TIMESTAMP_PREFIX = re.compile(r"<timestamp>[\s\S]*?</timestamp>\s*")
|
||||
_CURSOR_TIMESTAMP_VALUE = re.compile(
|
||||
r"<timestamp>\s*"
|
||||
r"([A-Za-z]+,\s+[A-Za-z]{3}\s+\d{1,2},\s+\d{4},\s+\d{1,2}:\d{2}\s+[AP]M)"
|
||||
r"\s+\(UTC([+-]\d{1,2})(?::(\d{2}))?\)\s*</timestamp>",
|
||||
)
|
||||
_CURSOR_IMAGE_FILES_PREFIX = re.compile(r"\[Image\]\s*<image_files>[\s\S]*?</image_files>\s*")
|
||||
_CURSOR_BARE_IMAGE_PREFIX = re.compile(r"(?:\[Image\]\s*)+")
|
||||
|
||||
@@ -128,24 +123,6 @@ def _strip_cursor_timestamp(content: str | None) -> str | None:
|
||||
return _CURSOR_TIMESTAMP_PREFIX.sub("", content, count=1) if content else None
|
||||
|
||||
|
||||
def _cursor_host_timestamp(content: str | None) -> datetime | None:
|
||||
"""Parse Cursor's user-visible timestamp envelope into UTC."""
|
||||
if not content:
|
||||
return None
|
||||
match = _CURSOR_TIMESTAMP_VALUE.search(content)
|
||||
if match is None:
|
||||
return None
|
||||
try:
|
||||
local = datetime.strptime(match[1], "%A, %b %d, %Y, %I:%M %p")
|
||||
hours = int(match[2])
|
||||
minutes = int(match[3] or "0")
|
||||
sign = -1 if hours < 0 else 1
|
||||
offset = timedelta(hours=hours, minutes=sign * minutes)
|
||||
return local.replace(tzinfo=timezone(offset)).astimezone(UTC)
|
||||
except (ValueError, OverflowError):
|
||||
return None
|
||||
|
||||
|
||||
def _strip_cursor_host_prefix(content: str | None) -> str | None:
|
||||
if not content:
|
||||
return content
|
||||
@@ -421,7 +398,10 @@ class CursorUnifiedSourceAdapter:
|
||||
if (
|
||||
isinstance(event, ScannedEvidence)
|
||||
and event.evidence.conversation_id in transcript_ids
|
||||
and not is_host_generated_user_turn(event.evidence.verbatim)
|
||||
and not (
|
||||
event.evidence.written_by is WrittenBy.AGENT
|
||||
and is_host_generated_user_turn(event.evidence.verbatim)
|
||||
)
|
||||
):
|
||||
yield ScanSkip(
|
||||
cursor=event.cursor,
|
||||
@@ -446,7 +426,10 @@ def _shadowed_by_transcript(event: ScanEvent, transcript_ids: set[str]) -> ScanE
|
||||
if (
|
||||
isinstance(event, ScannedEvidence)
|
||||
and event.evidence.conversation_id in transcript_ids
|
||||
and not is_host_generated_user_turn(event.evidence.verbatim)
|
||||
and not (
|
||||
event.evidence.written_by is WrittenBy.AGENT
|
||||
and is_host_generated_user_turn(event.evidence.verbatim)
|
||||
)
|
||||
):
|
||||
return ScanSkip(
|
||||
cursor=event.cursor,
|
||||
@@ -720,7 +703,6 @@ def _parse_transcript_record(
|
||||
project_name = path_parts[path_parts.index("projects") + 1] if "projects" in path_parts else ""
|
||||
session_name = context.relative_path.rsplit("/", 1)[-1].removesuffix(".jsonl")
|
||||
is_subagent = f"{project_name}\0{session_name}" in subagent_ids
|
||||
host_timestamp = _cursor_host_timestamp(content) if role == "user" else None
|
||||
created_at = _transcript_fallback_time(context)
|
||||
native_message_id = record.get("id") or message.get("id")
|
||||
message_id = (
|
||||
@@ -746,16 +728,7 @@ def _parse_transcript_record(
|
||||
created_at=created_at,
|
||||
workspace=workspace,
|
||||
record_type="cursor_transcript",
|
||||
origin_metadata=OriginMetadata(
|
||||
raw_meta={
|
||||
"time_basis": "file_mtime",
|
||||
**(
|
||||
{"turn_event_at": host_timestamp.isoformat()}
|
||||
if host_timestamp is not None
|
||||
else {}
|
||||
),
|
||||
}
|
||||
),
|
||||
origin_metadata=OriginMetadata(raw_meta={"time_basis": "file_mtime"}),
|
||||
),
|
||||
tool_blocks_evidence(message.get("content"), block_types=TOOL_BLOCKS),
|
||||
)
|
||||
@@ -846,11 +819,6 @@ def _read_store_messages(
|
||||
reason_code=ScanIssueCode.UNKNOWN_RECORD,
|
||||
)
|
||||
continue
|
||||
host_timestamp = _cursor_host_timestamp(content) if role == "user" else None
|
||||
fallback_timestamp = datetime.fromtimestamp(
|
||||
source_file.modified_ns / 1_000_000_000,
|
||||
tz=UTC,
|
||||
) + timedelta(microseconds=rowid)
|
||||
parsed = ParsedMessage(
|
||||
conversation_id=conversation_id,
|
||||
message_id=str(record.get("id") or row["id"]),
|
||||
@@ -861,19 +829,8 @@ def _read_store_messages(
|
||||
else written_by_user
|
||||
),
|
||||
verbatim=cleaned,
|
||||
created_at=fallback_timestamp,
|
||||
workspace=_store_workspace(source_file),
|
||||
record_type="cursor_store_blob",
|
||||
origin_metadata=OriginMetadata(
|
||||
raw_meta={
|
||||
"time_basis": "file_mtime",
|
||||
**(
|
||||
{"turn_event_at": host_timestamp.isoformat()}
|
||||
if host_timestamp is not None
|
||||
else {}
|
||||
),
|
||||
}
|
||||
),
|
||||
)
|
||||
yield ScannedEvidence(
|
||||
cursor=cursor,
|
||||
|
||||
@@ -57,8 +57,7 @@ _UNANSWERED_PREFIXES = (
|
||||
|
||||
|
||||
def _is_visible_user_row(role: str, written_by: str, verbatim: str) -> bool:
|
||||
del written_by # Legacy storage metadata; protocol role is authoritative.
|
||||
if role != "user":
|
||||
if role != "user" or written_by != WrittenBy.PERSON.value:
|
||||
return False
|
||||
return not verbatim.lstrip().lower().startswith(_UNANSWERED_PREFIXES)
|
||||
|
||||
@@ -469,54 +468,16 @@ class EvidenceStore:
|
||||
if anchored is not None:
|
||||
self._write_origins(connection, anchored, evidence.origin_meta, now)
|
||||
workspace, project = self._resolve_conversation_attribution(connection, evidence)
|
||||
existing = connection.execute(
|
||||
"SELECT role,revision FROM evidence WHERE evidence_id=?",
|
||||
(anchored,),
|
||||
).fetchone()
|
||||
if existing is None:
|
||||
raise RuntimeError("anchored evidence disappeared")
|
||||
if str(existing["role"]) != evidence.role:
|
||||
was_revised = True
|
||||
revision = int(existing["revision"]) + 1
|
||||
connection.execute(
|
||||
"""
|
||||
UPDATE evidence
|
||||
SET workspace=COALESCE(workspace,?),
|
||||
project=COALESCE(project,?),
|
||||
content_kind=?,role=?,created_at=COALESCE(?,created_at),
|
||||
verbatim=?,content_hash=?,revision=?,updated_at=?
|
||||
WHERE evidence_id=?
|
||||
""",
|
||||
(
|
||||
workspace,
|
||||
project,
|
||||
evidence.content_kind,
|
||||
evidence.role,
|
||||
evidence.created_at.isoformat() if evidence.created_at else None,
|
||||
evidence.verbatim,
|
||||
evidence.content_hash,
|
||||
revision,
|
||||
now,
|
||||
anchored,
|
||||
),
|
||||
)
|
||||
self._insert_current_version(
|
||||
connection,
|
||||
anchored,
|
||||
revision=revision,
|
||||
changed_at=now,
|
||||
)
|
||||
else:
|
||||
connection.execute(
|
||||
"""
|
||||
UPDATE evidence
|
||||
SET workspace=COALESCE(workspace,?),
|
||||
project=COALESCE(project,?),
|
||||
updated_at=?
|
||||
WHERE evidence_id=?
|
||||
""",
|
||||
(workspace, project, now, anchored),
|
||||
)
|
||||
connection.execute(
|
||||
"""
|
||||
UPDATE evidence
|
||||
SET workspace = COALESCE(workspace, ?),
|
||||
project = COALESCE(project, ?),
|
||||
updated_at = ?
|
||||
WHERE evidence_id = ?
|
||||
""",
|
||||
(workspace, project, now, anchored),
|
||||
)
|
||||
record = self._get_with_connection(connection, anchored)
|
||||
if record is None:
|
||||
raise RuntimeError("anchored evidence disappeared")
|
||||
@@ -524,11 +485,7 @@ class EvidenceStore:
|
||||
from one_memory import ingest_journal
|
||||
|
||||
ingest_journal.record(connection, anchored, cause="import")
|
||||
return PutEvidenceResult(
|
||||
record=record,
|
||||
was_created=False,
|
||||
was_revised=was_revised,
|
||||
)
|
||||
return PutEvidenceResult(record=record, was_created=False, was_revised=False)
|
||||
insert = connection.execute(
|
||||
"""
|
||||
INSERT INTO evidence (
|
||||
@@ -846,20 +803,27 @@ class EvidenceStore:
|
||||
and same_scope[0]["idempotency_key"] != evidence.idempotency_key
|
||||
):
|
||||
return str(same_scope[0]["evidence_id"])
|
||||
elif (
|
||||
legacy_id := next(
|
||||
(
|
||||
anchored
|
||||
for key in (
|
||||
*evidence.written_by_derived_v3_idempotency_keys,
|
||||
*evidence.older_written_by_idempotency_keys,
|
||||
)
|
||||
if (anchored := claim_legacy_key(key)) is not None
|
||||
),
|
||||
None,
|
||||
)
|
||||
) is not None:
|
||||
elif (legacy_id := claim_legacy_key(evidence.legacy_derived_idempotency_key)) is not None:
|
||||
return legacy_id
|
||||
if not evidence.message_id and not evidence.correlation_id:
|
||||
# BUG-03 moved the derived key off the capture time and onto the
|
||||
# pre-cleaning text. A replay of a row stored under the old scheme
|
||||
# has to be recognized as the same message.
|
||||
replayed = connection.execute(
|
||||
"""
|
||||
SELECT evidence_id FROM evidence
|
||||
WHERE idempotency_key = ? AND user_id = ? AND source_id = ?
|
||||
AND conversation_id = ?
|
||||
""",
|
||||
(
|
||||
evidence.prior_derived_idempotency_key,
|
||||
evidence.user_id,
|
||||
evidence.source_id,
|
||||
evidence.conversation_id,
|
||||
),
|
||||
).fetchone()
|
||||
if replayed is not None:
|
||||
return str(replayed["evidence_id"])
|
||||
if (
|
||||
EvidenceOrigin.HOOK in evidence.origins
|
||||
and evidence.message_id is None
|
||||
@@ -874,7 +838,7 @@ class EvidenceStore:
|
||||
ON hook.evidence_id=e.evidence_id AND hook.origin='hook'
|
||||
WHERE e.user_id=? AND e.source_id=? AND e.conversation_id=?
|
||||
AND (e.workspace IS ? OR e.workspace IS NULL OR ? IS NULL)
|
||||
AND e.role=? AND e.content_hash=?
|
||||
AND e.role=? AND e.written_by=? AND e.content_hash=?
|
||||
AND e.message_id IS NOT NULL AND hook.evidence_id IS NULL
|
||||
AND ABS((julianday(e.created_at)-julianday(?))*86400.0) <= 5.0
|
||||
ORDER BY ABS((julianday(e.created_at)-julianday(?))*86400.0), e.evidence_id
|
||||
@@ -887,6 +851,7 @@ class EvidenceStore:
|
||||
evidence.workspace,
|
||||
evidence.workspace,
|
||||
evidence.role,
|
||||
evidence.written_by,
|
||||
evidence.content_hash,
|
||||
evidence.created_at.isoformat(),
|
||||
evidence.created_at.isoformat(),
|
||||
@@ -907,7 +872,7 @@ class EvidenceStore:
|
||||
AND (e.workspace IS ? OR e.workspace IS NULL OR ? IS NULL)
|
||||
-- Unary plus keeps the coverage index from looking better than
|
||||
-- the message index, which narrows to this exact text.
|
||||
AND e.role = ? AND e.content_hash = ?
|
||||
AND e.role = ? AND +e.written_by = ? AND e.content_hash = ?
|
||||
AND e.message_id IS NULL
|
||||
""",
|
||||
(
|
||||
@@ -917,6 +882,7 @@ class EvidenceStore:
|
||||
evidence.workspace,
|
||||
evidence.workspace,
|
||||
evidence.role,
|
||||
evidence.written_by,
|
||||
evidence.content_hash,
|
||||
),
|
||||
).fetchall()
|
||||
@@ -1281,7 +1247,7 @@ class EvidenceStore:
|
||||
rows = connection.execute(
|
||||
"""
|
||||
SELECT evidence_id FROM evidence
|
||||
WHERE user_id = 'local' AND role = 'user'
|
||||
WHERE user_id = 'local' AND role = 'user' AND written_by = 'person'
|
||||
ORDER BY imported_at, evidence_id
|
||||
LIMIT ?
|
||||
""",
|
||||
@@ -1335,7 +1301,7 @@ class EvidenceStore:
|
||||
rows = connection.execute(
|
||||
f"""
|
||||
SELECT evidence_id FROM evidence
|
||||
WHERE user_id = 'local' AND role = 'user'
|
||||
WHERE user_id = 'local' AND role = 'user' AND written_by = 'person'
|
||||
AND ({predicates})
|
||||
ORDER BY imported_at, evidence_id
|
||||
LIMIT ?
|
||||
@@ -1367,7 +1333,7 @@ class EvidenceStore:
|
||||
rows = connection.execute(
|
||||
f"""
|
||||
SELECT evidence_id FROM evidence
|
||||
WHERE user_id = 'local' AND role = 'user'
|
||||
WHERE user_id = 'local' AND role = 'user' AND written_by = 'person'
|
||||
AND workspace IS NOT NULL
|
||||
AND ({predicates})
|
||||
ORDER BY imported_at, evidence_id
|
||||
@@ -1510,7 +1476,7 @@ class EvidenceStore:
|
||||
)
|
||||
values: list[object] = [user_id, *(source_ids or ())]
|
||||
visible_user = """
|
||||
role='user' AND NOT (
|
||||
role='user' AND written_by='person' AND NOT (
|
||||
lower(ltrim(verbatim)) LIKE '<codex_internal_context%' OR
|
||||
lower(ltrim(verbatim)) LIKE '<environment_context>%' OR
|
||||
lower(ltrim(verbatim)) LIKE '<heartbeat>%' OR
|
||||
@@ -1519,7 +1485,7 @@ class EvidenceStore:
|
||||
)
|
||||
"""
|
||||
selected_visible_user = """
|
||||
e.role='user' AND NOT (
|
||||
e.role='user' AND e.written_by='person' AND NOT (
|
||||
lower(ltrim(e.verbatim)) LIKE '<codex_internal_context%' OR
|
||||
lower(ltrim(e.verbatim)) LIKE '<environment_context>%' OR
|
||||
lower(ltrim(e.verbatim)) LIKE '<heartbeat>%' OR
|
||||
@@ -1672,7 +1638,7 @@ class EvidenceStore:
|
||||
if seed is None:
|
||||
return ConversationTurnPage((), {}, 0, offset, limit)
|
||||
visible_user = """
|
||||
role='user' AND NOT (
|
||||
role='user' AND written_by='person' AND NOT (
|
||||
lower(ltrim(verbatim)) LIKE '<codex_internal_context%' OR
|
||||
lower(ltrim(verbatim)) LIKE '<environment_context>%' OR
|
||||
lower(ltrim(verbatim)) LIKE '<heartbeat>%' OR
|
||||
@@ -1765,7 +1731,7 @@ class EvidenceStore:
|
||||
AND first.conversation_id = evidence.conversation_id
|
||||
AND first.workspace IS evidence.workspace
|
||||
AND (
|
||||
first.role = 'user'
|
||||
(first.role = 'user' AND first.written_by = 'person')
|
||||
OR first.content_kind = 'memory_content'
|
||||
)
|
||||
ORDER BY julianday(COALESCE(first.created_at, first.imported_at)),
|
||||
@@ -1796,67 +1762,6 @@ class EvidenceStore:
|
||||
records.append(record)
|
||||
return records
|
||||
|
||||
def release_sdk_staging(self, evidence_ids: tuple[str, ...]) -> int:
|
||||
"""Remove post-cutover queue staging after SDK success.
|
||||
|
||||
Historical rows referenced by legacy Governance remain for the
|
||||
compatibility release. SDK-authority imports create no such links and
|
||||
stop being MemHub raw storage once the job has SDK receipts.
|
||||
"""
|
||||
ids = tuple(dict.fromkeys(evidence_ids))
|
||||
if not ids:
|
||||
return 0
|
||||
placeholders = ",".join("?" for _ in ids)
|
||||
with self._connect(writing=True) as connection:
|
||||
has_governance = (
|
||||
connection.execute(
|
||||
"SELECT 1 FROM sqlite_master "
|
||||
"WHERE type='table' AND name='governed_memory_evidence'"
|
||||
).fetchone()
|
||||
is not None
|
||||
)
|
||||
legacy_guard = (
|
||||
"AND NOT EXISTS ("
|
||||
"SELECT 1 FROM governed_memory_evidence governed "
|
||||
"WHERE governed.evidence_id=evidence.evidence_id"
|
||||
")"
|
||||
if has_governance
|
||||
else ""
|
||||
)
|
||||
changed = connection.execute(
|
||||
f"DELETE FROM evidence WHERE evidence_id IN ({placeholders}) {legacy_guard}",
|
||||
ids,
|
||||
)
|
||||
return changed.rowcount
|
||||
|
||||
def release_completed_sdk_staging(self, *, limit: int = 2_048) -> int:
|
||||
"""Resume cleanup if Core stopped after job settlement."""
|
||||
if not 1 <= limit <= 100_000:
|
||||
raise ValueError("limit must be between 1 and 100000")
|
||||
with self._connect(writing=True) as connection:
|
||||
tables = {
|
||||
str(row[0])
|
||||
for row in connection.execute("SELECT name FROM sqlite_master WHERE type='table'")
|
||||
}
|
||||
if (
|
||||
not {
|
||||
"memory_write_jobs",
|
||||
"memory_write_job_evidence",
|
||||
}
|
||||
<= tables
|
||||
):
|
||||
return 0
|
||||
rows = connection.execute(
|
||||
"SELECT DISTINCT linked.evidence_id "
|
||||
"FROM memory_write_jobs job "
|
||||
"JOIN memory_write_job_evidence linked USING(job_id) "
|
||||
"WHERE job.state='completed' "
|
||||
"AND (job.sdk_raw_ids_json!='[]' OR job.sdk_memory_ids_json!='[]') "
|
||||
"ORDER BY linked.evidence_id LIMIT ?",
|
||||
(limit,),
|
||||
).fetchall()
|
||||
return self.release_sdk_staging(tuple(str(row[0]) for row in rows))
|
||||
|
||||
def stage_memory_dispatch(
|
||||
self,
|
||||
record: EvidenceRecord,
|
||||
@@ -2220,6 +2125,8 @@ class EvidenceStore:
|
||||
(
|
||||
existing["content_kind"] != evidence.content_kind,
|
||||
existing["role"] != evidence.role,
|
||||
evidence.written_by is not WrittenBy.UNKNOWN
|
||||
and existing["written_by"] != evidence.written_by,
|
||||
existing["content_hash"] != evidence.content_hash,
|
||||
)
|
||||
)
|
||||
|
||||
@@ -17,7 +17,6 @@ from one_memory.imports.jobs import (
|
||||
from one_memory.imports.models import EvidenceRecord
|
||||
from one_memory.imports.runner import ImportRunner
|
||||
from one_memory.imports.storage import EvidenceStore
|
||||
from one_memory.memory_server.durable_queue import DurableWriteQueue
|
||||
from one_memory.service_levels import (
|
||||
ResourceLimit,
|
||||
ServiceLevel,
|
||||
@@ -48,8 +47,6 @@ class ImportSupervisor:
|
||||
source_enabled: Callable[[str], bool] | None = None,
|
||||
prepare_adapter: Callable[[ImportJob, SourceAdapter], Awaitable[SourceAdapter]]
|
||||
| None = None,
|
||||
sdk_queue: DurableWriteQueue | None = None,
|
||||
sdk_authority: Callable[[], bool] | None = None,
|
||||
) -> None:
|
||||
if max_work_events_per_slice < 1:
|
||||
raise ValueError("max_work_events_per_slice must be positive")
|
||||
@@ -64,8 +61,6 @@ class ImportSupervisor:
|
||||
self.on_stable_evidence = on_stable_evidence
|
||||
self.source_enabled = source_enabled
|
||||
self.prepare_adapter = prepare_adapter
|
||||
self.sdk_queue = sdk_queue
|
||||
self.sdk_authority = sdk_authority or (lambda: False)
|
||||
self.max_work_events_per_slice = max_work_events_per_slice
|
||||
self.max_work_seconds_per_slice = max_work_seconds_per_slice
|
||||
self.retry_base_seconds = retry_base_seconds
|
||||
@@ -116,7 +111,7 @@ class ImportSupervisor:
|
||||
)
|
||||
)
|
||||
diagnostic_event(
|
||||
level="debug",
|
||||
level="info",
|
||||
component="import.supervisor",
|
||||
event="import.queued",
|
||||
outcome="queued",
|
||||
@@ -203,7 +198,7 @@ class ImportSupervisor:
|
||||
started = monotonic()
|
||||
failure_type: str | None = None
|
||||
diagnostic_event(
|
||||
level="debug",
|
||||
level="info",
|
||||
component="import.supervisor",
|
||||
event="import.started",
|
||||
outcome="started",
|
||||
@@ -235,8 +230,6 @@ class ImportSupervisor:
|
||||
job_store=self.job_store,
|
||||
on_evidence=self.on_evidence,
|
||||
on_stable_evidence=self.on_stable_evidence,
|
||||
sdk_queue=self.sdk_queue,
|
||||
sdk_authority=self.sdk_authority,
|
||||
)
|
||||
adapter = self.adapters[job.source_id]
|
||||
if self.prepare_adapter is not None:
|
||||
|
||||
Loaded 100 of 461 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user