Compare commits
13
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
8918098fa8 | ||
|
|
0afe787c95 | ||
|
|
67ae43f920 | ||
|
|
3ce89ea560 | ||
|
|
7b7fd217a8 | ||
|
|
7efe054e83 | ||
|
|
fdbcd9934d | ||
|
|
3171007b3c | ||
|
|
a04fcb50d5 | ||
|
|
a28ade2faf | ||
|
|
1c424928a2 | ||
|
|
688878171e | ||
|
|
a3f09f4cac |
No files matched your search
@@ -14,6 +14,25 @@ Hy Memory and MemHub are versioned and released independently.
|
|||||||
|
|
||||||
## Unreleased
|
## Unreleased
|
||||||
|
|
||||||
|
- Fixed governed-memory revisions and repeated evidence links failing when the
|
||||||
|
visibility dirty-marker trigger encounters the same memory twice. Startup
|
||||||
|
now replaces the affected legacy trigger without changing schema versions
|
||||||
|
or governed and visibility data.
|
||||||
|
- Serialized each SDK SQLite store's shared connection lifecycle so concurrent
|
||||||
|
history/cache operations cannot race with `close()` and crash the process.
|
||||||
|
- Scoped Coding reconciliation targets to the current HY authority,
|
||||||
|
conversation, and workspace, preventing near-duplicate sessions from
|
||||||
|
mutating an experience or working fact owned by another evidence context.
|
||||||
|
- Kept post-write authority verification failures available for read-only
|
||||||
|
reconciliation while preventing automatic or generic confirmed retries from
|
||||||
|
invoking the model again. Automatic reconciliation skips those sticky rows
|
||||||
|
before applying its batch limit, and linked durable jobs exclusively own
|
||||||
|
retry scheduling for their memory tasks.
|
||||||
|
- Clarified that `add(metadata=...)` carries legacy SDK processing options,
|
||||||
|
while persistent caller business context belongs in `meta`.
|
||||||
|
- Added an opt-in `get(..., include_raw=True)` path for reading `L1_RAW`
|
||||||
|
memories, including memories written in lite mode. The default remains
|
||||||
|
hidden for backward compatibility.
|
||||||
- Removed MemHub's task-scoped HyPermission store wrappers and global
|
- Removed MemHub's task-scoped HyPermission store wrappers and global
|
||||||
model-content consent switch. The trusted embedded Hy Memory SDK now uses
|
model-content consent switch. The trusted embedded Hy Memory SDK now uses
|
||||||
its native local stores directly.
|
its native local stores directly.
|
||||||
|
|||||||
@@ -32,7 +32,9 @@ English | [中文](README.zh.md)
|
|||||||
|
|
||||||
## Latest News
|
## Latest News
|
||||||
|
|
||||||
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-10]**: 🚀 **Released** **Hy Memory 2.0 (`hy-memory==2.0.3`)** on [PyPI](https://pypi.org/project/hy-memory/2.0.3/) and open-sourced the **Hy-Memory** monorepo together with **[MemHub](memhub/README.md)**!
|
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-10]**: 🚀 **Open-sourced** quantized **FP8** (**[Hy-Memory-30B-A3B-FP8](https://huggingface.co/tencent/Hy-Memory-30B-A3B-FP8)**) and **GGUF** (**[Hy-Memory-30B-A3B-GGUF](https://huggingface.co/tencent/Hy-Memory-30B-A3B-GGUF)**) versions of **Hy-Memory-30B-A3B** on Hugging Face!
|
||||||
|
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-10]**: 📦 **Released [MemHub v1.0.4](https://github.com/Tencent-Hunyuan/Hy-Memory/releases/tag/v1.0.4)** and **[memhub-cli 1.0.5](https://www.npmjs.com/package/memhub-cli/v/1.0.5)**, with a signed macOS desktop app and verified macOS/Linux Runtimes!
|
||||||
|
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-10]**: 🚀 **Released** **Hy Memory 2.0 (`hy-memory==2.0.4`)** on [PyPI](https://pypi.org/project/hy-memory/2.0.4/) and open-sourced the **Hy-Memory** monorepo together with **[MemHub](memhub/README.md)**!
|
||||||
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-10]**: 🚀 **Open-sourced** **[Hy-Memory-30B-A3B](https://huggingface.co/tencent/Hy-Memory-30B-A3B)** on Hugging Face along with its [technical report](https://arxiv.org/abs/2610.22196) on attributed reinforcement learning for long-term memory writing!
|
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-10]**: 🚀 **Open-sourced** **[Hy-Memory-30B-A3B](https://huggingface.co/tencent/Hy-Memory-30B-A3B)** on Hugging Face along with its [technical report](https://arxiv.org/abs/2610.22196) on attributed reinforcement learning for long-term memory writing!
|
||||||
* *<mark>[new]</mark>* 🔥🔥 **[2026-09]**: 📊 **Released** **[CMB: A Coding Memory Benchmark for Long-Horizon and Interleaved Coding Agents](docs/papers/CMB_A_Coding_Memory_Benchmark_for_Long_Horizon_and_Interleaved_Coding_Agents.pdf)** (submitted to **ICLR 2027**).
|
* *<mark>[new]</mark>* 🔥🔥 **[2026-09]**: 📊 **Released** **[CMB: A Coding Memory Benchmark for Long-Horizon and Interleaved Coding Agents](docs/papers/CMB_A_Coding_Memory_Benchmark_for_Long_Horizon_and_Interleaved_Coding_Agents.pdf)** (submitted to **ICLR 2027**).
|
||||||
* *<mark>[new]</mark>* 🔥🔥 **[2026-08]**: 🎉 **Accepted** at **EMNLP 2026**: **[Memory Beyond Recall: A Dual-Process Cognitive Memory System for Self-Evolving LLM Agents](https://arxiv.org/abs/2606.09483)**!
|
* *<mark>[new]</mark>* 🔥🔥 **[2026-08]**: 🎉 **Accepted** at **EMNLP 2026**: **[Memory Beyond Recall: A Dual-Process Cognitive Memory System for Self-Evolving LLM Agents](https://arxiv.org/abs/2606.09483)**!
|
||||||
|
|||||||
+3
-1
@@ -32,7 +32,9 @@
|
|||||||
|
|
||||||
## 最新动态
|
## 最新动态
|
||||||
|
|
||||||
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-10]**: 🚀 **正式发布** **Hy Memory 2.0 (`hy-memory==2.0.3`)**(已上线 [PyPI](https://pypi.org/project/hy-memory/2.0.3/)),并开源 **Hy-Memory** monorepo 与 **[MemHub](memhub/README.md)**!
|
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-10]**: 🚀 **模型开源** 在 Hugging Face 发布 **Hy-Memory-30B-A3B** 的 **FP8**(**[Hy-Memory-30B-A3B-FP8](https://huggingface.co/tencent/Hy-Memory-30B-A3B-FP8)**)与 **GGUF**(**[Hy-Memory-30B-A3B-GGUF](https://huggingface.co/tencent/Hy-Memory-30B-A3B-GGUF)**)量化版本模型!
|
||||||
|
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-10]**: 📦 **正式发布 [MemHub v1.0.4](https://github.com/Tencent-Hunyuan/Hy-Memory/releases/tag/v1.0.4)** 与 **[memhub-cli 1.0.5](https://www.npmjs.com/package/memhub-cli/v/1.0.5)**,提供已签名的 macOS 桌面应用及经过验证的 macOS/Linux Runtime!
|
||||||
|
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-10]**: 🚀 **正式发布** **Hy Memory 2.0 (`hy-memory==2.0.4`)**(已上线 [PyPI](https://pypi.org/project/hy-memory/2.0.4/)),并开源 **Hy-Memory** monorepo 与 **[MemHub](memhub/README.md)**!
|
||||||
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-10]**: 🚀 **模型开源** 在 Hugging Face 发布 **[Hy-Memory-30B-A3B](https://huggingface.co/tencent/Hy-Memory-30B-A3B)** 长期记忆写入模型及[技术报告](https://arxiv.org/abs/2610.22196)!
|
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-10]**: 🚀 **模型开源** 在 Hugging Face 发布 **[Hy-Memory-30B-A3B](https://huggingface.co/tencent/Hy-Memory-30B-A3B)** 长期记忆写入模型及[技术报告](https://arxiv.org/abs/2610.22196)!
|
||||||
* *<mark>[new]</mark>* 🔥🔥 **[2026-09]**: 📊 **基准与方法发布** 推出长周期与交错式编程记忆评测基准 **[CMB: A Coding Memory Benchmark for Long-Horizon and Interleaved Coding Agents](docs/papers/CMB_A_Coding_Memory_Benchmark_for_Long_Horizon_and_Interleaved_Coding_Agents.pdf)**(已投稿 **ICLR 2027**)。
|
* *<mark>[new]</mark>* 🔥🔥 **[2026-09]**: 📊 **基准与方法发布** 推出长周期与交错式编程记忆评测基准 **[CMB: A Coding Memory Benchmark for Long-Horizon and Interleaved Coding Agents](docs/papers/CMB_A_Coding_Memory_Benchmark_for_Long_Horizon_and_Interleaved_Coding_Agents.pdf)**(已投稿 **ICLR 2027**)。
|
||||||
* *<mark>[new]</mark>* 🔥🔥 **[2026-08]**: 🎉 **论文录用** 双系统认知记忆架构论文 **[Memory Beyond Recall: A Dual-Process Cognitive Memory System for Self-Evolving LLM Agents](https://arxiv.org/abs/2606.09483)** 被 **EMNLP 2026** 接收!
|
* *<mark>[new]</mark>* 🔥🔥 **[2026-08]**: 🎉 **论文录用** 双系统认知记忆架构论文 **[Memory Beyond Recall: A Dual-Process Cognitive Memory System for Self-Evolving LLM Agents](https://arxiv.org/abs/2606.09483)** 被 **EMNLP 2026** 接收!
|
||||||
|
|||||||
@@ -106,6 +106,15 @@ result = client.add(
|
|||||||
)
|
)
|
||||||
```
|
```
|
||||||
|
|
||||||
|
`metadata` 与 `meta` 不是同一字段:
|
||||||
|
|
||||||
|
- `metadata` 是兼容旧调用方保留的 SDK 处理选项,会进入内部
|
||||||
|
`WriteRequest.extra`,不保证持久化或随查询结果返回。
|
||||||
|
- `meta` 是调用方的业务上下文,会持久化并在 `search()`、`get()` 等结果中返回。
|
||||||
|
同一记忆被多次合并时,各次 `meta` 会按写入顺序保存在列表中。
|
||||||
|
|
||||||
|
需要以后读回的数据必须放在 `meta`,不要放在 `metadata`。
|
||||||
|
|
||||||
`scene` 的公开取值:
|
`scene` 的公开取值:
|
||||||
|
|
||||||
- `None`:普通文本直接走 chat;messages 含 tool 时执行快速场景判断。
|
- `None`:普通文本直接走 chat;messages 含 tool 时执行快速场景判断。
|
||||||
@@ -263,6 +272,8 @@ catalog 也可以在没有基础 L0 时形成只读的合成画像。
|
|||||||
|
|
||||||
```python
|
```python
|
||||||
memory = client.get("memory_id")
|
memory = client.get("memory_id")
|
||||||
|
# lite 模式写入的是 L1_RAW;按 ID 读取时必须显式选择返回原文。
|
||||||
|
raw_memory = client.get("memory_id", include_raw=True)
|
||||||
result = client.update("memory_id", "更新后的内容")
|
result = client.update("memory_id", "更新后的内容")
|
||||||
result = client.delete("memory_id")
|
result = client.delete("memory_id")
|
||||||
result = client.delete_all(user_id="user_001")
|
result = client.delete_all(user_id="user_001")
|
||||||
@@ -282,6 +293,10 @@ result = client.clone_user("user_001", "user_002", agent_id=None)
|
|||||||
对应异步方法为 `async_get()`、`async_update()`、`async_delete()`、
|
对应异步方法为 `async_get()`、`async_update()`、`async_delete()`、
|
||||||
`async_delete_all()`、`async_list_memories()`、`async_clone_user()`。
|
`async_delete_all()`、`async_list_memories()`、`async_clone_user()`。
|
||||||
|
|
||||||
|
`get()` / `async_get()` 默认继续隐藏 `L1_RAW`,因此不会改变现有调用方的原文访问
|
||||||
|
边界。只有显式传入 `include_raw=True` 才返回该层;这也是 lite 模式按 ID 读取
|
||||||
|
写入内容及其 `meta` 的方式。MemHub 的 HTTP 读取接口不开放此参数。
|
||||||
|
|
||||||
普通 `list_memories()` 返回 `vdb` 桶,并在 Graph 有结果时附带 `graph`;传入
|
普通 `list_memories()` 返回 `vdb` 桶,并在 Graph 有结果时附带 `graph`;传入
|
||||||
`workspace_id` 时返回生产力记忆的 `scene="productivity"` 和 `experiences`。
|
`workspace_id` 时返回生产力记忆的 `scene="productivity"` 和 `experiences`。
|
||||||
|
|
||||||
|
|||||||
+39
-12
@@ -1597,7 +1597,9 @@ class HyMemoryClient:
|
|||||||
session_id: Session ID(三级隔离 key)
|
session_id: Session ID(三级隔离 key)
|
||||||
source: 调用来源,例如 openclaw / cursor / codex;与 agent_id 正交。
|
source: 调用来源,例如 openclaw / cursor / codex;与 agent_id 正交。
|
||||||
写入前执行 strip().lower(),默认 "default"。
|
写入前执行 strip().lower(),默认 "default"。
|
||||||
metadata: 自定义元数据
|
metadata: 旧版 SDK 处理选项,会并入内部 WriteRequest.extra。
|
||||||
|
不保证持久化或随查询返回;调用方需要保存并读回的业务字段
|
||||||
|
请使用 meta。
|
||||||
memory_at: 记忆时间戳(不传则用当前时间),用于导入历史记忆
|
memory_at: 记忆时间戳(不传则用当前时间),用于导入历史记忆
|
||||||
scene: 显式指定写入路由,跳过 LLM 自动判定。
|
scene: 显式指定写入路由,跳过 LLM 自动判定。
|
||||||
- None(默认):含 tool 时 LLM 判 chat/working/coding;
|
- None(默认):含 tool 时 LLM 判 chat/working/coding;
|
||||||
@@ -1623,8 +1625,8 @@ class HyMemoryClient:
|
|||||||
branch: Coding memory 路径专用。分支名。同 workspace_id。chat 路径忽略。
|
branch: Coding memory 路径专用。分支名。同 workspace_id。chat 路径忽略。
|
||||||
meta: 调用方自定义业务 meta,随本次产出的记忆存储、查询时原样返回。
|
meta: 调用方自定义业务 meta,随本次产出的记忆存储、查询时原样返回。
|
||||||
不 embed、不参与召回、不作过滤条件。
|
不 embed、不参与召回、不作过滤条件。
|
||||||
与 metadata 不同:metadata 会并入 extra(含 SDK 内部控制键),
|
与 metadata 不同:metadata 是旧版内部处理选项,
|
||||||
meta 独立存放,只做透传。
|
meta 独立持久化,只做业务透传。
|
||||||
语义为「该次 write 时刻的上下文」,故存为 list——同一节点被
|
语义为「该次 write 时刻的上下文」,故存为 list——同一节点被
|
||||||
多次写入合并时会追加而非覆盖,详见 get() 返回的 meta 字段。
|
多次写入合并时会追加而非覆盖,详见 get() 返回的 meta 字段。
|
||||||
|
|
||||||
@@ -1791,17 +1793,26 @@ class HyMemoryClient:
|
|||||||
query_embedding=query_embedding,
|
query_embedding=query_embedding,
|
||||||
))
|
))
|
||||||
|
|
||||||
def get(self, memory_id: str) -> Optional[Dict[str, Any]]:
|
def get(
|
||||||
|
self,
|
||||||
|
memory_id: str,
|
||||||
|
*,
|
||||||
|
include_raw: bool = False,
|
||||||
|
) -> Optional[Dict[str, Any]]:
|
||||||
"""
|
"""
|
||||||
获取单条记忆。
|
获取单条记忆。
|
||||||
|
|
||||||
Args:
|
Args:
|
||||||
memory_id: 记忆 ID
|
memory_id: 记忆 ID
|
||||||
|
include_raw: 是否允许返回 L1_RAW 原始记忆。默认 False,保持原有隐藏行为;
|
||||||
|
lite 模式的记忆均为 L1_RAW,需显式设为 True 才能按 ID 读取。
|
||||||
|
|
||||||
Returns:
|
Returns:
|
||||||
{"memory_id", "content", "layer", "request_id", "elapsed_ms"} 或 None(不存在时)
|
{"memory_id", "content", "layer", "request_id", "elapsed_ms"} 或 None(不存在时)
|
||||||
"""
|
"""
|
||||||
return self._loop_thread.run(self.async_get(memory_id))
|
return self._loop_thread.run(
|
||||||
|
self.async_get(memory_id, include_raw=include_raw)
|
||||||
|
)
|
||||||
|
|
||||||
def update(
|
def update(
|
||||||
self,
|
self,
|
||||||
@@ -3650,12 +3661,21 @@ class HyMemoryClient:
|
|||||||
# ================================================================
|
# ================================================================
|
||||||
|
|
||||||
@_ensure_internal_loop
|
@_ensure_internal_loop
|
||||||
async def async_get(self, memory_id: str) -> Optional[Dict[str, Any]]:
|
async def async_get(
|
||||||
|
self,
|
||||||
|
memory_id: str,
|
||||||
|
*,
|
||||||
|
include_raw: bool = False,
|
||||||
|
) -> Optional[Dict[str, Any]]:
|
||||||
"""
|
"""
|
||||||
获取单条记忆(异步)。
|
获取单条记忆(异步)。
|
||||||
|
|
||||||
VDB + Graph 双路并行查询,哪个有结果返回哪个。
|
VDB + Graph 双路并行查询,哪个有结果返回哪个。
|
||||||
|
|
||||||
|
Args:
|
||||||
|
memory_id: 记忆 ID
|
||||||
|
include_raw: 是否允许返回 L1_RAW 原始记忆,默认 False。
|
||||||
|
|
||||||
Returns:
|
Returns:
|
||||||
{"memory_id", "content", "layer", "source", "request_id", "elapsed_ms"} 或 None
|
{"memory_id", "content", "layer", "source", "request_id", "elapsed_ms"} 或 None
|
||||||
"""
|
"""
|
||||||
@@ -3683,17 +3703,24 @@ class HyMemoryClient:
|
|||||||
|
|
||||||
vdb_node, graph_node = await _asyncio_get.gather(_get_vdb(), _get_graph())
|
vdb_node, graph_node = await _asyncio_get.gather(_get_vdb(), _get_graph())
|
||||||
|
|
||||||
# L1_RAW 不对外可见:按 ID 直取走主键查询、绕过向量库的 layer filter,
|
# L1_RAW 默认不对外可见:按 ID 直取走主键查询、绕过向量库的 layer
|
||||||
# 故在此统一挡掉。async_get 是 GET /v1/memories/{id} 与
|
# filter,故在此统一挡掉。SDK 调用方可通过 include_raw=True 显式读取;
|
||||||
# /v1/memories/batch 两个对外接口的共同必经处,而内部写链路不经过它
|
# HTTP 接口不传该参数,继续保持隐藏。内部写链路不经过此方法。
|
||||||
# (writer 写后校验直接调 vector_store.get_by_id),挡在此处不影响内部。
|
|
||||||
#
|
#
|
||||||
# 置于 gather 之后、埋点之前:命中 raw 仍照常落 GET 埋点(保留可观测性,
|
# 置于 gather 之后、埋点之前:命中 raw 仍照常落 GET 埋点(保留可观测性,
|
||||||
# 便于发现有调用方拿 raw id 试探),只是不把内容返回给调用方。
|
# 便于发现有调用方拿 raw id 试探),只是不把内容返回给调用方。
|
||||||
# 对调用方的表现与"该 id 不存在"完全一致,不额外暴露"存在但被挡"的信息。
|
# 对调用方的表现与"该 id 不存在"完全一致,不额外暴露"存在但被挡"的信息。
|
||||||
if vdb_node is not None and vdb_node.layer == MemoryLayer.L1_RAW:
|
if (
|
||||||
|
not include_raw
|
||||||
|
and vdb_node is not None
|
||||||
|
and vdb_node.layer == MemoryLayer.L1_RAW
|
||||||
|
):
|
||||||
vdb_node = None
|
vdb_node = None
|
||||||
if graph_node is not None and getattr(graph_node, "layer", None) == MemoryLayer.L1_RAW:
|
if (
|
||||||
|
not include_raw
|
||||||
|
and graph_node is not None
|
||||||
|
and getattr(graph_node, "layer", None) == MemoryLayer.L1_RAW
|
||||||
|
):
|
||||||
graph_node = None
|
graph_node = None
|
||||||
|
|
||||||
elapsed = round((time.perf_counter() - t0) * 1000, 2)
|
elapsed = round((time.perf_counter() - t0) * 1000, 2)
|
||||||
|
|||||||
@@ -466,7 +466,13 @@ class WorkingFactReconciler:
|
|||||||
return [], []
|
return [], []
|
||||||
|
|
||||||
candidates = await self._retrieve_candidates(
|
candidates = await self._retrieve_candidates(
|
||||||
drafts, user_id=user_id, source=source,
|
drafts,
|
||||||
|
user_id=user_id,
|
||||||
|
agent_id=agent_id,
|
||||||
|
session_id=session_id,
|
||||||
|
workspace_id=workspace_id,
|
||||||
|
branch=branch,
|
||||||
|
source=source,
|
||||||
)
|
)
|
||||||
|
|
||||||
try:
|
try:
|
||||||
@@ -499,7 +505,15 @@ class WorkingFactReconciler:
|
|||||||
# ------------------------------------------------------------
|
# ------------------------------------------------------------
|
||||||
|
|
||||||
async def _retrieve_candidates(
|
async def _retrieve_candidates(
|
||||||
self, drafts: List[WorkingFactDraft], *, user_id: str, source: str
|
self,
|
||||||
|
drafts: List[WorkingFactDraft],
|
||||||
|
*,
|
||||||
|
user_id: str,
|
||||||
|
agent_id: str,
|
||||||
|
session_id: Optional[str],
|
||||||
|
workspace_id: Optional[str],
|
||||||
|
branch: Optional[str],
|
||||||
|
source: str,
|
||||||
) -> List[List[WorkingFact]]:
|
) -> List[List[WorkingFact]]:
|
||||||
"""每条 draft 用 statement 召回同 user 的 top-N active 事实。
|
"""每条 draft 用 statement 召回同 user 的 top-N active 事实。
|
||||||
|
|
||||||
@@ -512,6 +526,10 @@ class WorkingFactReconciler:
|
|||||||
关掉时整份召回原样交给 LLM。适用于 subject 不是稳定实体名的数据集(自由文本、
|
关掉时整份召回原样交给 LLM。适用于 subject 不是稳定实体名的数据集(自由文本、
|
||||||
位置标签),那里精确字符串相等会把同实体判成不同主体,反而拦掉真实更新。
|
位置标签),那里精确字符串相等会把同实体判成不同主体,反而拦掉真实更新。
|
||||||
"""
|
"""
|
||||||
|
# Empty/legacy session metadata remains readable, but cannot prove
|
||||||
|
# conversation authority for a destructive SUPERSEDE.
|
||||||
|
if not session_id:
|
||||||
|
return [[] for _ in drafts]
|
||||||
try:
|
try:
|
||||||
embeddings = await self.embed_service.embed_batch(
|
embeddings = await self.embed_service.embed_batch(
|
||||||
[d.statement for d in drafts]
|
[d.statement for d in drafts]
|
||||||
@@ -525,6 +543,10 @@ class WorkingFactReconciler:
|
|||||||
try:
|
try:
|
||||||
hits = await self.store.search_facts(
|
hits = await self.store.search_facts(
|
||||||
emb, user_id=user_id, top=self.candidates_per_draft,
|
emb, user_id=user_id, top=self.candidates_per_draft,
|
||||||
|
agent_id=agent_id,
|
||||||
|
session_id=session_id,
|
||||||
|
workspace_id=workspace_id,
|
||||||
|
branch=branch,
|
||||||
# 写入侧同样接词面通道:候选选错了 LLM 就没有机会判对。这里不传
|
# 写入侧同样接词面通道:候选选错了 LLM 就没有机会判对。这里不传
|
||||||
# min_score——写入侧要的是"有哪些可能的前身",让 LLM 看更宽的名单,
|
# min_score——写入侧要的是"有哪些可能的前身",让 LLM 看更宽的名单,
|
||||||
# 阈值过滤是读取侧的事。
|
# 阈值过滤是读取侧的事。
|
||||||
@@ -794,6 +816,15 @@ class WorkingFactReconciler:
|
|||||||
old = await self.store.get_fact(old_id, user_id=user_id)
|
old = await self.store.get_fact(old_id, user_id=user_id)
|
||||||
if old is None:
|
if old is None:
|
||||||
continue
|
continue
|
||||||
|
if (
|
||||||
|
old.agent_id != agent_id
|
||||||
|
or old.session_id != session_id
|
||||||
|
or old.workspace_id != workspace_id
|
||||||
|
):
|
||||||
|
logger.warning(
|
||||||
|
f"[fact-reconcile] refused cross-authority SUPERSEDE: {old.fact_id}"
|
||||||
|
)
|
||||||
|
continue
|
||||||
# 闸门在候选阶段已经拦过一道,这里再拦一次:候选是按 statement 召回的,
|
# 闸门在候选阶段已经拦过一道,这里再拦一次:候选是按 statement 召回的,
|
||||||
# 而 LLM 只能覆盖候选名单里的 id,理论上到不了这儿。真到了就说明前面
|
# 而 LLM 只能覆盖候选名单里的 id,理论上到不了这儿。真到了就说明前面
|
||||||
# 哪一环漏了,宁可让这次覆盖不生效——多留一条旧事实是可恢复的,删掉
|
# 哪一环漏了,宁可让这次覆盖不生效——多留一条旧事实是可恢复的,删掉
|
||||||
|
|||||||
@@ -240,7 +240,7 @@ class CodingMemoryReconciler:
|
|||||||
)
|
)
|
||||||
|
|
||||||
# 3. 校验 + 执行
|
# 3. 校验 + 执行
|
||||||
executed = await self._execute(ops, drafts)
|
executed = await self._execute(ops, drafts, candidates_per_draft)
|
||||||
logger.info(
|
logger.info(
|
||||||
f"[coding-reconcile] drafts={len(drafts)} ops={len(executed)} "
|
f"[coding-reconcile] drafts={len(drafts)} ops={len(executed)} "
|
||||||
+ ", ".join(f"{a}={sum(1 for o in executed if o.action == a)}"
|
+ ", ".join(f"{a}={sum(1 for o in executed if o.action == a)}"
|
||||||
@@ -263,6 +263,12 @@ class CodingMemoryReconciler:
|
|||||||
"""
|
"""
|
||||||
result: List[List[CodingMemory]] = []
|
result: List[List[CodingMemory]] = []
|
||||||
for d in drafts:
|
for d in drafts:
|
||||||
|
# A coding mutation must be attributable to one conversation.
|
||||||
|
# Empty/legacy session metadata stays readable through ordinary
|
||||||
|
# search, but it cannot authorize UPDATE/DELETE.
|
||||||
|
if not d.session_id:
|
||||||
|
result.append([])
|
||||||
|
continue
|
||||||
try:
|
try:
|
||||||
# task 为主入口,search_keys 为附加入口(去重、去空)
|
# task 为主入口,search_keys 为附加入口(去重、去空)
|
||||||
queries = [d.task] + [k for k in (d.search_keys or []) if k and k != d.task]
|
queries = [d.task] + [k for k in (d.search_keys or []) if k and k != d.task]
|
||||||
@@ -274,6 +280,8 @@ class CodingMemoryReconciler:
|
|||||||
hits = await self.store.search_by_query_embedding(
|
hits = await self.store.search_by_query_embedding(
|
||||||
emb,
|
emb,
|
||||||
user_id=d.user_id,
|
user_id=d.user_id,
|
||||||
|
agent_id=d.agent_id,
|
||||||
|
session_id=d.session_id,
|
||||||
workspace_id=d.workspace_id,
|
workspace_id=d.workspace_id,
|
||||||
branch=d.branch,
|
branch=d.branch,
|
||||||
top=self.candidates_per_draft * 4,
|
top=self.candidates_per_draft * 4,
|
||||||
@@ -415,6 +423,7 @@ class CodingMemoryReconciler:
|
|||||||
self,
|
self,
|
||||||
ops: List[ReconcileOp],
|
ops: List[ReconcileOp],
|
||||||
drafts: List[CodingMemoryDraft],
|
drafts: List[CodingMemoryDraft],
|
||||||
|
candidates_per_draft: List[List[CodingMemory]],
|
||||||
) -> List[ReconcileOp]:
|
) -> List[ReconcileOp]:
|
||||||
"""
|
"""
|
||||||
校验 + 执行 ops。返回成功执行的 ops(filter 掉无效 op)。
|
校验 + 执行 ops。返回成功执行的 ops(filter 掉无效 op)。
|
||||||
@@ -454,6 +463,11 @@ class CodingMemoryReconciler:
|
|||||||
if not op.target_experience_id:
|
if not op.target_experience_id:
|
||||||
logger.warning(f"[coding-reconcile] drop UPDATE missing target: {op.to_dict()}")
|
logger.warning(f"[coding-reconcile] drop UPDATE missing target: {op.to_dict()}")
|
||||||
continue
|
continue
|
||||||
|
if not self._target_was_offered(op, candidates_per_draft):
|
||||||
|
logger.warning(
|
||||||
|
f"[coding-reconcile] drop UPDATE outside authority scope: {op.to_dict()}"
|
||||||
|
)
|
||||||
|
continue
|
||||||
if op.draft_idx in consumed:
|
if op.draft_idx in consumed:
|
||||||
logger.warning(f"[coding-reconcile] drop UPDATE on consumed draft: {op.to_dict()}")
|
logger.warning(f"[coding-reconcile] drop UPDATE on consumed draft: {op.to_dict()}")
|
||||||
continue
|
continue
|
||||||
@@ -476,6 +490,11 @@ class CodingMemoryReconciler:
|
|||||||
if not op.target_experience_id:
|
if not op.target_experience_id:
|
||||||
logger.warning(f"[coding-reconcile] drop DELETE missing target: {op.to_dict()}")
|
logger.warning(f"[coding-reconcile] drop DELETE missing target: {op.to_dict()}")
|
||||||
continue
|
continue
|
||||||
|
if not self._target_was_offered(op, candidates_per_draft):
|
||||||
|
logger.warning(
|
||||||
|
f"[coding-reconcile] drop DELETE outside authority scope: {op.to_dict()}"
|
||||||
|
)
|
||||||
|
continue
|
||||||
# DELETE 需要知道 user_id;通过 draft_idx(如有)或第一个 draft 推断
|
# DELETE 需要知道 user_id;通过 draft_idx(如有)或第一个 draft 推断
|
||||||
user_id = self._infer_user_id(op, drafts)
|
user_id = self._infer_user_id(op, drafts)
|
||||||
if not user_id:
|
if not user_id:
|
||||||
@@ -512,6 +531,19 @@ class CodingMemoryReconciler:
|
|||||||
|
|
||||||
return executed
|
return executed
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _target_was_offered(
|
||||||
|
op: ReconcileOp,
|
||||||
|
candidates_per_draft: List[List[CodingMemory]],
|
||||||
|
) -> bool:
|
||||||
|
"""Only mutate a target frozen into this draft's scoped candidate set."""
|
||||||
|
if op.draft_idx is None or not (0 <= op.draft_idx < len(candidates_per_draft)):
|
||||||
|
return False
|
||||||
|
return any(
|
||||||
|
candidate.experience_id == op.target_experience_id
|
||||||
|
for candidate in candidates_per_draft[op.draft_idx]
|
||||||
|
)
|
||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
def _infer_user_id(
|
def _infer_user_id(
|
||||||
op: ReconcileOp, drafts: List[CodingMemoryDraft]
|
op: ReconcileOp, drafts: List[CodingMemoryDraft]
|
||||||
|
|||||||
+67
-15
@@ -529,10 +529,20 @@ class CodingMemoryStore:
|
|||||||
"""覆盖式更新一条 coding memory。"""
|
"""覆盖式更新一条 coding memory。"""
|
||||||
# 拿旧 keys 长度,先删旧 VDB rows
|
# 拿旧 keys 长度,先删旧 VDB rows
|
||||||
prev = await self.get_by_id(memory.experience_id, user_id=memory.user_id)
|
prev = await self.get_by_id(memory.experience_id, user_id=memory.user_id)
|
||||||
prev_key_count = (1 + len(prev.search_keys)) if prev else 0
|
if prev is None:
|
||||||
|
raise ValueError("coding_memory_update_target_missing")
|
||||||
|
if (
|
||||||
|
not prev.session_id
|
||||||
|
or not memory.session_id
|
||||||
|
or prev.agent_id != memory.agent_id
|
||||||
|
or prev.session_id != memory.session_id
|
||||||
|
or prev.workspace_id != memory.workspace_id
|
||||||
|
):
|
||||||
|
raise ValueError("coding_memory_update_authority_mismatch")
|
||||||
|
prev_key_count = 1 + len(prev.search_keys)
|
||||||
|
|
||||||
memory.updated_at = datetime.utcnow()
|
memory.updated_at = datetime.utcnow()
|
||||||
if memory.created_at is None and prev is not None:
|
if memory.created_at is None:
|
||||||
memory.created_at = prev.created_at
|
memory.created_at = prev.created_at
|
||||||
await _run_in_pool(self._upsert_sql_sync, memory)
|
await _run_in_pool(self._upsert_sql_sync, memory)
|
||||||
# 删旧 VDB
|
# 删旧 VDB
|
||||||
@@ -671,6 +681,8 @@ class CodingMemoryStore:
|
|||||||
query_embedding: List[float],
|
query_embedding: List[float],
|
||||||
user_id: str,
|
user_id: str,
|
||||||
*,
|
*,
|
||||||
|
agent_id: Optional[str] = None,
|
||||||
|
session_id: Optional[str] = None,
|
||||||
workspace_id: Optional[str] = None,
|
workspace_id: Optional[str] = None,
|
||||||
branch: Optional[str] = None,
|
branch: Optional[str] = None,
|
||||||
top: int = 40,
|
top: int = 40,
|
||||||
@@ -688,21 +700,28 @@ class CodingMemoryStore:
|
|||||||
},
|
},
|
||||||
...
|
...
|
||||||
]
|
]
|
||||||
workspace_id / branch 是作用域过滤,规则见 _scope_where:不传则纯语义召回
|
agent_id / session_id 是写授权作用域,传入时精确匹配;workspace_id / branch
|
||||||
(行为与加过滤前完全一致);传了则只保留 SQLite 里落在作用域内的经验,并按
|
的规则见 _scope_where。不传任何作用域则维持纯语义召回;传了则只保留 SQLite
|
||||||
作用域的选择率放大向量取数,避免"先取 top-N 再过滤"把召回吃掉。
|
里落在作用域内的经验,并按选择率放大向量取数,避免"先取 top-N 再过滤"把
|
||||||
|
召回吃掉。
|
||||||
"""
|
"""
|
||||||
if self._vec is None:
|
if self._vec is None:
|
||||||
return []
|
return []
|
||||||
|
|
||||||
allowed: Optional[Set[str]] = None
|
allowed: Optional[Set[str]] = None
|
||||||
if workspace_id:
|
if agent_id or session_id or workspace_id:
|
||||||
allowed, total = await _run_in_pool(
|
allowed, total = await _run_in_pool(
|
||||||
self._scoped_experience_ids_sync, user_id, workspace_id, branch
|
self._authority_scoped_experience_ids_sync,
|
||||||
|
user_id,
|
||||||
|
agent_id,
|
||||||
|
session_id,
|
||||||
|
workspace_id,
|
||||||
|
branch,
|
||||||
)
|
)
|
||||||
if not allowed:
|
if not allowed:
|
||||||
logger.info(
|
logger.info(
|
||||||
f"[coding-store] no experience in scope: "
|
f"[coding-store] no experience in scope: "
|
||||||
|
f"agent_id={agent_id!r} session_id={session_id!r} "
|
||||||
f"workspace_id={workspace_id!r} branch={branch!r} "
|
f"workspace_id={workspace_id!r} branch={branch!r} "
|
||||||
f"(user has {total}); skip vdb search"
|
f"(user has {total}); skip vdb search"
|
||||||
)
|
)
|
||||||
@@ -778,6 +797,8 @@ class CodingMemoryStore:
|
|||||||
query_embedding: List[float],
|
query_embedding: List[float],
|
||||||
user_id: str,
|
user_id: str,
|
||||||
*,
|
*,
|
||||||
|
agent_id: Optional[str] = None,
|
||||||
|
session_id: Optional[str] = None,
|
||||||
workspace_id: Optional[str] = None,
|
workspace_id: Optional[str] = None,
|
||||||
branch: Optional[str] = None,
|
branch: Optional[str] = None,
|
||||||
top: int = 5,
|
top: int = 5,
|
||||||
@@ -791,8 +812,9 @@ class CodingMemoryStore:
|
|||||||
SUPERSEDE 时向量行已被删掉,理论上召不到 superseded;这里再按 SQLite 的
|
SUPERSEDE 时向量行已被删掉,理论上召不到 superseded;这里再按 SQLite 的
|
||||||
status 过一道,避免两边不一致时把过期结论放出去。
|
status 过一道,避免两边不一致时把过期结论放出去。
|
||||||
|
|
||||||
workspace_id / branch 是作用域过滤,规则见 _scope_where,语义与经验层一致:
|
agent_id / session_id 是写授权作用域,传入时精确匹配;workspace_id / branch
|
||||||
不传不过滤,传了按 SQLite 里的值筛,并按选择率放大向量取数。
|
的规则见 _scope_where。不传任何作用域则不过滤,传了按 SQLite 里的值筛,并
|
||||||
|
按选择率放大向量取数。
|
||||||
|
|
||||||
向量库按 top 的 4 倍过采样:向量行和 SQLite 行不是一个事务里写的,中间挂掉
|
向量库按 top 的 4 倍过采样:向量行和 SQLite 行不是一个事务里写的,中间挂掉
|
||||||
就会留下"向量还在、SQLite 没了"的孤儿,软删时删向量失败也会留下 superseded
|
就会留下"向量还在、SQLite 没了"的孤儿,软删时删向量失败也会留下 superseded
|
||||||
@@ -809,13 +831,19 @@ class CodingMemoryStore:
|
|||||||
return []
|
return []
|
||||||
|
|
||||||
fetch = top * 4
|
fetch = top * 4
|
||||||
if workspace_id:
|
if agent_id or session_id or workspace_id:
|
||||||
scoped, total = await _run_in_pool(
|
scoped, total = await _run_in_pool(
|
||||||
self._count_active_facts_sync, user_id, workspace_id, branch
|
self._count_active_facts_sync,
|
||||||
|
user_id,
|
||||||
|
agent_id,
|
||||||
|
session_id,
|
||||||
|
workspace_id,
|
||||||
|
branch,
|
||||||
)
|
)
|
||||||
if not scoped:
|
if not scoped:
|
||||||
logger.info(
|
logger.info(
|
||||||
f"[coding-store] no active fact in scope: "
|
f"[coding-store] no active fact in scope: "
|
||||||
|
f"agent_id={agent_id!r} session_id={session_id!r} "
|
||||||
f"workspace_id={workspace_id!r} branch={branch!r} "
|
f"workspace_id={workspace_id!r} branch={branch!r} "
|
||||||
f"(user has {total}); skip fact vdb search"
|
f"(user has {total}); skip fact vdb search"
|
||||||
)
|
)
|
||||||
@@ -852,6 +880,8 @@ class CodingMemoryStore:
|
|||||||
alive = [
|
alive = [
|
||||||
(f, scores[f.fact_id]) for f in facts
|
(f, scores[f.fact_id]) for f in facts
|
||||||
if f.status == "active"
|
if f.status == "active"
|
||||||
|
and (not agent_id or f.agent_id == agent_id)
|
||||||
|
and (not session_id or f.session_id == session_id)
|
||||||
and _scope_matches(f.workspace_id, f.branch, workspace_id, branch)
|
and _scope_matches(f.workspace_id, f.branch, workspace_id, branch)
|
||||||
]
|
]
|
||||||
kept = [(f, s) for f, s in alive if s >= min_score]
|
kept = [(f, s) for f, s in alive if s >= min_score]
|
||||||
@@ -1050,14 +1080,25 @@ class CodingMemoryStore:
|
|||||||
by_id = {r["experience_id"]: _row_to_memory(r) for r in rows}
|
by_id = {r["experience_id"]: _row_to_memory(r) for r in rows}
|
||||||
return [by_id[eid] for eid in experience_ids if eid in by_id]
|
return [by_id[eid] for eid in experience_ids if eid in by_id]
|
||||||
|
|
||||||
def _scoped_experience_ids_sync(
|
def _authority_scoped_experience_ids_sync(
|
||||||
self, user_id: str, workspace_id: str, branch: Optional[str]
|
self,
|
||||||
|
user_id: str,
|
||||||
|
agent_id: Optional[str],
|
||||||
|
session_id: Optional[str],
|
||||||
|
workspace_id: Optional[str],
|
||||||
|
branch: Optional[str],
|
||||||
) -> Tuple[Set[str], int]:
|
) -> Tuple[Set[str], int]:
|
||||||
"""返回 (落在作用域内的 experience_id 集合, 该 user 的经验总数)。
|
"""返回 (落在授权作用域内的 experience_id 集合, 该 user 的经验总数)。
|
||||||
|
|
||||||
一次查询同时拿到过滤结果和选择率分母:前者供向量命中做交集,后者定放大倍数。
|
一次查询同时拿到过滤结果和选择率分母:前者供向量命中做交集,后者定放大倍数。
|
||||||
"""
|
"""
|
||||||
scope_sql, scope_params = _scope_where(workspace_id, branch)
|
scope_sql, scope_params = _scope_where(workspace_id, branch)
|
||||||
|
if agent_id:
|
||||||
|
scope_sql = f"agent_id = ? AND ({scope_sql})"
|
||||||
|
scope_params = [agent_id, *scope_params]
|
||||||
|
if session_id:
|
||||||
|
scope_sql = f"session_id = ? AND ({scope_sql})"
|
||||||
|
scope_params = [session_id, *scope_params]
|
||||||
with self._lock:
|
with self._lock:
|
||||||
assert self._conn is not None
|
assert self._conn is not None
|
||||||
cur = self._conn.execute(
|
cur = self._conn.execute(
|
||||||
@@ -1069,13 +1110,24 @@ class CodingMemoryStore:
|
|||||||
return {r["experience_id"] for r in rows if r["in_scope"]}, len(rows)
|
return {r["experience_id"] for r in rows if r["in_scope"]}, len(rows)
|
||||||
|
|
||||||
def _count_active_facts_sync(
|
def _count_active_facts_sync(
|
||||||
self, user_id: str, workspace_id: str, branch: Optional[str]
|
self,
|
||||||
|
user_id: str,
|
||||||
|
agent_id: Optional[str],
|
||||||
|
session_id: Optional[str],
|
||||||
|
workspace_id: Optional[str],
|
||||||
|
branch: Optional[str],
|
||||||
) -> Tuple[int, int]:
|
) -> Tuple[int, int]:
|
||||||
"""返回 (作用域内 active 事实数, 该 user 的 active 事实总数)。
|
"""返回 (作用域内 active 事实数, 该 user 的 active 事实总数)。
|
||||||
|
|
||||||
事实层的过滤在 get_facts 拿到对象之后用 _scope_matches 做,这里只要选择率。
|
事实层的过滤在 get_facts 拿到对象之后用 _scope_matches 做,这里只要选择率。
|
||||||
"""
|
"""
|
||||||
scope_sql, scope_params = _scope_where(workspace_id, branch)
|
scope_sql, scope_params = _scope_where(workspace_id, branch)
|
||||||
|
if agent_id:
|
||||||
|
scope_sql = f"agent_id = ? AND ({scope_sql})"
|
||||||
|
scope_params = [agent_id, *scope_params]
|
||||||
|
if session_id:
|
||||||
|
scope_sql = f"session_id = ? AND ({scope_sql})"
|
||||||
|
scope_params = [session_id, *scope_params]
|
||||||
with self._lock:
|
with self._lock:
|
||||||
assert self._conn is not None
|
assert self._conn is not None
|
||||||
row = self._conn.execute(
|
row = self._conn.execute(
|
||||||
|
|||||||
+136
-125
@@ -7,7 +7,7 @@ Agent Memory V2 - SqliteCache
|
|||||||
设计模式跟随 history_store.py:
|
设计模式跟随 history_store.py:
|
||||||
- 标准库 sqlite3 + _run_in_sqlite_pool()
|
- 标准库 sqlite3 + _run_in_sqlite_pool()
|
||||||
- WAL 模式 + check_same_thread=False
|
- WAL 模式 + check_same_thread=False
|
||||||
- threading.Lock 保护写操作
|
- 每实例 threading.RLock 保护共享连接的全部操作与关闭
|
||||||
|
|
||||||
三张表:
|
三张表:
|
||||||
- memory_operations / pipeline_logs / system_metrics: 审计与观测
|
- memory_operations / pipeline_logs / system_metrics: 审计与观测
|
||||||
@@ -122,7 +122,10 @@ class SqliteCache(CacheBase):
|
|||||||
def __init__(self, config: MemoryConfig):
|
def __init__(self, config: MemoryConfig):
|
||||||
self.config = config
|
self.config = config
|
||||||
self._conn: Optional[sqlite3.Connection] = None
|
self._conn: Optional[sqlite3.Connection] = None
|
||||||
self._lock = threading.Lock()
|
# One connection is shared by the executor workers. Use the same lock
|
||||||
|
# for reads, writes, initialization and close so close() cannot free
|
||||||
|
# the connection while another worker is inside sqlite3.
|
||||||
|
self._lock = threading.RLock()
|
||||||
|
|
||||||
# 读取 db_path
|
# 读取 db_path
|
||||||
cache_cfg = getattr(config, "cache", None)
|
cache_cfg = getattr(config, "cache", None)
|
||||||
@@ -144,35 +147,37 @@ class SqliteCache(CacheBase):
|
|||||||
logger.debug(f"SqliteCache initialized: {self._db_path}")
|
logger.debug(f"SqliteCache initialized: {self._db_path}")
|
||||||
|
|
||||||
def _init_sync(self) -> None:
|
def _init_sync(self) -> None:
|
||||||
db_dir = os.path.dirname(self._db_path)
|
with self._lock:
|
||||||
if db_dir:
|
db_dir = os.path.dirname(self._db_path)
|
||||||
os.makedirs(db_dir, exist_ok=True)
|
if db_dir:
|
||||||
self._conn = sqlite3.connect(self._db_path, check_same_thread=False)
|
os.makedirs(db_dir, exist_ok=True)
|
||||||
self._conn.row_factory = sqlite3.Row
|
self._conn = sqlite3.connect(self._db_path, check_same_thread=False)
|
||||||
self._conn.execute("PRAGMA journal_mode=WAL;")
|
self._conn.row_factory = sqlite3.Row
|
||||||
self._conn.executescript(_CREATE_TABLES_SQL)
|
self._conn.execute("PRAGMA journal_mode=WAL;")
|
||||||
self._migrate_pipeline_logs_columns()
|
self._conn.executescript(_CREATE_TABLES_SQL)
|
||||||
self._conn.commit()
|
self._migrate_pipeline_logs_columns()
|
||||||
|
self._conn.commit()
|
||||||
|
|
||||||
def _migrate_pipeline_logs_columns(self) -> None:
|
def _migrate_pipeline_logs_columns(self) -> None:
|
||||||
"""给旧库的 pipeline_logs_v2 补 namespace/session/event time 列。"""
|
"""给旧库的 pipeline_logs_v2 补 namespace/session/event time 列。"""
|
||||||
try:
|
with self._lock:
|
||||||
cols = {
|
try:
|
||||||
row["name"]
|
cols = {
|
||||||
for row in self._conn.execute("PRAGMA table_info(pipeline_logs_v2)")
|
row["name"]
|
||||||
}
|
for row in self._conn.execute("PRAGMA table_info(pipeline_logs_v2)")
|
||||||
_text_cols = ("namespace", "session_id")
|
}
|
||||||
for col in _text_cols:
|
_text_cols = ("namespace", "session_id")
|
||||||
if col not in cols:
|
for col in _text_cols:
|
||||||
|
if col not in cols:
|
||||||
|
self._conn.execute(
|
||||||
|
f"ALTER TABLE pipeline_logs_v2 ADD COLUMN {col} TEXT NOT NULL DEFAULT ''"
|
||||||
|
)
|
||||||
|
if "event_at_ms" not in cols:
|
||||||
self._conn.execute(
|
self._conn.execute(
|
||||||
f"ALTER TABLE pipeline_logs_v2 ADD COLUMN {col} TEXT NOT NULL DEFAULT ''"
|
"ALTER TABLE pipeline_logs_v2 ADD COLUMN event_at_ms INTEGER NOT NULL DEFAULT 0"
|
||||||
)
|
)
|
||||||
if "event_at_ms" not in cols:
|
except Exception as e:
|
||||||
self._conn.execute(
|
logger.debug(f"[sqlite] pipeline_logs_v2 column migration skipped: {e}")
|
||||||
"ALTER TABLE pipeline_logs_v2 ADD COLUMN event_at_ms INTEGER NOT NULL DEFAULT 0"
|
|
||||||
)
|
|
||||||
except Exception as e:
|
|
||||||
logger.debug(f"[sqlite] pipeline_logs_v2 column migration skipped: {e}")
|
|
||||||
|
|
||||||
async def close(self) -> None:
|
async def close(self) -> None:
|
||||||
if self._conn is not None:
|
if self._conn is not None:
|
||||||
@@ -180,9 +185,10 @@ class SqliteCache(CacheBase):
|
|||||||
logger.info("SqliteCache closed")
|
logger.info("SqliteCache closed")
|
||||||
|
|
||||||
def _close_sync(self) -> None:
|
def _close_sync(self) -> None:
|
||||||
if self._conn:
|
with self._lock:
|
||||||
self._conn.close()
|
if self._conn:
|
||||||
self._conn = None
|
self._conn.close()
|
||||||
|
self._conn = None
|
||||||
|
|
||||||
# ================================================================
|
# ================================================================
|
||||||
# 统计
|
# 统计
|
||||||
@@ -255,39 +261,40 @@ class SqliteCache(CacheBase):
|
|||||||
"""
|
"""
|
||||||
try:
|
try:
|
||||||
def _query():
|
def _query():
|
||||||
conditions = []
|
with self._lock:
|
||||||
params = []
|
conditions = []
|
||||||
if request_id:
|
params = []
|
||||||
conditions.append("request_id = ?")
|
if request_id:
|
||||||
params.append(request_id)
|
conditions.append("request_id = ?")
|
||||||
if memory_id:
|
params.append(request_id)
|
||||||
conditions.append("memory_id = ?")
|
if memory_id:
|
||||||
params.append(memory_id)
|
conditions.append("memory_id = ?")
|
||||||
if user_id:
|
params.append(memory_id)
|
||||||
conditions.append("user_id = ?")
|
if user_id:
|
||||||
params.append(user_id)
|
conditions.append("user_id = ?")
|
||||||
|
params.append(user_id)
|
||||||
|
|
||||||
if not conditions:
|
if not conditions:
|
||||||
return []
|
return []
|
||||||
|
|
||||||
where = " AND ".join(conditions)
|
where = " AND ".join(conditions)
|
||||||
rows = self._conn.execute(
|
rows = self._conn.execute(
|
||||||
f"SELECT * FROM memory_operations_v2 WHERE {where} "
|
f"SELECT * FROM memory_operations_v2 WHERE {where} "
|
||||||
f"ORDER BY id DESC LIMIT ?",
|
f"ORDER BY id DESC LIMIT ?",
|
||||||
params + [limit],
|
params + [limit],
|
||||||
).fetchall()
|
).fetchall()
|
||||||
result = []
|
result = []
|
||||||
for r in rows:
|
for r in rows:
|
||||||
row = dict(r)
|
row = dict(r)
|
||||||
# 反序列化 supersedes JSON 字符串 → list
|
# 反序列化 supersedes JSON 字符串 → list
|
||||||
import json as _json
|
import json as _json
|
||||||
raw_sup = row.get("supersedes", "")
|
raw_sup = row.get("supersedes", "")
|
||||||
try:
|
try:
|
||||||
row["supersedes"] = _json.loads(raw_sup) if raw_sup else []
|
row["supersedes"] = _json.loads(raw_sup) if raw_sup else []
|
||||||
except Exception:
|
except Exception:
|
||||||
row["supersedes"] = []
|
row["supersedes"] = []
|
||||||
result.append(row)
|
result.append(row)
|
||||||
return result
|
return result
|
||||||
|
|
||||||
return await _run_in_sqlite_pool(_query)
|
return await _run_in_sqlite_pool(_query)
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
@@ -369,37 +376,38 @@ class SqliteCache(CacheBase):
|
|||||||
"""
|
"""
|
||||||
try:
|
try:
|
||||||
def _query():
|
def _query():
|
||||||
conditions = []
|
with self._lock:
|
||||||
params = []
|
conditions = []
|
||||||
if request_id:
|
params = []
|
||||||
conditions.append("request_id = ?")
|
if request_id:
|
||||||
params.append(request_id)
|
conditions.append("request_id = ?")
|
||||||
if user_id:
|
params.append(request_id)
|
||||||
conditions.append("user_id = ?")
|
if user_id:
|
||||||
params.append(user_id)
|
conditions.append("user_id = ?")
|
||||||
if step:
|
params.append(user_id)
|
||||||
conditions.append("step = ?")
|
if step:
|
||||||
params.append(step)
|
conditions.append("step = ?")
|
||||||
|
params.append(step)
|
||||||
|
|
||||||
if not conditions:
|
if not conditions:
|
||||||
return []
|
return []
|
||||||
|
|
||||||
where = " AND ".join(conditions)
|
where = " AND ".join(conditions)
|
||||||
rows = self._conn.execute(
|
rows = self._conn.execute(
|
||||||
f"SELECT * FROM pipeline_logs_v2 WHERE {where} "
|
f"SELECT * FROM pipeline_logs_v2 WHERE {where} "
|
||||||
f"ORDER BY id ASC LIMIT ?",
|
f"ORDER BY id ASC LIMIT ?",
|
||||||
params + [limit],
|
params + [limit],
|
||||||
).fetchall()
|
).fetchall()
|
||||||
out = [dict(r) for r in rows]
|
out = [dict(r) for r in rows]
|
||||||
# 派生可读的 event_at(ISO ms)供查询接口直接用;event_at_ms 为 0 时留空
|
# 派生可读的 event_at(ISO ms)供查询接口直接用;event_at_ms 为 0 时留空
|
||||||
from datetime import datetime as _dt
|
from datetime import datetime as _dt
|
||||||
for _r in out:
|
for _r in out:
|
||||||
_ms = _r.get("event_at_ms") or 0
|
_ms = _r.get("event_at_ms") or 0
|
||||||
_r["event_at"] = (
|
_r["event_at"] = (
|
||||||
_dt.fromtimestamp(_ms / 1000.0).isoformat(timespec="milliseconds")
|
_dt.fromtimestamp(_ms / 1000.0).isoformat(timespec="milliseconds")
|
||||||
if _ms else ""
|
if _ms else ""
|
||||||
)
|
)
|
||||||
return out
|
return out
|
||||||
|
|
||||||
return await _run_in_sqlite_pool(_query)
|
return await _run_in_sqlite_pool(_query)
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
@@ -414,26 +422,27 @@ class SqliteCache(CacheBase):
|
|||||||
"""存储一个分钟桶的增量指标数据(UPSERT: 同一分钟多次写入会合并)"""
|
"""存储一个分钟桶的增量指标数据(UPSERT: 同一分钟多次写入会合并)"""
|
||||||
try:
|
try:
|
||||||
def _store():
|
def _store():
|
||||||
now = datetime.now().isoformat()
|
with self._lock:
|
||||||
# UPSERT: 如果同一分钟已存在,合并数据
|
now = datetime.now().isoformat()
|
||||||
existing = self._conn.execute(
|
# UPSERT: 如果同一分钟已存在,合并数据
|
||||||
"SELECT data FROM system_metrics WHERE minute_ts = ?",
|
existing = self._conn.execute(
|
||||||
(minute_ts,),
|
"SELECT data FROM system_metrics WHERE minute_ts = ?",
|
||||||
).fetchone()
|
(minute_ts,),
|
||||||
if existing:
|
).fetchone()
|
||||||
import json as _json
|
if existing:
|
||||||
old_data = _json.loads(existing["data"])
|
import json as _json
|
||||||
merged = self._merge_metric_buckets(old_data, data)
|
old_data = _json.loads(existing["data"])
|
||||||
self._conn.execute(
|
merged = self._merge_metric_buckets(old_data, data)
|
||||||
"UPDATE system_metrics SET data = ?, created_at = ? WHERE minute_ts = ?",
|
self._conn.execute(
|
||||||
(_json.dumps(merged, default=str), now, minute_ts),
|
"UPDATE system_metrics SET data = ?, created_at = ? WHERE minute_ts = ?",
|
||||||
)
|
(_json.dumps(merged, default=str), now, minute_ts),
|
||||||
else:
|
)
|
||||||
self._conn.execute(
|
else:
|
||||||
"INSERT INTO system_metrics (minute_ts, data, created_at) VALUES (?, ?, ?)",
|
self._conn.execute(
|
||||||
(minute_ts, json.dumps(data, default=str), now),
|
"INSERT INTO system_metrics (minute_ts, data, created_at) VALUES (?, ?, ?)",
|
||||||
)
|
(minute_ts, json.dumps(data, default=str), now),
|
||||||
self._conn.commit()
|
)
|
||||||
|
self._conn.commit()
|
||||||
await _run_in_sqlite_pool(_store)
|
await _run_in_sqlite_pool(_store)
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
logger.warning(f"store_metrics_minute failed: {e}")
|
logger.warning(f"store_metrics_minute failed: {e}")
|
||||||
@@ -442,17 +451,18 @@ class SqliteCache(CacheBase):
|
|||||||
"""读取指定时间范围内的分钟指标数据"""
|
"""读取指定时间范围内的分钟指标数据"""
|
||||||
try:
|
try:
|
||||||
def _load():
|
def _load():
|
||||||
rows = self._conn.execute(
|
with self._lock:
|
||||||
"SELECT data FROM system_metrics WHERE minute_ts >= ? AND minute_ts <= ? ORDER BY minute_ts",
|
rows = self._conn.execute(
|
||||||
(start_ts, end_ts),
|
"SELECT data FROM system_metrics WHERE minute_ts >= ? AND minute_ts <= ? ORDER BY minute_ts",
|
||||||
).fetchall()
|
(start_ts, end_ts),
|
||||||
results = []
|
).fetchall()
|
||||||
for row in rows:
|
results = []
|
||||||
try:
|
for row in rows:
|
||||||
results.append(json.loads(row["data"]))
|
try:
|
||||||
except (json.JSONDecodeError, TypeError):
|
results.append(json.loads(row["data"]))
|
||||||
pass
|
except (json.JSONDecodeError, TypeError):
|
||||||
return results
|
pass
|
||||||
|
return results
|
||||||
return await _run_in_sqlite_pool(_load)
|
return await _run_in_sqlite_pool(_load)
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
logger.warning(f"load_metrics_range failed: {e}")
|
logger.warning(f"load_metrics_range failed: {e}")
|
||||||
@@ -462,11 +472,12 @@ class SqliteCache(CacheBase):
|
|||||||
"""删除指定时间之前的 metrics 数据"""
|
"""删除指定时间之前的 metrics 数据"""
|
||||||
try:
|
try:
|
||||||
def _cleanup():
|
def _cleanup():
|
||||||
self._conn.execute(
|
with self._lock:
|
||||||
"DELETE FROM system_metrics WHERE minute_ts < ?",
|
self._conn.execute(
|
||||||
(before_ts,),
|
"DELETE FROM system_metrics WHERE minute_ts < ?",
|
||||||
)
|
(before_ts,),
|
||||||
self._conn.commit()
|
)
|
||||||
|
self._conn.commit()
|
||||||
await _run_in_sqlite_pool(_cleanup)
|
await _run_in_sqlite_pool(_cleanup)
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
logger.warning(f"cleanup_old_metrics failed: {e}")
|
logger.warning(f"cleanup_old_metrics failed: {e}")
|
||||||
|
|||||||
@@ -16,6 +16,7 @@ import json
|
|||||||
import logging
|
import logging
|
||||||
import os
|
import os
|
||||||
import sqlite3
|
import sqlite3
|
||||||
|
import threading
|
||||||
from datetime import datetime, timezone
|
from datetime import datetime, timezone
|
||||||
from typing import Dict, List, Any, Optional
|
from typing import Dict, List, Any, Optional
|
||||||
|
|
||||||
@@ -89,6 +90,10 @@ class HistoryStore:
|
|||||||
"""
|
"""
|
||||||
self._db_path = config.history.db_path
|
self._db_path = config.history.db_path
|
||||||
self._conn: Optional[sqlite3.Connection] = None
|
self._conn: Optional[sqlite3.Connection] = None
|
||||||
|
# sqlite3.Connection.close() must never race with an execute() on the
|
||||||
|
# same connection. The lock also serializes operations on this shared
|
||||||
|
# connection; separate HistoryStore instances remain independent.
|
||||||
|
self._conn_lock = threading.RLock()
|
||||||
|
|
||||||
async def initialize(self) -> None:
|
async def initialize(self) -> None:
|
||||||
"""创建表和索引(如果不存在)"""
|
"""创建表和索引(如果不存在)"""
|
||||||
@@ -97,20 +102,21 @@ class HistoryStore:
|
|||||||
|
|
||||||
def _init_sync(self) -> None:
|
def _init_sync(self) -> None:
|
||||||
"""同步初始化:创建目录、连接数据库、建表建索引"""
|
"""同步初始化:创建目录、连接数据库、建表建索引"""
|
||||||
# 确保目录存在
|
with self._conn_lock:
|
||||||
db_dir = os.path.dirname(self._db_path)
|
# 确保目录存在
|
||||||
if db_dir:
|
db_dir = os.path.dirname(self._db_path)
|
||||||
os.makedirs(db_dir, exist_ok=True)
|
if db_dir:
|
||||||
|
os.makedirs(db_dir, exist_ok=True)
|
||||||
|
|
||||||
self._conn = sqlite3.connect(self._db_path, check_same_thread=False)
|
self._conn = sqlite3.connect(self._db_path, check_same_thread=False)
|
||||||
self._conn.row_factory = sqlite3.Row
|
self._conn.row_factory = sqlite3.Row
|
||||||
# 开启 WAL 模式,提高并发读写性能
|
# 开启 WAL 模式,提高并发读写性能
|
||||||
self._conn.execute("PRAGMA journal_mode=WAL;")
|
self._conn.execute("PRAGMA journal_mode=WAL;")
|
||||||
|
|
||||||
self._conn.execute(_CREATE_TABLE_SQL)
|
self._conn.execute(_CREATE_TABLE_SQL)
|
||||||
for idx_sql in _CREATE_INDEXES_SQL:
|
for idx_sql in _CREATE_INDEXES_SQL:
|
||||||
self._conn.execute(idx_sql)
|
self._conn.execute(idx_sql)
|
||||||
self._conn.commit()
|
self._conn.commit()
|
||||||
|
|
||||||
# ================================================================
|
# ================================================================
|
||||||
# 记录操作
|
# 记录操作
|
||||||
@@ -218,13 +224,14 @@ class HistoryStore:
|
|||||||
return await _run_in_sqlite_pool(self._get_history_sync, memory_id)
|
return await _run_in_sqlite_pool(self._get_history_sync, memory_id)
|
||||||
|
|
||||||
def _get_history_sync(self, memory_id: str) -> List[Dict[str, Any]]:
|
def _get_history_sync(self, memory_id: str) -> List[Dict[str, Any]]:
|
||||||
if self._conn is None:
|
with self._conn_lock:
|
||||||
return []
|
if self._conn is None:
|
||||||
cursor = self._conn.execute(
|
return []
|
||||||
"SELECT * FROM memory_history WHERE memory_id = ? ORDER BY created_at ASC",
|
cursor = self._conn.execute(
|
||||||
(memory_id,),
|
"SELECT * FROM memory_history WHERE memory_id = ? ORDER BY created_at ASC",
|
||||||
)
|
(memory_id,),
|
||||||
return [dict(row) for row in cursor.fetchall()]
|
)
|
||||||
|
return [dict(row) for row in cursor.fetchall()]
|
||||||
|
|
||||||
async def get_recent(
|
async def get_recent(
|
||||||
self,
|
self,
|
||||||
@@ -244,14 +251,15 @@ class HistoryStore:
|
|||||||
return await _run_in_sqlite_pool(self._get_recent_sync, isolation_key, limit)
|
return await _run_in_sqlite_pool(self._get_recent_sync, isolation_key, limit)
|
||||||
|
|
||||||
def _get_recent_sync(self, isolation_key: str, limit: int) -> List[Dict[str, Any]]:
|
def _get_recent_sync(self, isolation_key: str, limit: int) -> List[Dict[str, Any]]:
|
||||||
if self._conn is None:
|
with self._conn_lock:
|
||||||
return []
|
if self._conn is None:
|
||||||
cursor = self._conn.execute(
|
return []
|
||||||
"SELECT * FROM memory_history WHERE isolation_key = ? "
|
cursor = self._conn.execute(
|
||||||
"ORDER BY created_at DESC LIMIT ?",
|
"SELECT * FROM memory_history WHERE isolation_key = ? "
|
||||||
(isolation_key, limit),
|
"ORDER BY created_at DESC LIMIT ?",
|
||||||
)
|
(isolation_key, limit),
|
||||||
return [dict(row) for row in cursor.fetchall()]
|
)
|
||||||
|
return [dict(row) for row in cursor.fetchall()]
|
||||||
|
|
||||||
async def count(self, isolation_key: str = "") -> int:
|
async def count(self, isolation_key: str = "") -> int:
|
||||||
"""
|
"""
|
||||||
@@ -266,16 +274,17 @@ class HistoryStore:
|
|||||||
return await _run_in_sqlite_pool(self._count_sync, isolation_key)
|
return await _run_in_sqlite_pool(self._count_sync, isolation_key)
|
||||||
|
|
||||||
def _count_sync(self, isolation_key: str) -> int:
|
def _count_sync(self, isolation_key: str) -> int:
|
||||||
if self._conn is None:
|
with self._conn_lock:
|
||||||
return 0
|
if self._conn is None:
|
||||||
if isolation_key:
|
return 0
|
||||||
cursor = self._conn.execute(
|
if isolation_key:
|
||||||
"SELECT COUNT(*) FROM memory_history WHERE isolation_key = ?",
|
cursor = self._conn.execute(
|
||||||
(isolation_key,),
|
"SELECT COUNT(*) FROM memory_history WHERE isolation_key = ?",
|
||||||
)
|
(isolation_key,),
|
||||||
else:
|
)
|
||||||
cursor = self._conn.execute("SELECT COUNT(*) FROM memory_history")
|
else:
|
||||||
return cursor.fetchone()[0]
|
cursor = self._conn.execute("SELECT COUNT(*) FROM memory_history")
|
||||||
|
return cursor.fetchone()[0]
|
||||||
|
|
||||||
async def close(self) -> None:
|
async def close(self) -> None:
|
||||||
"""关闭数据库连接"""
|
"""关闭数据库连接"""
|
||||||
@@ -284,9 +293,10 @@ class HistoryStore:
|
|||||||
logger.info("[HistoryStore] closed")
|
logger.info("[HistoryStore] closed")
|
||||||
|
|
||||||
def _close_sync(self) -> None:
|
def _close_sync(self) -> None:
|
||||||
if self._conn:
|
with self._conn_lock:
|
||||||
self._conn.close()
|
if self._conn:
|
||||||
self._conn = None
|
self._conn.close()
|
||||||
|
self._conn = None
|
||||||
|
|
||||||
# ================================================================
|
# ================================================================
|
||||||
# 内部
|
# 内部
|
||||||
@@ -339,33 +349,34 @@ class HistoryStore:
|
|||||||
role: str = "",
|
role: str = "",
|
||||||
extra: Optional[Dict[str, Any]] = None,
|
extra: Optional[Dict[str, Any]] = None,
|
||||||
) -> int:
|
) -> int:
|
||||||
if self._conn is None:
|
with self._conn_lock:
|
||||||
return -1
|
if self._conn is None:
|
||||||
now = _now_iso()
|
return -1
|
||||||
extra_json = json.dumps(extra or {}, ensure_ascii=False)
|
now = _now_iso()
|
||||||
cursor = self._conn.execute(
|
extra_json = json.dumps(extra or {}, ensure_ascii=False)
|
||||||
"""INSERT INTO memory_history
|
cursor = self._conn.execute(
|
||||||
(memory_id, isolation_key, event,
|
"""INSERT INTO memory_history
|
||||||
old_memory, new_memory, old_status, new_status,
|
(memory_id, isolation_key, event,
|
||||||
change_reason, layer, actor_id, role, extra,
|
old_memory, new_memory, old_status, new_status,
|
||||||
created_at, updated_at)
|
change_reason, layer, actor_id, role, extra,
|
||||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)""",
|
created_at, updated_at)
|
||||||
(
|
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)""",
|
||||||
memory_id,
|
(
|
||||||
isolation_key,
|
memory_id,
|
||||||
event,
|
isolation_key,
|
||||||
old_memory,
|
event,
|
||||||
new_memory,
|
old_memory,
|
||||||
old_status,
|
new_memory,
|
||||||
new_status,
|
old_status,
|
||||||
change_reason,
|
new_status,
|
||||||
layer,
|
change_reason,
|
||||||
actor_id,
|
layer,
|
||||||
role,
|
actor_id,
|
||||||
extra_json,
|
role,
|
||||||
now,
|
extra_json,
|
||||||
now,
|
now,
|
||||||
),
|
now,
|
||||||
)
|
),
|
||||||
self._conn.commit()
|
)
|
||||||
return cursor.lastrowid
|
self._conn.commit()
|
||||||
|
return cursor.lastrowid
|
||||||
@@ -12,7 +12,6 @@
|
|||||||
* It is generated rather than hand-written so the two cannot drift: this reads
|
* It is generated rather than hand-written so the two cannot drift: this reads
|
||||||
* the real package.json and only strips what is provably not needed at runtime.
|
* the real package.json and only strips what is provably not needed at runtime.
|
||||||
*/
|
*/
|
||||||
import { execFileSync } from "node:child_process";
|
|
||||||
import { mkdirSync, readFileSync, writeFileSync } from "node:fs";
|
import { mkdirSync, readFileSync, writeFileSync } from "node:fs";
|
||||||
import { createRequire } from "node:module";
|
import { createRequire } from "node:module";
|
||||||
import { dirname, resolve } from "node:path";
|
import { dirname, resolve } from "node:path";
|
||||||
@@ -20,12 +19,14 @@ import { fileURLToPath } from "node:url";
|
|||||||
|
|
||||||
const here = dirname(fileURLToPath(import.meta.url));
|
const here = dirname(fileURLToPath(import.meta.url));
|
||||||
const root = resolve(here, "..");
|
const root = resolve(here, "..");
|
||||||
const repositoryRoot = resolve(root, "../..");
|
|
||||||
const require = createRequire(import.meta.url);
|
const require = createRequire(import.meta.url);
|
||||||
const { resolveBuildVersion } = require("../../../scripts/release-version.cjs");
|
const { assertSemver } = require("../../../scripts/release-version.cjs");
|
||||||
const releaseVersion = resolveBuildVersion(process.env, repositoryRoot);
|
|
||||||
const pkgPath = resolve(root, "package.json");
|
const pkgPath = resolve(root, "package.json");
|
||||||
const pkg = JSON.parse(readFileSync(pkgPath, "utf8"));
|
const pkg = JSON.parse(readFileSync(pkgPath, "utf8"));
|
||||||
|
const publishVersion = assertSemver(
|
||||||
|
process.env.MEMHUB_NPM_VERSION?.trim() || pkg.version,
|
||||||
|
"MEMHUB_NPM_VERSION",
|
||||||
|
);
|
||||||
const publishName = process.env.MEMHUB_NPM_PACKAGE?.trim();
|
const publishName = process.env.MEMHUB_NPM_PACKAGE?.trim();
|
||||||
if (!publishName || !/^(?:@[a-z0-9._-]+\/)?[a-z0-9._-]+$/.test(publishName)) {
|
if (!publishName || !/^(?:@[a-z0-9._-]+\/)?[a-z0-9._-]+$/.test(publishName)) {
|
||||||
throw new Error(
|
throw new Error(
|
||||||
@@ -53,7 +54,7 @@ mkdirSync(outDir, { recursive: true });
|
|||||||
|
|
||||||
const published = { ...pkg };
|
const published = { ...pkg };
|
||||||
published.name = publishName;
|
published.name = publishName;
|
||||||
published.version = releaseVersion;
|
published.version = publishVersion;
|
||||||
for (const section of ["dependencies", "devDependencies", "peerDependencies"]) {
|
for (const section of ["dependencies", "devDependencies", "peerDependencies"]) {
|
||||||
if (!published[section]) continue;
|
if (!published[section]) continue;
|
||||||
published[section] = Object.fromEntries(
|
published[section] = Object.fromEntries(
|
||||||
|
|||||||
@@ -331,7 +331,7 @@ import {
|
|||||||
renderVersion,
|
renderVersion,
|
||||||
renderVersionDocument,
|
renderVersionDocument,
|
||||||
} from "./usage.js";
|
} from "./usage.js";
|
||||||
import { CLI_VERSION, PRODUCT } from "./version.js";
|
import { CLI_VERSION, PRODUCT, RUNTIME_RELEASE_VERSION } from "./version.js";
|
||||||
import { runWatchLoop, type WatchEvent, type WatchRead } from "./watch-loop.js";
|
import { runWatchLoop, type WatchEvent, type WatchRead } from "./watch-loop.js";
|
||||||
|
|
||||||
export interface RunControls {
|
export interface RunControls {
|
||||||
@@ -417,7 +417,8 @@ async function selectInstallSource(
|
|||||||
throw new Error("release_manifest_url_unavailable");
|
throw new Error("release_manifest_url_unavailable");
|
||||||
}
|
}
|
||||||
const manifestUrl =
|
const manifestUrl =
|
||||||
explicitManifestUrl ?? template.replaceAll("{version}", CLI_VERSION);
|
explicitManifestUrl ??
|
||||||
|
template.replaceAll("{version}", RUNTIME_RELEASE_VERSION);
|
||||||
const allowedHosts = parseAllowedReleaseHosts(
|
const allowedHosts = parseAllowedReleaseHosts(
|
||||||
environment.MEMHUB_RELEASE_ALLOWED_HOSTS,
|
environment.MEMHUB_RELEASE_ALLOWED_HOSTS,
|
||||||
);
|
);
|
||||||
@@ -462,7 +463,7 @@ async function resolveRemoteInstallPreview(
|
|||||||
): Promise<ResolvedRuntimeRelease> {
|
): Promise<ResolvedRuntimeRelease> {
|
||||||
return (controls.resolveRuntimeRelease ?? resolveRuntimeRelease)({
|
return (controls.resolveRuntimeRelease ?? resolveRuntimeRelease)({
|
||||||
manifestUrl: selection.manifestUrl,
|
manifestUrl: selection.manifestUrl,
|
||||||
expectedVersion: CLI_VERSION,
|
expectedVersion: RUNTIME_RELEASE_VERSION,
|
||||||
channel: selection.channel,
|
channel: selection.channel,
|
||||||
platform: process.platform,
|
platform: process.platform,
|
||||||
arch: process.arch,
|
arch: process.arch,
|
||||||
@@ -476,7 +477,7 @@ async function acquireRemoteInstallSource(
|
|||||||
): Promise<AcquiredRuntimeDistribution> {
|
): Promise<AcquiredRuntimeDistribution> {
|
||||||
return (controls.acquireRuntimeDistribution ?? acquireRuntimeDistribution)({
|
return (controls.acquireRuntimeDistribution ?? acquireRuntimeDistribution)({
|
||||||
manifestUrl: selection.manifestUrl,
|
manifestUrl: selection.manifestUrl,
|
||||||
expectedVersion: CLI_VERSION,
|
expectedVersion: RUNTIME_RELEASE_VERSION,
|
||||||
channel: selection.channel,
|
channel: selection.channel,
|
||||||
platform: process.platform,
|
platform: process.platform,
|
||||||
arch: process.arch,
|
arch: process.arch,
|
||||||
@@ -6520,7 +6521,7 @@ function parseInstallOptions(
|
|||||||
...(releaseManifestUrl ? { releaseManifestUrl } : {}),
|
...(releaseManifestUrl ? { releaseManifestUrl } : {}),
|
||||||
releaseChannel:
|
releaseChannel:
|
||||||
releaseChannel ??
|
releaseChannel ??
|
||||||
(CLI_VERSION.includes("-") ? "prerelease" : "stable"),
|
(RUNTIME_RELEASE_VERSION.includes("-") ? "prerelease" : "stable"),
|
||||||
installRoot: resolvedInstallRoot,
|
installRoot: resolvedInstallRoot,
|
||||||
home: resolvedHome,
|
home: resolvedHome,
|
||||||
uid: uid ?? process.getuid?.(),
|
uid: uid ?? process.getuid?.(),
|
||||||
|
|||||||
@@ -48,7 +48,11 @@ describe("publish manifest is production-safe", () => {
|
|||||||
execFileSync("node", ["scripts/prepare-publish-manifest.mjs"], {
|
execFileSync("node", ["scripts/prepare-publish-manifest.mjs"], {
|
||||||
cwd: root,
|
cwd: root,
|
||||||
stdio: "pipe",
|
stdio: "pipe",
|
||||||
env: { ...process.env, MEMHUB_NPM_PACKAGE: "@example/memhub-cli" },
|
env: {
|
||||||
|
...process.env,
|
||||||
|
MEMHUB_NPM_PACKAGE: "@example/memhub-cli",
|
||||||
|
MEMHUB_NPM_VERSION: "1.0.5",
|
||||||
|
},
|
||||||
});
|
});
|
||||||
const published = JSON.parse(
|
const published = JSON.parse(
|
||||||
readFileSync(path.join(root, "dist-publish", "package.json"), "utf8"),
|
readFileSync(path.join(root, "dist-publish", "package.json"), "utf8"),
|
||||||
@@ -56,6 +60,7 @@ describe("publish manifest is production-safe", () => {
|
|||||||
expect(published.dependencies).toBeUndefined();
|
expect(published.dependencies).toBeUndefined();
|
||||||
expect(published.private).toBe(false);
|
expect(published.private).toBe(false);
|
||||||
expect(published.name).toBe("@example/memhub-cli");
|
expect(published.name).toBe("@example/memhub-cli");
|
||||||
|
expect(published.version).toBe("1.0.5");
|
||||||
expect(published.bin).toMatchObject({
|
expect(published.bin).toMatchObject({
|
||||||
memhub: "dist/one-memory.cjs",
|
memhub: "dist/one-memory.cjs",
|
||||||
"memhub-cli": "dist/one-memory.cjs",
|
"memhub-cli": "dist/one-memory.cjs",
|
||||||
|
|||||||
@@ -1,5 +1,9 @@
|
|||||||
declare const __ONE_MEMORY_VERSION__: string;
|
declare const __ONE_MEMORY_VERSION__: string;
|
||||||
declare const __ONE_MEMORY_BUILD_NUMBER__: string;
|
declare const __ONE_MEMORY_BUILD_NUMBER__: string;
|
||||||
|
declare const __MEMHUB_DEFAULT_RUNTIME_VERSION__: string;
|
||||||
|
|
||||||
|
const SEMVER =
|
||||||
|
/^(0|[1-9]\d*)\.(0|[1-9]\d*)\.(0|[1-9]\d*)(?:-[0-9A-Za-z-]+(?:\.[0-9A-Za-z-]+)*)?(?:\+[0-9A-Za-z-]+(?:\.[0-9A-Za-z-]+)*)?$/;
|
||||||
|
|
||||||
const embeddedVersion =
|
const embeddedVersion =
|
||||||
typeof __ONE_MEMORY_VERSION__ === "string"
|
typeof __ONE_MEMORY_VERSION__ === "string"
|
||||||
@@ -9,13 +13,18 @@ const embeddedBuildNumber =
|
|||||||
typeof __ONE_MEMORY_BUILD_NUMBER__ === "string"
|
typeof __ONE_MEMORY_BUILD_NUMBER__ === "string"
|
||||||
? __ONE_MEMORY_BUILD_NUMBER__.trim()
|
? __ONE_MEMORY_BUILD_NUMBER__.trim()
|
||||||
: "";
|
: "";
|
||||||
|
const embeddedRuntimeVersion =
|
||||||
|
typeof __MEMHUB_DEFAULT_RUNTIME_VERSION__ === "string"
|
||||||
|
? __MEMHUB_DEFAULT_RUNTIME_VERSION__.trim()
|
||||||
|
: "";
|
||||||
|
|
||||||
export const CLI_VERSION =
|
export const CLI_VERSION = SEMVER.test(embeddedVersion)
|
||||||
/^(0|[1-9]\d*)\.(0|[1-9]\d*)\.(0|[1-9]\d*)(?:-[0-9A-Za-z-]+(?:\.[0-9A-Za-z-]+)*)?(?:\+[0-9A-Za-z-]+(?:\.[0-9A-Za-z-]+)*)?$/.test(
|
? embeddedVersion
|
||||||
embeddedVersion,
|
: "development";
|
||||||
)
|
|
||||||
? embeddedVersion
|
export const RUNTIME_RELEASE_VERSION = SEMVER.test(embeddedRuntimeVersion)
|
||||||
: "development";
|
? embeddedRuntimeVersion
|
||||||
|
: CLI_VERSION;
|
||||||
|
|
||||||
export const CLI_BUILD_NUMBER = /^\d+$/.test(embeddedBuildNumber)
|
export const CLI_BUILD_NUMBER = /^\d+$/.test(embeddedBuildNumber)
|
||||||
? Number(embeddedBuildNumber)
|
? Number(embeddedBuildNumber)
|
||||||
|
|||||||
@@ -2,11 +2,22 @@ import { createRequire } from "node:module";
|
|||||||
import { defineConfig } from "tsup";
|
import { defineConfig } from "tsup";
|
||||||
|
|
||||||
const require = createRequire(import.meta.url);
|
const require = createRequire(import.meta.url);
|
||||||
const { resolveBuildVersion } =
|
const { assertSemver, resolveBuildVersion } =
|
||||||
require("../../scripts/release-version.cjs") as {
|
require("../../scripts/release-version.cjs") as {
|
||||||
|
assertSemver(value: string, label?: string): string;
|
||||||
resolveBuildVersion(environment?: NodeJS.ProcessEnv, root?: string): string;
|
resolveBuildVersion(environment?: NodeJS.ProcessEnv, root?: string): string;
|
||||||
};
|
};
|
||||||
const releaseVersion = resolveBuildVersion(process.env);
|
const releaseVersion = resolveBuildVersion(process.env);
|
||||||
|
const packageVersion = (require("./package.json") as { version: string })
|
||||||
|
.version;
|
||||||
|
const cliVersion = assertSemver(
|
||||||
|
process.env.MEMHUB_NPM_VERSION?.trim() || packageVersion,
|
||||||
|
"MEMHUB_NPM_VERSION",
|
||||||
|
);
|
||||||
|
const runtimeReleaseVersion = assertSemver(
|
||||||
|
process.env.MEMHUB_DEFAULT_RUNTIME_VERSION?.trim() || releaseVersion,
|
||||||
|
"MEMHUB_DEFAULT_RUNTIME_VERSION",
|
||||||
|
);
|
||||||
|
|
||||||
export default defineConfig({
|
export default defineConfig({
|
||||||
entry: { "one-memory": "src/entry.ts" },
|
entry: { "one-memory": "src/entry.ts" },
|
||||||
@@ -15,7 +26,7 @@ export default defineConfig({
|
|||||||
outDir: "dist",
|
outDir: "dist",
|
||||||
clean: true,
|
clean: true,
|
||||||
define: {
|
define: {
|
||||||
__ONE_MEMORY_VERSION__: JSON.stringify(releaseVersion),
|
__ONE_MEMORY_VERSION__: JSON.stringify(cliVersion),
|
||||||
__ONE_MEMORY_BUILD_NUMBER__: JSON.stringify(
|
__ONE_MEMORY_BUILD_NUMBER__: JSON.stringify(
|
||||||
process.env.ONE_MEMORY_BUILD_NUMBER ?? "unknown",
|
process.env.ONE_MEMORY_BUILD_NUMBER ?? "unknown",
|
||||||
),
|
),
|
||||||
@@ -25,6 +36,7 @@ export default defineConfig({
|
|||||||
__MEMHUB_DEFAULT_RELEASE_MANIFEST_URL_TEMPLATE__: JSON.stringify(
|
__MEMHUB_DEFAULT_RELEASE_MANIFEST_URL_TEMPLATE__: JSON.stringify(
|
||||||
process.env.MEMHUB_DEFAULT_RELEASE_MANIFEST_URL_TEMPLATE ?? "",
|
process.env.MEMHUB_DEFAULT_RELEASE_MANIFEST_URL_TEMPLATE ?? "",
|
||||||
),
|
),
|
||||||
|
__MEMHUB_DEFAULT_RUNTIME_VERSION__: JSON.stringify(runtimeReleaseVersion),
|
||||||
},
|
},
|
||||||
// The shared operations package ships TypeScript sources, so it must be
|
// The shared operations package ships TypeScript sources, so it must be
|
||||||
// inlined into the CJS bundle the same way runtime-control is; otherwise the
|
// inlined into the CJS bundle the same way runtime-control is; otherwise the
|
||||||
|
|||||||
@@ -11,6 +11,7 @@ export default defineConfig({
|
|||||||
__ONE_MEMORY_BUILD_NUMBER__: JSON.stringify("unknown"),
|
__ONE_MEMORY_BUILD_NUMBER__: JSON.stringify("unknown"),
|
||||||
__ONE_MEMORY_COMMIT__: JSON.stringify("unknown"),
|
__ONE_MEMORY_COMMIT__: JSON.stringify("unknown"),
|
||||||
__MEMHUB_DEFAULT_RELEASE_MANIFEST_URL_TEMPLATE__: JSON.stringify(""),
|
__MEMHUB_DEFAULT_RELEASE_MANIFEST_URL_TEMPLATE__: JSON.stringify(""),
|
||||||
|
__MEMHUB_DEFAULT_RUNTIME_VERSION__: JSON.stringify(version),
|
||||||
},
|
},
|
||||||
test: {
|
test: {
|
||||||
testTimeout: 120_000,
|
testTimeout: 120_000,
|
||||||
|
|||||||
@@ -185,7 +185,12 @@ from one_memory.memory_queue import (
|
|||||||
resolve_extraction_state,
|
resolve_extraction_state,
|
||||||
)
|
)
|
||||||
from one_memory.memory_server.config_state import ConfigStore
|
from one_memory.memory_server.config_state import ConfigStore
|
||||||
from one_memory.memory_tasks import MemoryTask, MemoryTaskStatus, MemoryTaskStore
|
from one_memory.memory_tasks import (
|
||||||
|
NON_RETRYABLE_POST_WRITE_CODES,
|
||||||
|
MemoryTask,
|
||||||
|
MemoryTaskStatus,
|
||||||
|
MemoryTaskStore,
|
||||||
|
)
|
||||||
from one_memory.model_presets import seed_preset_profile
|
from one_memory.model_presets import seed_preset_profile
|
||||||
from one_memory.model_probe import ModelProber, ModelProbeResult
|
from one_memory.model_probe import ModelProber, ModelProbeResult
|
||||||
from one_memory.model_profiles import (
|
from one_memory.model_profiles import (
|
||||||
@@ -1082,8 +1087,14 @@ def create_app(
|
|||||||
drain_budget = production_drain_budget()
|
drain_budget = production_drain_budget()
|
||||||
|
|
||||||
async def reconcile_unknown_writes() -> None:
|
async def reconcile_unknown_writes() -> None:
|
||||||
await app.state.memory_processing_supervisor.reconcile_unknown_outputs()
|
await app.state.memory_processing_supervisor.reconcile_unknown_outputs(
|
||||||
unknown = await asyncio.to_thread(memory_write_queue.unknown_jobs, limit=128)
|
exclude_error_codes=NON_RETRYABLE_POST_WRITE_CODES,
|
||||||
|
)
|
||||||
|
unknown = await asyncio.to_thread(
|
||||||
|
memory_write_queue.unknown_jobs,
|
||||||
|
limit=128,
|
||||||
|
exclude_error_codes=NON_RETRYABLE_POST_WRITE_CODES,
|
||||||
|
)
|
||||||
for job in unknown:
|
for job in unknown:
|
||||||
await asyncio.to_thread(
|
await asyncio.to_thread(
|
||||||
memory_write_queue.resolve_unknown_from_linked_output,
|
memory_write_queue.resolve_unknown_from_linked_output,
|
||||||
@@ -2793,6 +2804,12 @@ def create_app(
|
|||||||
# D75: work queued for the previous model continues with this one.
|
# D75: work queued for the previous model continues with this one.
|
||||||
# Only tasks that never reached the model are re-bound.
|
# Only tasks that never reached the model are re-bound.
|
||||||
await asyncio.to_thread(app.state.memory_task_store.rebind_unstarted_profile, profile)
|
await asyncio.to_thread(app.state.memory_task_store.rebind_unstarted_profile, profile)
|
||||||
|
# A missing profile is an event-driven wait, not a provider outage.
|
||||||
|
# Release those durable rows only now that activation has changed the
|
||||||
|
# condition; the ordinary drain poll will claim them without a restart.
|
||||||
|
await asyncio.to_thread(
|
||||||
|
app.state.memory_write_queue.readmit_model_profile_waiters
|
||||||
|
)
|
||||||
# Not awaited: the catch-up reads every conversation ever imported, and
|
# Not awaited: the catch-up reads every conversation ever imported, and
|
||||||
# the answer to "which profile is active" does not depend on it.
|
# the answer to "which profile is active" does not depend on it.
|
||||||
_start_backfill(app)
|
_start_backfill(app)
|
||||||
@@ -5062,10 +5079,11 @@ def _source_writes_on(app: FastAPI, source_id: str) -> bool:
|
|||||||
|
|
||||||
|
|
||||||
def _sedimentation_claim_enabled(app: FastAPI, source_id: str) -> bool:
|
def _sedimentation_claim_enabled(app: FastAPI, source_id: str) -> bool:
|
||||||
"""Claim queued model work only while both global and source gates allow it."""
|
"""Claim queued model work only when policy, source and model all admit it."""
|
||||||
|
|
||||||
return (
|
return (
|
||||||
_config_store(app).sedimentation_enabled()
|
_config_store(app).sedimentation_enabled()
|
||||||
|
and _model_profile_store(app).active_identity() is not None
|
||||||
and _history_import_gate_released(app)
|
and _history_import_gate_released(app)
|
||||||
and _source_writes_on(app, source_id)
|
and _source_writes_on(app, source_id)
|
||||||
)
|
)
|
||||||
|
|||||||
@@ -282,11 +282,24 @@ class PollingLiveSyncSupervisor:
|
|||||||
and job.status
|
and job.status
|
||||||
in {ImportStatus.PENDING, ImportStatus.RUNNING, ImportStatus.INTERRUPTED}
|
in {ImportStatus.PENDING, ImportStatus.RUNNING, ImportStatus.INTERRUPTED}
|
||||||
]
|
]
|
||||||
|
bounded_history_incomplete = any(
|
||||||
|
job.source_id == source_id
|
||||||
|
and job.operation == "history"
|
||||||
|
and job.history_after is not None
|
||||||
|
and job.status is not ImportStatus.COMPLETED
|
||||||
|
for job in source_jobs
|
||||||
|
)
|
||||||
# Establish the realtime EOF independently even while history owns
|
# Establish the realtime EOF independently even while history owns
|
||||||
# this source. Otherwise the P1 job starts from the P2 checkpoint
|
# this source. Otherwise the P1 job starts from the P2 checkpoint
|
||||||
# (or from zero) and becomes a second full-history scan.
|
# (or from zero) and becomes a second full-history scan.
|
||||||
adapter = self.import_supervisor.adapters[source_id]
|
adapter = self.import_supervisor.adapters[source_id]
|
||||||
await self._live_cursors(source_id, adapter)
|
live_cursors = await self._live_cursors(source_id, adapter)
|
||||||
|
# A bounded onboarding history is the authority for everything up
|
||||||
|
# to its EOF. An adapter without a native live baseline has no safe
|
||||||
|
# realtime cursor until that history finishes; starting a sync from
|
||||||
|
# zero here would bypass `history_after` and import the old corpus.
|
||||||
|
if bounded_history_incomplete and not live_cursors:
|
||||||
|
return None
|
||||||
# Only one scan of a source may be in flight: they share the
|
# Only one scan of a source may be in flight: they share the
|
||||||
# cursor, and two readers means every file read twice. With a
|
# cursor, and two readers means every file read twice. With a
|
||||||
# catch-up still holding the source there is no free moment to ask
|
# catch-up still holding the source there is no free moment to ask
|
||||||
|
|||||||
@@ -1522,14 +1522,24 @@ class MemoryProcessingSupervisor:
|
|||||||
await self._refresh_read_bridge(completed[-1].profile_snapshot)
|
await self._refresh_read_bridge(completed[-1].profile_snapshot)
|
||||||
return tuple(completed)
|
return tuple(completed)
|
||||||
|
|
||||||
async def reconcile_unknown_outputs(self, *, limit: int = 128) -> int:
|
async def reconcile_unknown_outputs(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
limit: int = 128,
|
||||||
|
exclude_error_codes: tuple[str, ...] = (),
|
||||||
|
) -> int:
|
||||||
"""Publish HY outputs that can be proven by an unknown task's request id.
|
"""Publish HY outputs that can be proven by an unknown task's request id.
|
||||||
|
|
||||||
This inquiry is read-only on HY and never invokes the model. Rows with
|
This inquiry is read-only on HY and never invokes the model. Rows with
|
||||||
no exact match remain unknown for the person's explicit retry choice.
|
no exact match remain unknown for the person's explicit retry choice.
|
||||||
"""
|
"""
|
||||||
reconciled = 0
|
reconciled = 0
|
||||||
for task in await in_the_background(self.tasks.unknown_tasks, limit=limit):
|
tasks = await in_the_background(
|
||||||
|
self.tasks.unknown_tasks,
|
||||||
|
limit=limit,
|
||||||
|
exclude_error_codes=exclude_error_codes,
|
||||||
|
)
|
||||||
|
for task in tasks:
|
||||||
runner = self.runner_factory()
|
runner = self.runner_factory()
|
||||||
bridge: MemoryBridgeProtocol | None = None
|
bridge: MemoryBridgeProtocol | None = None
|
||||||
try:
|
try:
|
||||||
|
|||||||
@@ -38,10 +38,11 @@ from one_memory.memory_server.migrations import memory_write_jobs_sqlite_schema
|
|||||||
from one_memory.memory_server.queue import build_reclaim_sqlite_statements
|
from one_memory.memory_server.queue import build_reclaim_sqlite_statements
|
||||||
from one_memory.memory_server.registry import WriteRecord
|
from one_memory.memory_server.registry import WriteRecord
|
||||||
from one_memory.memory_tasks import (
|
from one_memory.memory_tasks import (
|
||||||
POLICY_STOPPED_CODES as _TASK_POLICY_STOPPED_CODES,
|
NON_RETRYABLE_POST_WRITE_CODES,
|
||||||
|
MemoryTaskStatus,
|
||||||
)
|
)
|
||||||
from one_memory.memory_tasks import (
|
from one_memory.memory_tasks import (
|
||||||
MemoryTaskStatus,
|
POLICY_STOPPED_CODES as _TASK_POLICY_STOPPED_CODES,
|
||||||
)
|
)
|
||||||
from one_memory.service_levels import ServiceLevel, service_level_context
|
from one_memory.service_levels import ServiceLevel, service_level_context
|
||||||
|
|
||||||
@@ -1404,11 +1405,12 @@ class DurableWriteQueue:
|
|||||||
) -> int:
|
) -> int:
|
||||||
"""Re-admit `waiting_for_model` jobs whose wait has elapsed.
|
"""Re-admit `waiting_for_model` jobs whose wait has elapsed.
|
||||||
|
|
||||||
A model outage clears on its own, so this state is retried on a timer
|
A model outage clears on its own, so transient waiters retry on a timer
|
||||||
rather than waiting for a grant event like `waiting_for_permission`.
|
rather than waiting for a grant event like `waiting_for_permission`.
|
||||||
Keyed on `updated_at` so repeated calls while a job is still parked do
|
`model_profile_not_active` is excluded: only explicit activation can
|
||||||
not reset its own clock; the attempt consumed by the parking claim is
|
change that condition. Keyed on `updated_at` so repeated calls while a
|
||||||
returned, the same way `readmit_parked` does.
|
job is still parked do not reset its own clock; the attempt consumed by
|
||||||
|
the parking claim is returned, the same way `readmit_parked` does.
|
||||||
"""
|
"""
|
||||||
now = now or datetime.now(UTC)
|
now = now or datetime.now(UTC)
|
||||||
threshold = (now - after).isoformat()
|
threshold = (now - after).isoformat()
|
||||||
@@ -1417,8 +1419,33 @@ class DurableWriteQueue:
|
|||||||
"UPDATE memory_write_jobs SET state=?, lease_owner=NULL, "
|
"UPDATE memory_write_jobs SET state=?, lease_owner=NULL, "
|
||||||
"lease_expires_at=NULL, attempts=CASE WHEN attempts>0 "
|
"lease_expires_at=NULL, attempts=CASE WHEN attempts>0 "
|
||||||
"THEN attempts-1 ELSE 0 END, last_error=NULL, updated_at=? "
|
"THEN attempts-1 ELSE 0 END, last_error=NULL, updated_at=? "
|
||||||
"WHERE state=? AND updated_at <= ? AND hy_invoked=0",
|
"WHERE state=? AND updated_at <= ? AND hy_invoked=0 "
|
||||||
(PENDING, now.isoformat(), WAITING_FOR_MODEL, threshold),
|
"AND (last_error IS NULL OR last_error<>?)",
|
||||||
|
(
|
||||||
|
PENDING,
|
||||||
|
now.isoformat(),
|
||||||
|
WAITING_FOR_MODEL,
|
||||||
|
threshold,
|
||||||
|
"model_profile_not_active",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
return changed.rowcount
|
||||||
|
|
||||||
|
def readmit_model_profile_waiters(self) -> int:
|
||||||
|
"""Re-admit no-profile work only after model activation changes the condition."""
|
||||||
|
now = datetime.now(UTC).isoformat()
|
||||||
|
with main_database(self.database_path).write() as connection, connection:
|
||||||
|
changed = connection.execute(
|
||||||
|
"UPDATE memory_write_jobs SET state=?, lease_owner=NULL, "
|
||||||
|
"lease_expires_at=NULL, attempts=CASE WHEN attempts>0 "
|
||||||
|
"THEN attempts-1 ELSE 0 END, last_error=NULL, updated_at=? "
|
||||||
|
"WHERE state=? AND last_error=? AND hy_invoked=0",
|
||||||
|
(
|
||||||
|
PENDING,
|
||||||
|
now,
|
||||||
|
WAITING_FOR_MODEL,
|
||||||
|
"model_profile_not_active",
|
||||||
|
),
|
||||||
)
|
)
|
||||||
return changed.rowcount
|
return changed.rowcount
|
||||||
|
|
||||||
@@ -1516,9 +1543,13 @@ class DurableWriteQueue:
|
|||||||
connection,
|
connection,
|
||||||
):
|
):
|
||||||
row = connection.execute(
|
row = connection.execute(
|
||||||
"SELECT state FROM memory_write_jobs WHERE job_id=?", (job_id,)
|
"SELECT state,last_error FROM memory_write_jobs WHERE job_id=?", (job_id,)
|
||||||
).fetchone()
|
).fetchone()
|
||||||
if row is None or str(row["state"]) != UNKNOWN:
|
if (
|
||||||
|
row is None
|
||||||
|
or str(row["state"]) != UNKNOWN
|
||||||
|
or str(row["last_error"]) in NON_RETRYABLE_POST_WRITE_CODES
|
||||||
|
):
|
||||||
return 0
|
return 0
|
||||||
connection.execute(
|
connection.execute(
|
||||||
"""
|
"""
|
||||||
@@ -1675,8 +1706,15 @@ class DurableWriteQueue:
|
|||||||
rows = connection.execute(
|
rows = connection.execute(
|
||||||
"SELECT job_id FROM memory_write_jobs "
|
"SELECT job_id FROM memory_write_jobs "
|
||||||
"WHERE state=? AND attempts=1 AND attempts < max_attempts "
|
"WHERE state=? AND attempts=1 AND attempts < max_attempts "
|
||||||
|
f"AND COALESCE(last_error, '') NOT IN "
|
||||||
|
f"({','.join('?' for _ in NON_RETRYABLE_POST_WRITE_CODES)}) "
|
||||||
"AND updated_at <= ? ORDER BY updated_at, job_id LIMIT ?",
|
"AND updated_at <= ? ORDER BY updated_at, job_id LIMIT ?",
|
||||||
(UNKNOWN, older_than.isoformat(), limit),
|
(
|
||||||
|
UNKNOWN,
|
||||||
|
*NON_RETRYABLE_POST_WRITE_CODES,
|
||||||
|
older_than.isoformat(),
|
||||||
|
limit,
|
||||||
|
),
|
||||||
).fetchall()
|
).fetchall()
|
||||||
job_ids = tuple(str(row["job_id"]) for row in rows)
|
job_ids = tuple(str(row["job_id"]) for row in rows)
|
||||||
if not job_ids:
|
if not job_ids:
|
||||||
@@ -1740,6 +1778,7 @@ class DurableWriteQueue:
|
|||||||
*,
|
*,
|
||||||
limit: int = 128,
|
limit: int = 128,
|
||||||
after_cursor: tuple[str, str] | None = None,
|
after_cursor: tuple[str, str] | None = None,
|
||||||
|
exclude_error_codes: tuple[str, ...] = (),
|
||||||
) -> tuple[UnknownWriteJob, ...]:
|
) -> tuple[UnknownWriteJob, ...]:
|
||||||
"""List unknown rows that still need an inquiry, in stable order.
|
"""List unknown rows that still need an inquiry, in stable order.
|
||||||
|
|
||||||
@@ -1756,6 +1795,9 @@ class DurableWriteQueue:
|
|||||||
"""
|
"""
|
||||||
if type(limit) is not int or limit < 1:
|
if type(limit) is not int or limit < 1:
|
||||||
raise ValueError("limit must be positive")
|
raise ValueError("limit must be positive")
|
||||||
|
excluded = tuple(dict.fromkeys(exclude_error_codes))
|
||||||
|
if not all(isinstance(code, str) and code for code in excluded):
|
||||||
|
raise ValueError("exclude_error_codes must contain non-empty strings")
|
||||||
cursor_clause = ""
|
cursor_clause = ""
|
||||||
params: tuple[Any, ...] = ()
|
params: tuple[Any, ...] = ()
|
||||||
if after_cursor is not None:
|
if after_cursor is not None:
|
||||||
@@ -1768,14 +1810,20 @@ class DurableWriteQueue:
|
|||||||
updated_at, job_id = after_cursor
|
updated_at, job_id = after_cursor
|
||||||
cursor_clause = "AND (updated_at, job_id) > (?, ?)"
|
cursor_clause = "AND (updated_at, job_id) > (?, ?)"
|
||||||
params = (updated_at, job_id)
|
params = (updated_at, job_id)
|
||||||
|
if excluded:
|
||||||
|
placeholders = ",".join("?" for _ in excluded)
|
||||||
|
excluded_clause = f"AND COALESCE(last_error, '') NOT IN ({placeholders})"
|
||||||
|
else:
|
||||||
|
excluded_clause = ""
|
||||||
with main_database(self.database_path).read(row_factory=sqlite3.Row) as connection:
|
with main_database(self.database_path).read(row_factory=sqlite3.Row) as connection:
|
||||||
rows = connection.execute(
|
rows = connection.execute(
|
||||||
"SELECT job_id, idempotency_key, created_at, updated_at, last_error "
|
"SELECT job_id, idempotency_key, created_at, updated_at, last_error "
|
||||||
"FROM memory_write_jobs "
|
"FROM memory_write_jobs "
|
||||||
"WHERE state=? "
|
"WHERE state=? "
|
||||||
f"{cursor_clause} "
|
f"{cursor_clause} "
|
||||||
|
f"{excluded_clause} "
|
||||||
"ORDER BY updated_at, job_id LIMIT ?",
|
"ORDER BY updated_at, job_id LIMIT ?",
|
||||||
(UNKNOWN, *params, limit),
|
(UNKNOWN, *params, *excluded, limit),
|
||||||
).fetchall()
|
).fetchall()
|
||||||
return tuple(
|
return tuple(
|
||||||
UnknownWriteJob(
|
UnknownWriteJob(
|
||||||
@@ -2071,7 +2119,12 @@ async def execute_job(
|
|||||||
# Never `logger.exception` here: it prints the exception's message,
|
# Never `logger.exception` here: it prints the exception's message,
|
||||||
# which may be a provider response or the user's own text. Log the
|
# which may be a provider response or the user's own text. Log the
|
||||||
# already-sanitised code instead, matching `hy-failure-classification`.
|
# already-sanitised code instead, matching `hy-failure-classification`.
|
||||||
logger.error("durable_write_job_failed job_id=%s error_code=%s", job_id, code)
|
if code == "model_profile_not_active":
|
||||||
|
logger.debug("durable_write_job_parked job_id=%s error_code=%s", job_id, code)
|
||||||
|
elif state in PARKED_STATES:
|
||||||
|
logger.info("durable_write_job_parked job_id=%s error_code=%s", job_id, code)
|
||||||
|
else:
|
||||||
|
logger.error("durable_write_job_failed job_id=%s error_code=%s", job_id, code)
|
||||||
return
|
return
|
||||||
await in_the_background(queue.settle, job_id, state=COMPLETED, owner=owner)
|
await in_the_background(queue.settle, job_id, state=COMPLETED, owner=owner)
|
||||||
|
|
||||||
|
|||||||
@@ -67,6 +67,12 @@ POLICY_STOPPED_CODES = (
|
|||||||
"hy_memory_evidence_changed",
|
"hy_memory_evidence_changed",
|
||||||
)
|
)
|
||||||
|
|
||||||
|
#: HY completed a write, but Core could not prove that the selected mutation
|
||||||
|
#: target belonged to this task's frozen authority. Reconciliation may inspect
|
||||||
|
#: HY by request id, but neither automatic nor user-confirmed generic retry may
|
||||||
|
#: invoke the model again for these codes.
|
||||||
|
NON_RETRYABLE_POST_WRITE_CODES = ("hy_memory_write_verification_authority_invalid",)
|
||||||
|
|
||||||
|
|
||||||
class MemoryTask(BaseModel):
|
class MemoryTask(BaseModel):
|
||||||
model_config = ConfigDict(frozen=True, extra="forbid")
|
model_config = ConfigDict(frozen=True, extra="forbid")
|
||||||
@@ -116,6 +122,24 @@ PROFILE_STOP_CODES = (
|
|||||||
ACTIVE_WINDOW_DEFER_LIMIT = 2048
|
ACTIVE_WINDOW_DEFER_LIMIT = 2048
|
||||||
|
|
||||||
|
|
||||||
|
def _task_only_unknown_clause(connection: sqlite3.Connection) -> str:
|
||||||
|
"""Exclude tasks whose Evidence is already owned by the durable queue."""
|
||||||
|
linked_table = connection.execute(
|
||||||
|
"SELECT 1 FROM sqlite_master WHERE type='table' AND name='memory_write_job_evidence'"
|
||||||
|
).fetchone()
|
||||||
|
if linked_table is None:
|
||||||
|
return ""
|
||||||
|
return """
|
||||||
|
AND NOT EXISTS (
|
||||||
|
SELECT 1
|
||||||
|
FROM memory_task_inputs AS inputs, json_each(inputs.refs_json) AS ref
|
||||||
|
JOIN memory_write_job_evidence AS linked
|
||||||
|
ON linked.evidence_id=json_extract(ref.value, '$.evidence_id')
|
||||||
|
WHERE inputs.task_id=memory_tasks.task_id
|
||||||
|
)
|
||||||
|
"""
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True, slots=True)
|
@dataclass(frozen=True, slots=True)
|
||||||
class ActiveMemoryTask:
|
class ActiveMemoryTask:
|
||||||
"""One task plus only the evidence fields the scheduler needs."""
|
"""One task plus only the evidence fields the scheduler needs."""
|
||||||
@@ -1314,7 +1338,7 @@ class MemoryTaskStore:
|
|||||||
live_clause = f"task.task_id IN ({live_placeholders})" if live_ids else "0"
|
live_clause = f"task.task_id IN ({live_placeholders})" if live_ids else "0"
|
||||||
not_live_clause = f"task.task_id NOT IN ({live_placeholders})" if live_ids else "1"
|
not_live_clause = f"task.task_id NOT IN ({live_placeholders})" if live_ids else "1"
|
||||||
excluded_clause = f"task.task_id NOT IN ({excluded_placeholders})" if excluded else "1"
|
excluded_clause = f"task.task_id NOT IN ({excluded_placeholders})" if excluded else "1"
|
||||||
common = f"""
|
common_base = f"""
|
||||||
task.status IN (?,?,?,?)
|
task.status IN (?,?,?,?)
|
||||||
AND coalesce(task.last_error_code, '') NOT IN (
|
AND coalesce(task.last_error_code, '') NOT IN (
|
||||||
'hy_memory_input_exceeds_window', 'model_content_outcome_unknown'
|
'hy_memory_input_exceeds_window', 'model_content_outcome_unknown'
|
||||||
@@ -1333,7 +1357,33 @@ class MemoryTaskStore:
|
|||||||
"""
|
"""
|
||||||
moment = "coalesce(evidence.created_at, evidence.imported_at)"
|
moment = "coalesce(evidence.created_at, evidence.imported_at)"
|
||||||
|
|
||||||
|
def common_for(connection: sqlite3.Connection) -> str:
|
||||||
|
has_durable_queue = connection.execute(
|
||||||
|
"SELECT COUNT(*) FROM sqlite_master WHERE type='table' "
|
||||||
|
"AND name IN ('memory_write_jobs','memory_write_job_evidence')"
|
||||||
|
).fetchone()[0]
|
||||||
|
if int(has_durable_queue) != 2:
|
||||||
|
return common_base
|
||||||
|
return (
|
||||||
|
common_base
|
||||||
|
+ """
|
||||||
|
AND NOT EXISTS (
|
||||||
|
SELECT 1
|
||||||
|
FROM memory_task_inputs AS inputs, json_each(inputs.refs_json) AS ref
|
||||||
|
JOIN memory_write_job_evidence AS linked
|
||||||
|
ON linked.evidence_id=json_extract(ref.value, '$.evidence_id')
|
||||||
|
JOIN memory_write_jobs AS job ON job.job_id=linked.job_id
|
||||||
|
WHERE inputs.task_id=task.task_id
|
||||||
|
AND job.state IN (
|
||||||
|
'pending', 'running', 'unknown',
|
||||||
|
'waiting_for_permission', 'waiting_for_model'
|
||||||
|
)
|
||||||
|
)
|
||||||
|
"""
|
||||||
|
)
|
||||||
|
|
||||||
def run(connection: sqlite3.Connection, *, defer: bool) -> list[sqlite3.Row]:
|
def run(connection: sqlite3.Connection, *, defer: bool) -> list[sqlite3.Row]:
|
||||||
|
common = common_for(connection)
|
||||||
in_window = (
|
in_window = (
|
||||||
"1"
|
"1"
|
||||||
if extraction_since is None
|
if extraction_since is None
|
||||||
@@ -1430,6 +1480,7 @@ class MemoryTaskStore:
|
|||||||
def has_defer_candidate(connection: sqlite3.Connection) -> bool:
|
def has_defer_candidate(connection: sqlite3.Connection) -> bool:
|
||||||
if extraction_since is None:
|
if extraction_since is None:
|
||||||
return False
|
return False
|
||||||
|
common = common_for(connection)
|
||||||
row = connection.execute(
|
row = connection.execute(
|
||||||
f"""
|
f"""
|
||||||
SELECT 1
|
SELECT 1
|
||||||
@@ -2146,6 +2197,7 @@ class MemoryTaskStore:
|
|||||||
*,
|
*,
|
||||||
limit: int = 128,
|
limit: int = 128,
|
||||||
after_cursor: tuple[str, str] | None = None,
|
after_cursor: tuple[str, str] | None = None,
|
||||||
|
exclude_error_codes: tuple[str, ...] = (),
|
||||||
) -> tuple[MemoryTask, ...]:
|
) -> tuple[MemoryTask, ...]:
|
||||||
"""List `unknown` tasks that still need an external inquiry.
|
"""List `unknown` tasks that still need an external inquiry.
|
||||||
|
|
||||||
@@ -2160,6 +2212,9 @@ class MemoryTaskStore:
|
|||||||
"""
|
"""
|
||||||
if type(limit) is not int or limit < 1:
|
if type(limit) is not int or limit < 1:
|
||||||
raise ValueError("limit must be positive")
|
raise ValueError("limit must be positive")
|
||||||
|
excluded = tuple(dict.fromkeys(exclude_error_codes))
|
||||||
|
if not all(isinstance(code, str) and code for code in excluded):
|
||||||
|
raise ValueError("exclude_error_codes must contain non-empty strings")
|
||||||
if after_cursor is not None:
|
if after_cursor is not None:
|
||||||
if (
|
if (
|
||||||
not isinstance(after_cursor, tuple)
|
not isinstance(after_cursor, tuple)
|
||||||
@@ -2173,13 +2228,19 @@ class MemoryTaskStore:
|
|||||||
else:
|
else:
|
||||||
cursor_clause = ""
|
cursor_clause = ""
|
||||||
params = ()
|
params = ()
|
||||||
|
if excluded:
|
||||||
|
placeholders = ",".join("?" for _ in excluded)
|
||||||
|
excluded_clause = f"AND COALESCE(last_error_code, '') NOT IN ({placeholders})"
|
||||||
|
else:
|
||||||
|
excluded_clause = ""
|
||||||
with self._connect() as connection:
|
with self._connect() as connection:
|
||||||
rows = connection.execute(
|
rows = connection.execute(
|
||||||
"SELECT * FROM memory_tasks "
|
"SELECT * FROM memory_tasks "
|
||||||
"WHERE status=? "
|
"WHERE status=? "
|
||||||
f"{cursor_clause} "
|
f"{cursor_clause} "
|
||||||
|
f"{excluded_clause} "
|
||||||
"ORDER BY updated_at, task_id LIMIT ?",
|
"ORDER BY updated_at, task_id LIMIT ?",
|
||||||
(MemoryTaskStatus.UNKNOWN, *params, limit),
|
(MemoryTaskStatus.UNKNOWN, *params, *excluded, limit),
|
||||||
).fetchall()
|
).fetchall()
|
||||||
return tuple(self._to_task(row) for row in rows)
|
return tuple(self._to_task(row) for row in rows)
|
||||||
|
|
||||||
@@ -2211,20 +2272,26 @@ class MemoryTaskStore:
|
|||||||
"""
|
"""
|
||||||
now = datetime.now(UTC).isoformat()
|
now = datetime.now(UTC).isoformat()
|
||||||
with self._connect(writing=True) as connection:
|
with self._connect(writing=True) as connection:
|
||||||
|
task_only = _task_only_unknown_clause(connection)
|
||||||
|
blocked = ",".join("?" for _ in NON_RETRYABLE_POST_WRITE_CODES)
|
||||||
changed = connection.execute(
|
changed = connection.execute(
|
||||||
"""
|
f"""
|
||||||
UPDATE memory_tasks
|
UPDATE memory_tasks
|
||||||
SET status=?, runner_token=NULL, attempts=0,
|
SET status=?, runner_token=NULL, attempts=0,
|
||||||
last_error_code='model_content_manual_retry_requested', updated_at=?
|
last_error_code='model_content_manual_retry_requested', updated_at=?
|
||||||
WHERE task_id IN (
|
WHERE task_id IN (
|
||||||
SELECT task_id FROM memory_tasks
|
SELECT task_id FROM memory_tasks
|
||||||
WHERE status=? ORDER BY updated_at, task_id LIMIT ?
|
WHERE status=?
|
||||||
|
AND COALESCE(last_error_code, '') NOT IN ({blocked})
|
||||||
|
{task_only}
|
||||||
|
ORDER BY updated_at, task_id LIMIT ?
|
||||||
)
|
)
|
||||||
""",
|
""",
|
||||||
(
|
(
|
||||||
MemoryTaskStatus.PENDING,
|
MemoryTaskStatus.PENDING,
|
||||||
now,
|
now,
|
||||||
MemoryTaskStatus.UNKNOWN,
|
MemoryTaskStatus.UNKNOWN,
|
||||||
|
*NON_RETRYABLE_POST_WRITE_CODES,
|
||||||
limit,
|
limit,
|
||||||
),
|
),
|
||||||
)
|
)
|
||||||
@@ -2246,14 +2313,18 @@ class MemoryTaskStore:
|
|||||||
raise ValueError("limit must be positive")
|
raise ValueError("limit must be positive")
|
||||||
now = datetime.now(UTC).isoformat()
|
now = datetime.now(UTC).isoformat()
|
||||||
with self._connect(writing=True) as connection:
|
with self._connect(writing=True) as connection:
|
||||||
|
task_only = _task_only_unknown_clause(connection)
|
||||||
|
blocked = ",".join("?" for _ in NON_RETRYABLE_POST_WRITE_CODES)
|
||||||
changed = connection.execute(
|
changed = connection.execute(
|
||||||
"""
|
f"""
|
||||||
UPDATE memory_tasks
|
UPDATE memory_tasks
|
||||||
SET status=?, runner_token=NULL,
|
SET status=?, runner_token=NULL,
|
||||||
last_error_code='model_content_unknown_auto_retry', updated_at=?
|
last_error_code='model_content_unknown_auto_retry', updated_at=?
|
||||||
WHERE task_id IN (
|
WHERE task_id IN (
|
||||||
SELECT task_id FROM memory_tasks
|
SELECT task_id FROM memory_tasks
|
||||||
WHERE status=? AND attempts=1 AND updated_at <= ?
|
WHERE status=? AND attempts=1 AND updated_at <= ?
|
||||||
|
AND COALESCE(last_error_code, '') NOT IN ({blocked})
|
||||||
|
{task_only}
|
||||||
ORDER BY updated_at, task_id LIMIT ?
|
ORDER BY updated_at, task_id LIMIT ?
|
||||||
)
|
)
|
||||||
""",
|
""",
|
||||||
@@ -2262,6 +2333,7 @@ class MemoryTaskStore:
|
|||||||
now,
|
now,
|
||||||
MemoryTaskStatus.UNKNOWN,
|
MemoryTaskStatus.UNKNOWN,
|
||||||
older_than.isoformat(),
|
older_than.isoformat(),
|
||||||
|
*NON_RETRYABLE_POST_WRITE_CODES,
|
||||||
limit,
|
limit,
|
||||||
),
|
),
|
||||||
)
|
)
|
||||||
|
|||||||
@@ -200,10 +200,12 @@ def _trigger_sql() -> list[tuple[str, str]]:
|
|||||||
CREATE TRIGGER IF NOT EXISTS memory_visibility_dirty_on_link_update
|
CREATE TRIGGER IF NOT EXISTS memory_visibility_dirty_on_link_update
|
||||||
AFTER UPDATE ON governed_memory_evidence
|
AFTER UPDATE ON governed_memory_evidence
|
||||||
BEGIN
|
BEGIN
|
||||||
INSERT OR IGNORE INTO memory_visibility_dirty (memory_id)
|
INSERT INTO memory_visibility_dirty (memory_id)
|
||||||
VALUES (NEW.memory_id);
|
VALUES (NEW.memory_id)
|
||||||
INSERT OR IGNORE INTO memory_visibility_dirty (memory_id)
|
ON CONFLICT(memory_id) DO NOTHING;
|
||||||
VALUES (OLD.memory_id);
|
INSERT INTO memory_visibility_dirty (memory_id)
|
||||||
|
VALUES (OLD.memory_id)
|
||||||
|
ON CONFLICT(memory_id) DO NOTHING;
|
||||||
END;""",
|
END;""",
|
||||||
),
|
),
|
||||||
(
|
(
|
||||||
@@ -294,6 +296,10 @@ def install_triggers(db: sqlite3.Connection) -> None:
|
|||||||
str(row["name"])
|
str(row["name"])
|
||||||
for row in cursor.execute("SELECT name FROM sqlite_master WHERE type='table'")
|
for row in cursor.execute("SELECT name FROM sqlite_master WHERE type='table'")
|
||||||
}
|
}
|
||||||
|
# CREATE TRIGGER IF NOT EXISTS cannot repair the legacy definition. Replace
|
||||||
|
# this derived-cache trigger transactionally on every startup so old
|
||||||
|
# libraries get the row-level conflict handling without a schema migration.
|
||||||
|
cursor.execute("DROP TRIGGER IF EXISTS memory_visibility_dirty_on_link_update")
|
||||||
for table, statement in _trigger_sql():
|
for table, statement in _trigger_sql():
|
||||||
if table not in present:
|
if table not in present:
|
||||||
continue
|
continue
|
||||||
@@ -303,11 +309,15 @@ def install_triggers(db: sqlite3.Connection) -> None:
|
|||||||
def startup(database_path: Path) -> None:
|
def startup(database_path: Path) -> None:
|
||||||
"""Idempotent: create the derived tables and triggers. No version change.
|
"""Idempotent: create the derived tables and triggers. No version change.
|
||||||
|
|
||||||
Runs through the write connection because the triggers are schema. It is
|
Runs through the write connection because the triggers are schema. The
|
||||||
one short transaction and happens once per start.
|
trigger replacement is one short transaction and happens once per start.
|
||||||
"""
|
"""
|
||||||
with main_database(database_path).write() as db:
|
with main_database(database_path).write() as db, db:
|
||||||
ensure_tables(db)
|
ensure_tables(db)
|
||||||
|
# ensure_tables uses executescript(), which commits any pending legacy
|
||||||
|
# transaction. Begin only afterward so replacing the installed trigger
|
||||||
|
# is one atomic schema change and a failed CREATE restores the old one.
|
||||||
|
db.execute("BEGIN IMMEDIATE")
|
||||||
install_triggers(db)
|
install_triggers(db)
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -94,6 +94,67 @@ class SettledFixtureAdapter(IncrementalFixtureAdapter):
|
|||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
|
class BoundedOnboardingAdapter:
|
||||||
|
"""No native baseline: bounded history must establish the live cursor."""
|
||||||
|
|
||||||
|
descriptor = IncrementalFixtureAdapter.descriptor
|
||||||
|
|
||||||
|
def __init__(self) -> None:
|
||||||
|
self.include_new = False
|
||||||
|
|
||||||
|
async def detect(self) -> bool:
|
||||||
|
return True
|
||||||
|
|
||||||
|
async def scan(self, options: ScanOptions):
|
||||||
|
current = options.cursors.get("e" * 64)
|
||||||
|
checkpoint = current.checkpoint if current is not None else None
|
||||||
|
if checkpoint not in {"byte:10", "byte:20"}:
|
||||||
|
yield ScannedEvidence(
|
||||||
|
evidence=EvidenceInput(
|
||||||
|
source_id="fixture",
|
||||||
|
conversation_id="old-conversation",
|
||||||
|
message_id="old-message",
|
||||||
|
role="user",
|
||||||
|
written_by=WrittenBy.PERSON,
|
||||||
|
created_at=datetime(2020, 1, 1, tzinfo=UTC),
|
||||||
|
verbatim="old history",
|
||||||
|
origin_meta={EvidenceOrigin.FILE: OriginMetadata()},
|
||||||
|
),
|
||||||
|
cursor=SourceCursor(
|
||||||
|
file_key="e" * 64,
|
||||||
|
file_identity="bounded",
|
||||||
|
checkpoint="byte:10",
|
||||||
|
checkpoint_anchor="f" * 64,
|
||||||
|
observed_size=20,
|
||||||
|
observed_modified_ns=1,
|
||||||
|
),
|
||||||
|
)
|
||||||
|
checkpoint = "byte:10"
|
||||||
|
if self.include_new and checkpoint != "byte:20":
|
||||||
|
yield ScannedEvidence(
|
||||||
|
evidence=EvidenceInput(
|
||||||
|
source_id="fixture",
|
||||||
|
conversation_id="new-conversation",
|
||||||
|
message_id="new-message",
|
||||||
|
role="user",
|
||||||
|
written_by=WrittenBy.PERSON,
|
||||||
|
created_at=datetime(2026, 10, 18, tzinfo=UTC),
|
||||||
|
verbatim="new append",
|
||||||
|
origin_meta={EvidenceOrigin.FILE: OriginMetadata()},
|
||||||
|
),
|
||||||
|
cursor=SourceCursor(
|
||||||
|
file_key="e" * 64,
|
||||||
|
file_identity="bounded",
|
||||||
|
checkpoint="byte:20",
|
||||||
|
checkpoint_anchor="1" * 64,
|
||||||
|
observed_size=20,
|
||||||
|
observed_modified_ns=2,
|
||||||
|
read_to_end=True,
|
||||||
|
),
|
||||||
|
)
|
||||||
|
yield ScanProgressEvent(phase=ScanPhase.DONE)
|
||||||
|
|
||||||
|
|
||||||
class HoldingHistoryAdapter:
|
class HoldingHistoryAdapter:
|
||||||
"""First scan blocks until cancelled; later scans yield one new record."""
|
"""First scan blocks until cancelled; later scans yield one new record."""
|
||||||
|
|
||||||
@@ -328,6 +389,56 @@ def test_live_sync_does_not_resume_a_history_import_the_user_paused(tmp_path) ->
|
|||||||
asyncio.run(scenario())
|
asyncio.run(scenario())
|
||||||
|
|
||||||
|
|
||||||
|
def test_bounded_onboarding_history_establishes_cursor_before_first_sync(tmp_path) -> None:
|
||||||
|
async def scenario() -> None:
|
||||||
|
database = tmp_path / "one-memory.sqlite3"
|
||||||
|
evidence = EvidenceStore(database)
|
||||||
|
jobs = ImportJobStore(database)
|
||||||
|
adapter = BoundedOnboardingAdapter()
|
||||||
|
imports = ImportSupervisor(
|
||||||
|
adapters={"fixture": adapter},
|
||||||
|
evidence_store=evidence,
|
||||||
|
job_store=jobs,
|
||||||
|
)
|
||||||
|
live_sync = PollingLiveSyncSupervisor(
|
||||||
|
source_ids=("fixture",),
|
||||||
|
import_supervisor=imports,
|
||||||
|
job_store=jobs,
|
||||||
|
)
|
||||||
|
history = imports.create_job("fixture", operation="history")
|
||||||
|
with sqlite3.connect(database) as connection, connection:
|
||||||
|
connection.execute(
|
||||||
|
"UPDATE import_jobs SET history_after=? WHERE job_id=?",
|
||||||
|
(datetime(2026, 10, 15, tzinfo=UTC).isoformat(), history.job_id),
|
||||||
|
)
|
||||||
|
|
||||||
|
# With no native baseline, sync must wait instead of scanning from zero
|
||||||
|
# and bypassing the onboarding history window.
|
||||||
|
assert await live_sync.trigger("fixture") is None
|
||||||
|
assert not [job for job in jobs.list_jobs() if job.operation == "sync"]
|
||||||
|
|
||||||
|
imports.enqueue(history.job_id)
|
||||||
|
completed = await imports.wait(history.job_id)
|
||||||
|
assert completed.status is ImportStatus.COMPLETED
|
||||||
|
assert completed.stored == 0
|
||||||
|
assert completed.skipped == 1
|
||||||
|
assert evidence.count() == 0
|
||||||
|
|
||||||
|
# Once bounded history has advanced the cursor to EOF, normal realtime
|
||||||
|
# appends are admitted without retaining the three-day cutoff forever.
|
||||||
|
adapter.include_new = True
|
||||||
|
synced = await live_sync.trigger("fixture")
|
||||||
|
assert synced is not None
|
||||||
|
assert synced.status is ImportStatus.COMPLETED
|
||||||
|
assert synced.stored == 1
|
||||||
|
records = evidence.list_extractable()
|
||||||
|
assert [record.verbatim for record in records] == ["new append"]
|
||||||
|
await live_sync.shutdown()
|
||||||
|
await imports.shutdown()
|
||||||
|
|
||||||
|
asyncio.run(scenario())
|
||||||
|
|
||||||
|
|
||||||
def test_a_sync_left_running_after_its_worker_died_is_started_again(tmp_path) -> None:
|
def test_a_sync_left_running_after_its_worker_died_is_started_again(tmp_path) -> None:
|
||||||
"""A running row with no live worker used to be returned as 'already syncing'.
|
"""A running row with no live worker used to be returned as 'already syncing'.
|
||||||
|
|
||||||
|
|||||||
@@ -4,7 +4,7 @@ import logging
|
|||||||
import sqlite3
|
import sqlite3
|
||||||
import threading
|
import threading
|
||||||
import time
|
import time
|
||||||
from datetime import UTC, datetime
|
from datetime import UTC, datetime, timedelta
|
||||||
|
|
||||||
import pytest
|
import pytest
|
||||||
|
|
||||||
@@ -56,7 +56,12 @@ from one_memory.memory_processing import (
|
|||||||
)
|
)
|
||||||
from one_memory.memory_queue import classify
|
from one_memory.memory_queue import classify
|
||||||
from one_memory.memory_server.config_state import ConfigStore
|
from one_memory.memory_server.config_state import ConfigStore
|
||||||
from one_memory.memory_tasks import MemoryTaskKind, MemoryTaskStatus, MemoryTaskStore
|
from one_memory.memory_tasks import (
|
||||||
|
NON_RETRYABLE_POST_WRITE_CODES,
|
||||||
|
MemoryTaskKind,
|
||||||
|
MemoryTaskStatus,
|
||||||
|
MemoryTaskStore,
|
||||||
|
)
|
||||||
from one_memory.model_profiles import (
|
from one_memory.model_profiles import (
|
||||||
ModelCapabilities,
|
ModelCapabilities,
|
||||||
ModelProfileInput,
|
ModelProfileInput,
|
||||||
@@ -2613,6 +2618,123 @@ def test_an_unknown_task_without_inputs_does_not_end_the_reconcile_pass(tmp_path
|
|||||||
assert [memory.memory_id for memory in governance.list(user_id="local")] == ["hy-recovered"]
|
assert [memory.memory_id for memory in governance.list(user_id="local")] == ["hy-recovered"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_sticky_authority_unknowns_do_not_starve_ordinary_reconciliation(tmp_path) -> None:
|
||||||
|
stores = _setup(tmp_path)
|
||||||
|
evidence, profiles, tasks, governance, secrets, profile = stores
|
||||||
|
record = _put(
|
||||||
|
evidence,
|
||||||
|
message_id="fair-unknown",
|
||||||
|
role="user",
|
||||||
|
written_by=WrittenBy.PERSON,
|
||||||
|
content="ordinary unknown remains reconcilable",
|
||||||
|
)
|
||||||
|
sticky = [
|
||||||
|
tasks.create(
|
||||||
|
request_id=f"sticky-authority-{index}",
|
||||||
|
user_id=record.user_id,
|
||||||
|
kind=MemoryTaskKind.EXTRACT,
|
||||||
|
input_ref=record.evidence_id,
|
||||||
|
profile=profile,
|
||||||
|
input_records=(record,),
|
||||||
|
)
|
||||||
|
for index in range(129)
|
||||||
|
]
|
||||||
|
ordinary = tasks.create(
|
||||||
|
request_id="ordinary-unknown-after-sticky",
|
||||||
|
user_id=record.user_id,
|
||||||
|
kind=MemoryTaskKind.EXTRACT,
|
||||||
|
input_ref=record.evidence_id,
|
||||||
|
profile=profile,
|
||||||
|
input_records=(record,),
|
||||||
|
)
|
||||||
|
anchor = datetime(2026, 1, 1, tzinfo=UTC)
|
||||||
|
with sqlite3.connect(tasks.database_path) as connection, connection:
|
||||||
|
connection.executescript(
|
||||||
|
"""
|
||||||
|
CREATE TABLE source_grants (
|
||||||
|
user_id TEXT NOT NULL, source_id TEXT NOT NULL,
|
||||||
|
root_path TEXT NOT NULL, candidate_kind TEXT NOT NULL,
|
||||||
|
granted_at TEXT NOT NULL, revoked_at TEXT,
|
||||||
|
PRIMARY KEY (user_id, source_id)
|
||||||
|
);
|
||||||
|
"""
|
||||||
|
)
|
||||||
|
connection.execute(
|
||||||
|
"INSERT INTO source_grants VALUES (?,?,?,?,?,NULL)",
|
||||||
|
(
|
||||||
|
record.user_id,
|
||||||
|
record.source_id,
|
||||||
|
"/authorized",
|
||||||
|
"default",
|
||||||
|
datetime.now(UTC).isoformat(),
|
||||||
|
),
|
||||||
|
)
|
||||||
|
for index, task in enumerate(sticky):
|
||||||
|
connection.execute(
|
||||||
|
"UPDATE memory_tasks SET status='unknown',last_error_code=?,updated_at=? "
|
||||||
|
"WHERE task_id=?",
|
||||||
|
(
|
||||||
|
"hy_memory_write_verification_authority_invalid",
|
||||||
|
(anchor + timedelta(seconds=index)).isoformat(),
|
||||||
|
task.task_id,
|
||||||
|
),
|
||||||
|
)
|
||||||
|
connection.execute(
|
||||||
|
"UPDATE memory_tasks SET status='unknown',last_error_code=?,updated_at=? "
|
||||||
|
"WHERE task_id=?",
|
||||||
|
(
|
||||||
|
"model_content_outcome_unknown",
|
||||||
|
(anchor + timedelta(minutes=3)).isoformat(),
|
||||||
|
ordinary.task_id,
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
class InquiryOnlyBridge(FakeBridge):
|
||||||
|
def __init__(self) -> None:
|
||||||
|
super().__init__(
|
||||||
|
outputs=(
|
||||||
|
HyMemoryOutput(
|
||||||
|
memory_id="ordinary-recovered",
|
||||||
|
content="ordinary unknown remains reconcilable",
|
||||||
|
layer="L2_FACT",
|
||||||
|
owner="user",
|
||||||
|
evidence_ids=(record.evidence_id,),
|
||||||
|
),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
self.inquiries: list[str] = []
|
||||||
|
|
||||||
|
async def outputs_for_request(self, **kwargs) -> tuple[HyMemoryOutput, ...]:
|
||||||
|
self.inquiries.append(str(kwargs["request_id"]))
|
||||||
|
return self.outputs or ()
|
||||||
|
|
||||||
|
bridge = InquiryOnlyBridge()
|
||||||
|
supervisor = MemoryProcessingSupervisor(
|
||||||
|
tasks=tasks,
|
||||||
|
profiles=profiles,
|
||||||
|
secrets=secrets,
|
||||||
|
evidence=evidence,
|
||||||
|
governance=governance,
|
||||||
|
data_dir=tmp_path,
|
||||||
|
runner_factory=lambda: _runner(tmp_path, stores, FakeBridge()),
|
||||||
|
bridge_factory=lambda _profile, _secret, _data_dir: bridge,
|
||||||
|
)
|
||||||
|
|
||||||
|
assert (
|
||||||
|
asyncio.run(
|
||||||
|
supervisor.reconcile_unknown_outputs(
|
||||||
|
limit=128,
|
||||||
|
exclude_error_codes=NON_RETRYABLE_POST_WRITE_CODES,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
== 1
|
||||||
|
)
|
||||||
|
assert bridge.inquiries == ["ordinary-unknown-after-sticky"]
|
||||||
|
assert bridge.requests == []
|
||||||
|
assert tasks.require(ordinary.task_id).status is MemoryTaskStatus.COMPLETED
|
||||||
|
assert all(tasks.require(task.task_id).status is MemoryTaskStatus.UNKNOWN for task in sticky)
|
||||||
|
|
||||||
|
|
||||||
def test_two_workspaces_with_one_conversation_id_prepare_apart(tmp_path) -> None:
|
def test_two_workspaces_with_one_conversation_id_prepare_apart(tmp_path) -> None:
|
||||||
"""Cursor can reuse a conversation id across projects.
|
"""Cursor can reuse a conversation id across projects.
|
||||||
|
|
||||||
@@ -3441,6 +3563,58 @@ def test_digest_scopes_separate_applications_and_dates(tmp_path) -> None:
|
|||||||
assert len({_hy_agent_id((record,)) for record in records}) == 3
|
assert len({_hy_agent_id((record,)) for record in records}) == 3
|
||||||
|
|
||||||
|
|
||||||
|
def test_workspace_requests_carry_conversation_authority_beyond_agent_scope(tmp_path) -> None:
|
||||||
|
"""Same-day workspace scopes collide by design; session_id must separate writes."""
|
||||||
|
evidence, _profiles, tasks, _governance, _secrets, profile = _setup(tmp_path)
|
||||||
|
first = _put(
|
||||||
|
evidence,
|
||||||
|
message_id="conversation-a",
|
||||||
|
role="user",
|
||||||
|
written_by=WrittenBy.PERSON,
|
||||||
|
content="same workspace, first conversation",
|
||||||
|
conversation_id="conversation-a",
|
||||||
|
workspace="repo-a",
|
||||||
|
created_at=datetime(2026, 10, 18, 8, tzinfo=UTC),
|
||||||
|
)
|
||||||
|
second = _put(
|
||||||
|
evidence,
|
||||||
|
message_id="conversation-b",
|
||||||
|
role="user",
|
||||||
|
written_by=WrittenBy.PERSON,
|
||||||
|
content="same workspace, second conversation",
|
||||||
|
conversation_id="conversation-b",
|
||||||
|
workspace="repo-a",
|
||||||
|
created_at=datetime(2026, 10, 18, 9, tzinfo=UTC),
|
||||||
|
)
|
||||||
|
next_day = _put(
|
||||||
|
evidence,
|
||||||
|
message_id="conversation-a-next-day",
|
||||||
|
role="user",
|
||||||
|
written_by=WrittenBy.PERSON,
|
||||||
|
content="same conversation on the next day",
|
||||||
|
conversation_id="conversation-a",
|
||||||
|
workspace="repo-a",
|
||||||
|
created_at=datetime(2026, 10, 19, 8, tzinfo=UTC),
|
||||||
|
)
|
||||||
|
|
||||||
|
requests = []
|
||||||
|
for record in (first, second, next_day):
|
||||||
|
task = tasks.create(
|
||||||
|
request_id=f"authority:{record.evidence_id}",
|
||||||
|
user_id=record.user_id,
|
||||||
|
kind=MemoryTaskKind.EXTRACT,
|
||||||
|
input_ref=record.evidence_id,
|
||||||
|
profile=profile,
|
||||||
|
input_records=(record,),
|
||||||
|
)
|
||||||
|
requests.append(MemoryTaskRunner._request(task, (record,)))
|
||||||
|
|
||||||
|
assert requests[0].agent_id == requests[1].agent_id
|
||||||
|
assert requests[0].session_id == "conversation-a"
|
||||||
|
assert requests[1].session_id == "conversation-b"
|
||||||
|
assert requests[2].agent_id != requests[0].agent_id
|
||||||
|
|
||||||
|
|
||||||
def test_cancelled_multi_source_lock_acquisition_releases_earlier_source(tmp_path) -> None:
|
def test_cancelled_multi_source_lock_acquisition_releases_earlier_source(tmp_path) -> None:
|
||||||
evidence, profiles, tasks, governance, secrets, _profile = _setup(tmp_path)
|
evidence, profiles, tasks, governance, secrets, _profile = _setup(tmp_path)
|
||||||
first = _put(
|
first = _put(
|
||||||
|
|||||||
@@ -5,6 +5,7 @@ from __future__ import annotations
|
|||||||
import asyncio
|
import asyncio
|
||||||
import contextlib
|
import contextlib
|
||||||
import json
|
import json
|
||||||
|
import logging
|
||||||
import sqlite3
|
import sqlite3
|
||||||
import threading
|
import threading
|
||||||
import time
|
import time
|
||||||
@@ -1369,24 +1370,30 @@ def test_a_machine_error_code_is_preserved(tmp_path) -> None:
|
|||||||
assert safe_error_code(ValueError("some sentence with spaces")) == "ValueError"
|
assert safe_error_code(ValueError("some sentence with spaces")) == "ValueError"
|
||||||
|
|
||||||
|
|
||||||
def test_an_inactive_profile_parks_for_model_before_any_send(tmp_path) -> None:
|
def test_an_inactive_profile_parks_without_error_logging(tmp_path, caplog) -> None:
|
||||||
"""A pre-bridge model-not-ready code parks and keeps the intent."""
|
"""A pre-bridge model-not-ready code parks and keeps the intent."""
|
||||||
path = tmp_path / "one-memory.sqlite3"
|
path = tmp_path / "one-memory.sqlite3"
|
||||||
queue = DurableWriteQueue(path)
|
queue = DurableWriteQueue(path)
|
||||||
record = queue.submit(_request())
|
record = queue.submit(_request())
|
||||||
_claim_through_drain(queue, record.job_id)
|
_claim_through_drain(queue, record.job_id)
|
||||||
|
|
||||||
asyncio.run(
|
with caplog.at_level(
|
||||||
execute_job(
|
logging.DEBUG,
|
||||||
record.job_id,
|
logger="one_memory.memory_server.durable_queue",
|
||||||
queue=queue,
|
):
|
||||||
supervisor=_Recorder(error=RuntimeError("model_profile_not_active")),
|
asyncio.run(
|
||||||
evidence=_EvidenceStore({"e1"}),
|
execute_job(
|
||||||
|
record.job_id,
|
||||||
|
queue=queue,
|
||||||
|
supervisor=_Recorder(error=RuntimeError("model_profile_not_active")),
|
||||||
|
evidence=_EvidenceStore({"e1"}),
|
||||||
|
)
|
||||||
)
|
)
|
||||||
)
|
|
||||||
|
|
||||||
assert _state(path, record.job_id) == WAITING_FOR_MODEL
|
assert _state(path, record.job_id) == WAITING_FOR_MODEL
|
||||||
assert _rows(path)[0]["hy_invoked"] == 0
|
assert _rows(path)[0]["hy_invoked"] == 0
|
||||||
|
assert "durable_write_job_failed" not in caplog.text
|
||||||
|
assert "durable_write_job_parked" in caplog.text
|
||||||
|
|
||||||
|
|
||||||
def test_an_authorization_stop_cancels_instead_of_failing(tmp_path) -> None:
|
def test_an_authorization_stop_cancels_instead_of_failing(tmp_path) -> None:
|
||||||
@@ -1546,7 +1553,7 @@ def test_parking_does_not_burn_the_retry_budget(tmp_path) -> None:
|
|||||||
assert queue.get("idem-1").state == PENDING
|
assert queue.get("idem-1").state == PENDING
|
||||||
|
|
||||||
|
|
||||||
def test_a_model_waiter_is_readmitted_once_its_wait_elapses(tmp_path) -> None:
|
def test_a_transient_model_waiter_is_readmitted_once_its_wait_elapses(tmp_path) -> None:
|
||||||
"""A model outage clears on its own; nothing re-admits it otherwise.
|
"""A model outage clears on its own; nothing re-admits it otherwise.
|
||||||
|
|
||||||
`waiting_for_permission` needs a grant event, but `waiting_for_model` is
|
`waiting_for_permission` needs a grant event, but `waiting_for_model` is
|
||||||
@@ -1564,7 +1571,7 @@ def test_a_model_waiter_is_readmitted_once_its_wait_elapses(tmp_path) -> None:
|
|||||||
queue.settle(
|
queue.settle(
|
||||||
record.job_id,
|
record.job_id,
|
||||||
state=WAITING_FOR_MODEL,
|
state=WAITING_FOR_MODEL,
|
||||||
error_code="model_profile_not_active",
|
error_code="model_secret_unavailable",
|
||||||
hy_invoked=False,
|
hy_invoked=False,
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -1581,8 +1588,8 @@ def test_a_model_waiter_is_readmitted_once_its_wait_elapses(tmp_path) -> None:
|
|||||||
assert _rows(path)[0]["attempts"] == 0
|
assert _rows(path)[0]["attempts"] == 0
|
||||||
|
|
||||||
|
|
||||||
def test_readmitting_a_model_waiter_clears_its_stale_detail(tmp_path) -> None:
|
def test_no_profile_waiter_is_readmitted_only_after_activation(tmp_path) -> None:
|
||||||
"""The timer readmission path must clear the old blocking reason too."""
|
"""No timer can create a model profile; activation owns this transition."""
|
||||||
path = tmp_path / "one-memory.sqlite3"
|
path = tmp_path / "one-memory.sqlite3"
|
||||||
queue = DurableWriteQueue(path)
|
queue = DurableWriteQueue(path)
|
||||||
record = queue.submit(_request())
|
record = queue.submit(_request())
|
||||||
@@ -1594,13 +1601,50 @@ def test_readmitting_a_model_waiter_clears_its_stale_detail(tmp_path) -> None:
|
|||||||
)
|
)
|
||||||
now = datetime.now(UTC)
|
now = datetime.now(UTC)
|
||||||
|
|
||||||
assert queue.readmit_model_waiters(now=now + timedelta(minutes=2)) == 1
|
assert queue.readmit_model_waiters(now=now + timedelta(minutes=2)) == 0
|
||||||
|
parked = queue.get("idem-1")
|
||||||
|
assert parked.state == WAITING_FOR_MODEL
|
||||||
|
assert parked.detail == "model_profile_not_active"
|
||||||
|
|
||||||
|
assert queue.readmit_model_profile_waiters() == 1
|
||||||
readmitted = queue.get("idem-1")
|
readmitted = queue.get("idem-1")
|
||||||
assert readmitted.state == PENDING
|
assert readmitted.state == PENDING
|
||||||
assert readmitted.detail == ""
|
assert readmitted.detail == ""
|
||||||
|
|
||||||
|
|
||||||
|
def test_no_profile_backlog_stays_parked_across_maintenance_cycles(tmp_path) -> None:
|
||||||
|
path = tmp_path / "one-memory.sqlite3"
|
||||||
|
queue = DurableWriteQueue(path)
|
||||||
|
for index in range(100):
|
||||||
|
record = queue.submit(_request(key=f"no-profile-{index}"))
|
||||||
|
queue.settle(
|
||||||
|
record.job_id,
|
||||||
|
state=WAITING_FOR_MODEL,
|
||||||
|
error_code="model_profile_not_active",
|
||||||
|
hy_invoked=False,
|
||||||
|
)
|
||||||
|
with sqlite3.connect(path) as connection, connection:
|
||||||
|
connection.execute(
|
||||||
|
"UPDATE memory_write_jobs SET attempts=1,"
|
||||||
|
"updated_at='2000-01-01T00:00:00+00:00'"
|
||||||
|
)
|
||||||
|
|
||||||
|
first = datetime.now(UTC)
|
||||||
|
assert queue.readmit_model_waiters(now=first, after=timedelta(0)) == 0
|
||||||
|
assert (
|
||||||
|
queue.readmit_model_waiters(
|
||||||
|
now=first + timedelta(minutes=2),
|
||||||
|
after=timedelta(0),
|
||||||
|
)
|
||||||
|
== 0
|
||||||
|
)
|
||||||
|
with sqlite3.connect(path) as connection:
|
||||||
|
assert connection.execute(
|
||||||
|
"SELECT state,attempts,COUNT(*) FROM memory_write_jobs "
|
||||||
|
"GROUP BY state,attempts"
|
||||||
|
).fetchall() == [(WAITING_FOR_MODEL, 1, 100)]
|
||||||
|
|
||||||
|
|
||||||
def test_readmit_parked_cannot_resurrect_a_terminal_row(tmp_path) -> None:
|
def test_readmit_parked_cannot_resurrect_a_terminal_row(tmp_path) -> None:
|
||||||
"""Only parked states may be re-admitted, never a finished write.
|
"""Only parked states may be re-admitted, never a finished write.
|
||||||
|
|
||||||
@@ -3472,6 +3516,41 @@ def test_unknown_jobs_does_not_resolve_or_make_a_row_claimable(tmp_path) -> None
|
|||||||
assert queue.resolve_unknown(found[0].job_id, found=False, detail="not in HY") == DEAD
|
assert queue.resolve_unknown(found[0].job_id, found=False, detail="not in HY") == DEAD
|
||||||
|
|
||||||
|
|
||||||
|
def test_unknown_jobs_can_exclude_sticky_authority_failures_before_limit(tmp_path) -> None:
|
||||||
|
path = tmp_path / "one-memory.sqlite3"
|
||||||
|
queue = DurableWriteQueue(path)
|
||||||
|
sticky = [
|
||||||
|
queue.submit(_request(key=f"sticky-{index}", evidence=(f"sticky-e{index}",)))
|
||||||
|
for index in range(129)
|
||||||
|
]
|
||||||
|
ordinary = queue.submit(_request(key="ordinary", evidence=("ordinary-e",)))
|
||||||
|
anchor = datetime(2026, 1, 1, tzinfo=UTC)
|
||||||
|
with sqlite3.connect(path) as connection, connection:
|
||||||
|
for index, record in enumerate(sticky):
|
||||||
|
connection.execute(
|
||||||
|
"UPDATE memory_write_jobs SET state='unknown',attempts=1,hy_invoked=1,"
|
||||||
|
"last_error=?,updated_at=? WHERE job_id=?",
|
||||||
|
(
|
||||||
|
"hy_memory_write_verification_authority_invalid",
|
||||||
|
(anchor + timedelta(seconds=index)).isoformat(),
|
||||||
|
record.job_id,
|
||||||
|
),
|
||||||
|
)
|
||||||
|
connection.execute(
|
||||||
|
"UPDATE memory_write_jobs SET state='unknown',attempts=1,hy_invoked=1,"
|
||||||
|
"last_error='model_content_outcome_unknown',updated_at=? WHERE job_id=?",
|
||||||
|
((anchor + timedelta(minutes=3)).isoformat(), ordinary.job_id),
|
||||||
|
)
|
||||||
|
|
||||||
|
found = queue.unknown_jobs(
|
||||||
|
limit=128,
|
||||||
|
exclude_error_codes=("hy_memory_write_verification_authority_invalid",),
|
||||||
|
)
|
||||||
|
|
||||||
|
assert [job.job_id for job in found] == [ordinary.job_id]
|
||||||
|
assert all(queue.get(record.request_id).state == UNKNOWN for record in sticky)
|
||||||
|
|
||||||
|
|
||||||
def test_unknown_jobs_tolerates_a_row_resolved_between_pages(tmp_path) -> None:
|
def test_unknown_jobs_tolerates_a_row_resolved_between_pages(tmp_path) -> None:
|
||||||
"""Discovery is read-only and may overlap an inquiry from another worker.
|
"""Discovery is read-only and may overlap an inquiry from another worker.
|
||||||
|
|
||||||
@@ -4852,15 +4931,16 @@ def test_a_failure_after_the_call_still_parks_as_unknown(tmp_path) -> None:
|
|||||||
record = queue.submit(_request(key="after-call", source="opencode"))
|
record = queue.submit(_request(key="after-call", source="opencode"))
|
||||||
claimer = DurableWriteDrain(queue, execute=_never_execute, owner="after-call-owner")
|
claimer = DurableWriteDrain(queue, execute=_never_execute, owner="after-call-owner")
|
||||||
assert claimer.claim(limit=1) == (record.job_id,)
|
assert claimer.claim(limit=1) == (record.job_id,)
|
||||||
|
recorder = _Recorder(
|
||||||
|
error=HyMemoryBridgeError("hy_memory_write_verification_authority_invalid"),
|
||||||
|
invoke_hy=True,
|
||||||
|
)
|
||||||
|
|
||||||
asyncio.run(
|
asyncio.run(
|
||||||
execute_job(
|
execute_job(
|
||||||
record.job_id,
|
record.job_id,
|
||||||
queue=queue,
|
queue=queue,
|
||||||
supervisor=_Recorder(
|
supervisor=recorder,
|
||||||
error=HyMemoryBridgeError("hy_memory_write_verification_authority_invalid"),
|
|
||||||
invoke_hy=True,
|
|
||||||
),
|
|
||||||
evidence=_EvidenceStore({"e1"}),
|
evidence=_EvidenceStore({"e1"}),
|
||||||
owner="after-call-owner",
|
owner="after-call-owner",
|
||||||
)
|
)
|
||||||
@@ -4874,6 +4954,16 @@ def test_a_failure_after_the_call_still_parks_as_unknown(tmp_path) -> None:
|
|||||||
|
|
||||||
assert state == UNKNOWN
|
assert state == UNKNOWN
|
||||||
assert invoked == 1
|
assert invoked == 1
|
||||||
|
assert len(recorder.seen) == 1
|
||||||
|
assert (
|
||||||
|
queue.retry_first_unknown_after_reconcile(
|
||||||
|
older_than=datetime.now(UTC) + timedelta(minutes=1)
|
||||||
|
)
|
||||||
|
== 0
|
||||||
|
)
|
||||||
|
assert queue.retry_unknown_confirmed(record.job_id) == 0
|
||||||
|
assert DurableWriteDrain(queue, execute=_never_execute).claim(limit=1) == ()
|
||||||
|
assert len(recorder.seen) == 1
|
||||||
|
|
||||||
|
|
||||||
def test_event_time_not_enqueue_order_drives_the_window(
|
def test_event_time_not_enqueue_order_drives_the_window(
|
||||||
|
|||||||
@@ -1113,6 +1113,109 @@ def test_task_only_unknown_is_automatically_retried_only_once(tmp_path) -> None:
|
|||||||
assert store.require(task.task_id).status is MemoryTaskStatus.UNKNOWN
|
assert store.require(task.task_id).status is MemoryTaskStatus.UNKNOWN
|
||||||
|
|
||||||
|
|
||||||
|
def test_authority_invalid_task_is_never_reinvoked(tmp_path) -> None:
|
||||||
|
store, profile = _profile_and_tasks(tmp_path)
|
||||||
|
task = store.create(
|
||||||
|
request_id="authority-invalid",
|
||||||
|
user_id="local",
|
||||||
|
kind=MemoryTaskKind.EXTRACT,
|
||||||
|
input_ref="authority-evidence",
|
||||||
|
profile=profile,
|
||||||
|
)
|
||||||
|
with sqlite3.connect(store.database_path) as connection, connection:
|
||||||
|
connection.execute(
|
||||||
|
"UPDATE memory_tasks SET status=?,attempts=1,last_error_code=?,updated_at=? "
|
||||||
|
"WHERE task_id=?",
|
||||||
|
(
|
||||||
|
MemoryTaskStatus.UNKNOWN,
|
||||||
|
"hy_memory_write_verification_authority_invalid",
|
||||||
|
"2026-01-01T00:00:00+00:00",
|
||||||
|
task.task_id,
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
cutoff = datetime(2026, 1, 1, 0, 5, tzinfo=UTC)
|
||||||
|
assert store.retry_first_unknown_after_reconcile(older_than=cutoff) == 0
|
||||||
|
assert store.retry_unknown_confirmed() == 0
|
||||||
|
retained = store.require(task.task_id)
|
||||||
|
assert retained.status is MemoryTaskStatus.UNKNOWN
|
||||||
|
assert retained.attempts == 1
|
||||||
|
assert retained.last_error_code == "hy_memory_write_verification_authority_invalid"
|
||||||
|
|
||||||
|
|
||||||
|
def test_linked_unknown_is_retried_only_by_the_durable_job(tmp_path) -> None:
|
||||||
|
from one_memory.memory_server.contracts import BatchIdentity, WriteRequest
|
||||||
|
from one_memory.memory_server.durable_queue import DurableWriteQueue
|
||||||
|
|
||||||
|
store, profile = _profile_and_tasks(tmp_path)
|
||||||
|
evidence = EvidenceStore(store.database_path)
|
||||||
|
record = evidence.put(
|
||||||
|
EvidenceInput(
|
||||||
|
source_id="cursor",
|
||||||
|
conversation_id="linked-conversation",
|
||||||
|
message_id="linked-message",
|
||||||
|
role="user",
|
||||||
|
written_by=WrittenBy.PERSON,
|
||||||
|
verbatim="linked unknown",
|
||||||
|
created_at=datetime(2026, 1, 1, tzinfo=UTC),
|
||||||
|
origin_meta={EvidenceOrigin.FILE: OriginMetadata()},
|
||||||
|
)
|
||||||
|
).record
|
||||||
|
task = store.create(
|
||||||
|
request_id="linked-task",
|
||||||
|
user_id="local",
|
||||||
|
kind=MemoryTaskKind.EXTRACT,
|
||||||
|
input_ref=record.evidence_id,
|
||||||
|
profile=profile,
|
||||||
|
input_records=(record,),
|
||||||
|
)
|
||||||
|
queue = DurableWriteQueue(store.database_path, evidence=evidence)
|
||||||
|
job = queue.submit(
|
||||||
|
WriteRequest(
|
||||||
|
internal_session_id="linked-session",
|
||||||
|
batch_identity=BatchIdentity(
|
||||||
|
internal_session_id="linked-session",
|
||||||
|
first_round=1,
|
||||||
|
last_round=1,
|
||||||
|
),
|
||||||
|
evidence_ids=[record.evidence_id],
|
||||||
|
idempotency_key="linked-job",
|
||||||
|
source_id="cursor",
|
||||||
|
)
|
||||||
|
)
|
||||||
|
with sqlite3.connect(store.database_path) as connection, connection:
|
||||||
|
connection.execute(
|
||||||
|
"UPDATE memory_tasks SET status=?,attempts=1,last_error_code=?,updated_at=? "
|
||||||
|
"WHERE task_id=?",
|
||||||
|
(
|
||||||
|
MemoryTaskStatus.UNKNOWN,
|
||||||
|
"model_content_outcome_unknown",
|
||||||
|
"2026-01-01T00:00:00+00:00",
|
||||||
|
task.task_id,
|
||||||
|
),
|
||||||
|
)
|
||||||
|
connection.execute(
|
||||||
|
"UPDATE memory_write_jobs SET state='unknown',attempts=1,hy_invoked=1,"
|
||||||
|
"last_error='model_content_outcome_unknown',updated_at=? WHERE job_id=?",
|
||||||
|
("2026-01-01T00:00:00+00:00", job.job_id),
|
||||||
|
)
|
||||||
|
|
||||||
|
cutoff = datetime(2026, 1, 1, 0, 5, tzinfo=UTC)
|
||||||
|
assert store.retry_first_unknown_after_reconcile(older_than=cutoff) == 0
|
||||||
|
assert store.require(task.task_id).status is MemoryTaskStatus.UNKNOWN
|
||||||
|
assert queue.retry_first_unknown_after_reconcile(older_than=cutoff) == 1
|
||||||
|
assert store.require(task.task_id).status is MemoryTaskStatus.PENDING
|
||||||
|
assert queue.get(job.request_id).state == "pending"
|
||||||
|
assert (
|
||||||
|
store.active_window(
|
||||||
|
now=datetime(2026, 1, 2, tzinfo=UTC),
|
||||||
|
fresh_window=timedelta(days=1),
|
||||||
|
extraction_window=None,
|
||||||
|
)
|
||||||
|
== []
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def test_orphan_recovery_preserves_active_running_tasks(tmp_path) -> None:
|
def test_orphan_recovery_preserves_active_running_tasks(tmp_path) -> None:
|
||||||
store, profile = _profile_and_tasks(tmp_path)
|
store, profile = _profile_and_tasks(tmp_path)
|
||||||
active = store.create(
|
active = store.create(
|
||||||
|
|||||||
@@ -8,7 +8,14 @@ import pytest
|
|||||||
from one_memory.imports.grants import GRANT_SCHEMA
|
from one_memory.imports.grants import GRANT_SCHEMA
|
||||||
from one_memory.imports.models import EvidenceInput, EvidenceOrigin, OriginMetadata, WrittenBy
|
from one_memory.imports.models import EvidenceInput, EvidenceOrigin, OriginMetadata, WrittenBy
|
||||||
from one_memory.imports.storage import EvidenceStore
|
from one_memory.imports.storage import EvidenceStore
|
||||||
from one_memory.memory_governance import MemoryDraft, MemoryGovernanceStore, MemoryKind
|
from one_memory.memory_governance import (
|
||||||
|
HyCreate,
|
||||||
|
HyLink,
|
||||||
|
HyRevise,
|
||||||
|
MemoryDraft,
|
||||||
|
MemoryGovernanceStore,
|
||||||
|
MemoryKind,
|
||||||
|
)
|
||||||
from one_memory.memory_tasks import (
|
from one_memory.memory_tasks import (
|
||||||
MemoryTaskOutput,
|
MemoryTaskOutput,
|
||||||
MemoryTaskStore,
|
MemoryTaskStore,
|
||||||
@@ -19,6 +26,7 @@ from one_memory.memory_visibility import (
|
|||||||
install_triggers,
|
install_triggers,
|
||||||
pending,
|
pending,
|
||||||
recompute,
|
recompute,
|
||||||
|
startup,
|
||||||
)
|
)
|
||||||
from one_memory.model_profiles import (
|
from one_memory.model_profiles import (
|
||||||
ModelCapabilities,
|
ModelCapabilities,
|
||||||
@@ -336,6 +344,197 @@ def test_starting_the_app_creates_the_derived_tables(tmp_path):
|
|||||||
asyncio.run(scenario())
|
asyncio.run(scenario())
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("visibility_mode", ["exact", "derived"])
|
||||||
|
def test_hy_revise_and_repeated_link_reuse_the_same_evidence(library, monkeypatch, visibility_mode):
|
||||||
|
"""The outer evidence UPSERT must not override the trigger's duplicate handling."""
|
||||||
|
database, evidence, _tasks, governance, _profile = library
|
||||||
|
monkeypatch.setenv("ONE_MEMORY_READ_VISIBILITY", visibility_mode)
|
||||||
|
startup(database)
|
||||||
|
record = evidence.put(
|
||||||
|
EvidenceInput(
|
||||||
|
source_id="cursor",
|
||||||
|
conversation_id="trigger-regression",
|
||||||
|
message_id="trigger-regression",
|
||||||
|
correlation_id="trigger-regression",
|
||||||
|
role="user",
|
||||||
|
written_by=WrittenBy.PERSON,
|
||||||
|
verbatim="用户住在杭州。",
|
||||||
|
origin_meta={EvidenceOrigin.FILE: OriginMetadata()},
|
||||||
|
)
|
||||||
|
).record
|
||||||
|
created = governance.write_from_hy(
|
||||||
|
HyCreate(
|
||||||
|
draft=MemoryDraft(
|
||||||
|
user_id="local",
|
||||||
|
kind=MemoryKind.CHAT,
|
||||||
|
content="用户住在杭州。",
|
||||||
|
scopes=("all-apps",),
|
||||||
|
evidence_ids=(record.evidence_id,),
|
||||||
|
),
|
||||||
|
request_id="trigger-create",
|
||||||
|
memory_id="trigger-memory",
|
||||||
|
),
|
||||||
|
user_id="local",
|
||||||
|
derivation=governance.hy_derivation((record.evidence_id,)),
|
||||||
|
)
|
||||||
|
revision = HyRevise(
|
||||||
|
memory_id=created.memory_id,
|
||||||
|
request_id="trigger-revise",
|
||||||
|
expected_revision=created.revision,
|
||||||
|
content="用户住在杭州并在那里工作。",
|
||||||
|
evidence_ids=(record.evidence_id,),
|
||||||
|
)
|
||||||
|
revised = governance.write_from_hy(
|
||||||
|
revision,
|
||||||
|
user_id="local",
|
||||||
|
derivation=governance.hy_derivation((record.evidence_id,)),
|
||||||
|
)
|
||||||
|
replayed = governance.write_from_hy(
|
||||||
|
revision,
|
||||||
|
user_id="local",
|
||||||
|
derivation=governance.hy_derivation((record.evidence_id,)),
|
||||||
|
)
|
||||||
|
assert replayed == revised
|
||||||
|
link = HyLink(memory_id=created.memory_id, evidence_id=record.evidence_id)
|
||||||
|
for _ in range(2):
|
||||||
|
governance.write_from_hy(
|
||||||
|
link,
|
||||||
|
user_id="local",
|
||||||
|
derivation=governance.hy_derivation((record.evidence_id,)),
|
||||||
|
)
|
||||||
|
|
||||||
|
current = governance.get(created.memory_id, user_id="local")
|
||||||
|
assert current is not None
|
||||||
|
assert current.content == "用户住在杭州并在那里工作。"
|
||||||
|
assert current.revision == created.revision + 1
|
||||||
|
with main_database_read(database) as db:
|
||||||
|
assert db.execute(
|
||||||
|
"SELECT memory_id, COUNT(*) FROM memory_visibility_dirty GROUP BY memory_id"
|
||||||
|
).fetchall() == [(created.memory_id, 1)]
|
||||||
|
|
||||||
|
if visibility_mode == "derived":
|
||||||
|
monkeypatch.setenv("ONE_MEMORY_READ_VISIBILITY", "exact")
|
||||||
|
exact = {memory.memory_id for memory in SourceReadAccess(database).filter([current])}
|
||||||
|
with main_database_write(database) as db:
|
||||||
|
assert recompute(db, [created.memory_id], readable_sql=_MEMORY_READABLE) == 1
|
||||||
|
db.commit()
|
||||||
|
monkeypatch.setenv("ONE_MEMORY_READ_VISIBILITY", "derived")
|
||||||
|
derived = {memory.memory_id for memory in SourceReadAccess(database).filter([current])}
|
||||||
|
assert derived == exact
|
||||||
|
with main_database_read(database) as db:
|
||||||
|
assert created.memory_id not in pending(db)
|
||||||
|
|
||||||
|
|
||||||
|
def test_startup_atomically_replaces_the_legacy_link_update_trigger(tmp_path, monkeypatch):
|
||||||
|
import one_memory.memory_visibility as visibility_module
|
||||||
|
from one_memory.schema_gate import guard_schema_version
|
||||||
|
|
||||||
|
database = tmp_path / "one-memory.sqlite3"
|
||||||
|
expected_version = guard_schema_version(database)
|
||||||
|
evidence = EvidenceStore(database)
|
||||||
|
governance = MemoryGovernanceStore(database)
|
||||||
|
record = evidence.put(
|
||||||
|
EvidenceInput(
|
||||||
|
source_id="cursor",
|
||||||
|
conversation_id="legacy-trigger",
|
||||||
|
message_id="legacy-trigger",
|
||||||
|
correlation_id="legacy-trigger",
|
||||||
|
role="user",
|
||||||
|
written_by=WrittenBy.PERSON,
|
||||||
|
verbatim="升级失败不能改变这条原文。",
|
||||||
|
origin_meta={EvidenceOrigin.FILE: OriginMetadata()},
|
||||||
|
)
|
||||||
|
).record
|
||||||
|
memory = as_person(governance).create(
|
||||||
|
MemoryDraft(
|
||||||
|
user_id="local",
|
||||||
|
kind=MemoryKind.CHAT,
|
||||||
|
content="升级失败不能改变这条记忆。",
|
||||||
|
scopes=("all-apps",),
|
||||||
|
evidence_ids=(record.evidence_id,),
|
||||||
|
),
|
||||||
|
request_id="legacy-trigger-memory",
|
||||||
|
)
|
||||||
|
startup(database)
|
||||||
|
with sqlite3.connect(database) as db:
|
||||||
|
db.execute("DROP TRIGGER memory_visibility_dirty_on_link_update")
|
||||||
|
db.executescript(
|
||||||
|
"""
|
||||||
|
CREATE TRIGGER memory_visibility_dirty_on_link_update
|
||||||
|
AFTER UPDATE ON governed_memory_evidence
|
||||||
|
BEGIN
|
||||||
|
INSERT OR IGNORE INTO memory_visibility_dirty (memory_id)
|
||||||
|
VALUES (NEW.memory_id);
|
||||||
|
INSERT OR IGNORE INTO memory_visibility_dirty (memory_id)
|
||||||
|
VALUES (OLD.memory_id);
|
||||||
|
END;
|
||||||
|
"""
|
||||||
|
)
|
||||||
|
db.execute(
|
||||||
|
"INSERT INTO memory_visibility (memory_id, visible, reason, computed_at) "
|
||||||
|
"VALUES (?, 1, 'ok', '2026-10-19T00:00:00+00:00')",
|
||||||
|
(memory.memory_id,),
|
||||||
|
)
|
||||||
|
db.execute(
|
||||||
|
"INSERT INTO memory_visibility_dirty (memory_id) VALUES (?)",
|
||||||
|
(memory.memory_id,),
|
||||||
|
)
|
||||||
|
legacy_trigger = db.execute(
|
||||||
|
"SELECT sql FROM sqlite_master "
|
||||||
|
"WHERE type='trigger' AND name='memory_visibility_dirty_on_link_update'"
|
||||||
|
).fetchone()[0]
|
||||||
|
before = tuple(db.iterdump())
|
||||||
|
|
||||||
|
real_trigger_sql = visibility_module._trigger_sql
|
||||||
|
|
||||||
|
def fail_after_drop() -> list[tuple[str, str]]:
|
||||||
|
statements = real_trigger_sql()
|
||||||
|
return [
|
||||||
|
(table, "CREATE TRIGGER")
|
||||||
|
if "CREATE TRIGGER IF NOT EXISTS memory_visibility_dirty_on_link_update" in statement
|
||||||
|
else (table, statement)
|
||||||
|
for table, statement in statements
|
||||||
|
]
|
||||||
|
|
||||||
|
with monkeypatch.context() as failure:
|
||||||
|
failure.setattr(visibility_module, "_trigger_sql", fail_after_drop)
|
||||||
|
with pytest.raises(sqlite3.OperationalError):
|
||||||
|
startup(database)
|
||||||
|
|
||||||
|
with sqlite3.connect(database) as db:
|
||||||
|
assert tuple(db.iterdump()) == before
|
||||||
|
assert (
|
||||||
|
db.execute(
|
||||||
|
"SELECT sql FROM sqlite_master "
|
||||||
|
"WHERE type='trigger' AND name='memory_visibility_dirty_on_link_update'"
|
||||||
|
).fetchone()[0]
|
||||||
|
== legacy_trigger
|
||||||
|
)
|
||||||
|
assert db.execute("SELECT MAX(version) FROM schema_migrations").fetchone()[0] == (
|
||||||
|
expected_version
|
||||||
|
)
|
||||||
|
|
||||||
|
startup(database)
|
||||||
|
startup(database)
|
||||||
|
|
||||||
|
with sqlite3.connect(database) as db:
|
||||||
|
trigger_sql = db.execute(
|
||||||
|
"SELECT sql FROM sqlite_master "
|
||||||
|
"WHERE type='trigger' AND name='memory_visibility_dirty_on_link_update'"
|
||||||
|
).fetchone()[0]
|
||||||
|
assert trigger_sql.count("ON CONFLICT(memory_id) DO NOTHING") == 2
|
||||||
|
assert "INSERT OR IGNORE" not in trigger_sql
|
||||||
|
assert db.execute("SELECT MAX(version) FROM schema_migrations").fetchone()[0] == (
|
||||||
|
expected_version
|
||||||
|
)
|
||||||
|
assert db.execute("SELECT * FROM memory_visibility").fetchall() == [
|
||||||
|
(memory.memory_id, 1, "ok", "2026-10-19T00:00:00+00:00")
|
||||||
|
]
|
||||||
|
assert db.execute("SELECT * FROM memory_visibility_dirty").fetchall() == [
|
||||||
|
(memory.memory_id,)
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
def test_the_audit_notices_a_conclusion_that_no_longer_holds(library):
|
def test_the_audit_notices_a_conclusion_that_no_longer_holds(library):
|
||||||
"""The ongoing proof: recompute a slice and compare with what is stored.
|
"""The ongoing proof: recompute a slice and compare with what is stored.
|
||||||
|
|
||||||
|
|||||||
@@ -259,12 +259,17 @@ def test_activation_survives_a_corpus_already_imported(tmp_path) -> None:
|
|||||||
),
|
),
|
||||||
secrets=secrets,
|
secrets=secrets,
|
||||||
)
|
)
|
||||||
|
readmissions: list[bool] = []
|
||||||
|
app.state.memory_write_queue.readmit_model_profile_waiters = (
|
||||||
|
lambda: readmissions.append(True) or 0
|
||||||
|
)
|
||||||
|
|
||||||
activated = await client.post(f"/api/v1/model-profiles/{profile_id}/activate")
|
activated = await client.post(f"/api/v1/model-profiles/{profile_id}/activate")
|
||||||
|
|
||||||
assert activated.status_code == 200
|
assert activated.status_code == 200
|
||||||
assert activated.json()["is_active"] is True
|
assert activated.json()["is_active"] is True
|
||||||
assert API_KEY not in activated.text
|
assert API_KEY not in activated.text
|
||||||
|
assert readmissions == [True]
|
||||||
# Started, not waited for: the walk reads every conversation.
|
# Started, not waited for: the walk reads every conversation.
|
||||||
assert not app.state.memory_backfill.done()
|
assert not app.state.memory_backfill.done()
|
||||||
|
|
||||||
|
|||||||
@@ -111,6 +111,9 @@ def test_the_manual_source_is_admitted_by_grant_without_a_toggle(tmp_path) -> No
|
|||||||
app = SimpleNamespace(
|
app = SimpleNamespace(
|
||||||
state=SimpleNamespace(
|
state=SimpleNamespace(
|
||||||
config_store=config,
|
config_store=config,
|
||||||
|
model_profile_store=SimpleNamespace(
|
||||||
|
active_identity=lambda: ("profile", 1)
|
||||||
|
),
|
||||||
source_grant_store=SimpleNamespace(
|
source_grant_store=SimpleNamespace(
|
||||||
receipt=lambda **_kwargs: SimpleNamespace(revoked_at=None)
|
receipt=lambda **_kwargs: SimpleNamespace(revoked_at=None)
|
||||||
),
|
),
|
||||||
@@ -125,6 +128,21 @@ def test_the_manual_source_is_admitted_by_grant_without_a_toggle(tmp_path) -> No
|
|||||||
assert _sedimentation_claim_enabled(app, "one-memory-manual") is False
|
assert _sedimentation_claim_enabled(app, "one-memory-manual") is False
|
||||||
|
|
||||||
|
|
||||||
|
def test_claim_gate_waits_without_polling_when_no_model_is_active(tmp_path) -> None:
|
||||||
|
config = ConfigStore(tmp_path)
|
||||||
|
app = SimpleNamespace(
|
||||||
|
state=SimpleNamespace(
|
||||||
|
config_store=config,
|
||||||
|
model_profile_store=SimpleNamespace(active_identity=lambda: None),
|
||||||
|
source_grant_store=SimpleNamespace(
|
||||||
|
receipt=lambda **_kwargs: SimpleNamespace(revoked_at=None)
|
||||||
|
),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
assert _sedimentation_claim_enabled(app, "one-memory-manual") is False
|
||||||
|
|
||||||
|
|
||||||
def test_first_import_gate_waits_for_every_selected_history_job() -> None:
|
def test_first_import_gate_waits_for_every_selected_history_job() -> None:
|
||||||
state = SimpleNamespace(
|
state = SimpleNamespace(
|
||||||
status="importing",
|
status="importing",
|
||||||
|
|||||||
@@ -174,20 +174,20 @@ it("continues to step3 while sample GET is pending without calling completion AP
|
|||||||
expect(screen.queryByText(/^已生成.*记忆/)).toBeNull();
|
expect(screen.queryByText(/^已生成.*记忆/)).toBeNull();
|
||||||
});
|
});
|
||||||
it.each([
|
it.each([
|
||||||
"pending",
|
["pending", "已扫描 100 条,正在筛选"],
|
||||||
"paused",
|
["paused", "已扫描 100 条,尚未保存"],
|
||||||
"interrupted",
|
["interrupted", "已扫描 100 条,尚未保存"],
|
||||||
"failed",
|
["failed", "已扫描 100 条,尚未保存"],
|
||||||
"cancelled",
|
["cancelled", "已扫描 100 条,尚未保存"],
|
||||||
"completed",
|
["completed", "已扫描 100 条,本次没有可保存的对话"],
|
||||||
] as const)(
|
] as const)(
|
||||||
"shows %s without mutating jobs or fetching missing content",
|
"shows %s without mutating jobs or fetching missing content",
|
||||||
(status) => {
|
(status, expected) => {
|
||||||
setup({
|
setup({
|
||||||
jobs: [{ ...job, status, stored: 0 }],
|
jobs: [{ ...job, status, stored: 0 }],
|
||||||
sources: [{ ...source, imported_messages: 0 }],
|
sources: [{ ...source, imported_messages: 0 }],
|
||||||
});
|
});
|
||||||
expect(screen.getByText("已扫描 100 条,正在保存")).toBeTruthy();
|
expect(screen.getByText(expected)).toBeTruthy();
|
||||||
expect(api.receipt).not.toHaveBeenCalled();
|
expect(api.receipt).not.toHaveBeenCalled();
|
||||||
},
|
},
|
||||||
);
|
);
|
||||||
|
|||||||
@@ -38,6 +38,71 @@ it("uses the selected job's message count, never the source total or fake percen
|
|||||||
expect(container.textContent).not.toContain("%");
|
expect(container.textContent).not.toContain("%");
|
||||||
expect(container.textContent).not.toContain("持续导入中");
|
expect(container.textContent).not.toContain("持续导入中");
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it("does not describe scanned-but-skipped history as saved", () => {
|
||||||
|
const { container } = render(
|
||||||
|
<R6ImportProgress
|
||||||
|
sourceIds={["cursor"]}
|
||||||
|
sources={[source]}
|
||||||
|
jobs={[
|
||||||
|
{
|
||||||
|
...job,
|
||||||
|
status: "completed",
|
||||||
|
discovered: 12_375,
|
||||||
|
stored: 0,
|
||||||
|
skipped: 12_375,
|
||||||
|
history_after: "2026-10-15T00:00:00Z",
|
||||||
|
},
|
||||||
|
]}
|
||||||
|
/>,
|
||||||
|
);
|
||||||
|
expect(
|
||||||
|
screen.getByText("已扫描 12,375 条,本次没有符合时间范围的对话"),
|
||||||
|
).toBeTruthy();
|
||||||
|
expect(screen.getByText("导入完成")).toBeTruthy();
|
||||||
|
expect(container.textContent).not.toContain("正在保存");
|
||||||
|
expect(container.textContent).not.toContain("已保存 12,375");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("calls an active bounded scan filtering rather than saving", () => {
|
||||||
|
render(
|
||||||
|
<R6ImportProgress
|
||||||
|
sourceIds={["cursor"]}
|
||||||
|
sources={[source]}
|
||||||
|
jobs={[
|
||||||
|
{
|
||||||
|
...job,
|
||||||
|
discovered: 3_522,
|
||||||
|
stored: 0,
|
||||||
|
history_after: "2026-10-15T00:00:00Z",
|
||||||
|
},
|
||||||
|
]}
|
||||||
|
/>,
|
||||||
|
);
|
||||||
|
expect(screen.getByText("已扫描 3,522 条,正在筛选最近 3 天")).toBeTruthy();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("reports only stored rows as saved after a completed partial import", () => {
|
||||||
|
const { container } = render(
|
||||||
|
<R6ImportProgress
|
||||||
|
sourceIds={["cursor"]}
|
||||||
|
sources={[source]}
|
||||||
|
jobs={[
|
||||||
|
{
|
||||||
|
...job,
|
||||||
|
status: "completed",
|
||||||
|
discovered: 25_177,
|
||||||
|
stored: 20_826,
|
||||||
|
skipped: 4_351,
|
||||||
|
},
|
||||||
|
]}
|
||||||
|
/>,
|
||||||
|
);
|
||||||
|
expect(screen.getByText("已保存 20,826 条消息")).toBeTruthy();
|
||||||
|
expect(screen.getByText("导入完成")).toBeTruthy();
|
||||||
|
expect(container.textContent).not.toContain("25,177");
|
||||||
|
});
|
||||||
|
|
||||||
it.each(["paused", "interrupted", "failed", "completed", "cancelled"] as const)(
|
it.each(["paused", "interrupted", "failed", "completed", "cancelled"] as const)(
|
||||||
"does not animate %s as active work",
|
"does not animate %s as active work",
|
||||||
(status) => {
|
(status) => {
|
||||||
|
|||||||
@@ -3,6 +3,21 @@ import { useEffect, useLayoutEffect, useRef } from "react";
|
|||||||
import { AppMark } from "./app-mark";
|
import { AppMark } from "./app-mark";
|
||||||
import "./r6-import-progress.css";
|
import "./r6-import-progress.css";
|
||||||
|
|
||||||
|
function progressText(job: ImportJob): string | null {
|
||||||
|
if (job.stored > 0) return `已保存 ${job.stored.toLocaleString()} 条消息`;
|
||||||
|
if (job.discovered <= 0) return null;
|
||||||
|
const scanned = `已扫描 ${job.discovered.toLocaleString()} 条`;
|
||||||
|
if (job.status === "completed")
|
||||||
|
return job.history_after
|
||||||
|
? `${scanned},本次没有符合时间范围的对话`
|
||||||
|
: `${scanned},本次没有可保存的对话`;
|
||||||
|
if (job.status === "pending" || job.status === "running")
|
||||||
|
return job.history_after
|
||||||
|
? `${scanned},正在筛选最近 3 天`
|
||||||
|
: `${scanned},正在筛选`;
|
||||||
|
return `${scanned},尚未保存`;
|
||||||
|
}
|
||||||
|
|
||||||
export function R6ImportProgress({
|
export function R6ImportProgress({
|
||||||
sourceIds,
|
sourceIds,
|
||||||
sources,
|
sources,
|
||||||
@@ -86,8 +101,9 @@ export function R6ImportProgress({
|
|||||||
interrupted: "已中断",
|
interrupted: "已中断",
|
||||||
failed: "导入遇到问题",
|
failed: "导入遇到问题",
|
||||||
cancelled: "已取消",
|
cancelled: "已取消",
|
||||||
completed: "已保存",
|
completed: "导入完成",
|
||||||
};
|
};
|
||||||
|
const progress = job ? progressText(job) : null;
|
||||||
return (
|
return (
|
||||||
<article
|
<article
|
||||||
className="r6-import-progress-card"
|
className="r6-import-progress-card"
|
||||||
@@ -102,10 +118,8 @@ export function R6ImportProgress({
|
|||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
<h2>{source?.display_name ?? id}</h2>
|
<h2>{source?.display_name ?? id}</h2>
|
||||||
{job && job.stored > 0 ? (
|
{progress ? (
|
||||||
<p>{`已保存 ${job.stored.toLocaleString()} 条消息`}</p>
|
<p>{progress}</p>
|
||||||
) : job && job.discovered > 0 ? (
|
|
||||||
<p>{`已扫描 ${job.discovered.toLocaleString()} 条,正在保存`}</p>
|
|
||||||
) : noImportedHistory ? (
|
) : noImportedHistory ? (
|
||||||
<p>没有可保存的历史对话</p>
|
<p>没有可保存的历史对话</p>
|
||||||
) : null}
|
) : null}
|
||||||
|
|||||||
@@ -312,6 +312,48 @@ describe("remote Runtime acquisition", () => {
|
|||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it("allows the exact GitHub Release asset redirect hosts by default", async () => {
|
||||||
|
const value = await fixture();
|
||||||
|
const manifestUrl =
|
||||||
|
"https://github.com/example/project/releases/download/v1.2.3/release-manifest.json";
|
||||||
|
const artifactUrl =
|
||||||
|
"https://github.com/example/project/releases/download/v1.2.3/runtime.tar.gz";
|
||||||
|
const manifestAssetUrl =
|
||||||
|
"https://release-assets.githubusercontent.com/manifest-download";
|
||||||
|
const runtimeAssetUrl =
|
||||||
|
"https://release-assets.githubusercontent.com/runtime-download";
|
||||||
|
const release = {
|
||||||
|
...value.release,
|
||||||
|
artifacts: [{ ...value.artifact, url: artifactUrl }],
|
||||||
|
};
|
||||||
|
const temporaryParent = await mkdtemp(
|
||||||
|
path.join(os.tmpdir(), "memhub-github-release-parent-"),
|
||||||
|
);
|
||||||
|
roots.push(temporaryParent);
|
||||||
|
|
||||||
|
const acquired = await acquireRuntimeDistribution({
|
||||||
|
manifestUrl,
|
||||||
|
expectedVersion: "1.2.3",
|
||||||
|
channel: "stable",
|
||||||
|
platform: "linux",
|
||||||
|
arch: "x64",
|
||||||
|
temporaryParent,
|
||||||
|
httpGet: httpFixture({
|
||||||
|
[manifestUrl]: response(new Uint8Array(), 302, {
|
||||||
|
location: manifestAssetUrl,
|
||||||
|
}),
|
||||||
|
[manifestAssetUrl]: response(Buffer.from(JSON.stringify(release))),
|
||||||
|
[artifactUrl]: response(new Uint8Array(), 302, {
|
||||||
|
location: runtimeAssetUrl,
|
||||||
|
}),
|
||||||
|
[runtimeAssetUrl]: response(value.archiveBytes),
|
||||||
|
}),
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(acquired.release.version).toBe("1.2.3");
|
||||||
|
await acquired.cleanup();
|
||||||
|
});
|
||||||
|
|
||||||
it("rejects an archive whose internal commit drifts from the Release", async () => {
|
it("rejects an archive whose internal commit drifts from the Release", async () => {
|
||||||
const value = await fixture();
|
const value = await fixture();
|
||||||
const release = { ...value.release, git_commit: "b".repeat(40) };
|
const release = { ...value.release, git_commit: "b".repeat(40) };
|
||||||
@@ -368,7 +410,6 @@ describe("remote Runtime acquisition", () => {
|
|||||||
);
|
);
|
||||||
|
|
||||||
it("rejects a redirect to an untrusted host", async () => {
|
it("rejects a redirect to an untrusted host", async () => {
|
||||||
const value = await fixture();
|
|
||||||
await expect(
|
await expect(
|
||||||
acquireRuntimeDistribution({
|
acquireRuntimeDistribution({
|
||||||
manifestUrl: "https://downloads.example.test/release-manifest.json",
|
manifestUrl: "https://downloads.example.test/release-manifest.json",
|
||||||
|
|||||||
@@ -38,6 +38,10 @@ const SHA256 = /^[a-f0-9]{64}$/;
|
|||||||
const COMMIT = /^[0-9a-f]{40}$/;
|
const COMMIT = /^[0-9a-f]{40}$/;
|
||||||
const SAFE_FILE_NAME = /^[0-9A-Za-z][0-9A-Za-z._-]{0,255}$/;
|
const SAFE_FILE_NAME = /^[0-9A-Za-z][0-9A-Za-z._-]{0,255}$/;
|
||||||
const REDIRECTS = new Set([301, 302, 303, 307, 308]);
|
const REDIRECTS = new Set([301, 302, 303, 307, 308]);
|
||||||
|
const GITHUB_RELEASE_REDIRECT_HOSTS = [
|
||||||
|
"release-assets.githubusercontent.com",
|
||||||
|
"objects.githubusercontent.com",
|
||||||
|
] as const;
|
||||||
|
|
||||||
export type ReleaseChannel = "stable" | "prerelease";
|
export type ReleaseChannel = "stable" | "prerelease";
|
||||||
export type RuntimeReleasePlatform = "darwin" | "linux";
|
export type RuntimeReleasePlatform = "darwin" | "linux";
|
||||||
@@ -330,8 +334,12 @@ export async function resolveRuntimeRelease(options: {
|
|||||||
maxRedirects?: number;
|
maxRedirects?: number;
|
||||||
}): Promise<ResolvedRuntimeRelease> {
|
}): Promise<ResolvedRuntimeRelease> {
|
||||||
const manifestUrl = trustedUrl(options.manifestUrl);
|
const manifestUrl = trustedUrl(options.manifestUrl);
|
||||||
|
const defaultHosts =
|
||||||
|
normalizedHost(manifestUrl.hostname) === "github.com"
|
||||||
|
? [manifestUrl.hostname, ...GITHUB_RELEASE_REDIRECT_HOSTS]
|
||||||
|
: [manifestUrl.hostname];
|
||||||
const allowedHosts = new Set(
|
const allowedHosts = new Set(
|
||||||
(options.allowedHosts ?? [manifestUrl.hostname]).map(normalizedHost),
|
(options.allowedHosts ?? defaultHosts).map(normalizedHost),
|
||||||
);
|
);
|
||||||
if (!allowedHosts.has(normalizedHost(manifestUrl.hostname))) {
|
if (!allowedHosts.has(normalizedHost(manifestUrl.hostname))) {
|
||||||
throw new Error("release_url_untrusted");
|
throw new Error("release_url_untrusted");
|
||||||
|
|||||||
@@ -8,6 +8,7 @@ import {
|
|||||||
mkdir,
|
mkdir,
|
||||||
readdir,
|
readdir,
|
||||||
readFile,
|
readFile,
|
||||||
|
readlink,
|
||||||
realpath,
|
realpath,
|
||||||
rm,
|
rm,
|
||||||
stat,
|
stat,
|
||||||
@@ -320,6 +321,12 @@ async function distributionBytes(source) {
|
|||||||
|
|
||||||
async function sha256File(file) {
|
async function sha256File(file) {
|
||||||
const digest = createHash("sha256");
|
const digest = createHash("sha256");
|
||||||
|
const info = await lstat(file);
|
||||||
|
if (info.isSymbolicLink()) {
|
||||||
|
digest.update("symlink\0");
|
||||||
|
digest.update(await readlink(file));
|
||||||
|
return digest.digest("hex");
|
||||||
|
}
|
||||||
for await (const chunk of createReadStream(file)) digest.update(chunk);
|
for await (const chunk of createReadStream(file)) digest.update(chunk);
|
||||||
return digest.digest("hex");
|
return digest.digest("hex");
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -6,6 +6,7 @@ import {
|
|||||||
readdir,
|
readdir,
|
||||||
readFile,
|
readFile,
|
||||||
rm,
|
rm,
|
||||||
|
symlink,
|
||||||
writeFile,
|
writeFile,
|
||||||
} from "node:fs/promises";
|
} from "node:fs/promises";
|
||||||
import os from "node:os";
|
import os from "node:os";
|
||||||
@@ -121,6 +122,35 @@ test("builds versioned Runtime archives, checksums and a downloader-compatible m
|
|||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
|
test("verifies Runtime symlinks with the same identity rule as the installer", async (t) => {
|
||||||
|
const root = await mkdtemp(path.join(os.tmpdir(), "memhub-release-assets-"));
|
||||||
|
t.after(() => rm(root, { recursive: true, force: true }));
|
||||||
|
const source = await distribution(root, "linux", "x64");
|
||||||
|
const link = "memhub";
|
||||||
|
await symlink(link, path.join(source, "bin", "node"));
|
||||||
|
const manifestPath = path.join(source, "manifest.json");
|
||||||
|
const manifest = JSON.parse(await readFile(manifestPath, "utf8"));
|
||||||
|
manifest.files["bin/node"] = sha256(`symlink\0${link}`);
|
||||||
|
await writeFile(manifestPath, `${JSON.stringify(manifest, null, 2)}\n`);
|
||||||
|
|
||||||
|
const release = await buildReleaseAssets({
|
||||||
|
output: path.join(root, "release"),
|
||||||
|
version: "0.1.0-internal.1",
|
||||||
|
channel: "prerelease",
|
||||||
|
assetBaseUrl: "https://git.example.test/release/downloads/",
|
||||||
|
distributions: [source],
|
||||||
|
});
|
||||||
|
|
||||||
|
assert.ok(
|
||||||
|
release.artifacts.some(
|
||||||
|
(artifact) =>
|
||||||
|
artifact.kind === "runtime" &&
|
||||||
|
artifact.platform === "linux" &&
|
||||||
|
artifact.arch === "x64",
|
||||||
|
),
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
test("refuses version, build and duplicate target drift", async (t) => {
|
test("refuses version, build and duplicate target drift", async (t) => {
|
||||||
const root = await mkdtemp(path.join(os.tmpdir(), "memhub-release-assets-"));
|
const root = await mkdtemp(path.join(os.tmpdir(), "memhub-release-assets-"));
|
||||||
t.after(() => rm(root, { recursive: true, force: true }));
|
t.after(() => rm(root, { recursive: true, force: true }));
|
||||||
|
|||||||
@@ -23,7 +23,8 @@ export async function publishCliAfterRelease(options, dependencies = {}) {
|
|||||||
const statImpl = dependencies.statImpl ?? stat;
|
const statImpl = dependencies.statImpl ?? stat;
|
||||||
const buildEnvironment = {
|
const buildEnvironment = {
|
||||||
...environment,
|
...environment,
|
||||||
ONE_MEMORY_VERSION: options.version,
|
MEMHUB_NPM_VERSION: options.version,
|
||||||
|
MEMHUB_DEFAULT_RUNTIME_VERSION: options.runtimeVersion,
|
||||||
ONE_MEMORY_BUILD_NUMBER: String(options.buildNumber),
|
ONE_MEMORY_BUILD_NUMBER: String(options.buildNumber),
|
||||||
ONE_MEMORY_COMMIT: options.gitCommit,
|
ONE_MEMORY_COMMIT: options.gitCommit,
|
||||||
};
|
};
|
||||||
@@ -31,7 +32,7 @@ export async function publishCliAfterRelease(options, dependencies = {}) {
|
|||||||
const versionReceipt = await verifyVersion(
|
const versionReceipt = await verifyVersion(
|
||||||
[
|
[
|
||||||
"--version",
|
"--version",
|
||||||
options.version,
|
options.runtimeVersion,
|
||||||
"--channel",
|
"--channel",
|
||||||
options.channel,
|
options.channel,
|
||||||
"--tag",
|
"--tag",
|
||||||
@@ -43,7 +44,7 @@ export async function publishCliAfterRelease(options, dependencies = {}) {
|
|||||||
);
|
);
|
||||||
const releaseReceipt = await verifyRelease({
|
const releaseReceipt = await verifyRelease({
|
||||||
manifestUrl: options.manifestUrl,
|
manifestUrl: options.manifestUrl,
|
||||||
version: options.version,
|
version: options.runtimeVersion,
|
||||||
gitCommit: options.gitCommit,
|
gitCommit: options.gitCommit,
|
||||||
channel: options.channel,
|
channel: options.channel,
|
||||||
platform: options.platform,
|
platform: options.platform,
|
||||||
@@ -106,7 +107,8 @@ export async function publishCliAfterRelease(options, dependencies = {}) {
|
|||||||
return {
|
return {
|
||||||
schema_version: 1,
|
schema_version: 1,
|
||||||
product: "memhub-cli-publication",
|
product: "memhub-cli-publication",
|
||||||
version: versionReceipt.version,
|
version: options.version,
|
||||||
|
runtime_version: releaseReceipt.version,
|
||||||
build_number: versionReceipt.build_number,
|
build_number: versionReceipt.build_number,
|
||||||
tag: versionReceipt.tag,
|
tag: versionReceipt.tag,
|
||||||
release: releaseReceipt,
|
release: releaseReceipt,
|
||||||
@@ -196,6 +198,11 @@ function parseArgs(arguments_) {
|
|||||||
options.manifestUrl = value;
|
options.manifestUrl = value;
|
||||||
else if (flag === "--version" && options.version === undefined)
|
else if (flag === "--version" && options.version === undefined)
|
||||||
options.version = value;
|
options.version = value;
|
||||||
|
else if (
|
||||||
|
flag === "--runtime-version" &&
|
||||||
|
options.runtimeVersion === undefined
|
||||||
|
)
|
||||||
|
options.runtimeVersion = value;
|
||||||
else if (flag === "--git-commit" && options.gitCommit === undefined)
|
else if (flag === "--git-commit" && options.gitCommit === undefined)
|
||||||
options.gitCommit = value;
|
options.gitCommit = value;
|
||||||
else if (flag === "--channel" && options.channel === undefined)
|
else if (flag === "--channel" && options.channel === undefined)
|
||||||
@@ -219,6 +226,7 @@ function parseArgs(arguments_) {
|
|||||||
if (
|
if (
|
||||||
!options.manifestUrl ||
|
!options.manifestUrl ||
|
||||||
!options.version ||
|
!options.version ||
|
||||||
|
!options.runtimeVersion ||
|
||||||
!options.gitCommit ||
|
!options.gitCommit ||
|
||||||
!options.channel ||
|
!options.channel ||
|
||||||
!options.tag ||
|
!options.tag ||
|
||||||
@@ -230,7 +238,7 @@ function parseArgs(arguments_) {
|
|||||||
!options.npmTag
|
!options.npmTag
|
||||||
) {
|
) {
|
||||||
throw new Error(
|
throw new Error(
|
||||||
"usage: publish-cli-after-release.mjs --manifest-url <https-url> --version <SemVer> --git-commit <sha> --channel <stable|prerelease> --tag <vSemVer> --build-number <integer> --platform <darwin|linux> --arch <x64|arm64> --registry <https-url> --npm-tag <tag> [--access <public|restricted>] [--provenance] [--allowed-host <hostname>] [--dry-run]",
|
"usage: publish-cli-after-release.mjs --manifest-url <https-url> --version <CLI SemVer> --runtime-version <Runtime SemVer> --git-commit <sha> --channel <stable|prerelease> --tag <vSemVer> --build-number <integer> --platform <darwin|linux> --arch <x64|arm64> --registry <https-url> --npm-tag <tag> [--access <public|restricted>] [--provenance] [--allowed-host <hostname>] [--dry-run]",
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
return options;
|
return options;
|
||||||
|
|||||||
@@ -6,6 +6,9 @@ import test from "node:test";
|
|||||||
|
|
||||||
import { publishCliAfterRelease } from "./publish-cli-after-release.mjs";
|
import { publishCliAfterRelease } from "./publish-cli-after-release.mjs";
|
||||||
|
|
||||||
|
const CLI_VERSION = "0.1.1";
|
||||||
|
const RUNTIME_VERSION = "0.1.0-internal.1";
|
||||||
|
|
||||||
async function fixture(t) {
|
async function fixture(t) {
|
||||||
const root = await mkdtemp(path.join(os.tmpdir(), "memhub-publish-gate-"));
|
const root = await mkdtemp(path.join(os.tmpdir(), "memhub-publish-gate-"));
|
||||||
t.after(() => rm(root, { recursive: true, force: true }));
|
t.after(() => rm(root, { recursive: true, force: true }));
|
||||||
@@ -15,10 +18,10 @@ async function fixture(t) {
|
|||||||
await mkdir(output, { recursive: true });
|
await mkdir(output, { recursive: true });
|
||||||
await writeFile(
|
await writeFile(
|
||||||
path.join(publish, "package.json"),
|
path.join(publish, "package.json"),
|
||||||
`${JSON.stringify({ name: "@example/memhub-cli", version: "0.1.0-internal.1", private: false })}\n`,
|
`${JSON.stringify({ name: "@example/memhub-cli", version: CLI_VERSION, private: false })}\n`,
|
||||||
);
|
);
|
||||||
await writeFile(
|
await writeFile(
|
||||||
path.join(output, "example-memhub-cli-0.1.0-internal.1.tgz"),
|
path.join(output, `example-memhub-cli-${CLI_VERSION}.tgz`),
|
||||||
"tarball",
|
"tarball",
|
||||||
);
|
);
|
||||||
return root;
|
return root;
|
||||||
@@ -26,10 +29,11 @@ async function fixture(t) {
|
|||||||
|
|
||||||
const options = {
|
const options = {
|
||||||
manifestUrl: "https://releases.example.test/release-manifest.json",
|
manifestUrl: "https://releases.example.test/release-manifest.json",
|
||||||
version: "0.1.0-internal.1",
|
version: CLI_VERSION,
|
||||||
|
runtimeVersion: RUNTIME_VERSION,
|
||||||
gitCommit: "a".repeat(40),
|
gitCommit: "a".repeat(40),
|
||||||
channel: "prerelease",
|
channel: "prerelease",
|
||||||
tag: "v0.1.0-internal.1",
|
tag: `v${RUNTIME_VERSION}`,
|
||||||
buildNumber: 1,
|
buildNumber: 1,
|
||||||
platform: "linux",
|
platform: "linux",
|
||||||
arch: "x64",
|
arch: "x64",
|
||||||
@@ -49,15 +53,24 @@ test("verifies the Runtime before building and publishing the CLI", async (t) =>
|
|||||||
{
|
{
|
||||||
verifyVersion: async () => {
|
verifyVersion: async () => {
|
||||||
events.push("version");
|
events.push("version");
|
||||||
return { version: options.version, build_number: 1, tag: options.tag };
|
return {
|
||||||
|
version: options.runtimeVersion,
|
||||||
|
build_number: 1,
|
||||||
|
tag: options.tag,
|
||||||
|
};
|
||||||
},
|
},
|
||||||
verifyRelease: async (verifyOptions) => {
|
verifyRelease: async (verifyOptions) => {
|
||||||
events.push("release");
|
events.push("release");
|
||||||
assert.equal(verifyOptions.gitCommit, options.gitCommit);
|
assert.equal(verifyOptions.gitCommit, options.gitCommit);
|
||||||
return { version: options.version, sha256: "a".repeat(64) };
|
assert.equal(verifyOptions.version, options.runtimeVersion);
|
||||||
|
return { version: options.runtimeVersion, sha256: "a".repeat(64) };
|
||||||
},
|
},
|
||||||
runCommand: (command, arguments_, commandOptions) => {
|
runCommand: (command, arguments_, commandOptions) => {
|
||||||
assert.equal(commandOptions.env.ONE_MEMORY_VERSION, options.version);
|
assert.equal(commandOptions.env.MEMHUB_NPM_VERSION, options.version);
|
||||||
|
assert.equal(
|
||||||
|
commandOptions.env.MEMHUB_DEFAULT_RUNTIME_VERSION,
|
||||||
|
options.runtimeVersion,
|
||||||
|
);
|
||||||
assert.equal(
|
assert.equal(
|
||||||
commandOptions.env.ONE_MEMORY_BUILD_NUMBER,
|
commandOptions.env.ONE_MEMORY_BUILD_NUMBER,
|
||||||
String(options.buildNumber),
|
String(options.buildNumber),
|
||||||
@@ -74,6 +87,8 @@ test("verifies the Runtime before building and publishing the CLI", async (t) =>
|
|||||||
assert.match(events[3], /--access public/);
|
assert.match(events[3], /--access public/);
|
||||||
assert.match(events[3], /--provenance/);
|
assert.match(events[3], /--provenance/);
|
||||||
assert.equal(receipt.npm_package, "@example/memhub-cli");
|
assert.equal(receipt.npm_package, "@example/memhub-cli");
|
||||||
|
assert.equal(receipt.version, CLI_VERSION);
|
||||||
|
assert.equal(receipt.runtime_version, RUNTIME_VERSION);
|
||||||
assert.equal(receipt.dry_run, true);
|
assert.equal(receipt.dry_run, true);
|
||||||
assert.equal(receipt.npm_access, "public");
|
assert.equal(receipt.npm_access, "public");
|
||||||
assert.equal(receipt.provenance, true);
|
assert.equal(receipt.provenance, true);
|
||||||
@@ -87,7 +102,7 @@ test("never builds or invokes npm when the Runtime release is unavailable", asyn
|
|||||||
{ ...options, root },
|
{ ...options, root },
|
||||||
{
|
{
|
||||||
verifyVersion: async () => ({
|
verifyVersion: async () => ({
|
||||||
version: options.version,
|
version: options.runtimeVersion,
|
||||||
build_number: 1,
|
build_number: 1,
|
||||||
tag: options.tag,
|
tag: options.tag,
|
||||||
}),
|
}),
|
||||||
@@ -115,11 +130,11 @@ test("rejects a package identity that differs from the configured publish name",
|
|||||||
},
|
},
|
||||||
{
|
{
|
||||||
verifyVersion: async () => ({
|
verifyVersion: async () => ({
|
||||||
version: options.version,
|
version: options.runtimeVersion,
|
||||||
build_number: 1,
|
build_number: 1,
|
||||||
tag: options.tag,
|
tag: options.tag,
|
||||||
}),
|
}),
|
||||||
verifyRelease: async () => ({ version: options.version }),
|
verifyRelease: async () => ({ version: options.runtimeVersion }),
|
||||||
runCommand: () => undefined,
|
runCommand: () => undefined,
|
||||||
},
|
},
|
||||||
),
|
),
|
||||||
|
|||||||
@@ -107,6 +107,7 @@ class TestSupersede:
|
|||||||
async def _seed_old(self, store, *, derived=None):
|
async def _seed_old(self, store, *, derived=None):
|
||||||
old = WorkingFact(
|
old = WorkingFact(
|
||||||
fact_id="f_old", user_id="u1",
|
fact_id="f_old", user_id="u1",
|
||||||
|
session_id="test-session",
|
||||||
statement=(
|
statement=(
|
||||||
"用户要修复 CI 的 npm run build;检查 .github/workflows/ci.yml 后"
|
"用户要修复 CI 的 npm run build;检查 .github/workflows/ci.yml 后"
|
||||||
"定位到 Node 18 报 ERR_OSSL_EVP_UNSUPPORTED,尚未修改。"
|
"定位到 Node 18 报 ERR_OSSL_EVP_UNSUPPORTED,尚未修改。"
|
||||||
@@ -129,7 +130,7 @@ class TestSupersede:
|
|||||||
[_draft(
|
[_draft(
|
||||||
"用户继续修复同一 CI 构建;把 .github/workflows/ci.yml 升到 Node 20,"
|
"用户继续修复同一 CI 构建;把 .github/workflows/ci.yml 升到 Node 20,"
|
||||||
"npm run build 已通过。"
|
"npm run build 已通过。"
|
||||||
)], user_id="u1",
|
)], user_id="u1", session_id="test-session",
|
||||||
)
|
)
|
||||||
|
|
||||||
new_id = facts[0].fact_id
|
new_id = facts[0].fact_id
|
||||||
@@ -152,7 +153,7 @@ class TestSupersede:
|
|||||||
[_draft(
|
[_draft(
|
||||||
"用户继续修复同一 CI 构建;把 .github/workflows/ci.yml 升到 Node 20,"
|
"用户继续修复同一 CI 构建;把 .github/workflows/ci.yml 升到 Node 20,"
|
||||||
"npm run build 已通过。"
|
"npm run build 已通过。"
|
||||||
)], user_id="u1",
|
)], user_id="u1", session_id="test-session",
|
||||||
)
|
)
|
||||||
new_id = facts[0].fact_id
|
new_id = facts[0].fact_id
|
||||||
|
|
||||||
@@ -184,7 +185,9 @@ class TestSupersede:
|
|||||||
coding_store, mock_embed,
|
coding_store, mock_embed,
|
||||||
_llm([{"draft_idx": 0, "supersedes": [target or prev_id]}]),
|
_llm([{"draft_idx": 0, "supersedes": [target or prev_id]}]),
|
||||||
)
|
)
|
||||||
facts, _ = await rec.reconcile([_draft(stmt)], user_id="u1")
|
facts, _ = await rec.reconcile(
|
||||||
|
[_draft(stmt)], user_id="u1", session_id="test-session"
|
||||||
|
)
|
||||||
prev_id = facts[0].fact_id
|
prev_id = facts[0].fact_id
|
||||||
|
|
||||||
lineage = await coding_store.get_fact_ancestors(prev_id, user_id="u1")
|
lineage = await coding_store.get_fact_ancestors(prev_id, user_id="u1")
|
||||||
@@ -208,6 +211,7 @@ class TestSupersede:
|
|||||||
"配置完成但尚未发布。"
|
"配置完成但尚未发布。"
|
||||||
)],
|
)],
|
||||||
user_id="u1",
|
user_id="u1",
|
||||||
|
session_id="test-session",
|
||||||
)
|
)
|
||||||
|
|
||||||
prompt = llm.calls[0]["prompt"]
|
prompt = llm.calls[0]["prompt"]
|
||||||
@@ -234,7 +238,9 @@ class TestSupersede:
|
|||||||
coding_store, mock_embed, _llm([{"draft_idx": 0, "supersedes": ["f_old"]}]),
|
coding_store, mock_embed, _llm([{"draft_idx": 0, "supersedes": ["f_old"]}]),
|
||||||
)
|
)
|
||||||
await rec.reconcile(
|
await rec.reconcile(
|
||||||
[_draft("config 位于 /etc/app/config.yaml")], user_id="u1",
|
[_draft("config 位于 /etc/app/config.yaml")],
|
||||||
|
user_id="u1",
|
||||||
|
session_id="test-session",
|
||||||
)
|
)
|
||||||
|
|
||||||
# 拿旧说法原文去搜,只能落到新事实(靠两者本身相似),
|
# 拿旧说法原文去搜,只能落到新事实(靠两者本身相似),
|
||||||
@@ -256,6 +262,34 @@ class TestSupersede:
|
|||||||
old = await coding_store.get_fact("f_old", user_id="u1")
|
old = await coding_store.get_fact("f_old", user_id="u1")
|
||||||
assert old.status == "active"
|
assert old.status == "active"
|
||||||
|
|
||||||
|
async def test_near_duplicate_fact_cannot_cross_authority_scope(
|
||||||
|
self, coding_store, mock_embed
|
||||||
|
):
|
||||||
|
await coding_store.insert_fact(WorkingFact(
|
||||||
|
fact_id="f_old",
|
||||||
|
user_id="u1",
|
||||||
|
agent_id="scope:2026-10-18",
|
||||||
|
session_id="conversation-a",
|
||||||
|
workspace_id="repo-a",
|
||||||
|
statement="CI build now passes after upgrading Node",
|
||||||
|
))
|
||||||
|
llm = _llm([{"draft_idx": 0, "supersedes": ["f_old"]}])
|
||||||
|
rec = WorkingFactReconciler(coding_store, mock_embed, llm)
|
||||||
|
|
||||||
|
_, ops = await rec.reconcile(
|
||||||
|
[_draft("CI build now passes after upgrading Node")],
|
||||||
|
user_id="u1",
|
||||||
|
agent_id="scope:2026-10-19",
|
||||||
|
session_id="conversation-b",
|
||||||
|
workspace_id="repo-a",
|
||||||
|
)
|
||||||
|
|
||||||
|
old = await coding_store.get_fact("f_old", user_id="u1")
|
||||||
|
assert old is not None
|
||||||
|
assert old.status == "active"
|
||||||
|
assert [op.action for op in ops] == ["INSERT"]
|
||||||
|
assert len(llm.calls) == 0
|
||||||
|
|
||||||
async def test_old_fact_superseded_only_once(self, coding_store, mock_embed):
|
async def test_old_fact_superseded_only_once(self, coding_store, mock_embed):
|
||||||
"""两条新事实都声称覆盖同一条旧事实时,只让第一条生效。
|
"""两条新事实都声称覆盖同一条旧事实时,只让第一条生效。
|
||||||
|
|
||||||
@@ -274,6 +308,7 @@ class TestSupersede:
|
|||||||
_draft("config 位于 /srv/app/config.yaml"),
|
_draft("config 位于 /srv/app/config.yaml"),
|
||||||
],
|
],
|
||||||
user_id="u1",
|
user_id="u1",
|
||||||
|
session_id="test-session",
|
||||||
)
|
)
|
||||||
assert len(facts) == 2
|
assert len(facts) == 2
|
||||||
assert sum(1 for o in ops if o.action == "SUPERSEDE") == 1
|
assert sum(1 for o in ops if o.action == "SUPERSEDE") == 1
|
||||||
@@ -288,6 +323,7 @@ class TestChainSummary:
|
|||||||
async def _seed_old(self, store):
|
async def _seed_old(self, store):
|
||||||
await store.insert_fact(WorkingFact(
|
await store.insert_fact(WorkingFact(
|
||||||
fact_id="f_old", user_id="u1",
|
fact_id="f_old", user_id="u1",
|
||||||
|
session_id="test-session",
|
||||||
statement=(
|
statement=(
|
||||||
"用户要迁移 app config;确认当前读取 /opt/app/config.yaml,"
|
"用户要迁移 app config;确认当前读取 /opt/app/config.yaml,"
|
||||||
"尚未修改配置加载逻辑。"
|
"尚未修改配置加载逻辑。"
|
||||||
@@ -310,7 +346,7 @@ class TestChainSummary:
|
|||||||
[_draft(
|
[_draft(
|
||||||
"用户继续迁移 app config;将加载路径改为 /etc/app/config.yaml,"
|
"用户继续迁移 app config;将加载路径改为 /etc/app/config.yaml,"
|
||||||
"配置加载测试通过。"
|
"配置加载测试通过。"
|
||||||
)], user_id="u1",
|
)], user_id="u1", session_id="test-session",
|
||||||
)
|
)
|
||||||
|
|
||||||
stored = await coding_store.get_fact(facts[0].fact_id, user_id="u1")
|
stored = await coding_store.get_fact(facts[0].fact_id, user_id="u1")
|
||||||
@@ -326,7 +362,7 @@ class TestChainSummary:
|
|||||||
[_draft(
|
[_draft(
|
||||||
"用户继续迁移 app config;将加载路径改为 /etc/app/config.yaml,"
|
"用户继续迁移 app config;将加载路径改为 /etc/app/config.yaml,"
|
||||||
"配置加载测试通过。"
|
"配置加载测试通过。"
|
||||||
)], user_id="u1",
|
)], user_id="u1", session_id="test-session",
|
||||||
)
|
)
|
||||||
|
|
||||||
assert len(llm.calls) == 2, "SUPERSEDE 应触发第二次调用"
|
assert len(llm.calls) == 2, "SUPERSEDE 应触发第二次调用"
|
||||||
@@ -344,7 +380,11 @@ class TestChainSummary:
|
|||||||
llm = _llm([])
|
llm = _llm([])
|
||||||
rec = WorkingFactReconciler(coding_store, mock_embed, llm)
|
rec = WorkingFactReconciler(coding_store, mock_embed, llm)
|
||||||
|
|
||||||
await rec.reconcile([_draft("另一个话题的结论")], user_id="u1")
|
await rec.reconcile(
|
||||||
|
[_draft("另一个话题的结论")],
|
||||||
|
user_id="u1",
|
||||||
|
session_id="test-session",
|
||||||
|
)
|
||||||
assert len(llm.calls) == 1
|
assert len(llm.calls) == 1
|
||||||
|
|
||||||
async def test_prompt_carries_full_lineage(self, coding_store, mock_embed):
|
async def test_prompt_carries_full_lineage(self, coding_store, mock_embed):
|
||||||
@@ -357,7 +397,7 @@ class TestChainSummary:
|
|||||||
[_draft(
|
[_draft(
|
||||||
"用户继续迁移 app config;将加载路径改为 /etc/app/config.yaml,"
|
"用户继续迁移 app config;将加载路径改为 /etc/app/config.yaml,"
|
||||||
"配置加载测试通过。"
|
"配置加载测试通过。"
|
||||||
)], user_id="u1",
|
)], user_id="u1", session_id="test-session",
|
||||||
)
|
)
|
||||||
|
|
||||||
llm2 = _llm([{"draft_idx": 0, "supersedes": [facts[0].fact_id]}])
|
llm2 = _llm([{"draft_idx": 0, "supersedes": [facts[0].fact_id]}])
|
||||||
@@ -366,7 +406,7 @@ class TestChainSummary:
|
|||||||
[_draft(
|
[_draft(
|
||||||
"用户继续完善同一 config 迁移;改由环境变量 APP_CONFIG 指定路径,"
|
"用户继续完善同一 config 迁移;改由环境变量 APP_CONFIG 指定路径,"
|
||||||
"回归测试通过。"
|
"回归测试通过。"
|
||||||
)], user_id="u1",
|
)], user_id="u1", session_id="test-session",
|
||||||
)
|
)
|
||||||
|
|
||||||
prompt = llm2.calls[1]["prompt"]
|
prompt = llm2.calls[1]["prompt"]
|
||||||
@@ -393,7 +433,7 @@ class TestChainSummary:
|
|||||||
[_draft(
|
[_draft(
|
||||||
"用户继续迁移 app config;将加载路径改为 /etc/app/config.yaml,"
|
"用户继续迁移 app config;将加载路径改为 /etc/app/config.yaml,"
|
||||||
"配置加载测试通过。"
|
"配置加载测试通过。"
|
||||||
)], user_id="u1",
|
)], user_id="u1", session_id="test-session",
|
||||||
)
|
)
|
||||||
|
|
||||||
assert sum(1 for o in ops if o.action == "SUPERSEDE") == 1
|
assert sum(1 for o in ops if o.action == "SUPERSEDE") == 1
|
||||||
@@ -426,7 +466,7 @@ class TestChainSummary:
|
|||||||
[_draft(
|
[_draft(
|
||||||
"用户继续迁移 app config;将加载路径改为 /etc/app/config.yaml,"
|
"用户继续迁移 app config;将加载路径改为 /etc/app/config.yaml,"
|
||||||
"配置加载测试通过。"
|
"配置加载测试通过。"
|
||||||
)], user_id="u1",
|
)], user_id="u1", session_id="test-session",
|
||||||
)
|
)
|
||||||
assert facts[0].fact_chain_summary == ""
|
assert facts[0].fact_chain_summary == ""
|
||||||
|
|
||||||
@@ -436,7 +476,7 @@ class TestChainSummary:
|
|||||||
[_draft(
|
[_draft(
|
||||||
"用户继续完善同一 config 迁移;改由环境变量 APP_CONFIG 指定路径,"
|
"用户继续完善同一 config 迁移;改由环境变量 APP_CONFIG 指定路径,"
|
||||||
"回归测试通过。"
|
"回归测试通过。"
|
||||||
)], user_id="u1",
|
)], user_id="u1", session_id="test-session",
|
||||||
)
|
)
|
||||||
|
|
||||||
prompt = llm2.calls[1]["prompt"]
|
prompt = llm2.calls[1]["prompt"]
|
||||||
@@ -450,7 +490,8 @@ class TestChainSummary:
|
|||||||
total = MAX_FACT_CHAIN_ENTRIES + 3
|
total = MAX_FACT_CHAIN_ENTRIES + 3
|
||||||
for i in range(total):
|
for i in range(total):
|
||||||
await coding_store.insert_fact(WorkingFact(
|
await coding_store.insert_fact(WorkingFact(
|
||||||
fact_id=f"g{i}", user_id="u1", statement=f"第 {i} 代结论",
|
fact_id=f"g{i}", user_id="u1", session_id="test-session",
|
||||||
|
statement=f"第 {i} 代结论",
|
||||||
created_at=datetime(2026, 1, i + 1),
|
created_at=datetime(2026, 1, i + 1),
|
||||||
))
|
))
|
||||||
if prev:
|
if prev:
|
||||||
@@ -459,7 +500,9 @@ class TestChainSummary:
|
|||||||
|
|
||||||
llm = _llm([{"draft_idx": 0, "supersedes": [prev]}])
|
llm = _llm([{"draft_idx": 0, "supersedes": [prev]}])
|
||||||
rec = WorkingFactReconciler(coding_store, mock_embed, llm)
|
rec = WorkingFactReconciler(coding_store, mock_embed, llm)
|
||||||
await rec.reconcile([_draft("最新结论")], user_id="u1")
|
await rec.reconcile(
|
||||||
|
[_draft("最新结论")], user_id="u1", session_id="test-session"
|
||||||
|
)
|
||||||
|
|
||||||
prompt = llm.calls[1]["prompt"]
|
prompt = llm.calls[1]["prompt"]
|
||||||
assert "第 0 代结论" not in prompt, "最老的几代该被截掉"
|
assert "第 0 代结论" not in prompt, "最老的几代该被截掉"
|
||||||
@@ -478,7 +521,7 @@ class TestChainSummary:
|
|||||||
[_draft(
|
[_draft(
|
||||||
"用户继续迁移 app config;将加载路径改为 /etc/app/config.yaml,"
|
"用户继续迁移 app config;将加载路径改为 /etc/app/config.yaml,"
|
||||||
"配置加载测试通过。"
|
"配置加载测试通过。"
|
||||||
)], user_id="u1",
|
)], user_id="u1", session_id="test-session",
|
||||||
)
|
)
|
||||||
entries = llm.calls[1]["prompt"].split(
|
entries = llm.calls[1]["prompt"].split(
|
||||||
"ENTRIES\n═══════════════════════════════════════════════════════════════════\n",
|
"ENTRIES\n═══════════════════════════════════════════════════════════════════\n",
|
||||||
@@ -497,7 +540,7 @@ class TestChainSummary:
|
|||||||
[_draft(
|
[_draft(
|
||||||
"用户继续迁移 app config;将加载路径改为 /etc/app/config.yaml,"
|
"用户继续迁移 app config;将加载路径改为 /etc/app/config.yaml,"
|
||||||
"配置加载测试通过。"
|
"配置加载测试通过。"
|
||||||
)], user_id="u1",
|
)], user_id="u1", session_id="test-session",
|
||||||
)
|
)
|
||||||
|
|
||||||
rec2 = WorkingFactReconciler(
|
rec2 = WorkingFactReconciler(
|
||||||
@@ -509,7 +552,7 @@ class TestChainSummary:
|
|||||||
[_draft(
|
[_draft(
|
||||||
"用户继续完善同一 config 迁移;改由环境变量 APP_CONFIG 指定路径,"
|
"用户继续完善同一 config 迁移;改由环境变量 APP_CONFIG 指定路径,"
|
||||||
"回归测试通过。"
|
"回归测试通过。"
|
||||||
)], user_id="u1",
|
)], user_id="u1", session_id="test-session",
|
||||||
)
|
)
|
||||||
|
|
||||||
stored = await coding_store.get_fact(facts2[0].fact_id, user_id="u1")
|
stored = await coding_store.get_fact(facts2[0].fact_id, user_id="u1")
|
||||||
@@ -526,7 +569,7 @@ class TestChainSummary:
|
|||||||
[_draft(
|
[_draft(
|
||||||
"用户继续迁移 app config;将加载路径改为 /etc/app/config.yaml,"
|
"用户继续迁移 app config;将加载路径改为 /etc/app/config.yaml,"
|
||||||
"配置加载测试通过。"
|
"配置加载测试通过。"
|
||||||
)], user_id="u1",
|
)], user_id="u1", session_id="test-session",
|
||||||
)
|
)
|
||||||
|
|
||||||
assert len(llm.calls) == 1
|
assert len(llm.calls) == 1
|
||||||
@@ -564,7 +607,8 @@ class TestFailSafe:
|
|||||||
async def test_llm_failure_still_inserts(self, coding_store, mock_embed):
|
async def test_llm_failure_still_inserts(self, coding_store, mock_embed):
|
||||||
"""LLM 挂了宁可多留一条过期事实,也不能因为一次调用失败丢结论。"""
|
"""LLM 挂了宁可多留一条过期事实,也不能因为一次调用失败丢结论。"""
|
||||||
await coding_store.insert_fact(WorkingFact(
|
await coding_store.insert_fact(WorkingFact(
|
||||||
fact_id="f_old", user_id="u1", statement="config 位于 /opt/app/config.yaml",
|
fact_id="f_old", user_id="u1", session_id="test-session",
|
||||||
|
statement="config 位于 /opt/app/config.yaml",
|
||||||
))
|
))
|
||||||
rec = WorkingFactReconciler(coding_store, mock_embed, FailingLLMProvider())
|
rec = WorkingFactReconciler(coding_store, mock_embed, FailingLLMProvider())
|
||||||
|
|
||||||
@@ -578,7 +622,8 @@ class TestFailSafe:
|
|||||||
|
|
||||||
async def test_unparsable_llm_output(self, coding_store, mock_embed):
|
async def test_unparsable_llm_output(self, coding_store, mock_embed):
|
||||||
await coding_store.insert_fact(WorkingFact(
|
await coding_store.insert_fact(WorkingFact(
|
||||||
fact_id="f_old", user_id="u1", statement="config 位于 /opt/app/config.yaml",
|
fact_id="f_old", user_id="u1", session_id="test-session",
|
||||||
|
statement="config 位于 /opt/app/config.yaml",
|
||||||
))
|
))
|
||||||
rec = WorkingFactReconciler(
|
rec = WorkingFactReconciler(
|
||||||
coding_store, mock_embed, MockLLMProvider("不是 JSON 的一段话")
|
coding_store, mock_embed, MockLLMProvider("不是 JSON 的一段话")
|
||||||
@@ -592,7 +637,8 @@ class TestFailSafe:
|
|||||||
|
|
||||||
async def test_markdown_fence_parsed(self, coding_store, mock_embed):
|
async def test_markdown_fence_parsed(self, coding_store, mock_embed):
|
||||||
await coding_store.insert_fact(WorkingFact(
|
await coding_store.insert_fact(WorkingFact(
|
||||||
fact_id="f_old", user_id="u1", statement="config 位于 /opt/app/config.yaml",
|
fact_id="f_old", user_id="u1", session_id="test-session",
|
||||||
|
statement="config 位于 /opt/app/config.yaml",
|
||||||
))
|
))
|
||||||
llm = MockLLMProvider([
|
llm = MockLLMProvider([
|
||||||
"```json\n" + _ops_resp([{"draft_idx": 0, "supersedes": ["f_old"]}]) + "\n```",
|
"```json\n" + _ops_resp([{"draft_idx": 0, "supersedes": ["f_old"]}]) + "\n```",
|
||||||
@@ -601,7 +647,9 @@ class TestFailSafe:
|
|||||||
rec = WorkingFactReconciler(coding_store, mock_embed, llm)
|
rec = WorkingFactReconciler(coding_store, mock_embed, llm)
|
||||||
|
|
||||||
_, ops = await rec.reconcile(
|
_, ops = await rec.reconcile(
|
||||||
[_draft("config 位于 /etc/app/config.yaml")], user_id="u1",
|
[_draft("config 位于 /etc/app/config.yaml")],
|
||||||
|
user_id="u1",
|
||||||
|
session_id="test-session",
|
||||||
)
|
)
|
||||||
assert sum(1 for o in ops if o.action == "SUPERSEDE") == 1
|
assert sum(1 for o in ops if o.action == "SUPERSEDE") == 1
|
||||||
|
|
||||||
@@ -618,7 +666,8 @@ class TestSubjectGateSwitch:
|
|||||||
|
|
||||||
async def _seed(self, store, fact_id, subject, statement):
|
async def _seed(self, store, fact_id, subject, statement):
|
||||||
await store.insert_fact(WorkingFact(
|
await store.insert_fact(WorkingFact(
|
||||||
fact_id=fact_id, user_id="u1", subject=subject, statement=statement,
|
fact_id=fact_id, user_id="u1", session_id="test-session",
|
||||||
|
subject=subject, statement=statement,
|
||||||
))
|
))
|
||||||
|
|
||||||
async def test_gate_on_blocks_cross_subject_supersede(self, coding_store, mock_embed):
|
async def test_gate_on_blocks_cross_subject_supersede(self, coding_store, mock_embed):
|
||||||
@@ -632,6 +681,7 @@ class TestSubjectGateSwitch:
|
|||||||
_, ops = await rec.reconcile(
|
_, ops = await rec.reconcile(
|
||||||
[WorkingFactDraft(subject="Bart", statement="Bart 的三日行程已排定,预算待定。")],
|
[WorkingFactDraft(subject="Bart", statement="Bart 的三日行程已排定,预算待定。")],
|
||||||
user_id="u1",
|
user_id="u1",
|
||||||
|
session_id="test-session",
|
||||||
)
|
)
|
||||||
assert [o.action for o in ops] == ["INSERT"]
|
assert [o.action for o in ops] == ["INSERT"]
|
||||||
assert (await coding_store.get_fact("f_emma", user_id="u1")).status == "active"
|
assert (await coding_store.get_fact("f_emma", user_id="u1")).status == "active"
|
||||||
@@ -647,6 +697,7 @@ class TestSubjectGateSwitch:
|
|||||||
facts, ops = await rec.reconcile(
|
facts, ops = await rec.reconcile(
|
||||||
[WorkingFactDraft(subject="Bart", statement="Bart 的三日行程已排定,预算待定。")],
|
[WorkingFactDraft(subject="Bart", statement="Bart 的三日行程已排定,预算待定。")],
|
||||||
user_id="u1",
|
user_id="u1",
|
||||||
|
session_id="test-session",
|
||||||
)
|
)
|
||||||
assert [o.action for o in ops] == ["SUPERSEDE", "INSERT"]
|
assert [o.action for o in ops] == ["SUPERSEDE", "INSERT"]
|
||||||
old = await coding_store.get_fact("f_emma", user_id="u1")
|
old = await coding_store.get_fact("f_emma", user_id="u1")
|
||||||
@@ -667,7 +718,8 @@ class TestSubjectGateSwitch:
|
|||||||
[WorkingFactDraft(subject="Bart", statement="Bart 的行程。")],
|
[WorkingFactDraft(subject="Bart", statement="Bart 的行程。")],
|
||||||
[[old]],
|
[[old]],
|
||||||
{0: ["f_emma"]},
|
{0: ["f_emma"]},
|
||||||
user_id="u1", agent_id="a1", session_id=None, workspace_id=None,
|
user_id="u1", agent_id="default_agent",
|
||||||
|
session_id="test-session", workspace_id=None,
|
||||||
)
|
)
|
||||||
assert "SUPERSEDE" in [o.action for o in ops]
|
assert "SUPERSEDE" in [o.action for o in ops]
|
||||||
|
|
||||||
@@ -687,7 +739,8 @@ class TestSubjectGateSwitch:
|
|||||||
[WorkingFactDraft(subject="deploy.sh", statement="新的部署脚本结论。")],
|
[WorkingFactDraft(subject="deploy.sh", statement="新的部署脚本结论。")],
|
||||||
[[old]],
|
[[old]],
|
||||||
{0: [f"f_noneed_{gate}"]},
|
{0: [f"f_noneed_{gate}"]},
|
||||||
user_id="u1", agent_id="a1", session_id=None, workspace_id=None,
|
user_id="u1", agent_id="default_agent",
|
||||||
|
session_id="test-session", workspace_id=None,
|
||||||
)
|
)
|
||||||
assert "SUPERSEDE" in [o.action for o in ops], f"gate={gate} 时空 subject 被误拦"
|
assert "SUPERSEDE" in [o.action for o in ops], f"gate={gate} 时空 subject 被误拦"
|
||||||
|
|
||||||
@@ -714,6 +767,7 @@ class TestDetailsInheritance:
|
|||||||
async def test_ancestor_details_copied_verbatim(self, coding_store, mock_embed):
|
async def test_ancestor_details_copied_verbatim(self, coding_store, mock_embed):
|
||||||
await coding_store.insert_fact(WorkingFact(
|
await coding_store.insert_fact(WorkingFact(
|
||||||
fact_id="f_old", user_id="u1", subject="Emma",
|
fact_id="f_old", user_id="u1", subject="Emma",
|
||||||
|
session_id="test-session",
|
||||||
statement="Emma 的行程草案已出,预算待核。",
|
statement="Emma 的行程草案已出,预算待核。",
|
||||||
details="日均餐费 $32.4-$57.6\n酒店评分 3.2-3.8",
|
details="日均餐费 $32.4-$57.6\n酒店评分 3.2-3.8",
|
||||||
))
|
))
|
||||||
@@ -726,6 +780,7 @@ class TestDetailsInheritance:
|
|||||||
[WorkingFactDraft(subject="Emma", statement="Emma 的行程已定稿。",
|
[WorkingFactDraft(subject="Emma", statement="Emma 的行程已定稿。",
|
||||||
details="航班 CA1234")],
|
details="航班 CA1234")],
|
||||||
user_id="u1",
|
user_id="u1",
|
||||||
|
session_id="test-session",
|
||||||
)
|
)
|
||||||
stored = await coding_store.get_fact(facts[0].fact_id, user_id="u1")
|
stored = await coding_store.get_fact(facts[0].fact_id, user_id="u1")
|
||||||
# 本代的 details 在,祖先的具体取值也一字不差地在
|
# 本代的 details 在,祖先的具体取值也一字不差地在
|
||||||
@@ -742,6 +797,7 @@ class TestDetailsInheritance:
|
|||||||
"""
|
"""
|
||||||
await coding_store.insert_fact(WorkingFact(
|
await coding_store.insert_fact(WorkingFact(
|
||||||
fact_id="f_g1", user_id="u1", subject="T",
|
fact_id="f_g1", user_id="u1", subject="T",
|
||||||
|
session_id="test-session",
|
||||||
statement="第一代结论。", details="V1=" + "a" * 300,
|
statement="第一代结论。", details="V1=" + "a" * 300,
|
||||||
))
|
))
|
||||||
prev = "f_g1"
|
prev = "f_g1"
|
||||||
@@ -756,6 +812,7 @@ class TestDetailsInheritance:
|
|||||||
[WorkingFactDraft(subject="T", statement=f"第{gen}代结论。",
|
[WorkingFactDraft(subject="T", statement=f"第{gen}代结论。",
|
||||||
details=f"V{gen}=" + "a" * 300)],
|
details=f"V{gen}=" + "a" * 300)],
|
||||||
user_id="u1",
|
user_id="u1",
|
||||||
|
session_id="test-session",
|
||||||
)
|
)
|
||||||
prev = facts[0].fact_id
|
prev = facts[0].fact_id
|
||||||
stored = await coding_store.get_fact(prev, user_id="u1")
|
stored = await coding_store.get_fact(prev, user_id="u1")
|
||||||
@@ -772,6 +829,7 @@ class TestDetailsInheritance:
|
|||||||
"""预算是硬的,但被丢掉的代数必须写出来,不静默截断。"""
|
"""预算是硬的,但被丢掉的代数必须写出来,不静默截断。"""
|
||||||
await coding_store.insert_fact(WorkingFact(
|
await coding_store.insert_fact(WorkingFact(
|
||||||
fact_id="f_old", user_id="u1", subject="T",
|
fact_id="f_old", user_id="u1", subject="T",
|
||||||
|
session_id="test-session",
|
||||||
statement="旧结论。", details="X=" + "b" * 8000,
|
statement="旧结论。", details="X=" + "b" * 8000,
|
||||||
))
|
))
|
||||||
rec = WorkingFactReconciler(
|
rec = WorkingFactReconciler(
|
||||||
@@ -782,6 +840,7 @@ class TestDetailsInheritance:
|
|||||||
facts, _ = await rec.reconcile(
|
facts, _ = await rec.reconcile(
|
||||||
[WorkingFactDraft(subject="T", statement="新结论。", details="Y=1")],
|
[WorkingFactDraft(subject="T", statement="新结论。", details="Y=1")],
|
||||||
user_id="u1",
|
user_id="u1",
|
||||||
|
session_id="test-session",
|
||||||
)
|
)
|
||||||
stored = await coding_store.get_fact(facts[0].fact_id, user_id="u1")
|
stored = await coding_store.get_fact(facts[0].fact_id, user_id="u1")
|
||||||
assert "…(truncated)" in stored.details
|
assert "…(truncated)" in stored.details
|
||||||
@@ -790,6 +849,7 @@ class TestDetailsInheritance:
|
|||||||
async def test_switch_off_keeps_details_untouched(self, coding_store, mock_embed):
|
async def test_switch_off_keeps_details_untouched(self, coding_store, mock_embed):
|
||||||
await coding_store.insert_fact(WorkingFact(
|
await coding_store.insert_fact(WorkingFact(
|
||||||
fact_id="f_old", user_id="u1", subject="T",
|
fact_id="f_old", user_id="u1", subject="T",
|
||||||
|
session_id="test-session",
|
||||||
statement="旧结论。", details="祖先取值 Z=9",
|
statement="旧结论。", details="祖先取值 Z=9",
|
||||||
))
|
))
|
||||||
rec = WorkingFactReconciler(
|
rec = WorkingFactReconciler(
|
||||||
@@ -800,6 +860,7 @@ class TestDetailsInheritance:
|
|||||||
facts, ops = await rec.reconcile(
|
facts, ops = await rec.reconcile(
|
||||||
[WorkingFactDraft(subject="T", statement="新结论。", details="Y=1")],
|
[WorkingFactDraft(subject="T", statement="新结论。", details="Y=1")],
|
||||||
user_id="u1",
|
user_id="u1",
|
||||||
|
session_id="test-session",
|
||||||
)
|
)
|
||||||
# 覆盖照样发生,只是不过继
|
# 覆盖照样发生,只是不过继
|
||||||
assert "SUPERSEDE" in [o.action for o in ops]
|
assert "SUPERSEDE" in [o.action for o in ops]
|
||||||
@@ -814,6 +875,7 @@ class TestDetailsInheritance:
|
|||||||
"""
|
"""
|
||||||
await coding_store.insert_fact(WorkingFact(
|
await coding_store.insert_fact(WorkingFact(
|
||||||
fact_id="f_old", user_id="u1", subject="T",
|
fact_id="f_old", user_id="u1", subject="T",
|
||||||
|
session_id="test-session",
|
||||||
statement="旧结论。", details="无关噪声 " + "q" * 2000,
|
statement="旧结论。", details="无关噪声 " + "q" * 2000,
|
||||||
))
|
))
|
||||||
rec = WorkingFactReconciler(
|
rec = WorkingFactReconciler(
|
||||||
@@ -824,6 +886,7 @@ class TestDetailsInheritance:
|
|||||||
facts, _ = await rec.reconcile(
|
facts, _ = await rec.reconcile(
|
||||||
[WorkingFactDraft(subject="T", statement="新结论。", details="Y=1")],
|
[WorkingFactDraft(subject="T", statement="新结论。", details="Y=1")],
|
||||||
user_id="u1",
|
user_id="u1",
|
||||||
|
session_id="test-session",
|
||||||
)
|
)
|
||||||
new_id = facts[0].fact_id
|
new_id = facts[0].fact_id
|
||||||
emb = await coding_store.embed_service.embed("T 新结论。")
|
emb = await coding_store.embed_service.embed("T 新结论。")
|
||||||
|
|||||||
@@ -35,7 +35,8 @@ class TestReconcilerActions:
|
|||||||
async def test_update(self, coding_store, mock_embed):
|
async def test_update(self, coding_store, mock_embed):
|
||||||
# 种入 m1
|
# 种入 m1
|
||||||
await coding_store.insert(CodingMemory(
|
await coding_store.insert(CodingMemory(
|
||||||
experience_id="m1", user_id="u1", task="T1",
|
experience_id="m1", user_id="u1", agent_id="scope-day-1",
|
||||||
|
workspace_id="repo-a", session_id="conversation-1", task="T1",
|
||||||
search_keys=["old"], solution="old",
|
search_keys=["old"], solution="old",
|
||||||
))
|
))
|
||||||
llm = MockLLMProvider(_llm_resp([
|
llm = MockLLMProvider(_llm_resp([
|
||||||
@@ -44,23 +45,146 @@ class TestReconcilerActions:
|
|||||||
]))
|
]))
|
||||||
rec = CodingMemoryReconciler(coding_store, mock_embed, llm)
|
rec = CodingMemoryReconciler(coding_store, mock_embed, llm)
|
||||||
d = CodingMemoryDraft(task="T1", search_keys=["new1"], solution="new",
|
d = CodingMemoryDraft(task="T1", search_keys=["new1"], solution="new",
|
||||||
user_id="u1")
|
user_id="u1", agent_id="scope-day-1",
|
||||||
|
workspace_id="repo-a", session_id="conversation-1")
|
||||||
ops = await rec.reconcile([d])
|
ops = await rec.reconcile([d])
|
||||||
assert len(ops) == 1 and ops[0].action == "UPDATE"
|
assert len(ops) == 1 and ops[0].action == "UPDATE"
|
||||||
got = await coding_store.get_by_id("m1", "u1")
|
got = await coding_store.get_by_id("m1", "u1")
|
||||||
assert got.solution == "new"
|
assert got.solution == "new"
|
||||||
assert got.search_keys == ["new1"]
|
assert got.search_keys == ["new1"]
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
(
|
||||||
|
"old_agent",
|
||||||
|
"old_workspace",
|
||||||
|
"old_session",
|
||||||
|
"new_agent",
|
||||||
|
"new_workspace",
|
||||||
|
"new_session",
|
||||||
|
),
|
||||||
|
[
|
||||||
|
(
|
||||||
|
"scope:same-day",
|
||||||
|
"repo-a",
|
||||||
|
"conversation-a",
|
||||||
|
"scope:same-day",
|
||||||
|
"repo-a",
|
||||||
|
"conversation-b",
|
||||||
|
),
|
||||||
|
(
|
||||||
|
"scope:2026-10-18",
|
||||||
|
"repo-a",
|
||||||
|
"conversation-a",
|
||||||
|
"scope:2026-10-19",
|
||||||
|
"repo-a",
|
||||||
|
"conversation-a",
|
||||||
|
),
|
||||||
|
(
|
||||||
|
"shared-scope",
|
||||||
|
"repo-a",
|
||||||
|
"conversation-a",
|
||||||
|
"shared-scope",
|
||||||
|
"repo-b",
|
||||||
|
"conversation-a",
|
||||||
|
),
|
||||||
|
],
|
||||||
|
)
|
||||||
|
async def test_update_target_must_match_frozen_authority_scope(
|
||||||
|
self,
|
||||||
|
coding_store,
|
||||||
|
mock_embed,
|
||||||
|
old_agent,
|
||||||
|
old_workspace,
|
||||||
|
old_session,
|
||||||
|
new_agent,
|
||||||
|
new_workspace,
|
||||||
|
new_session,
|
||||||
|
):
|
||||||
|
"""Near duplicates cannot cross conversation, date, or workspace authority."""
|
||||||
|
await coding_store.insert(CodingMemory(
|
||||||
|
experience_id="old-target", user_id="u1", agent_id=old_agent,
|
||||||
|
workspace_id=old_workspace, session_id=old_session,
|
||||||
|
task="fix flaky release", solution="old solution",
|
||||||
|
))
|
||||||
|
llm = MockLLMProvider(_llm_resp([
|
||||||
|
{
|
||||||
|
"draft_idx": 0,
|
||||||
|
"action": "UPDATE",
|
||||||
|
"target_experience_id": "old-target",
|
||||||
|
"reason": "near duplicate",
|
||||||
|
},
|
||||||
|
]))
|
||||||
|
rec = CodingMemoryReconciler(coding_store, mock_embed, llm)
|
||||||
|
draft = CodingMemoryDraft(
|
||||||
|
task="fix flaky release",
|
||||||
|
solution="new solution",
|
||||||
|
user_id="u1",
|
||||||
|
agent_id=new_agent,
|
||||||
|
workspace_id=new_workspace,
|
||||||
|
session_id=new_session,
|
||||||
|
)
|
||||||
|
|
||||||
|
ops = await rec.reconcile([draft])
|
||||||
|
|
||||||
|
old = await coding_store.get_by_id("old-target", "u1")
|
||||||
|
assert old is not None
|
||||||
|
assert old.solution == "old solution"
|
||||||
|
assert old.agent_id == old_agent
|
||||||
|
assert old.workspace_id == old_workspace
|
||||||
|
assert old.session_id == old_session
|
||||||
|
assert len(ops) == 1
|
||||||
|
assert ops[0].action == "ADD"
|
||||||
|
assert ops[0].target_experience_id != "old-target"
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("legacy_agent", ["", "default_agent", "scope:legacy"])
|
||||||
|
async def test_legacy_target_without_conversation_falls_back_to_add(
|
||||||
|
self, coding_store, mock_embed, legacy_agent
|
||||||
|
):
|
||||||
|
await coding_store.insert(CodingMemory(
|
||||||
|
experience_id="legacy-target",
|
||||||
|
user_id="u1",
|
||||||
|
agent_id=legacy_agent,
|
||||||
|
session_id=None,
|
||||||
|
workspace_id="repo-a",
|
||||||
|
task="upgrade dependency",
|
||||||
|
solution="legacy solution",
|
||||||
|
))
|
||||||
|
llm = MockLLMProvider(_llm_resp([
|
||||||
|
{
|
||||||
|
"draft_idx": 0,
|
||||||
|
"action": "UPDATE",
|
||||||
|
"target_experience_id": "legacy-target",
|
||||||
|
},
|
||||||
|
]))
|
||||||
|
rec = CodingMemoryReconciler(coding_store, mock_embed, llm)
|
||||||
|
draft = CodingMemoryDraft(
|
||||||
|
task="upgrade dependency",
|
||||||
|
solution="current solution",
|
||||||
|
user_id="u1",
|
||||||
|
agent_id=legacy_agent,
|
||||||
|
session_id="current-conversation",
|
||||||
|
workspace_id="repo-a",
|
||||||
|
)
|
||||||
|
|
||||||
|
ops = await rec.reconcile([draft])
|
||||||
|
|
||||||
|
legacy = await coding_store.get_by_id("legacy-target", "u1")
|
||||||
|
assert legacy is not None
|
||||||
|
assert legacy.solution == "legacy solution"
|
||||||
|
assert legacy.session_id is None
|
||||||
|
assert [op.action for op in ops] == ["ADD"]
|
||||||
|
|
||||||
async def test_skip(self, coding_store, mock_embed):
|
async def test_skip(self, coding_store, mock_embed):
|
||||||
await coding_store.insert(CodingMemory(
|
await coding_store.insert(CodingMemory(
|
||||||
experience_id="m1", user_id="u1", task="X", solution="Y",
|
experience_id="m1", user_id="u1", task="X", solution="Y",
|
||||||
|
session_id="conversation-1",
|
||||||
))
|
))
|
||||||
llm = MockLLMProvider(_llm_resp([
|
llm = MockLLMProvider(_llm_resp([
|
||||||
{"draft_idx": 0, "action": "SKIP", "reason": "duplicate"},
|
{"draft_idx": 0, "action": "SKIP", "reason": "duplicate"},
|
||||||
]))
|
]))
|
||||||
rec = CodingMemoryReconciler(coding_store, mock_embed, llm)
|
rec = CodingMemoryReconciler(coding_store, mock_embed, llm)
|
||||||
d = CodingMemoryDraft(task="X", solution="Y",
|
d = CodingMemoryDraft(task="X", solution="Y",
|
||||||
user_id="u1")
|
user_id="u1", session_id="conversation-1")
|
||||||
ops = await rec.reconcile([d])
|
ops = await rec.reconcile([d])
|
||||||
assert len(ops) == 1 and ops[0].action == "SKIP"
|
assert len(ops) == 1 and ops[0].action == "SKIP"
|
||||||
# store 不变
|
# store 不变
|
||||||
@@ -71,7 +195,7 @@ class TestReconcilerActions:
|
|||||||
# 旧记忆要被 DELETE
|
# 旧记忆要被 DELETE
|
||||||
await coding_store.insert(CodingMemory(
|
await coding_store.insert(CodingMemory(
|
||||||
experience_id="m_dep", user_id="u1", task="use old.sh",
|
experience_id="m_dep", user_id="u1", task="use old.sh",
|
||||||
solution="old script",
|
solution="old script", session_id="conversation-1",
|
||||||
))
|
))
|
||||||
llm = MockLLMProvider(_llm_resp([
|
llm = MockLLMProvider(_llm_resp([
|
||||||
{"draft_idx": 0, "action": "ADD"},
|
{"draft_idx": 0, "action": "ADD"},
|
||||||
@@ -80,7 +204,7 @@ class TestReconcilerActions:
|
|||||||
]))
|
]))
|
||||||
rec = CodingMemoryReconciler(coding_store, mock_embed, llm)
|
rec = CodingMemoryReconciler(coding_store, mock_embed, llm)
|
||||||
d = CodingMemoryDraft(task="use new CLI", solution="new way",
|
d = CodingMemoryDraft(task="use new CLI", solution="new way",
|
||||||
user_id="u1")
|
user_id="u1", session_id="conversation-1")
|
||||||
ops = await rec.reconcile([d])
|
ops = await rec.reconcile([d])
|
||||||
assert sorted(op.action for op in ops) == ["ADD", "DELETE"]
|
assert sorted(op.action for op in ops) == ["ADD", "DELETE"]
|
||||||
# 旧的没了
|
# 旧的没了
|
||||||
|
|||||||
@@ -67,7 +67,8 @@ class TestStoreCRUD:
|
|||||||
|
|
||||||
async def test_update_overrides(self, coding_store, mock_embed):
|
async def test_update_overrides(self, coding_store, mock_embed):
|
||||||
m = CodingMemory(experience_id="e1", user_id="u1", task="T",
|
m = CodingMemory(experience_id="e1", user_id="u1", task="T",
|
||||||
search_keys=["old1", "old2"], solution="old")
|
search_keys=["old1", "old2"], solution="old",
|
||||||
|
session_id="conversation-1")
|
||||||
await coding_store.insert(m)
|
await coding_store.insert(m)
|
||||||
|
|
||||||
# 旧 aspect 应可命中
|
# 旧 aspect 应可命中
|
||||||
@@ -90,6 +91,59 @@ class TestStoreCRUD:
|
|||||||
assert got.solution == "new"
|
assert got.solution == "new"
|
||||||
assert got.search_keys == ["new1"]
|
assert got.search_keys == ["new1"]
|
||||||
|
|
||||||
|
async def test_update_rejects_authority_scope_change(self, coding_store):
|
||||||
|
await coding_store.insert(CodingMemory(
|
||||||
|
experience_id="e1", user_id="u1", agent_id="scope-a",
|
||||||
|
workspace_id="repo-a", session_id="conversation-1",
|
||||||
|
task="T", solution="old",
|
||||||
|
))
|
||||||
|
replacement = CodingMemory(
|
||||||
|
experience_id="e1", user_id="u1", agent_id="scope-b",
|
||||||
|
workspace_id="repo-a", session_id="conversation-1",
|
||||||
|
task="T", solution="new",
|
||||||
|
)
|
||||||
|
|
||||||
|
with pytest.raises(ValueError, match="coding_memory_update_authority_mismatch"):
|
||||||
|
await coding_store.update(replacement)
|
||||||
|
|
||||||
|
stored = await coding_store.get_by_id("e1", user_id="u1")
|
||||||
|
assert stored is not None
|
||||||
|
assert stored.agent_id == "scope-a"
|
||||||
|
assert stored.solution == "old"
|
||||||
|
|
||||||
|
async def test_update_rejects_conversation_change_and_legacy_empty_session(
|
||||||
|
self, coding_store
|
||||||
|
):
|
||||||
|
for experience_id, old_session in (
|
||||||
|
("different-conversation", "conversation-1"),
|
||||||
|
("legacy-empty-session", None),
|
||||||
|
):
|
||||||
|
await coding_store.insert(CodingMemory(
|
||||||
|
experience_id=experience_id,
|
||||||
|
user_id="u1",
|
||||||
|
agent_id="scope-a",
|
||||||
|
workspace_id="repo-a",
|
||||||
|
session_id=old_session,
|
||||||
|
task="T",
|
||||||
|
solution="old",
|
||||||
|
))
|
||||||
|
replacement = CodingMemory(
|
||||||
|
experience_id=experience_id,
|
||||||
|
user_id="u1",
|
||||||
|
agent_id="scope-a",
|
||||||
|
workspace_id="repo-a",
|
||||||
|
session_id="conversation-2",
|
||||||
|
task="T",
|
||||||
|
solution="new",
|
||||||
|
)
|
||||||
|
|
||||||
|
with pytest.raises(ValueError, match="coding_memory_update_authority_mismatch"):
|
||||||
|
await coding_store.update(replacement)
|
||||||
|
|
||||||
|
stored = await coding_store.get_by_id(experience_id, user_id="u1")
|
||||||
|
assert stored is not None
|
||||||
|
assert stored.solution == "old"
|
||||||
|
|
||||||
async def test_delete(self, coding_store):
|
async def test_delete(self, coding_store):
|
||||||
m = CodingMemory(experience_id="e1", user_id="u1", task="T",
|
m = CodingMemory(experience_id="e1", user_id="u1", task="T",
|
||||||
search_keys=["k"], solution="S")
|
search_keys=["k"], solution="S")
|
||||||
|
|||||||
@@ -335,7 +335,7 @@ class TestDualLayerWrite:
|
|||||||
from hy_memory.coding.types import CodingMemory
|
from hy_memory.coding.types import CodingMemory
|
||||||
await coding_store.insert(CodingMemory(
|
await coding_store.insert(CodingMemory(
|
||||||
experience_id="e1", user_id="u1", task="老 task", solution="旧 solution",
|
experience_id="e1", user_id="u1", task="老 task", solution="旧 solution",
|
||||||
source_fact_ids=["f_before"],
|
source_fact_ids=["f_before"], session_id="test-session",
|
||||||
))
|
))
|
||||||
llm = MockLLMProvider([
|
llm = MockLLMProvider([
|
||||||
_extract_resp(
|
_extract_resp(
|
||||||
@@ -348,7 +348,9 @@ class TestDualLayerWrite:
|
|||||||
])
|
])
|
||||||
writer = self._writer(coding_store, mock_embed, llm)
|
writer = self._writer(coding_store, mock_embed, llm)
|
||||||
|
|
||||||
resp = await writer.write(_coding_messages(), user_id="u1")
|
resp = await writer.write(
|
||||||
|
_coding_messages(), user_id="u1", session_id="test-session"
|
||||||
|
)
|
||||||
exp = await coding_store.get_by_id("e1", user_id="u1")
|
exp = await coding_store.get_by_id("e1", user_id="u1")
|
||||||
assert exp.solution == "新 solution"
|
assert exp.solution == "新 solution"
|
||||||
assert exp.source_fact_ids == ["f_before", resp["fact_ids"][0]]
|
assert exp.source_fact_ids == ["f_before", resp["fact_ids"][0]]
|
||||||
|
|||||||
@@ -12,8 +12,9 @@ L1_RAW 对外读路径不可见:闸门焊在向量库唯一查询入口。
|
|||||||
hybrid_search_native / list_by_user 四个查询方法的共同必经处,
|
hybrid_search_native / list_by_user 四个查询方法的共同必经处,
|
||||||
上层无法绕开,新增接口自动受保护。
|
上层无法绕开,新增接口自动受保护。
|
||||||
|
|
||||||
按 ID 直取走主键查询、不经过 filter,故在 `client.async_get` 处单独挡
|
按 ID 直取走主键查询、不经过 filter,故在 `client.async_get` 处默认挡住。
|
||||||
(GET /v1/memories/{id} 与 /v1/memories/batch 的共同必经处)。
|
Python SDK 调用方可显式传 `include_raw=True` 读取;HTTP 接口不传该参数,
|
||||||
|
继续保持原有隐藏行为。
|
||||||
|
|
||||||
内部管线需读原始对话时显式 allow_raw_layer=True 放行:
|
内部管线需读原始对话时显式 allow_raw_layer=True 放行:
|
||||||
- `writer._get_recent_dialogue_history`:提取前拼多轮对话历史喂 LLM
|
- `writer._get_recent_dialogue_history`:提取前拼多轮对话历史喂 LLM
|
||||||
@@ -124,28 +125,63 @@ class TestListNoLongerFiltersInProcess:
|
|||||||
"list 又出现进程内过滤 raw 的写法,应交由 vdb 侧 filter 完成"
|
"list 又出现进程内过滤 raw 的写法,应交由 vdb 侧 filter 完成"
|
||||||
|
|
||||||
|
|
||||||
class TestGetByIdPathBlocksRaw:
|
class TestGetByIdRawOptIn:
|
||||||
"""按 ID 直取:走主键查询不经 filter,须在 async_get 处挡住。"""
|
"""按 ID 直取默认隐藏 raw,只有 SDK 显式选择时才返回。"""
|
||||||
|
|
||||||
def test_async_get_drops_raw_node(self):
|
@staticmethod
|
||||||
|
def _client(node):
|
||||||
|
from hy_memory.client import HyMemoryClient
|
||||||
|
|
||||||
|
class Store:
|
||||||
|
async def get_by_id(self, memory_id):
|
||||||
|
return node if memory_id == node.node_id else None
|
||||||
|
|
||||||
|
async def no_op_log(**kwargs):
|
||||||
|
return None
|
||||||
|
|
||||||
|
client = HyMemoryClient.__new__(HyMemoryClient)
|
||||||
|
client._vector_store = Store()
|
||||||
|
client._graph_store = None
|
||||||
|
client._backend_identity = lambda: {}
|
||||||
|
client._maybe_store_op_log = no_op_log
|
||||||
|
return client
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _raw_node():
|
||||||
|
return MemoryNode(
|
||||||
|
node_id="raw-1",
|
||||||
|
user_id="u1",
|
||||||
|
agent_id="a1",
|
||||||
|
session_id="s1",
|
||||||
|
content="原始对话",
|
||||||
|
layer=MemoryLayer.L1_RAW,
|
||||||
|
status=MemoryStatus.ACTIVE,
|
||||||
|
custom={"meta": [{"source": "test"}]},
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_default_still_hides_raw(self):
|
||||||
|
from hy_memory.client import HyMemoryClient
|
||||||
|
|
||||||
|
client = self._client(self._raw_node())
|
||||||
|
raw_get = HyMemoryClient.async_get.__wrapped__
|
||||||
|
assert run(raw_get(client, "raw-1")) is None
|
||||||
|
|
||||||
|
def test_explicit_include_raw_returns_lite_memory(self):
|
||||||
|
from hy_memory.client import HyMemoryClient
|
||||||
|
|
||||||
|
client = self._client(self._raw_node())
|
||||||
|
raw_get = HyMemoryClient.async_get.__wrapped__
|
||||||
|
result = run(raw_get(client, "raw-1", include_raw=True))
|
||||||
|
|
||||||
|
assert result["memory_id"] == "raw-1"
|
||||||
|
assert result["content"] == "原始对话"
|
||||||
|
assert result["layer"] == MemoryLayer.L1_RAW.value
|
||||||
|
assert result["meta"] == [{"source": "test"}]
|
||||||
|
|
||||||
|
def test_public_get_exposes_keyword_only_opt_in(self):
|
||||||
import inspect
|
import inspect
|
||||||
from hy_memory.client import HyMemoryClient
|
from hy_memory.client import HyMemoryClient
|
||||||
src = inspect.getsource(HyMemoryClient.async_get)
|
|
||||||
assert "MemoryLayer.L1_RAW" in src, \
|
|
||||||
"async_get 未挡 raw,batch 与按 ID 单取会漏出原始对话"
|
|
||||||
|
|
||||||
def test_block_happens_before_return(self):
|
param = inspect.signature(HyMemoryClient.get).parameters["include_raw"]
|
||||||
"""挡的位置必须在两处 return 之前,否则等于没挡。"""
|
assert param.default is False
|
||||||
import inspect
|
assert param.kind is inspect.Parameter.KEYWORD_ONLY
|
||||||
from hy_memory.client import HyMemoryClient
|
|
||||||
lines = inspect.getsource(HyMemoryClient.async_get).splitlines()
|
|
||||||
block_at = next(
|
|
||||||
i for i, l in enumerate(lines)
|
|
||||||
if "MemoryLayer.L1_RAW" in l and "vdb_node" in l
|
|
||||||
)
|
|
||||||
first_return_at = next(
|
|
||||||
i for i, l in enumerate(lines)
|
|
||||||
if l.strip().startswith("return {")
|
|
||||||
)
|
|
||||||
assert block_at < first_return_at, \
|
|
||||||
f"挡 raw 在第 {block_at} 行、首个 return 在第 {first_return_at} 行,顺序错了"
|
|
||||||
@@ -0,0 +1,136 @@
|
|||||||
|
import asyncio
|
||||||
|
import threading
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from hy_memory.config import MemoryConfig
|
||||||
|
from hy_memory.data.cache_sqlite import SqliteCache
|
||||||
|
from hy_memory.data.history_store import HistoryStore
|
||||||
|
|
||||||
|
|
||||||
|
class _BlockingCursor:
|
||||||
|
def fetchone(self):
|
||||||
|
return (1,)
|
||||||
|
|
||||||
|
def fetchall(self):
|
||||||
|
return []
|
||||||
|
|
||||||
|
|
||||||
|
class _BlockingConnection:
|
||||||
|
"""Hold execute() open so a concurrent close() can be observed."""
|
||||||
|
|
||||||
|
def __init__(self):
|
||||||
|
self.entered = threading.Event()
|
||||||
|
self.release = threading.Event()
|
||||||
|
self.closed = threading.Event()
|
||||||
|
|
||||||
|
def execute(self, *_args, **_kwargs):
|
||||||
|
self.entered.set()
|
||||||
|
if not self.release.wait(timeout=2):
|
||||||
|
raise TimeoutError("test did not release the blocking query")
|
||||||
|
if self.closed.is_set():
|
||||||
|
raise AssertionError("connection was closed while execute() was active")
|
||||||
|
return _BlockingCursor()
|
||||||
|
|
||||||
|
def close(self):
|
||||||
|
self.closed.set()
|
||||||
|
|
||||||
|
|
||||||
|
def _config(tmp_path) -> MemoryConfig:
|
||||||
|
config = MemoryConfig()
|
||||||
|
config.history.db_path = str(tmp_path / "history.db")
|
||||||
|
config.cache.db_path = str(tmp_path / "cache.db")
|
||||||
|
return config
|
||||||
|
|
||||||
|
|
||||||
|
async def _wait_for(event: threading.Event) -> None:
|
||||||
|
assert await asyncio.wait_for(asyncio.to_thread(event.wait, 1), timeout=2)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_history_close_waits_for_active_query(tmp_path):
|
||||||
|
store = HistoryStore(_config(tmp_path))
|
||||||
|
connection = _BlockingConnection()
|
||||||
|
store._conn = connection
|
||||||
|
|
||||||
|
query = asyncio.create_task(store.count())
|
||||||
|
await _wait_for(connection.entered)
|
||||||
|
closing = asyncio.create_task(store.close())
|
||||||
|
await asyncio.sleep(0.05)
|
||||||
|
|
||||||
|
assert not connection.closed.is_set()
|
||||||
|
connection.release.set()
|
||||||
|
assert await query == 1
|
||||||
|
await closing
|
||||||
|
assert connection.closed.is_set()
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_cache_close_waits_for_active_query(tmp_path):
|
||||||
|
store = SqliteCache(_config(tmp_path))
|
||||||
|
connection = _BlockingConnection()
|
||||||
|
store._conn = connection
|
||||||
|
|
||||||
|
query = asyncio.create_task(store.load_metrics_range("0", "9"))
|
||||||
|
await _wait_for(connection.entered)
|
||||||
|
closing = asyncio.create_task(store.close())
|
||||||
|
await asyncio.sleep(0.05)
|
||||||
|
|
||||||
|
assert not connection.closed.is_set()
|
||||||
|
connection.release.set()
|
||||||
|
assert await query == []
|
||||||
|
await closing
|
||||||
|
assert connection.closed.is_set()
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_history_concurrent_reads_and_writes_are_serialized(tmp_path):
|
||||||
|
store = HistoryStore(_config(tmp_path))
|
||||||
|
await store.initialize()
|
||||||
|
try:
|
||||||
|
inserted = await asyncio.gather(
|
||||||
|
*(
|
||||||
|
store.record_add(
|
||||||
|
memory_id=f"memory-{index}",
|
||||||
|
content="content",
|
||||||
|
layer="l2_fact",
|
||||||
|
isolation_key="user",
|
||||||
|
)
|
||||||
|
for index in range(200)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
assert all(row_id > 0 for row_id in inserted)
|
||||||
|
assert len(set(inserted)) == 200
|
||||||
|
|
||||||
|
counts = await asyncio.gather(*(store.count() for _ in range(100)))
|
||||||
|
assert counts == [200] * 100
|
||||||
|
finally:
|
||||||
|
await store.close()
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_cache_concurrent_pipeline_log_reads_and_writes_are_serialized(tmp_path):
|
||||||
|
store = SqliteCache(_config(tmp_path))
|
||||||
|
await store.initialize()
|
||||||
|
try:
|
||||||
|
written = await asyncio.gather(
|
||||||
|
*(
|
||||||
|
store.store_pipeline_log(
|
||||||
|
request_id="request",
|
||||||
|
user_id="user",
|
||||||
|
agent_id="agent",
|
||||||
|
step="EXTRACT",
|
||||||
|
prompt=f"prompt-{index}",
|
||||||
|
response="response",
|
||||||
|
)
|
||||||
|
for index in range(200)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
assert written == [True] * 200
|
||||||
|
|
||||||
|
pages = await asyncio.gather(
|
||||||
|
*(store.get_pipeline_logs(request_id="request", limit=200) for _ in range(50))
|
||||||
|
)
|
||||||
|
assert all(len(page) == 200 for page in pages)
|
||||||
|
finally:
|
||||||
|
await store.close()
|
||||||
Reference in new issue
Block a user