Compare commits

...
Author SHA1 Message Date
kkjzkangandCursor 8918098fa8 docs: announce Hy-Memory-30B-A3B FP8 and GGUF quantized models
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-20 11:22:16 +08:00
0afe787c95 fix: stop authority-invalid HY retries (#78)
* fix: prevent authority-invalid HY retries

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix: align coding authority with conversations

Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: alvinfei <alvinfei@tencent.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-19 15:30:32 +08:00
67ae43f920 fix(memhub): repair visibility dirty trigger conflicts (#77)
* fix(memhub): repair visibility link trigger

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(memhub): make trigger upgrade atomic

Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: alvinfei <alvinfei@tencent.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-19 15:30:11 +08:00
3ce89ea560 fix(sdk): serialize sqlite connection lifecycle (#73)
Co-authored-by: alvinfei <alvinfei@tencent.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-19 13:02:22 +08:00
7b7fd217a8 fix(sdk): clarify metadata and allow opt-in raw get (#65)
Co-authored-by: alvinfei <alvinfei@tencent.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-18 22:30:23 +08:00
7efe054e83 fix(core): park writes until a model is active (#63)
Co-authored-by: alvinfei <alvinfei@tencent.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-18 21:43:24 +08:00
fdbcd9934d fix(web): distinguish scanned and saved imports (#60)
Co-authored-by: alvinfei <alvinfei@tencent.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-18 21:10:10 +08:00
3171007b3c fix(core): keep initial sync behind bounded history (#59)
Co-authored-by: alvinfei <alvinfei@tencent.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-18 21:00:54 +08:00
32 changed files with 1916 additions and 346 deletions

No files matched your search

+19
View File
@@ -14,6 +14,25 @@ Hy Memory and MemHub are versioned and released independently.
## Unreleased
- Fixed governed-memory revisions and repeated evidence links failing when the
visibility dirty-marker trigger encounters the same memory twice. Startup
now replaces the affected legacy trigger without changing schema versions
or governed and visibility data.
- Serialized each SDK SQLite store's shared connection lifecycle so concurrent
history/cache operations cannot race with `close()` and crash the process.
- Scoped Coding reconciliation targets to the current HY authority,
conversation, and workspace, preventing near-duplicate sessions from
mutating an experience or working fact owned by another evidence context.
- Kept post-write authority verification failures available for read-only
reconciliation while preventing automatic or generic confirmed retries from
invoking the model again. Automatic reconciliation skips those sticky rows
before applying its batch limit, and linked durable jobs exclusively own
retry scheduling for their memory tasks.
- Clarified that `add(metadata=...)` carries legacy SDK processing options,
while persistent caller business context belongs in `meta`.
- Added an opt-in `get(..., include_raw=True)` path for reading `L1_RAW`
memories, including memories written in lite mode. The default remains
hidden for backward compatibility.
- Removed MemHub's task-scoped HyPermission store wrappers and global
model-content consent switch. The trusted embedded Hy Memory SDK now uses
its native local stores directly.
+1
View File
@@ -32,6 +32,7 @@ English | [中文](README.zh.md)
## Latest News
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-10]**: 🚀 **Open-sourced** quantized **FP8** (**[Hy-Memory-30B-A3B-FP8](https://huggingface.co/tencent/Hy-Memory-30B-A3B-FP8)**) and **GGUF** (**[Hy-Memory-30B-A3B-GGUF](https://huggingface.co/tencent/Hy-Memory-30B-A3B-GGUF)**) versions of **Hy-Memory-30B-A3B** on Hugging Face!
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-10]**: 📦 **Released [MemHub v1.0.4](https://github.com/Tencent-Hunyuan/Hy-Memory/releases/tag/v1.0.4)** and **[memhub-cli 1.0.5](https://www.npmjs.com/package/memhub-cli/v/1.0.5)**, with a signed macOS desktop app and verified macOS/Linux Runtimes!
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-10]**: 🚀 **Released** **Hy Memory 2.0 (`hy-memory==2.0.4`)** on [PyPI](https://pypi.org/project/hy-memory/2.0.4/) and open-sourced the **Hy-Memory** monorepo together with **[MemHub](memhub/README.md)**!
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-10]**: 🚀 **Open-sourced** **[Hy-Memory-30B-A3B](https://huggingface.co/tencent/Hy-Memory-30B-A3B)** on Hugging Face along with its [technical report](https://arxiv.org/abs/2610.22196) on attributed reinforcement learning for long-term memory writing!
+1
View File
@@ -32,6 +32,7 @@
## 最新动态
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-10]**: 🚀 **模型开源** 在 Hugging Face 发布 **Hy-Memory-30B-A3B** 的 **FP8**(**[Hy-Memory-30B-A3B-FP8](https://huggingface.co/tencent/Hy-Memory-30B-A3B-FP8)**)与 **GGUF**(**[Hy-Memory-30B-A3B-GGUF](https://huggingface.co/tencent/Hy-Memory-30B-A3B-GGUF)**)量化版本模型!
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-10]**: 📦 **正式发布 [MemHub v1.0.4](https://github.com/Tencent-Hunyuan/Hy-Memory/releases/tag/v1.0.4)** 与 **[memhub-cli 1.0.5](https://www.npmjs.com/package/memhub-cli/v/1.0.5)**,提供已签名的 macOS 桌面应用及经过验证的 macOS/Linux Runtime!
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-10]**: 🚀 **正式发布** **Hy Memory 2.0 (`hy-memory==2.0.4`)**(已上线 [PyPI](https://pypi.org/project/hy-memory/2.0.4/)),并开源 **Hy-Memory** monorepo 与 **[MemHub](memhub/README.md)**!
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-10]**: 🚀 **模型开源** 在 Hugging Face 发布 **[Hy-Memory-30B-A3B](https://huggingface.co/tencent/Hy-Memory-30B-A3B)** 长期记忆写入模型及[技术报告](https://arxiv.org/abs/2610.22196)!
+15
View File
@@ -106,6 +106,15 @@ result = client.add(
)
```
`metadata` 与 `meta` 不是同一字段:
- `metadata` 是兼容旧调用方保留的 SDK 处理选项,会进入内部
`WriteRequest.extra`,不保证持久化或随查询结果返回。
- `meta` 是调用方的业务上下文,会持久化并在 `search()`、`get()` 等结果中返回。
同一记忆被多次合并时,各次 `meta` 会按写入顺序保存在列表中。
需要以后读回的数据必须放在 `meta`,不要放在 `metadata`。
`scene` 的公开取值:
- `None`:普通文本直接走 chat;messages 含 tool 时执行快速场景判断。
@@ -263,6 +272,8 @@ catalog 也可以在没有基础 L0 时形成只读的合成画像。
```python
memory = client.get("memory_id")
# lite 模式写入的是 L1_RAW;按 ID 读取时必须显式选择返回原文。
raw_memory = client.get("memory_id", include_raw=True)
result = client.update("memory_id", "更新后的内容")
result = client.delete("memory_id")
result = client.delete_all(user_id="user_001")
@@ -282,6 +293,10 @@ result = client.clone_user("user_001", "user_002", agent_id=None)
对应异步方法为 `async_get()`、`async_update()`、`async_delete()`、
`async_delete_all()`、`async_list_memories()`、`async_clone_user()`。
`get()` / `async_get()` 默认继续隐藏 `L1_RAW`,因此不会改变现有调用方的原文访问
边界。只有显式传入 `include_raw=True` 才返回该层;这也是 lite 模式按 ID 读取
写入内容及其 `meta` 的方式。MemHub 的 HTTP 读取接口不开放此参数。
普通 `list_memories()` 返回 `vdb` 桶,并在 Graph 有结果时附带 `graph`;传入
`workspace_id` 时返回生产力记忆的 `scene="productivity"` 和 `experiences`。
+39 -12
View File
@@ -1597,7 +1597,9 @@ class HyMemoryClient:
session_id: Session ID(三级隔离 key)
source: 调用来源,例如 openclaw / cursor / codex;与 agent_id 正交。
写入前执行 strip().lower(),默认 "default"。
metadata: 自定义元数据
metadata: 旧版 SDK 处理选项,会并入内部 WriteRequest.extra。
不保证持久化或随查询返回;调用方需要保存并读回的业务字段
请使用 meta。
memory_at: 记忆时间戳(不传则用当前时间),用于导入历史记忆
scene: 显式指定写入路由,跳过 LLM 自动判定。
- None(默认):含 tool 时 LLM 判 chat/working/coding;
@@ -1623,8 +1625,8 @@ class HyMemoryClient:
branch: Coding memory 路径专用。分支名。同 workspace_id。chat 路径忽略。
meta: 调用方自定义业务 meta,随本次产出的记忆存储、查询时原样返回。
不 embed、不参与召回、不作过滤条件。
与 metadata 不同:metadata 会并入 extra(含 SDK 内部控制键),
meta 独立存放,只做透传。
与 metadata 不同:metadata 是旧版内部处理选项,
meta 独立持久化,只做业务透传。
语义为「该次 write 时刻的上下文」,故存为 list——同一节点被
多次写入合并时会追加而非覆盖,详见 get() 返回的 meta 字段。
@@ -1791,17 +1793,26 @@ class HyMemoryClient:
query_embedding=query_embedding,
))
def get(self, memory_id: str) -> Optional[Dict[str, Any]]:
def get(
self,
memory_id: str,
*,
include_raw: bool = False,
) -> Optional[Dict[str, Any]]:
"""
获取单条记忆。
Args:
memory_id: 记忆 ID
include_raw: 是否允许返回 L1_RAW 原始记忆。默认 False,保持原有隐藏行为;
lite 模式的记忆均为 L1_RAW,需显式设为 True 才能按 ID 读取。
Returns:
{"memory_id", "content", "layer", "request_id", "elapsed_ms"} 或 None(不存在时)
"""
return self._loop_thread.run(self.async_get(memory_id))
return self._loop_thread.run(
self.async_get(memory_id, include_raw=include_raw)
)
def update(
self,
@@ -3650,12 +3661,21 @@ class HyMemoryClient:
# ================================================================
@_ensure_internal_loop
async def async_get(self, memory_id: str) -> Optional[Dict[str, Any]]:
async def async_get(
self,
memory_id: str,
*,
include_raw: bool = False,
) -> Optional[Dict[str, Any]]:
"""
获取单条记忆(异步)。
VDB + Graph 双路并行查询,哪个有结果返回哪个。
Args:
memory_id: 记忆 ID
include_raw: 是否允许返回 L1_RAW 原始记忆,默认 False。
Returns:
{"memory_id", "content", "layer", "source", "request_id", "elapsed_ms"} 或 None
"""
@@ -3683,17 +3703,24 @@ class HyMemoryClient:
vdb_node, graph_node = await _asyncio_get.gather(_get_vdb(), _get_graph())
# L1_RAW 不对外可见:按 ID 直取走主键查询、绕过向量库的 layer filter,
# 故在此统一挡掉。async_get 是 GET /v1/memories/{id} 与
# /v1/memories/batch 两个对外接口的共同必经处,而内部写链路不经过它
# (writer 写后校验直接调 vector_store.get_by_id),挡在此处不影响内部。
# L1_RAW 默认不对外可见:按 ID 直取走主键查询、绕过向量库的 layer
# filter,故在此统一挡掉。SDK 调用方可通过 include_raw=True 显式读取;
# HTTP 接口不传该参数,继续保持隐藏。内部写链路不经过此方法。
#
# 置于 gather 之后、埋点之前:命中 raw 仍照常落 GET 埋点(保留可观测性,
# 便于发现有调用方拿 raw id 试探),只是不把内容返回给调用方。
# 对调用方的表现与"该 id 不存在"完全一致,不额外暴露"存在但被挡"的信息。
if vdb_node is not None and vdb_node.layer == MemoryLayer.L1_RAW:
if (
not include_raw
and vdb_node is not None
and vdb_node.layer == MemoryLayer.L1_RAW
):
vdb_node = None
if graph_node is not None and getattr(graph_node, "layer", None) == MemoryLayer.L1_RAW:
if (
not include_raw
and graph_node is not None
and getattr(graph_node, "layer", None) == MemoryLayer.L1_RAW
):
graph_node = None
elapsed = round((time.perf_counter() - t0) * 1000, 2)
+33 -2
View File
@@ -466,7 +466,13 @@ class WorkingFactReconciler:
return [], []
candidates = await self._retrieve_candidates(
drafts, user_id=user_id, source=source,
drafts,
user_id=user_id,
agent_id=agent_id,
session_id=session_id,
workspace_id=workspace_id,
branch=branch,
source=source,
)
try:
@@ -499,7 +505,15 @@ class WorkingFactReconciler:
# ------------------------------------------------------------
async def _retrieve_candidates(
self, drafts: List[WorkingFactDraft], *, user_id: str, source: str
self,
drafts: List[WorkingFactDraft],
*,
user_id: str,
agent_id: str,
session_id: Optional[str],
workspace_id: Optional[str],
branch: Optional[str],
source: str,
) -> List[List[WorkingFact]]:
"""每条 draft 用 statement 召回同 user 的 top-N active 事实。
@@ -512,6 +526,10 @@ class WorkingFactReconciler:
关掉时整份召回原样交给 LLM。适用于 subject 不是稳定实体名的数据集(自由文本、
位置标签),那里精确字符串相等会把同实体判成不同主体,反而拦掉真实更新。
"""
# Empty/legacy session metadata remains readable, but cannot prove
# conversation authority for a destructive SUPERSEDE.
if not session_id:
return [[] for _ in drafts]
try:
embeddings = await self.embed_service.embed_batch(
[d.statement for d in drafts]
@@ -525,6 +543,10 @@ class WorkingFactReconciler:
try:
hits = await self.store.search_facts(
emb, user_id=user_id, top=self.candidates_per_draft,
agent_id=agent_id,
session_id=session_id,
workspace_id=workspace_id,
branch=branch,
# 写入侧同样接词面通道:候选选错了 LLM 就没有机会判对。这里不传
# min_score——写入侧要的是"有哪些可能的前身",让 LLM 看更宽的名单,
# 阈值过滤是读取侧的事。
@@ -794,6 +816,15 @@ class WorkingFactReconciler:
old = await self.store.get_fact(old_id, user_id=user_id)
if old is None:
continue
if (
old.agent_id != agent_id
or old.session_id != session_id
or old.workspace_id != workspace_id
):
logger.warning(
f"[fact-reconcile] refused cross-authority SUPERSEDE: {old.fact_id}"
)
continue
# 闸门在候选阶段已经拦过一道,这里再拦一次:候选是按 statement 召回的,
# 而 LLM 只能覆盖候选名单里的 id,理论上到不了这儿。真到了就说明前面
# 哪一环漏了,宁可让这次覆盖不生效——多留一条旧事实是可恢复的,删掉
+33 -1
View File
@@ -240,7 +240,7 @@ class CodingMemoryReconciler:
)
# 3. 校验 + 执行
executed = await self._execute(ops, drafts)
executed = await self._execute(ops, drafts, candidates_per_draft)
logger.info(
f"[coding-reconcile] drafts={len(drafts)} ops={len(executed)} "
+ ", ".join(f"{a}={sum(1 for o in executed if o.action == a)}"
@@ -263,6 +263,12 @@ class CodingMemoryReconciler:
"""
result: List[List[CodingMemory]] = []
for d in drafts:
# A coding mutation must be attributable to one conversation.
# Empty/legacy session metadata stays readable through ordinary
# search, but it cannot authorize UPDATE/DELETE.
if not d.session_id:
result.append([])
continue
try:
# task 为主入口,search_keys 为附加入口(去重、去空)
queries = [d.task] + [k for k in (d.search_keys or []) if k and k != d.task]
@@ -274,6 +280,8 @@ class CodingMemoryReconciler:
hits = await self.store.search_by_query_embedding(
emb,
user_id=d.user_id,
agent_id=d.agent_id,
session_id=d.session_id,
workspace_id=d.workspace_id,
branch=d.branch,
top=self.candidates_per_draft * 4,
@@ -415,6 +423,7 @@ class CodingMemoryReconciler:
self,
ops: List[ReconcileOp],
drafts: List[CodingMemoryDraft],
candidates_per_draft: List[List[CodingMemory]],
) -> List[ReconcileOp]:
"""
校验 + 执行 ops。返回成功执行的 ops(filter 掉无效 op)。
@@ -454,6 +463,11 @@ class CodingMemoryReconciler:
if not op.target_experience_id:
logger.warning(f"[coding-reconcile] drop UPDATE missing target: {op.to_dict()}")
continue
if not self._target_was_offered(op, candidates_per_draft):
logger.warning(
f"[coding-reconcile] drop UPDATE outside authority scope: {op.to_dict()}"
)
continue
if op.draft_idx in consumed:
logger.warning(f"[coding-reconcile] drop UPDATE on consumed draft: {op.to_dict()}")
continue
@@ -476,6 +490,11 @@ class CodingMemoryReconciler:
if not op.target_experience_id:
logger.warning(f"[coding-reconcile] drop DELETE missing target: {op.to_dict()}")
continue
if not self._target_was_offered(op, candidates_per_draft):
logger.warning(
f"[coding-reconcile] drop DELETE outside authority scope: {op.to_dict()}"
)
continue
# DELETE 需要知道 user_id;通过 draft_idx(如有)或第一个 draft 推断
user_id = self._infer_user_id(op, drafts)
if not user_id:
@@ -512,6 +531,19 @@ class CodingMemoryReconciler:
return executed
@staticmethod
def _target_was_offered(
op: ReconcileOp,
candidates_per_draft: List[List[CodingMemory]],
) -> bool:
"""Only mutate a target frozen into this draft's scoped candidate set."""
if op.draft_idx is None or not (0 <= op.draft_idx < len(candidates_per_draft)):
return False
return any(
candidate.experience_id == op.target_experience_id
for candidate in candidates_per_draft[op.draft_idx]
)
@staticmethod
def _infer_user_id(
op: ReconcileOp, drafts: List[CodingMemoryDraft]
+67 -15
View File
@@ -529,10 +529,20 @@ class CodingMemoryStore:
"""覆盖式更新一条 coding memory。"""
# 拿旧 keys 长度,先删旧 VDB rows
prev = await self.get_by_id(memory.experience_id, user_id=memory.user_id)
prev_key_count = (1 + len(prev.search_keys)) if prev else 0
if prev is None:
raise ValueError("coding_memory_update_target_missing")
if (
not prev.session_id
or not memory.session_id
or prev.agent_id != memory.agent_id
or prev.session_id != memory.session_id
or prev.workspace_id != memory.workspace_id
):
raise ValueError("coding_memory_update_authority_mismatch")
prev_key_count = 1 + len(prev.search_keys)
memory.updated_at = datetime.utcnow()
if memory.created_at is None and prev is not None:
if memory.created_at is None:
memory.created_at = prev.created_at
await _run_in_pool(self._upsert_sql_sync, memory)
# 删旧 VDB
@@ -671,6 +681,8 @@ class CodingMemoryStore:
query_embedding: List[float],
user_id: str,
*,
agent_id: Optional[str] = None,
session_id: Optional[str] = None,
workspace_id: Optional[str] = None,
branch: Optional[str] = None,
top: int = 40,
@@ -688,21 +700,28 @@ class CodingMemoryStore:
},
...
]
workspace_id / branch 是作用域过滤,规则见 _scope_where:不传则纯语义召回
(行为与加过滤前完全一致);传了则只保留 SQLite 里落在作用域内的经验,并按
作用域的选择率放大向量取数,避免"先取 top-N 再过滤"把召回吃掉。
agent_id / session_id 是写授权作用域,传入时精确匹配;workspace_id / branch
的规则见 _scope_where。不传任何作用域则维持纯语义召回;传了则只保留 SQLite
里落在作用域内的经验,并按选择率放大向量取数,避免"先取 top-N 再过滤"把
召回吃掉。
"""
if self._vec is None:
return []
allowed: Optional[Set[str]] = None
if workspace_id:
if agent_id or session_id or workspace_id:
allowed, total = await _run_in_pool(
self._scoped_experience_ids_sync, user_id, workspace_id, branch
self._authority_scoped_experience_ids_sync,
user_id,
agent_id,
session_id,
workspace_id,
branch,
)
if not allowed:
logger.info(
f"[coding-store] no experience in scope: "
f"agent_id={agent_id!r} session_id={session_id!r} "
f"workspace_id={workspace_id!r} branch={branch!r} "
f"(user has {total}); skip vdb search"
)
@@ -778,6 +797,8 @@ class CodingMemoryStore:
query_embedding: List[float],
user_id: str,
*,
agent_id: Optional[str] = None,
session_id: Optional[str] = None,
workspace_id: Optional[str] = None,
branch: Optional[str] = None,
top: int = 5,
@@ -791,8 +812,9 @@ class CodingMemoryStore:
SUPERSEDE 时向量行已被删掉,理论上召不到 superseded;这里再按 SQLite 的
status 过一道,避免两边不一致时把过期结论放出去。
workspace_id / branch 是作用域过滤,规则见 _scope_where,语义与经验层一致:
不传不过滤,传了按 SQLite 里的值筛,并按选择率放大向量取数。
agent_id / session_id 是写授权作用域,传入时精确匹配;workspace_id / branch
的规则见 _scope_where。不传任何作用域则不过滤,传了按 SQLite 里的值筛,并
按选择率放大向量取数。
向量库按 top 的 4 倍过采样:向量行和 SQLite 行不是一个事务里写的,中间挂掉
就会留下"向量还在、SQLite 没了"的孤儿,软删时删向量失败也会留下 superseded
@@ -809,13 +831,19 @@ class CodingMemoryStore:
return []
fetch = top * 4
if workspace_id:
if agent_id or session_id or workspace_id:
scoped, total = await _run_in_pool(
self._count_active_facts_sync, user_id, workspace_id, branch
self._count_active_facts_sync,
user_id,
agent_id,
session_id,
workspace_id,
branch,
)
if not scoped:
logger.info(
f"[coding-store] no active fact in scope: "
f"agent_id={agent_id!r} session_id={session_id!r} "
f"workspace_id={workspace_id!r} branch={branch!r} "
f"(user has {total}); skip fact vdb search"
)
@@ -852,6 +880,8 @@ class CodingMemoryStore:
alive = [
(f, scores[f.fact_id]) for f in facts
if f.status == "active"
and (not agent_id or f.agent_id == agent_id)
and (not session_id or f.session_id == session_id)
and _scope_matches(f.workspace_id, f.branch, workspace_id, branch)
]
kept = [(f, s) for f, s in alive if s >= min_score]
@@ -1050,14 +1080,25 @@ class CodingMemoryStore:
by_id = {r["experience_id"]: _row_to_memory(r) for r in rows}
return [by_id[eid] for eid in experience_ids if eid in by_id]
def _scoped_experience_ids_sync(
self, user_id: str, workspace_id: str, branch: Optional[str]
def _authority_scoped_experience_ids_sync(
self,
user_id: str,
agent_id: Optional[str],
session_id: Optional[str],
workspace_id: Optional[str],
branch: Optional[str],
) -> Tuple[Set[str], int]:
"""返回 (落在作用域内的 experience_id 集合, 该 user 的经验总数)。
"""返回 (落在授权作用域内的 experience_id 集合, 该 user 的经验总数)。
一次查询同时拿到过滤结果和选择率分母:前者供向量命中做交集,后者定放大倍数。
"""
scope_sql, scope_params = _scope_where(workspace_id, branch)
if agent_id:
scope_sql = f"agent_id = ? AND ({scope_sql})"
scope_params = [agent_id, *scope_params]
if session_id:
scope_sql = f"session_id = ? AND ({scope_sql})"
scope_params = [session_id, *scope_params]
with self._lock:
assert self._conn is not None
cur = self._conn.execute(
@@ -1069,13 +1110,24 @@ class CodingMemoryStore:
return {r["experience_id"] for r in rows if r["in_scope"]}, len(rows)
def _count_active_facts_sync(
self, user_id: str, workspace_id: str, branch: Optional[str]
self,
user_id: str,
agent_id: Optional[str],
session_id: Optional[str],
workspace_id: Optional[str],
branch: Optional[str],
) -> Tuple[int, int]:
"""返回 (作用域内 active 事实数, 该 user 的 active 事实总数)。
事实层的过滤在 get_facts 拿到对象之后用 _scope_matches 做,这里只要选择率。
"""
scope_sql, scope_params = _scope_where(workspace_id, branch)
if agent_id:
scope_sql = f"agent_id = ? AND ({scope_sql})"
scope_params = [agent_id, *scope_params]
if session_id:
scope_sql = f"session_id = ? AND ({scope_sql})"
scope_params = [session_id, *scope_params]
with self._lock:
assert self._conn is not None
row = self._conn.execute(
+136 -125
View File
@@ -7,7 +7,7 @@ Agent Memory V2 - SqliteCache
设计模式跟随 history_store.py:
- 标准库 sqlite3 + _run_in_sqlite_pool()
- WAL 模式 + check_same_thread=False
- threading.Lock 保护写操作
- 每实例 threading.RLock 保护共享连接的全部操作与关闭
三张表:
- memory_operations / pipeline_logs / system_metrics: 审计与观测
@@ -122,7 +122,10 @@ class SqliteCache(CacheBase):
def __init__(self, config: MemoryConfig):
self.config = config
self._conn: Optional[sqlite3.Connection] = None
self._lock = threading.Lock()
# One connection is shared by the executor workers. Use the same lock
# for reads, writes, initialization and close so close() cannot free
# the connection while another worker is inside sqlite3.
self._lock = threading.RLock()
# 读取 db_path
cache_cfg = getattr(config, "cache", None)
@@ -144,35 +147,37 @@ class SqliteCache(CacheBase):
logger.debug(f"SqliteCache initialized: {self._db_path}")
def _init_sync(self) -> None:
db_dir = os.path.dirname(self._db_path)
if db_dir:
os.makedirs(db_dir, exist_ok=True)
self._conn = sqlite3.connect(self._db_path, check_same_thread=False)
self._conn.row_factory = sqlite3.Row
self._conn.execute("PRAGMA journal_mode=WAL;")
self._conn.executescript(_CREATE_TABLES_SQL)
self._migrate_pipeline_logs_columns()
self._conn.commit()
with self._lock:
db_dir = os.path.dirname(self._db_path)
if db_dir:
os.makedirs(db_dir, exist_ok=True)
self._conn = sqlite3.connect(self._db_path, check_same_thread=False)
self._conn.row_factory = sqlite3.Row
self._conn.execute("PRAGMA journal_mode=WAL;")
self._conn.executescript(_CREATE_TABLES_SQL)
self._migrate_pipeline_logs_columns()
self._conn.commit()
def _migrate_pipeline_logs_columns(self) -> None:
"""给旧库的 pipeline_logs_v2 补 namespace/session/event time 列。"""
try:
cols = {
row["name"]
for row in self._conn.execute("PRAGMA table_info(pipeline_logs_v2)")
}
_text_cols = ("namespace", "session_id")
for col in _text_cols:
if col not in cols:
with self._lock:
try:
cols = {
row["name"]
for row in self._conn.execute("PRAGMA table_info(pipeline_logs_v2)")
}
_text_cols = ("namespace", "session_id")
for col in _text_cols:
if col not in cols:
self._conn.execute(
f"ALTER TABLE pipeline_logs_v2 ADD COLUMN {col} TEXT NOT NULL DEFAULT ''"
)
if "event_at_ms" not in cols:
self._conn.execute(
f"ALTER TABLE pipeline_logs_v2 ADD COLUMN {col} TEXT NOT NULL DEFAULT ''"
"ALTER TABLE pipeline_logs_v2 ADD COLUMN event_at_ms INTEGER NOT NULL DEFAULT 0"
)
if "event_at_ms" not in cols:
self._conn.execute(
"ALTER TABLE pipeline_logs_v2 ADD COLUMN event_at_ms INTEGER NOT NULL DEFAULT 0"
)
except Exception as e:
logger.debug(f"[sqlite] pipeline_logs_v2 column migration skipped: {e}")
except Exception as e:
logger.debug(f"[sqlite] pipeline_logs_v2 column migration skipped: {e}")
async def close(self) -> None:
if self._conn is not None:
@@ -180,9 +185,10 @@ class SqliteCache(CacheBase):
logger.info("SqliteCache closed")
def _close_sync(self) -> None:
if self._conn:
self._conn.close()
self._conn = None
with self._lock:
if self._conn:
self._conn.close()
self._conn = None
# ================================================================
# 统计
@@ -255,39 +261,40 @@ class SqliteCache(CacheBase):
"""
try:
def _query():
conditions = []
params = []
if request_id:
conditions.append("request_id = ?")
params.append(request_id)
if memory_id:
conditions.append("memory_id = ?")
params.append(memory_id)
if user_id:
conditions.append("user_id = ?")
params.append(user_id)
with self._lock:
conditions = []
params = []
if request_id:
conditions.append("request_id = ?")
params.append(request_id)
if memory_id:
conditions.append("memory_id = ?")
params.append(memory_id)
if user_id:
conditions.append("user_id = ?")
params.append(user_id)
if not conditions:
return []
if not conditions:
return []
where = " AND ".join(conditions)
rows = self._conn.execute(
f"SELECT * FROM memory_operations_v2 WHERE {where} "
f"ORDER BY id DESC LIMIT ?",
params + [limit],
).fetchall()
result = []
for r in rows:
row = dict(r)
# 反序列化 supersedes JSON 字符串 → list
import json as _json
raw_sup = row.get("supersedes", "")
try:
row["supersedes"] = _json.loads(raw_sup) if raw_sup else []
except Exception:
row["supersedes"] = []
result.append(row)
return result
where = " AND ".join(conditions)
rows = self._conn.execute(
f"SELECT * FROM memory_operations_v2 WHERE {where} "
f"ORDER BY id DESC LIMIT ?",
params + [limit],
).fetchall()
result = []
for r in rows:
row = dict(r)
# 反序列化 supersedes JSON 字符串 → list
import json as _json
raw_sup = row.get("supersedes", "")
try:
row["supersedes"] = _json.loads(raw_sup) if raw_sup else []
except Exception:
row["supersedes"] = []
result.append(row)
return result
return await _run_in_sqlite_pool(_query)
except Exception as e:
@@ -369,37 +376,38 @@ class SqliteCache(CacheBase):
"""
try:
def _query():
conditions = []
params = []
if request_id:
conditions.append("request_id = ?")
params.append(request_id)
if user_id:
conditions.append("user_id = ?")
params.append(user_id)
if step:
conditions.append("step = ?")
params.append(step)
with self._lock:
conditions = []
params = []
if request_id:
conditions.append("request_id = ?")
params.append(request_id)
if user_id:
conditions.append("user_id = ?")
params.append(user_id)
if step:
conditions.append("step = ?")
params.append(step)
if not conditions:
return []
if not conditions:
return []
where = " AND ".join(conditions)
rows = self._conn.execute(
f"SELECT * FROM pipeline_logs_v2 WHERE {where} "
f"ORDER BY id ASC LIMIT ?",
params + [limit],
).fetchall()
out = [dict(r) for r in rows]
# 派生可读的 event_at(ISO ms)供查询接口直接用;event_at_ms 为 0 时留空
from datetime import datetime as _dt
for _r in out:
_ms = _r.get("event_at_ms") or 0
_r["event_at"] = (
_dt.fromtimestamp(_ms / 1000.0).isoformat(timespec="milliseconds")
if _ms else ""
)
return out
where = " AND ".join(conditions)
rows = self._conn.execute(
f"SELECT * FROM pipeline_logs_v2 WHERE {where} "
f"ORDER BY id ASC LIMIT ?",
params + [limit],
).fetchall()
out = [dict(r) for r in rows]
# 派生可读的 event_at(ISO ms)供查询接口直接用;event_at_ms 为 0 时留空
from datetime import datetime as _dt
for _r in out:
_ms = _r.get("event_at_ms") or 0
_r["event_at"] = (
_dt.fromtimestamp(_ms / 1000.0).isoformat(timespec="milliseconds")
if _ms else ""
)
return out
return await _run_in_sqlite_pool(_query)
except Exception as e:
@@ -414,26 +422,27 @@ class SqliteCache(CacheBase):
"""存储一个分钟桶的增量指标数据(UPSERT: 同一分钟多次写入会合并)"""
try:
def _store():
now = datetime.now().isoformat()
# UPSERT: 如果同一分钟已存在,合并数据
existing = self._conn.execute(
"SELECT data FROM system_metrics WHERE minute_ts = ?",
(minute_ts,),
).fetchone()
if existing:
import json as _json
old_data = _json.loads(existing["data"])
merged = self._merge_metric_buckets(old_data, data)
self._conn.execute(
"UPDATE system_metrics SET data = ?, created_at = ? WHERE minute_ts = ?",
(_json.dumps(merged, default=str), now, minute_ts),
)
else:
self._conn.execute(
"INSERT INTO system_metrics (minute_ts, data, created_at) VALUES (?, ?, ?)",
(minute_ts, json.dumps(data, default=str), now),
)
self._conn.commit()
with self._lock:
now = datetime.now().isoformat()
# UPSERT: 如果同一分钟已存在,合并数据
existing = self._conn.execute(
"SELECT data FROM system_metrics WHERE minute_ts = ?",
(minute_ts,),
).fetchone()
if existing:
import json as _json
old_data = _json.loads(existing["data"])
merged = self._merge_metric_buckets(old_data, data)
self._conn.execute(
"UPDATE system_metrics SET data = ?, created_at = ? WHERE minute_ts = ?",
(_json.dumps(merged, default=str), now, minute_ts),
)
else:
self._conn.execute(
"INSERT INTO system_metrics (minute_ts, data, created_at) VALUES (?, ?, ?)",
(minute_ts, json.dumps(data, default=str), now),
)
self._conn.commit()
await _run_in_sqlite_pool(_store)
except Exception as e:
logger.warning(f"store_metrics_minute failed: {e}")
@@ -442,17 +451,18 @@ class SqliteCache(CacheBase):
"""读取指定时间范围内的分钟指标数据"""
try:
def _load():
rows = self._conn.execute(
"SELECT data FROM system_metrics WHERE minute_ts >= ? AND minute_ts <= ? ORDER BY minute_ts",
(start_ts, end_ts),
).fetchall()
results = []
for row in rows:
try:
results.append(json.loads(row["data"]))
except (json.JSONDecodeError, TypeError):
pass
return results
with self._lock:
rows = self._conn.execute(
"SELECT data FROM system_metrics WHERE minute_ts >= ? AND minute_ts <= ? ORDER BY minute_ts",
(start_ts, end_ts),
).fetchall()
results = []
for row in rows:
try:
results.append(json.loads(row["data"]))
except (json.JSONDecodeError, TypeError):
pass
return results
return await _run_in_sqlite_pool(_load)
except Exception as e:
logger.warning(f"load_metrics_range failed: {e}")
@@ -462,11 +472,12 @@ class SqliteCache(CacheBase):
"""删除指定时间之前的 metrics 数据"""
try:
def _cleanup():
self._conn.execute(
"DELETE FROM system_metrics WHERE minute_ts < ?",
(before_ts,),
)
self._conn.commit()
with self._lock:
self._conn.execute(
"DELETE FROM system_metrics WHERE minute_ts < ?",
(before_ts,),
)
self._conn.commit()
await _run_in_sqlite_pool(_cleanup)
except Exception as e:
logger.warning(f"cleanup_old_metrics failed: {e}")
+81 -70
View File
@@ -16,6 +16,7 @@ import json
import logging
import os
import sqlite3
import threading
from datetime import datetime, timezone
from typing import Dict, List, Any, Optional
@@ -89,6 +90,10 @@ class HistoryStore:
"""
self._db_path = config.history.db_path
self._conn: Optional[sqlite3.Connection] = None
# sqlite3.Connection.close() must never race with an execute() on the
# same connection. The lock also serializes operations on this shared
# connection; separate HistoryStore instances remain independent.
self._conn_lock = threading.RLock()
async def initialize(self) -> None:
"""创建表和索引(如果不存在)"""
@@ -97,20 +102,21 @@ class HistoryStore:
def _init_sync(self) -> None:
"""同步初始化:创建目录、连接数据库、建表建索引"""
# 确保目录存在
db_dir = os.path.dirname(self._db_path)
if db_dir:
os.makedirs(db_dir, exist_ok=True)
with self._conn_lock:
# 确保目录存在
db_dir = os.path.dirname(self._db_path)
if db_dir:
os.makedirs(db_dir, exist_ok=True)
self._conn = sqlite3.connect(self._db_path, check_same_thread=False)
self._conn.row_factory = sqlite3.Row
# 开启 WAL 模式,提高并发读写性能
self._conn.execute("PRAGMA journal_mode=WAL;")
self._conn = sqlite3.connect(self._db_path, check_same_thread=False)
self._conn.row_factory = sqlite3.Row
# 开启 WAL 模式,提高并发读写性能
self._conn.execute("PRAGMA journal_mode=WAL;")
self._conn.execute(_CREATE_TABLE_SQL)
for idx_sql in _CREATE_INDEXES_SQL:
self._conn.execute(idx_sql)
self._conn.commit()
self._conn.execute(_CREATE_TABLE_SQL)
for idx_sql in _CREATE_INDEXES_SQL:
self._conn.execute(idx_sql)
self._conn.commit()
# ================================================================
# 记录操作
@@ -218,13 +224,14 @@ class HistoryStore:
return await _run_in_sqlite_pool(self._get_history_sync, memory_id)
def _get_history_sync(self, memory_id: str) -> List[Dict[str, Any]]:
if self._conn is None:
return []
cursor = self._conn.execute(
"SELECT * FROM memory_history WHERE memory_id = ? ORDER BY created_at ASC",
(memory_id,),
)
return [dict(row) for row in cursor.fetchall()]
with self._conn_lock:
if self._conn is None:
return []
cursor = self._conn.execute(
"SELECT * FROM memory_history WHERE memory_id = ? ORDER BY created_at ASC",
(memory_id,),
)
return [dict(row) for row in cursor.fetchall()]
async def get_recent(
self,
@@ -244,14 +251,15 @@ class HistoryStore:
return await _run_in_sqlite_pool(self._get_recent_sync, isolation_key, limit)
def _get_recent_sync(self, isolation_key: str, limit: int) -> List[Dict[str, Any]]:
if self._conn is None:
return []
cursor = self._conn.execute(
"SELECT * FROM memory_history WHERE isolation_key = ? "
"ORDER BY created_at DESC LIMIT ?",
(isolation_key, limit),
)
return [dict(row) for row in cursor.fetchall()]
with self._conn_lock:
if self._conn is None:
return []
cursor = self._conn.execute(
"SELECT * FROM memory_history WHERE isolation_key = ? "
"ORDER BY created_at DESC LIMIT ?",
(isolation_key, limit),
)
return [dict(row) for row in cursor.fetchall()]
async def count(self, isolation_key: str = "") -> int:
"""
@@ -266,16 +274,17 @@ class HistoryStore:
return await _run_in_sqlite_pool(self._count_sync, isolation_key)
def _count_sync(self, isolation_key: str) -> int:
if self._conn is None:
return 0
if isolation_key:
cursor = self._conn.execute(
"SELECT COUNT(*) FROM memory_history WHERE isolation_key = ?",
(isolation_key,),
)
else:
cursor = self._conn.execute("SELECT COUNT(*) FROM memory_history")
return cursor.fetchone()[0]
with self._conn_lock:
if self._conn is None:
return 0
if isolation_key:
cursor = self._conn.execute(
"SELECT COUNT(*) FROM memory_history WHERE isolation_key = ?",
(isolation_key,),
)
else:
cursor = self._conn.execute("SELECT COUNT(*) FROM memory_history")
return cursor.fetchone()[0]
async def close(self) -> None:
"""关闭数据库连接"""
@@ -284,9 +293,10 @@ class HistoryStore:
logger.info("[HistoryStore] closed")
def _close_sync(self) -> None:
if self._conn:
self._conn.close()
self._conn = None
with self._conn_lock:
if self._conn:
self._conn.close()
self._conn = None
# ================================================================
# 内部
@@ -339,33 +349,34 @@ class HistoryStore:
role: str = "",
extra: Optional[Dict[str, Any]] = None,
) -> int:
if self._conn is None:
return -1
now = _now_iso()
extra_json = json.dumps(extra or {}, ensure_ascii=False)
cursor = self._conn.execute(
"""INSERT INTO memory_history
(memory_id, isolation_key, event,
old_memory, new_memory, old_status, new_status,
change_reason, layer, actor_id, role, extra,
created_at, updated_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)""",
(
memory_id,
isolation_key,
event,
old_memory,
new_memory,
old_status,
new_status,
change_reason,
layer,
actor_id,
role,
extra_json,
now,
now,
),
)
self._conn.commit()
return cursor.lastrowid
with self._conn_lock:
if self._conn is None:
return -1
now = _now_iso()
extra_json = json.dumps(extra or {}, ensure_ascii=False)
cursor = self._conn.execute(
"""INSERT INTO memory_history
(memory_id, isolation_key, event,
old_memory, new_memory, old_status, new_status,
change_reason, layer, actor_id, role, extra,
created_at, updated_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)""",
(
memory_id,
isolation_key,
event,
old_memory,
new_memory,
old_status,
new_status,
change_reason,
layer,
actor_id,
role,
extra_json,
now,
now,
),
)
self._conn.commit()
return cursor.lastrowid
+22 -4
View File
@@ -185,7 +185,12 @@ from one_memory.memory_queue import (
resolve_extraction_state,
)
from one_memory.memory_server.config_state import ConfigStore
from one_memory.memory_tasks import MemoryTask, MemoryTaskStatus, MemoryTaskStore
from one_memory.memory_tasks import (
NON_RETRYABLE_POST_WRITE_CODES,
MemoryTask,
MemoryTaskStatus,
MemoryTaskStore,
)
from one_memory.model_presets import seed_preset_profile
from one_memory.model_probe import ModelProber, ModelProbeResult
from one_memory.model_profiles import (
@@ -1082,8 +1087,14 @@ def create_app(
drain_budget = production_drain_budget()
async def reconcile_unknown_writes() -> None:
await app.state.memory_processing_supervisor.reconcile_unknown_outputs()
unknown = await asyncio.to_thread(memory_write_queue.unknown_jobs, limit=128)
await app.state.memory_processing_supervisor.reconcile_unknown_outputs(
exclude_error_codes=NON_RETRYABLE_POST_WRITE_CODES,
)
unknown = await asyncio.to_thread(
memory_write_queue.unknown_jobs,
limit=128,
exclude_error_codes=NON_RETRYABLE_POST_WRITE_CODES,
)
for job in unknown:
await asyncio.to_thread(
memory_write_queue.resolve_unknown_from_linked_output,
@@ -2793,6 +2804,12 @@ def create_app(
# D75: work queued for the previous model continues with this one.
# Only tasks that never reached the model are re-bound.
await asyncio.to_thread(app.state.memory_task_store.rebind_unstarted_profile, profile)
# A missing profile is an event-driven wait, not a provider outage.
# Release those durable rows only now that activation has changed the
# condition; the ordinary drain poll will claim them without a restart.
await asyncio.to_thread(
app.state.memory_write_queue.readmit_model_profile_waiters
)
# Not awaited: the catch-up reads every conversation ever imported, and
# the answer to "which profile is active" does not depend on it.
_start_backfill(app)
@@ -5062,10 +5079,11 @@ def _source_writes_on(app: FastAPI, source_id: str) -> bool:
def _sedimentation_claim_enabled(app: FastAPI, source_id: str) -> bool:
"""Claim queued model work only while both global and source gates allow it."""
"""Claim queued model work only when policy, source and model all admit it."""
return (
_config_store(app).sedimentation_enabled()
and _model_profile_store(app).active_identity() is not None
and _history_import_gate_released(app)
and _source_writes_on(app, source_id)
)
@@ -282,11 +282,24 @@ class PollingLiveSyncSupervisor:
and job.status
in {ImportStatus.PENDING, ImportStatus.RUNNING, ImportStatus.INTERRUPTED}
]
bounded_history_incomplete = any(
job.source_id == source_id
and job.operation == "history"
and job.history_after is not None
and job.status is not ImportStatus.COMPLETED
for job in source_jobs
)
# Establish the realtime EOF independently even while history owns
# this source. Otherwise the P1 job starts from the P2 checkpoint
# (or from zero) and becomes a second full-history scan.
adapter = self.import_supervisor.adapters[source_id]
await self._live_cursors(source_id, adapter)
live_cursors = await self._live_cursors(source_id, adapter)
# A bounded onboarding history is the authority for everything up
# to its EOF. An adapter without a native live baseline has no safe
# realtime cursor until that history finishes; starting a sync from
# zero here would bypass `history_after` and import the old corpus.
if bounded_history_incomplete and not live_cursors:
return None
# Only one scan of a source may be in flight: they share the
# cursor, and two readers means every file read twice. With a
# catch-up still holding the source there is no free moment to ask
@@ -1522,14 +1522,24 @@ class MemoryProcessingSupervisor:
await self._refresh_read_bridge(completed[-1].profile_snapshot)
return tuple(completed)
async def reconcile_unknown_outputs(self, *, limit: int = 128) -> int:
async def reconcile_unknown_outputs(
self,
*,
limit: int = 128,
exclude_error_codes: tuple[str, ...] = (),
) -> int:
"""Publish HY outputs that can be proven by an unknown task's request id.
This inquiry is read-only on HY and never invokes the model. Rows with
no exact match remain unknown for the person's explicit retry choice.
"""
reconciled = 0
for task in await in_the_background(self.tasks.unknown_tasks, limit=limit):
tasks = await in_the_background(
self.tasks.unknown_tasks,
limit=limit,
exclude_error_codes=exclude_error_codes,
)
for task in tasks:
runner = self.runner_factory()
bridge: MemoryBridgeProtocol | None = None
try:
@@ -38,10 +38,11 @@ from one_memory.memory_server.migrations import memory_write_jobs_sqlite_schema
from one_memory.memory_server.queue import build_reclaim_sqlite_statements
from one_memory.memory_server.registry import WriteRecord
from one_memory.memory_tasks import (
POLICY_STOPPED_CODES as _TASK_POLICY_STOPPED_CODES,
NON_RETRYABLE_POST_WRITE_CODES,
MemoryTaskStatus,
)
from one_memory.memory_tasks import (
MemoryTaskStatus,
POLICY_STOPPED_CODES as _TASK_POLICY_STOPPED_CODES,
)
from one_memory.service_levels import ServiceLevel, service_level_context
@@ -1404,11 +1405,12 @@ class DurableWriteQueue:
) -> int:
"""Re-admit `waiting_for_model` jobs whose wait has elapsed.
A model outage clears on its own, so this state is retried on a timer
A model outage clears on its own, so transient waiters retry on a timer
rather than waiting for a grant event like `waiting_for_permission`.
Keyed on `updated_at` so repeated calls while a job is still parked do
not reset its own clock; the attempt consumed by the parking claim is
returned, the same way `readmit_parked` does.
`model_profile_not_active` is excluded: only explicit activation can
change that condition. Keyed on `updated_at` so repeated calls while a
job is still parked do not reset its own clock; the attempt consumed by
the parking claim is returned, the same way `readmit_parked` does.
"""
now = now or datetime.now(UTC)
threshold = (now - after).isoformat()
@@ -1417,8 +1419,33 @@ class DurableWriteQueue:
"UPDATE memory_write_jobs SET state=?, lease_owner=NULL, "
"lease_expires_at=NULL, attempts=CASE WHEN attempts>0 "
"THEN attempts-1 ELSE 0 END, last_error=NULL, updated_at=? "
"WHERE state=? AND updated_at <= ? AND hy_invoked=0",
(PENDING, now.isoformat(), WAITING_FOR_MODEL, threshold),
"WHERE state=? AND updated_at <= ? AND hy_invoked=0 "
"AND (last_error IS NULL OR last_error<>?)",
(
PENDING,
now.isoformat(),
WAITING_FOR_MODEL,
threshold,
"model_profile_not_active",
),
)
return changed.rowcount
def readmit_model_profile_waiters(self) -> int:
"""Re-admit no-profile work only after model activation changes the condition."""
now = datetime.now(UTC).isoformat()
with main_database(self.database_path).write() as connection, connection:
changed = connection.execute(
"UPDATE memory_write_jobs SET state=?, lease_owner=NULL, "
"lease_expires_at=NULL, attempts=CASE WHEN attempts>0 "
"THEN attempts-1 ELSE 0 END, last_error=NULL, updated_at=? "
"WHERE state=? AND last_error=? AND hy_invoked=0",
(
PENDING,
now,
WAITING_FOR_MODEL,
"model_profile_not_active",
),
)
return changed.rowcount
@@ -1516,9 +1543,13 @@ class DurableWriteQueue:
connection,
):
row = connection.execute(
"SELECT state FROM memory_write_jobs WHERE job_id=?", (job_id,)
"SELECT state,last_error FROM memory_write_jobs WHERE job_id=?", (job_id,)
).fetchone()
if row is None or str(row["state"]) != UNKNOWN:
if (
row is None
or str(row["state"]) != UNKNOWN
or str(row["last_error"]) in NON_RETRYABLE_POST_WRITE_CODES
):
return 0
connection.execute(
"""
@@ -1675,8 +1706,15 @@ class DurableWriteQueue:
rows = connection.execute(
"SELECT job_id FROM memory_write_jobs "
"WHERE state=? AND attempts=1 AND attempts < max_attempts "
f"AND COALESCE(last_error, '') NOT IN "
f"({','.join('?' for _ in NON_RETRYABLE_POST_WRITE_CODES)}) "
"AND updated_at <= ? ORDER BY updated_at, job_id LIMIT ?",
(UNKNOWN, older_than.isoformat(), limit),
(
UNKNOWN,
*NON_RETRYABLE_POST_WRITE_CODES,
older_than.isoformat(),
limit,
),
).fetchall()
job_ids = tuple(str(row["job_id"]) for row in rows)
if not job_ids:
@@ -1740,6 +1778,7 @@ class DurableWriteQueue:
*,
limit: int = 128,
after_cursor: tuple[str, str] | None = None,
exclude_error_codes: tuple[str, ...] = (),
) -> tuple[UnknownWriteJob, ...]:
"""List unknown rows that still need an inquiry, in stable order.
@@ -1756,6 +1795,9 @@ class DurableWriteQueue:
"""
if type(limit) is not int or limit < 1:
raise ValueError("limit must be positive")
excluded = tuple(dict.fromkeys(exclude_error_codes))
if not all(isinstance(code, str) and code for code in excluded):
raise ValueError("exclude_error_codes must contain non-empty strings")
cursor_clause = ""
params: tuple[Any, ...] = ()
if after_cursor is not None:
@@ -1768,14 +1810,20 @@ class DurableWriteQueue:
updated_at, job_id = after_cursor
cursor_clause = "AND (updated_at, job_id) > (?, ?)"
params = (updated_at, job_id)
if excluded:
placeholders = ",".join("?" for _ in excluded)
excluded_clause = f"AND COALESCE(last_error, '') NOT IN ({placeholders})"
else:
excluded_clause = ""
with main_database(self.database_path).read(row_factory=sqlite3.Row) as connection:
rows = connection.execute(
"SELECT job_id, idempotency_key, created_at, updated_at, last_error "
"FROM memory_write_jobs "
"WHERE state=? "
f"{cursor_clause} "
f"{excluded_clause} "
"ORDER BY updated_at, job_id LIMIT ?",
(UNKNOWN, *params, limit),
(UNKNOWN, *params, *excluded, limit),
).fetchall()
return tuple(
UnknownWriteJob(
@@ -2071,7 +2119,12 @@ async def execute_job(
# Never `logger.exception` here: it prints the exception's message,
# which may be a provider response or the user's own text. Log the
# already-sanitised code instead, matching `hy-failure-classification`.
logger.error("durable_write_job_failed job_id=%s error_code=%s", job_id, code)
if code == "model_profile_not_active":
logger.debug("durable_write_job_parked job_id=%s error_code=%s", job_id, code)
elif state in PARKED_STATES:
logger.info("durable_write_job_parked job_id=%s error_code=%s", job_id, code)
else:
logger.error("durable_write_job_failed job_id=%s error_code=%s", job_id, code)
return
await in_the_background(queue.settle, job_id, state=COMPLETED, owner=owner)
@@ -67,6 +67,12 @@ POLICY_STOPPED_CODES = (
"hy_memory_evidence_changed",
)
#: HY completed a write, but Core could not prove that the selected mutation
#: target belonged to this task's frozen authority. Reconciliation may inspect
#: HY by request id, but neither automatic nor user-confirmed generic retry may
#: invoke the model again for these codes.
NON_RETRYABLE_POST_WRITE_CODES = ("hy_memory_write_verification_authority_invalid",)
class MemoryTask(BaseModel):
model_config = ConfigDict(frozen=True, extra="forbid")
@@ -116,6 +122,24 @@ PROFILE_STOP_CODES = (
ACTIVE_WINDOW_DEFER_LIMIT = 2048
def _task_only_unknown_clause(connection: sqlite3.Connection) -> str:
"""Exclude tasks whose Evidence is already owned by the durable queue."""
linked_table = connection.execute(
"SELECT 1 FROM sqlite_master WHERE type='table' AND name='memory_write_job_evidence'"
).fetchone()
if linked_table is None:
return ""
return """
AND NOT EXISTS (
SELECT 1
FROM memory_task_inputs AS inputs, json_each(inputs.refs_json) AS ref
JOIN memory_write_job_evidence AS linked
ON linked.evidence_id=json_extract(ref.value, '$.evidence_id')
WHERE inputs.task_id=memory_tasks.task_id
)
"""
@dataclass(frozen=True, slots=True)
class ActiveMemoryTask:
"""One task plus only the evidence fields the scheduler needs."""
@@ -1314,7 +1338,7 @@ class MemoryTaskStore:
live_clause = f"task.task_id IN ({live_placeholders})" if live_ids else "0"
not_live_clause = f"task.task_id NOT IN ({live_placeholders})" if live_ids else "1"
excluded_clause = f"task.task_id NOT IN ({excluded_placeholders})" if excluded else "1"
common = f"""
common_base = f"""
task.status IN (?,?,?,?)
AND coalesce(task.last_error_code, '') NOT IN (
'hy_memory_input_exceeds_window', 'model_content_outcome_unknown'
@@ -1333,7 +1357,33 @@ class MemoryTaskStore:
"""
moment = "coalesce(evidence.created_at, evidence.imported_at)"
def common_for(connection: sqlite3.Connection) -> str:
has_durable_queue = connection.execute(
"SELECT COUNT(*) FROM sqlite_master WHERE type='table' "
"AND name IN ('memory_write_jobs','memory_write_job_evidence')"
).fetchone()[0]
if int(has_durable_queue) != 2:
return common_base
return (
common_base
+ """
AND NOT EXISTS (
SELECT 1
FROM memory_task_inputs AS inputs, json_each(inputs.refs_json) AS ref
JOIN memory_write_job_evidence AS linked
ON linked.evidence_id=json_extract(ref.value, '$.evidence_id')
JOIN memory_write_jobs AS job ON job.job_id=linked.job_id
WHERE inputs.task_id=task.task_id
AND job.state IN (
'pending', 'running', 'unknown',
'waiting_for_permission', 'waiting_for_model'
)
)
"""
)
def run(connection: sqlite3.Connection, *, defer: bool) -> list[sqlite3.Row]:
common = common_for(connection)
in_window = (
"1"
if extraction_since is None
@@ -1430,6 +1480,7 @@ class MemoryTaskStore:
def has_defer_candidate(connection: sqlite3.Connection) -> bool:
if extraction_since is None:
return False
common = common_for(connection)
row = connection.execute(
f"""
SELECT 1
@@ -2146,6 +2197,7 @@ class MemoryTaskStore:
*,
limit: int = 128,
after_cursor: tuple[str, str] | None = None,
exclude_error_codes: tuple[str, ...] = (),
) -> tuple[MemoryTask, ...]:
"""List `unknown` tasks that still need an external inquiry.
@@ -2160,6 +2212,9 @@ class MemoryTaskStore:
"""
if type(limit) is not int or limit < 1:
raise ValueError("limit must be positive")
excluded = tuple(dict.fromkeys(exclude_error_codes))
if not all(isinstance(code, str) and code for code in excluded):
raise ValueError("exclude_error_codes must contain non-empty strings")
if after_cursor is not None:
if (
not isinstance(after_cursor, tuple)
@@ -2173,13 +2228,19 @@ class MemoryTaskStore:
else:
cursor_clause = ""
params = ()
if excluded:
placeholders = ",".join("?" for _ in excluded)
excluded_clause = f"AND COALESCE(last_error_code, '') NOT IN ({placeholders})"
else:
excluded_clause = ""
with self._connect() as connection:
rows = connection.execute(
"SELECT * FROM memory_tasks "
"WHERE status=? "
f"{cursor_clause} "
f"{excluded_clause} "
"ORDER BY updated_at, task_id LIMIT ?",
(MemoryTaskStatus.UNKNOWN, *params, limit),
(MemoryTaskStatus.UNKNOWN, *params, *excluded, limit),
).fetchall()
return tuple(self._to_task(row) for row in rows)
@@ -2211,20 +2272,26 @@ class MemoryTaskStore:
"""
now = datetime.now(UTC).isoformat()
with self._connect(writing=True) as connection:
task_only = _task_only_unknown_clause(connection)
blocked = ",".join("?" for _ in NON_RETRYABLE_POST_WRITE_CODES)
changed = connection.execute(
"""
f"""
UPDATE memory_tasks
SET status=?, runner_token=NULL, attempts=0,
last_error_code='model_content_manual_retry_requested', updated_at=?
WHERE task_id IN (
SELECT task_id FROM memory_tasks
WHERE status=? ORDER BY updated_at, task_id LIMIT ?
WHERE status=?
AND COALESCE(last_error_code, '') NOT IN ({blocked})
{task_only}
ORDER BY updated_at, task_id LIMIT ?
)
""",
(
MemoryTaskStatus.PENDING,
now,
MemoryTaskStatus.UNKNOWN,
*NON_RETRYABLE_POST_WRITE_CODES,
limit,
),
)
@@ -2246,14 +2313,18 @@ class MemoryTaskStore:
raise ValueError("limit must be positive")
now = datetime.now(UTC).isoformat()
with self._connect(writing=True) as connection:
task_only = _task_only_unknown_clause(connection)
blocked = ",".join("?" for _ in NON_RETRYABLE_POST_WRITE_CODES)
changed = connection.execute(
"""
f"""
UPDATE memory_tasks
SET status=?, runner_token=NULL,
last_error_code='model_content_unknown_auto_retry', updated_at=?
WHERE task_id IN (
SELECT task_id FROM memory_tasks
WHERE status=? AND attempts=1 AND updated_at <= ?
AND COALESCE(last_error_code, '') NOT IN ({blocked})
{task_only}
ORDER BY updated_at, task_id LIMIT ?
)
""",
@@ -2262,6 +2333,7 @@ class MemoryTaskStore:
now,
MemoryTaskStatus.UNKNOWN,
older_than.isoformat(),
*NON_RETRYABLE_POST_WRITE_CODES,
limit,
),
)
@@ -200,10 +200,12 @@ def _trigger_sql() -> list[tuple[str, str]]:
CREATE TRIGGER IF NOT EXISTS memory_visibility_dirty_on_link_update
AFTER UPDATE ON governed_memory_evidence
BEGIN
INSERT OR IGNORE INTO memory_visibility_dirty (memory_id)
VALUES (NEW.memory_id);
INSERT OR IGNORE INTO memory_visibility_dirty (memory_id)
VALUES (OLD.memory_id);
INSERT INTO memory_visibility_dirty (memory_id)
VALUES (NEW.memory_id)
ON CONFLICT(memory_id) DO NOTHING;
INSERT INTO memory_visibility_dirty (memory_id)
VALUES (OLD.memory_id)
ON CONFLICT(memory_id) DO NOTHING;
END;""",
),
(
@@ -294,6 +296,10 @@ def install_triggers(db: sqlite3.Connection) -> None:
str(row["name"])
for row in cursor.execute("SELECT name FROM sqlite_master WHERE type='table'")
}
# CREATE TRIGGER IF NOT EXISTS cannot repair the legacy definition. Replace
# this derived-cache trigger transactionally on every startup so old
# libraries get the row-level conflict handling without a schema migration.
cursor.execute("DROP TRIGGER IF EXISTS memory_visibility_dirty_on_link_update")
for table, statement in _trigger_sql():
if table not in present:
continue
@@ -303,11 +309,15 @@ def install_triggers(db: sqlite3.Connection) -> None:
def startup(database_path: Path) -> None:
"""Idempotent: create the derived tables and triggers. No version change.
Runs through the write connection because the triggers are schema. It is
one short transaction and happens once per start.
Runs through the write connection because the triggers are schema. The
trigger replacement is one short transaction and happens once per start.
"""
with main_database(database_path).write() as db:
with main_database(database_path).write() as db, db:
ensure_tables(db)
# ensure_tables uses executescript(), which commits any pending legacy
# transaction. Begin only afterward so replacing the installed trigger
# is one atomic schema change and a failed CREATE restores the old one.
db.execute("BEGIN IMMEDIATE")
install_triggers(db)
+111
View File
@@ -94,6 +94,67 @@ class SettledFixtureAdapter(IncrementalFixtureAdapter):
}
class BoundedOnboardingAdapter:
"""No native baseline: bounded history must establish the live cursor."""
descriptor = IncrementalFixtureAdapter.descriptor
def __init__(self) -> None:
self.include_new = False
async def detect(self) -> bool:
return True
async def scan(self, options: ScanOptions):
current = options.cursors.get("e" * 64)
checkpoint = current.checkpoint if current is not None else None
if checkpoint not in {"byte:10", "byte:20"}:
yield ScannedEvidence(
evidence=EvidenceInput(
source_id="fixture",
conversation_id="old-conversation",
message_id="old-message",
role="user",
written_by=WrittenBy.PERSON,
created_at=datetime(2020, 1, 1, tzinfo=UTC),
verbatim="old history",
origin_meta={EvidenceOrigin.FILE: OriginMetadata()},
),
cursor=SourceCursor(
file_key="e" * 64,
file_identity="bounded",
checkpoint="byte:10",
checkpoint_anchor="f" * 64,
observed_size=20,
observed_modified_ns=1,
),
)
checkpoint = "byte:10"
if self.include_new and checkpoint != "byte:20":
yield ScannedEvidence(
evidence=EvidenceInput(
source_id="fixture",
conversation_id="new-conversation",
message_id="new-message",
role="user",
written_by=WrittenBy.PERSON,
created_at=datetime(2026, 10, 18, tzinfo=UTC),
verbatim="new append",
origin_meta={EvidenceOrigin.FILE: OriginMetadata()},
),
cursor=SourceCursor(
file_key="e" * 64,
file_identity="bounded",
checkpoint="byte:20",
checkpoint_anchor="1" * 64,
observed_size=20,
observed_modified_ns=2,
read_to_end=True,
),
)
yield ScanProgressEvent(phase=ScanPhase.DONE)
class HoldingHistoryAdapter:
"""First scan blocks until cancelled; later scans yield one new record."""
@@ -328,6 +389,56 @@ def test_live_sync_does_not_resume_a_history_import_the_user_paused(tmp_path) ->
asyncio.run(scenario())
def test_bounded_onboarding_history_establishes_cursor_before_first_sync(tmp_path) -> None:
async def scenario() -> None:
database = tmp_path / "one-memory.sqlite3"
evidence = EvidenceStore(database)
jobs = ImportJobStore(database)
adapter = BoundedOnboardingAdapter()
imports = ImportSupervisor(
adapters={"fixture": adapter},
evidence_store=evidence,
job_store=jobs,
)
live_sync = PollingLiveSyncSupervisor(
source_ids=("fixture",),
import_supervisor=imports,
job_store=jobs,
)
history = imports.create_job("fixture", operation="history")
with sqlite3.connect(database) as connection, connection:
connection.execute(
"UPDATE import_jobs SET history_after=? WHERE job_id=?",
(datetime(2026, 10, 15, tzinfo=UTC).isoformat(), history.job_id),
)
# With no native baseline, sync must wait instead of scanning from zero
# and bypassing the onboarding history window.
assert await live_sync.trigger("fixture") is None
assert not [job for job in jobs.list_jobs() if job.operation == "sync"]
imports.enqueue(history.job_id)
completed = await imports.wait(history.job_id)
assert completed.status is ImportStatus.COMPLETED
assert completed.stored == 0
assert completed.skipped == 1
assert evidence.count() == 0
# Once bounded history has advanced the cursor to EOF, normal realtime
# appends are admitted without retaining the three-day cutoff forever.
adapter.include_new = True
synced = await live_sync.trigger("fixture")
assert synced is not None
assert synced.status is ImportStatus.COMPLETED
assert synced.stored == 1
records = evidence.list_extractable()
assert [record.verbatim for record in records] == ["new append"]
await live_sync.shutdown()
await imports.shutdown()
asyncio.run(scenario())
def test_a_sync_left_running_after_its_worker_died_is_started_again(tmp_path) -> None:
"""A running row with no live worker used to be returned as 'already syncing'.
@@ -4,7 +4,7 @@ import logging
import sqlite3
import threading
import time
from datetime import UTC, datetime
from datetime import UTC, datetime, timedelta
import pytest
@@ -56,7 +56,12 @@ from one_memory.memory_processing import (
)
from one_memory.memory_queue import classify
from one_memory.memory_server.config_state import ConfigStore
from one_memory.memory_tasks import MemoryTaskKind, MemoryTaskStatus, MemoryTaskStore
from one_memory.memory_tasks import (
NON_RETRYABLE_POST_WRITE_CODES,
MemoryTaskKind,
MemoryTaskStatus,
MemoryTaskStore,
)
from one_memory.model_profiles import (
ModelCapabilities,
ModelProfileInput,
@@ -2613,6 +2618,123 @@ def test_an_unknown_task_without_inputs_does_not_end_the_reconcile_pass(tmp_path
assert [memory.memory_id for memory in governance.list(user_id="local")] == ["hy-recovered"]
def test_sticky_authority_unknowns_do_not_starve_ordinary_reconciliation(tmp_path) -> None:
stores = _setup(tmp_path)
evidence, profiles, tasks, governance, secrets, profile = stores
record = _put(
evidence,
message_id="fair-unknown",
role="user",
written_by=WrittenBy.PERSON,
content="ordinary unknown remains reconcilable",
)
sticky = [
tasks.create(
request_id=f"sticky-authority-{index}",
user_id=record.user_id,
kind=MemoryTaskKind.EXTRACT,
input_ref=record.evidence_id,
profile=profile,
input_records=(record,),
)
for index in range(129)
]
ordinary = tasks.create(
request_id="ordinary-unknown-after-sticky",
user_id=record.user_id,
kind=MemoryTaskKind.EXTRACT,
input_ref=record.evidence_id,
profile=profile,
input_records=(record,),
)
anchor = datetime(2026, 1, 1, tzinfo=UTC)
with sqlite3.connect(tasks.database_path) as connection, connection:
connection.executescript(
"""
CREATE TABLE source_grants (
user_id TEXT NOT NULL, source_id TEXT NOT NULL,
root_path TEXT NOT NULL, candidate_kind TEXT NOT NULL,
granted_at TEXT NOT NULL, revoked_at TEXT,
PRIMARY KEY (user_id, source_id)
);
"""
)
connection.execute(
"INSERT INTO source_grants VALUES (?,?,?,?,?,NULL)",
(
record.user_id,
record.source_id,
"/authorized",
"default",
datetime.now(UTC).isoformat(),
),
)
for index, task in enumerate(sticky):
connection.execute(
"UPDATE memory_tasks SET status='unknown',last_error_code=?,updated_at=? "
"WHERE task_id=?",
(
"hy_memory_write_verification_authority_invalid",
(anchor + timedelta(seconds=index)).isoformat(),
task.task_id,
),
)
connection.execute(
"UPDATE memory_tasks SET status='unknown',last_error_code=?,updated_at=? "
"WHERE task_id=?",
(
"model_content_outcome_unknown",
(anchor + timedelta(minutes=3)).isoformat(),
ordinary.task_id,
),
)
class InquiryOnlyBridge(FakeBridge):
def __init__(self) -> None:
super().__init__(
outputs=(
HyMemoryOutput(
memory_id="ordinary-recovered",
content="ordinary unknown remains reconcilable",
layer="L2_FACT",
owner="user",
evidence_ids=(record.evidence_id,),
),
)
)
self.inquiries: list[str] = []
async def outputs_for_request(self, **kwargs) -> tuple[HyMemoryOutput, ...]:
self.inquiries.append(str(kwargs["request_id"]))
return self.outputs or ()
bridge = InquiryOnlyBridge()
supervisor = MemoryProcessingSupervisor(
tasks=tasks,
profiles=profiles,
secrets=secrets,
evidence=evidence,
governance=governance,
data_dir=tmp_path,
runner_factory=lambda: _runner(tmp_path, stores, FakeBridge()),
bridge_factory=lambda _profile, _secret, _data_dir: bridge,
)
assert (
asyncio.run(
supervisor.reconcile_unknown_outputs(
limit=128,
exclude_error_codes=NON_RETRYABLE_POST_WRITE_CODES,
)
)
== 1
)
assert bridge.inquiries == ["ordinary-unknown-after-sticky"]
assert bridge.requests == []
assert tasks.require(ordinary.task_id).status is MemoryTaskStatus.COMPLETED
assert all(tasks.require(task.task_id).status is MemoryTaskStatus.UNKNOWN for task in sticky)
def test_two_workspaces_with_one_conversation_id_prepare_apart(tmp_path) -> None:
"""Cursor can reuse a conversation id across projects.
@@ -3441,6 +3563,58 @@ def test_digest_scopes_separate_applications_and_dates(tmp_path) -> None:
assert len({_hy_agent_id((record,)) for record in records}) == 3
def test_workspace_requests_carry_conversation_authority_beyond_agent_scope(tmp_path) -> None:
"""Same-day workspace scopes collide by design; session_id must separate writes."""
evidence, _profiles, tasks, _governance, _secrets, profile = _setup(tmp_path)
first = _put(
evidence,
message_id="conversation-a",
role="user",
written_by=WrittenBy.PERSON,
content="same workspace, first conversation",
conversation_id="conversation-a",
workspace="repo-a",
created_at=datetime(2026, 10, 18, 8, tzinfo=UTC),
)
second = _put(
evidence,
message_id="conversation-b",
role="user",
written_by=WrittenBy.PERSON,
content="same workspace, second conversation",
conversation_id="conversation-b",
workspace="repo-a",
created_at=datetime(2026, 10, 18, 9, tzinfo=UTC),
)
next_day = _put(
evidence,
message_id="conversation-a-next-day",
role="user",
written_by=WrittenBy.PERSON,
content="same conversation on the next day",
conversation_id="conversation-a",
workspace="repo-a",
created_at=datetime(2026, 10, 19, 8, tzinfo=UTC),
)
requests = []
for record in (first, second, next_day):
task = tasks.create(
request_id=f"authority:{record.evidence_id}",
user_id=record.user_id,
kind=MemoryTaskKind.EXTRACT,
input_ref=record.evidence_id,
profile=profile,
input_records=(record,),
)
requests.append(MemoryTaskRunner._request(task, (record,)))
assert requests[0].agent_id == requests[1].agent_id
assert requests[0].session_id == "conversation-a"
assert requests[1].session_id == "conversation-b"
assert requests[2].agent_id != requests[0].agent_id
def test_cancelled_multi_source_lock_acquisition_releases_earlier_source(tmp_path) -> None:
evidence, profiles, tasks, governance, secrets, _profile = _setup(tmp_path)
first = _put(
@@ -5,6 +5,7 @@ from __future__ import annotations
import asyncio
import contextlib
import json
import logging
import sqlite3
import threading
import time
@@ -1369,24 +1370,30 @@ def test_a_machine_error_code_is_preserved(tmp_path) -> None:
assert safe_error_code(ValueError("some sentence with spaces")) == "ValueError"
def test_an_inactive_profile_parks_for_model_before_any_send(tmp_path) -> None:
def test_an_inactive_profile_parks_without_error_logging(tmp_path, caplog) -> None:
"""A pre-bridge model-not-ready code parks and keeps the intent."""
path = tmp_path / "one-memory.sqlite3"
queue = DurableWriteQueue(path)
record = queue.submit(_request())
_claim_through_drain(queue, record.job_id)
asyncio.run(
execute_job(
record.job_id,
queue=queue,
supervisor=_Recorder(error=RuntimeError("model_profile_not_active")),
evidence=_EvidenceStore({"e1"}),
with caplog.at_level(
logging.DEBUG,
logger="one_memory.memory_server.durable_queue",
):
asyncio.run(
execute_job(
record.job_id,
queue=queue,
supervisor=_Recorder(error=RuntimeError("model_profile_not_active")),
evidence=_EvidenceStore({"e1"}),
)
)
)
assert _state(path, record.job_id) == WAITING_FOR_MODEL
assert _rows(path)[0]["hy_invoked"] == 0
assert "durable_write_job_failed" not in caplog.text
assert "durable_write_job_parked" in caplog.text
def test_an_authorization_stop_cancels_instead_of_failing(tmp_path) -> None:
@@ -1546,7 +1553,7 @@ def test_parking_does_not_burn_the_retry_budget(tmp_path) -> None:
assert queue.get("idem-1").state == PENDING
def test_a_model_waiter_is_readmitted_once_its_wait_elapses(tmp_path) -> None:
def test_a_transient_model_waiter_is_readmitted_once_its_wait_elapses(tmp_path) -> None:
"""A model outage clears on its own; nothing re-admits it otherwise.
`waiting_for_permission` needs a grant event, but `waiting_for_model` is
@@ -1564,7 +1571,7 @@ def test_a_model_waiter_is_readmitted_once_its_wait_elapses(tmp_path) -> None:
queue.settle(
record.job_id,
state=WAITING_FOR_MODEL,
error_code="model_profile_not_active",
error_code="model_secret_unavailable",
hy_invoked=False,
)
@@ -1581,8 +1588,8 @@ def test_a_model_waiter_is_readmitted_once_its_wait_elapses(tmp_path) -> None:
assert _rows(path)[0]["attempts"] == 0
def test_readmitting_a_model_waiter_clears_its_stale_detail(tmp_path) -> None:
"""The timer readmission path must clear the old blocking reason too."""
def test_no_profile_waiter_is_readmitted_only_after_activation(tmp_path) -> None:
"""No timer can create a model profile; activation owns this transition."""
path = tmp_path / "one-memory.sqlite3"
queue = DurableWriteQueue(path)
record = queue.submit(_request())
@@ -1594,13 +1601,50 @@ def test_readmitting_a_model_waiter_clears_its_stale_detail(tmp_path) -> None:
)
now = datetime.now(UTC)
assert queue.readmit_model_waiters(now=now + timedelta(minutes=2)) == 1
assert queue.readmit_model_waiters(now=now + timedelta(minutes=2)) == 0
parked = queue.get("idem-1")
assert parked.state == WAITING_FOR_MODEL
assert parked.detail == "model_profile_not_active"
assert queue.readmit_model_profile_waiters() == 1
readmitted = queue.get("idem-1")
assert readmitted.state == PENDING
assert readmitted.detail == ""
def test_no_profile_backlog_stays_parked_across_maintenance_cycles(tmp_path) -> None:
path = tmp_path / "one-memory.sqlite3"
queue = DurableWriteQueue(path)
for index in range(100):
record = queue.submit(_request(key=f"no-profile-{index}"))
queue.settle(
record.job_id,
state=WAITING_FOR_MODEL,
error_code="model_profile_not_active",
hy_invoked=False,
)
with sqlite3.connect(path) as connection, connection:
connection.execute(
"UPDATE memory_write_jobs SET attempts=1,"
"updated_at='2000-01-01T00:00:00+00:00'"
)
first = datetime.now(UTC)
assert queue.readmit_model_waiters(now=first, after=timedelta(0)) == 0
assert (
queue.readmit_model_waiters(
now=first + timedelta(minutes=2),
after=timedelta(0),
)
== 0
)
with sqlite3.connect(path) as connection:
assert connection.execute(
"SELECT state,attempts,COUNT(*) FROM memory_write_jobs "
"GROUP BY state,attempts"
).fetchall() == [(WAITING_FOR_MODEL, 1, 100)]
def test_readmit_parked_cannot_resurrect_a_terminal_row(tmp_path) -> None:
"""Only parked states may be re-admitted, never a finished write.
@@ -3472,6 +3516,41 @@ def test_unknown_jobs_does_not_resolve_or_make_a_row_claimable(tmp_path) -> None
assert queue.resolve_unknown(found[0].job_id, found=False, detail="not in HY") == DEAD
def test_unknown_jobs_can_exclude_sticky_authority_failures_before_limit(tmp_path) -> None:
path = tmp_path / "one-memory.sqlite3"
queue = DurableWriteQueue(path)
sticky = [
queue.submit(_request(key=f"sticky-{index}", evidence=(f"sticky-e{index}",)))
for index in range(129)
]
ordinary = queue.submit(_request(key="ordinary", evidence=("ordinary-e",)))
anchor = datetime(2026, 1, 1, tzinfo=UTC)
with sqlite3.connect(path) as connection, connection:
for index, record in enumerate(sticky):
connection.execute(
"UPDATE memory_write_jobs SET state='unknown',attempts=1,hy_invoked=1,"
"last_error=?,updated_at=? WHERE job_id=?",
(
"hy_memory_write_verification_authority_invalid",
(anchor + timedelta(seconds=index)).isoformat(),
record.job_id,
),
)
connection.execute(
"UPDATE memory_write_jobs SET state='unknown',attempts=1,hy_invoked=1,"
"last_error='model_content_outcome_unknown',updated_at=? WHERE job_id=?",
((anchor + timedelta(minutes=3)).isoformat(), ordinary.job_id),
)
found = queue.unknown_jobs(
limit=128,
exclude_error_codes=("hy_memory_write_verification_authority_invalid",),
)
assert [job.job_id for job in found] == [ordinary.job_id]
assert all(queue.get(record.request_id).state == UNKNOWN for record in sticky)
def test_unknown_jobs_tolerates_a_row_resolved_between_pages(tmp_path) -> None:
"""Discovery is read-only and may overlap an inquiry from another worker.
@@ -4852,15 +4931,16 @@ def test_a_failure_after_the_call_still_parks_as_unknown(tmp_path) -> None:
record = queue.submit(_request(key="after-call", source="opencode"))
claimer = DurableWriteDrain(queue, execute=_never_execute, owner="after-call-owner")
assert claimer.claim(limit=1) == (record.job_id,)
recorder = _Recorder(
error=HyMemoryBridgeError("hy_memory_write_verification_authority_invalid"),
invoke_hy=True,
)
asyncio.run(
execute_job(
record.job_id,
queue=queue,
supervisor=_Recorder(
error=HyMemoryBridgeError("hy_memory_write_verification_authority_invalid"),
invoke_hy=True,
),
supervisor=recorder,
evidence=_EvidenceStore({"e1"}),
owner="after-call-owner",
)
@@ -4874,6 +4954,16 @@ def test_a_failure_after_the_call_still_parks_as_unknown(tmp_path) -> None:
assert state == UNKNOWN
assert invoked == 1
assert len(recorder.seen) == 1
assert (
queue.retry_first_unknown_after_reconcile(
older_than=datetime.now(UTC) + timedelta(minutes=1)
)
== 0
)
assert queue.retry_unknown_confirmed(record.job_id) == 0
assert DurableWriteDrain(queue, execute=_never_execute).claim(limit=1) == ()
assert len(recorder.seen) == 1
def test_event_time_not_enqueue_order_drives_the_window(
+103
View File
@@ -1113,6 +1113,109 @@ def test_task_only_unknown_is_automatically_retried_only_once(tmp_path) -> None:
assert store.require(task.task_id).status is MemoryTaskStatus.UNKNOWN
def test_authority_invalid_task_is_never_reinvoked(tmp_path) -> None:
store, profile = _profile_and_tasks(tmp_path)
task = store.create(
request_id="authority-invalid",
user_id="local",
kind=MemoryTaskKind.EXTRACT,
input_ref="authority-evidence",
profile=profile,
)
with sqlite3.connect(store.database_path) as connection, connection:
connection.execute(
"UPDATE memory_tasks SET status=?,attempts=1,last_error_code=?,updated_at=? "
"WHERE task_id=?",
(
MemoryTaskStatus.UNKNOWN,
"hy_memory_write_verification_authority_invalid",
"2026-01-01T00:00:00+00:00",
task.task_id,
),
)
cutoff = datetime(2026, 1, 1, 0, 5, tzinfo=UTC)
assert store.retry_first_unknown_after_reconcile(older_than=cutoff) == 0
assert store.retry_unknown_confirmed() == 0
retained = store.require(task.task_id)
assert retained.status is MemoryTaskStatus.UNKNOWN
assert retained.attempts == 1
assert retained.last_error_code == "hy_memory_write_verification_authority_invalid"
def test_linked_unknown_is_retried_only_by_the_durable_job(tmp_path) -> None:
from one_memory.memory_server.contracts import BatchIdentity, WriteRequest
from one_memory.memory_server.durable_queue import DurableWriteQueue
store, profile = _profile_and_tasks(tmp_path)
evidence = EvidenceStore(store.database_path)
record = evidence.put(
EvidenceInput(
source_id="cursor",
conversation_id="linked-conversation",
message_id="linked-message",
role="user",
written_by=WrittenBy.PERSON,
verbatim="linked unknown",
created_at=datetime(2026, 1, 1, tzinfo=UTC),
origin_meta={EvidenceOrigin.FILE: OriginMetadata()},
)
).record
task = store.create(
request_id="linked-task",
user_id="local",
kind=MemoryTaskKind.EXTRACT,
input_ref=record.evidence_id,
profile=profile,
input_records=(record,),
)
queue = DurableWriteQueue(store.database_path, evidence=evidence)
job = queue.submit(
WriteRequest(
internal_session_id="linked-session",
batch_identity=BatchIdentity(
internal_session_id="linked-session",
first_round=1,
last_round=1,
),
evidence_ids=[record.evidence_id],
idempotency_key="linked-job",
source_id="cursor",
)
)
with sqlite3.connect(store.database_path) as connection, connection:
connection.execute(
"UPDATE memory_tasks SET status=?,attempts=1,last_error_code=?,updated_at=? "
"WHERE task_id=?",
(
MemoryTaskStatus.UNKNOWN,
"model_content_outcome_unknown",
"2026-01-01T00:00:00+00:00",
task.task_id,
),
)
connection.execute(
"UPDATE memory_write_jobs SET state='unknown',attempts=1,hy_invoked=1,"
"last_error='model_content_outcome_unknown',updated_at=? WHERE job_id=?",
("2026-01-01T00:00:00+00:00", job.job_id),
)
cutoff = datetime(2026, 1, 1, 0, 5, tzinfo=UTC)
assert store.retry_first_unknown_after_reconcile(older_than=cutoff) == 0
assert store.require(task.task_id).status is MemoryTaskStatus.UNKNOWN
assert queue.retry_first_unknown_after_reconcile(older_than=cutoff) == 1
assert store.require(task.task_id).status is MemoryTaskStatus.PENDING
assert queue.get(job.request_id).state == "pending"
assert (
store.active_window(
now=datetime(2026, 1, 2, tzinfo=UTC),
fresh_window=timedelta(days=1),
extraction_window=None,
)
== []
)
def test_orphan_recovery_preserves_active_running_tasks(tmp_path) -> None:
store, profile = _profile_and_tasks(tmp_path)
active = store.create(
@@ -8,7 +8,14 @@ import pytest
from one_memory.imports.grants import GRANT_SCHEMA
from one_memory.imports.models import EvidenceInput, EvidenceOrigin, OriginMetadata, WrittenBy
from one_memory.imports.storage import EvidenceStore
from one_memory.memory_governance import MemoryDraft, MemoryGovernanceStore, MemoryKind
from one_memory.memory_governance import (
HyCreate,
HyLink,
HyRevise,
MemoryDraft,
MemoryGovernanceStore,
MemoryKind,
)
from one_memory.memory_tasks import (
MemoryTaskOutput,
MemoryTaskStore,
@@ -19,6 +26,7 @@ from one_memory.memory_visibility import (
install_triggers,
pending,
recompute,
startup,
)
from one_memory.model_profiles import (
ModelCapabilities,
@@ -336,6 +344,197 @@ def test_starting_the_app_creates_the_derived_tables(tmp_path):
asyncio.run(scenario())
@pytest.mark.parametrize("visibility_mode", ["exact", "derived"])
def test_hy_revise_and_repeated_link_reuse_the_same_evidence(library, monkeypatch, visibility_mode):
"""The outer evidence UPSERT must not override the trigger's duplicate handling."""
database, evidence, _tasks, governance, _profile = library
monkeypatch.setenv("ONE_MEMORY_READ_VISIBILITY", visibility_mode)
startup(database)
record = evidence.put(
EvidenceInput(
source_id="cursor",
conversation_id="trigger-regression",
message_id="trigger-regression",
correlation_id="trigger-regression",
role="user",
written_by=WrittenBy.PERSON,
verbatim="用户住在杭州。",
origin_meta={EvidenceOrigin.FILE: OriginMetadata()},
)
).record
created = governance.write_from_hy(
HyCreate(
draft=MemoryDraft(
user_id="local",
kind=MemoryKind.CHAT,
content="用户住在杭州。",
scopes=("all-apps",),
evidence_ids=(record.evidence_id,),
),
request_id="trigger-create",
memory_id="trigger-memory",
),
user_id="local",
derivation=governance.hy_derivation((record.evidence_id,)),
)
revision = HyRevise(
memory_id=created.memory_id,
request_id="trigger-revise",
expected_revision=created.revision,
content="用户住在杭州并在那里工作。",
evidence_ids=(record.evidence_id,),
)
revised = governance.write_from_hy(
revision,
user_id="local",
derivation=governance.hy_derivation((record.evidence_id,)),
)
replayed = governance.write_from_hy(
revision,
user_id="local",
derivation=governance.hy_derivation((record.evidence_id,)),
)
assert replayed == revised
link = HyLink(memory_id=created.memory_id, evidence_id=record.evidence_id)
for _ in range(2):
governance.write_from_hy(
link,
user_id="local",
derivation=governance.hy_derivation((record.evidence_id,)),
)
current = governance.get(created.memory_id, user_id="local")
assert current is not None
assert current.content == "用户住在杭州并在那里工作。"
assert current.revision == created.revision + 1
with main_database_read(database) as db:
assert db.execute(
"SELECT memory_id, COUNT(*) FROM memory_visibility_dirty GROUP BY memory_id"
).fetchall() == [(created.memory_id, 1)]
if visibility_mode == "derived":
monkeypatch.setenv("ONE_MEMORY_READ_VISIBILITY", "exact")
exact = {memory.memory_id for memory in SourceReadAccess(database).filter([current])}
with main_database_write(database) as db:
assert recompute(db, [created.memory_id], readable_sql=_MEMORY_READABLE) == 1
db.commit()
monkeypatch.setenv("ONE_MEMORY_READ_VISIBILITY", "derived")
derived = {memory.memory_id for memory in SourceReadAccess(database).filter([current])}
assert derived == exact
with main_database_read(database) as db:
assert created.memory_id not in pending(db)
def test_startup_atomically_replaces_the_legacy_link_update_trigger(tmp_path, monkeypatch):
import one_memory.memory_visibility as visibility_module
from one_memory.schema_gate import guard_schema_version
database = tmp_path / "one-memory.sqlite3"
expected_version = guard_schema_version(database)
evidence = EvidenceStore(database)
governance = MemoryGovernanceStore(database)
record = evidence.put(
EvidenceInput(
source_id="cursor",
conversation_id="legacy-trigger",
message_id="legacy-trigger",
correlation_id="legacy-trigger",
role="user",
written_by=WrittenBy.PERSON,
verbatim="升级失败不能改变这条原文。",
origin_meta={EvidenceOrigin.FILE: OriginMetadata()},
)
).record
memory = as_person(governance).create(
MemoryDraft(
user_id="local",
kind=MemoryKind.CHAT,
content="升级失败不能改变这条记忆。",
scopes=("all-apps",),
evidence_ids=(record.evidence_id,),
),
request_id="legacy-trigger-memory",
)
startup(database)
with sqlite3.connect(database) as db:
db.execute("DROP TRIGGER memory_visibility_dirty_on_link_update")
db.executescript(
"""
CREATE TRIGGER memory_visibility_dirty_on_link_update
AFTER UPDATE ON governed_memory_evidence
BEGIN
INSERT OR IGNORE INTO memory_visibility_dirty (memory_id)
VALUES (NEW.memory_id);
INSERT OR IGNORE INTO memory_visibility_dirty (memory_id)
VALUES (OLD.memory_id);
END;
"""
)
db.execute(
"INSERT INTO memory_visibility (memory_id, visible, reason, computed_at) "
"VALUES (?, 1, 'ok', '2026-10-19T00:00:00+00:00')",
(memory.memory_id,),
)
db.execute(
"INSERT INTO memory_visibility_dirty (memory_id) VALUES (?)",
(memory.memory_id,),
)
legacy_trigger = db.execute(
"SELECT sql FROM sqlite_master "
"WHERE type='trigger' AND name='memory_visibility_dirty_on_link_update'"
).fetchone()[0]
before = tuple(db.iterdump())
real_trigger_sql = visibility_module._trigger_sql
def fail_after_drop() -> list[tuple[str, str]]:
statements = real_trigger_sql()
return [
(table, "CREATE TRIGGER")
if "CREATE TRIGGER IF NOT EXISTS memory_visibility_dirty_on_link_update" in statement
else (table, statement)
for table, statement in statements
]
with monkeypatch.context() as failure:
failure.setattr(visibility_module, "_trigger_sql", fail_after_drop)
with pytest.raises(sqlite3.OperationalError):
startup(database)
with sqlite3.connect(database) as db:
assert tuple(db.iterdump()) == before
assert (
db.execute(
"SELECT sql FROM sqlite_master "
"WHERE type='trigger' AND name='memory_visibility_dirty_on_link_update'"
).fetchone()[0]
== legacy_trigger
)
assert db.execute("SELECT MAX(version) FROM schema_migrations").fetchone()[0] == (
expected_version
)
startup(database)
startup(database)
with sqlite3.connect(database) as db:
trigger_sql = db.execute(
"SELECT sql FROM sqlite_master "
"WHERE type='trigger' AND name='memory_visibility_dirty_on_link_update'"
).fetchone()[0]
assert trigger_sql.count("ON CONFLICT(memory_id) DO NOTHING") == 2
assert "INSERT OR IGNORE" not in trigger_sql
assert db.execute("SELECT MAX(version) FROM schema_migrations").fetchone()[0] == (
expected_version
)
assert db.execute("SELECT * FROM memory_visibility").fetchall() == [
(memory.memory_id, 1, "ok", "2026-10-19T00:00:00+00:00")
]
assert db.execute("SELECT * FROM memory_visibility_dirty").fetchall() == [
(memory.memory_id,)
]
def test_the_audit_notices_a_conclusion_that_no_longer_holds(library):
"""The ongoing proof: recompute a slice and compare with what is stored.
@@ -259,12 +259,17 @@ def test_activation_survives_a_corpus_already_imported(tmp_path) -> None:
),
secrets=secrets,
)
readmissions: list[bool] = []
app.state.memory_write_queue.readmit_model_profile_waiters = (
lambda: readmissions.append(True) or 0
)
activated = await client.post(f"/api/v1/model-profiles/{profile_id}/activate")
assert activated.status_code == 200
assert activated.json()["is_active"] is True
assert API_KEY not in activated.text
assert readmissions == [True]
# Started, not waited for: the walk reads every conversation.
assert not app.state.memory_backfill.done()
@@ -111,6 +111,9 @@ def test_the_manual_source_is_admitted_by_grant_without_a_toggle(tmp_path) -> No
app = SimpleNamespace(
state=SimpleNamespace(
config_store=config,
model_profile_store=SimpleNamespace(
active_identity=lambda: ("profile", 1)
),
source_grant_store=SimpleNamespace(
receipt=lambda **_kwargs: SimpleNamespace(revoked_at=None)
),
@@ -125,6 +128,21 @@ def test_the_manual_source_is_admitted_by_grant_without_a_toggle(tmp_path) -> No
assert _sedimentation_claim_enabled(app, "one-memory-manual") is False
def test_claim_gate_waits_without_polling_when_no_model_is_active(tmp_path) -> None:
config = ConfigStore(tmp_path)
app = SimpleNamespace(
state=SimpleNamespace(
config_store=config,
model_profile_store=SimpleNamespace(active_identity=lambda: None),
source_grant_store=SimpleNamespace(
receipt=lambda **_kwargs: SimpleNamespace(revoked_at=None)
),
)
)
assert _sedimentation_claim_enabled(app, "one-memory-manual") is False
def test_first_import_gate_waits_for_every_selected_history_job() -> None:
state = SimpleNamespace(
status="importing",
@@ -174,20 +174,20 @@ it("continues to step3 while sample GET is pending without calling completion AP
expect(screen.queryByText(/^已生成.*记忆/)).toBeNull();
});
it.each([
"pending",
"paused",
"interrupted",
"failed",
"cancelled",
"completed",
["pending", "已扫描 100 条,正在筛选"],
["paused", "已扫描 100 条,尚未保存"],
["interrupted", "已扫描 100 条,尚未保存"],
["failed", "已扫描 100 条,尚未保存"],
["cancelled", "已扫描 100 条,尚未保存"],
["completed", "已扫描 100 条,本次没有可保存的对话"],
] as const)(
"shows %s without mutating jobs or fetching missing content",
(status) => {
(status, expected) => {
setup({
jobs: [{ ...job, status, stored: 0 }],
sources: [{ ...source, imported_messages: 0 }],
});
expect(screen.getByText("已扫描 100 条,正在保存")).toBeTruthy();
expect(screen.getByText(expected)).toBeTruthy();
expect(api.receipt).not.toHaveBeenCalled();
},
);
@@ -38,6 +38,71 @@ it("uses the selected job's message count, never the source total or fake percen
expect(container.textContent).not.toContain("%");
expect(container.textContent).not.toContain("持续导入中");
});
it("does not describe scanned-but-skipped history as saved", () => {
const { container } = render(
<R6ImportProgress
sourceIds={["cursor"]}
sources={[source]}
jobs={[
{
...job,
status: "completed",
discovered: 12_375,
stored: 0,
skipped: 12_375,
history_after: "2026-10-15T00:00:00Z",
},
]}
/>,
);
expect(
screen.getByText("已扫描 12,375 条,本次没有符合时间范围的对话"),
).toBeTruthy();
expect(screen.getByText("导入完成")).toBeTruthy();
expect(container.textContent).not.toContain("正在保存");
expect(container.textContent).not.toContain("已保存 12,375");
});
it("calls an active bounded scan filtering rather than saving", () => {
render(
<R6ImportProgress
sourceIds={["cursor"]}
sources={[source]}
jobs={[
{
...job,
discovered: 3_522,
stored: 0,
history_after: "2026-10-15T00:00:00Z",
},
]}
/>,
);
expect(screen.getByText("已扫描 3,522 条,正在筛选最近 3 天")).toBeTruthy();
});
it("reports only stored rows as saved after a completed partial import", () => {
const { container } = render(
<R6ImportProgress
sourceIds={["cursor"]}
sources={[source]}
jobs={[
{
...job,
status: "completed",
discovered: 25_177,
stored: 20_826,
skipped: 4_351,
},
]}
/>,
);
expect(screen.getByText("已保存 20,826 条消息")).toBeTruthy();
expect(screen.getByText("导入完成")).toBeTruthy();
expect(container.textContent).not.toContain("25,177");
});
it.each(["paused", "interrupted", "failed", "completed", "cancelled"] as const)(
"does not animate %s as active work",
(status) => {
+19 -5
View File
@@ -3,6 +3,21 @@ import { useEffect, useLayoutEffect, useRef } from "react";
import { AppMark } from "./app-mark";
import "./r6-import-progress.css";
function progressText(job: ImportJob): string | null {
if (job.stored > 0) return `已保存 ${job.stored.toLocaleString()} 条消息`;
if (job.discovered <= 0) return null;
const scanned = `已扫描 ${job.discovered.toLocaleString()} 条`;
if (job.status === "completed")
return job.history_after
? `${scanned},本次没有符合时间范围的对话`
: `${scanned},本次没有可保存的对话`;
if (job.status === "pending" || job.status === "running")
return job.history_after
? `${scanned},正在筛选最近 3 天`
: `${scanned},正在筛选`;
return `${scanned},尚未保存`;
}
export function R6ImportProgress({
sourceIds,
sources,
@@ -86,8 +101,9 @@ export function R6ImportProgress({
interrupted: "已中断",
failed: "导入遇到问题",
cancelled: "已取消",
completed: "已保存",
completed: "导入完成",
};
const progress = job ? progressText(job) : null;
return (
<article
className="r6-import-progress-card"
@@ -102,10 +118,8 @@ export function R6ImportProgress({
</div>
</div>
<h2>{source?.display_name ?? id}</h2>
{job && job.stored > 0 ? (
<p>{`已保存 ${job.stored.toLocaleString()} 条消息`}</p>
) : job && job.discovered > 0 ? (
<p>{`已扫描 ${job.discovered.toLocaleString()} 条,正在保存`}</p>
{progress ? (
<p>{progress}</p>
) : noImportedHistory ? (
<p>没有可保存的历史对话</p>
) : null}
+88 -25
View File
@@ -107,6 +107,7 @@ class TestSupersede:
async def _seed_old(self, store, *, derived=None):
old = WorkingFact(
fact_id="f_old", user_id="u1",
session_id="test-session",
statement=(
"用户要修复 CI 的 npm run build;检查 .github/workflows/ci.yml 后"
"定位到 Node 18 报 ERR_OSSL_EVP_UNSUPPORTED,尚未修改。"
@@ -129,7 +130,7 @@ class TestSupersede:
[_draft(
"用户继续修复同一 CI 构建;把 .github/workflows/ci.yml 升到 Node 20,"
"npm run build 已通过。"
)], user_id="u1",
)], user_id="u1", session_id="test-session",
)
new_id = facts[0].fact_id
@@ -152,7 +153,7 @@ class TestSupersede:
[_draft(
"用户继续修复同一 CI 构建;把 .github/workflows/ci.yml 升到 Node 20,"
"npm run build 已通过。"
)], user_id="u1",
)], user_id="u1", session_id="test-session",
)
new_id = facts[0].fact_id
@@ -184,7 +185,9 @@ class TestSupersede:
coding_store, mock_embed,
_llm([{"draft_idx": 0, "supersedes": [target or prev_id]}]),
)
facts, _ = await rec.reconcile([_draft(stmt)], user_id="u1")
facts, _ = await rec.reconcile(
[_draft(stmt)], user_id="u1", session_id="test-session"
)
prev_id = facts[0].fact_id
lineage = await coding_store.get_fact_ancestors(prev_id, user_id="u1")
@@ -208,6 +211,7 @@ class TestSupersede:
"配置完成但尚未发布。"
)],
user_id="u1",
session_id="test-session",
)
prompt = llm.calls[0]["prompt"]
@@ -234,7 +238,9 @@ class TestSupersede:
coding_store, mock_embed, _llm([{"draft_idx": 0, "supersedes": ["f_old"]}]),
)
await rec.reconcile(
[_draft("config 位于 /etc/app/config.yaml")], user_id="u1",
[_draft("config 位于 /etc/app/config.yaml")],
user_id="u1",
session_id="test-session",
)
# 拿旧说法原文去搜,只能落到新事实(靠两者本身相似),
@@ -256,6 +262,34 @@ class TestSupersede:
old = await coding_store.get_fact("f_old", user_id="u1")
assert old.status == "active"
async def test_near_duplicate_fact_cannot_cross_authority_scope(
self, coding_store, mock_embed
):
await coding_store.insert_fact(WorkingFact(
fact_id="f_old",
user_id="u1",
agent_id="scope:2026-10-18",
session_id="conversation-a",
workspace_id="repo-a",
statement="CI build now passes after upgrading Node",
))
llm = _llm([{"draft_idx": 0, "supersedes": ["f_old"]}])
rec = WorkingFactReconciler(coding_store, mock_embed, llm)
_, ops = await rec.reconcile(
[_draft("CI build now passes after upgrading Node")],
user_id="u1",
agent_id="scope:2026-10-19",
session_id="conversation-b",
workspace_id="repo-a",
)
old = await coding_store.get_fact("f_old", user_id="u1")
assert old is not None
assert old.status == "active"
assert [op.action for op in ops] == ["INSERT"]
assert len(llm.calls) == 0
async def test_old_fact_superseded_only_once(self, coding_store, mock_embed):
"""两条新事实都声称覆盖同一条旧事实时,只让第一条生效。
@@ -274,6 +308,7 @@ class TestSupersede:
_draft("config 位于 /srv/app/config.yaml"),
],
user_id="u1",
session_id="test-session",
)
assert len(facts) == 2
assert sum(1 for o in ops if o.action == "SUPERSEDE") == 1
@@ -288,6 +323,7 @@ class TestChainSummary:
async def _seed_old(self, store):
await store.insert_fact(WorkingFact(
fact_id="f_old", user_id="u1",
session_id="test-session",
statement=(
"用户要迁移 app config;确认当前读取 /opt/app/config.yaml,"
"尚未修改配置加载逻辑。"
@@ -310,7 +346,7 @@ class TestChainSummary:
[_draft(
"用户继续迁移 app config;将加载路径改为 /etc/app/config.yaml,"
"配置加载测试通过。"
)], user_id="u1",
)], user_id="u1", session_id="test-session",
)
stored = await coding_store.get_fact(facts[0].fact_id, user_id="u1")
@@ -326,7 +362,7 @@ class TestChainSummary:
[_draft(
"用户继续迁移 app config;将加载路径改为 /etc/app/config.yaml,"
"配置加载测试通过。"
)], user_id="u1",
)], user_id="u1", session_id="test-session",
)
assert len(llm.calls) == 2, "SUPERSEDE 应触发第二次调用"
@@ -344,7 +380,11 @@ class TestChainSummary:
llm = _llm([])
rec = WorkingFactReconciler(coding_store, mock_embed, llm)
await rec.reconcile([_draft("另一个话题的结论")], user_id="u1")
await rec.reconcile(
[_draft("另一个话题的结论")],
user_id="u1",
session_id="test-session",
)
assert len(llm.calls) == 1
async def test_prompt_carries_full_lineage(self, coding_store, mock_embed):
@@ -357,7 +397,7 @@ class TestChainSummary:
[_draft(
"用户继续迁移 app config;将加载路径改为 /etc/app/config.yaml,"
"配置加载测试通过。"
)], user_id="u1",
)], user_id="u1", session_id="test-session",
)
llm2 = _llm([{"draft_idx": 0, "supersedes": [facts[0].fact_id]}])
@@ -366,7 +406,7 @@ class TestChainSummary:
[_draft(
"用户继续完善同一 config 迁移;改由环境变量 APP_CONFIG 指定路径,"
"回归测试通过。"
)], user_id="u1",
)], user_id="u1", session_id="test-session",
)
prompt = llm2.calls[1]["prompt"]
@@ -393,7 +433,7 @@ class TestChainSummary:
[_draft(
"用户继续迁移 app config;将加载路径改为 /etc/app/config.yaml,"
"配置加载测试通过。"
)], user_id="u1",
)], user_id="u1", session_id="test-session",
)
assert sum(1 for o in ops if o.action == "SUPERSEDE") == 1
@@ -426,7 +466,7 @@ class TestChainSummary:
[_draft(
"用户继续迁移 app config;将加载路径改为 /etc/app/config.yaml,"
"配置加载测试通过。"
)], user_id="u1",
)], user_id="u1", session_id="test-session",
)
assert facts[0].fact_chain_summary == ""
@@ -436,7 +476,7 @@ class TestChainSummary:
[_draft(
"用户继续完善同一 config 迁移;改由环境变量 APP_CONFIG 指定路径,"
"回归测试通过。"
)], user_id="u1",
)], user_id="u1", session_id="test-session",
)
prompt = llm2.calls[1]["prompt"]
@@ -450,7 +490,8 @@ class TestChainSummary:
total = MAX_FACT_CHAIN_ENTRIES + 3
for i in range(total):
await coding_store.insert_fact(WorkingFact(
fact_id=f"g{i}", user_id="u1", statement=f"第 {i} 代结论",
fact_id=f"g{i}", user_id="u1", session_id="test-session",
statement=f"第 {i} 代结论",
created_at=datetime(2026, 1, i + 1),
))
if prev:
@@ -459,7 +500,9 @@ class TestChainSummary:
llm = _llm([{"draft_idx": 0, "supersedes": [prev]}])
rec = WorkingFactReconciler(coding_store, mock_embed, llm)
await rec.reconcile([_draft("最新结论")], user_id="u1")
await rec.reconcile(
[_draft("最新结论")], user_id="u1", session_id="test-session"
)
prompt = llm.calls[1]["prompt"]
assert "第 0 代结论" not in prompt, "最老的几代该被截掉"
@@ -478,7 +521,7 @@ class TestChainSummary:
[_draft(
"用户继续迁移 app config;将加载路径改为 /etc/app/config.yaml,"
"配置加载测试通过。"
)], user_id="u1",
)], user_id="u1", session_id="test-session",
)
entries = llm.calls[1]["prompt"].split(
"ENTRIES\n═══════════════════════════════════════════════════════════════════\n",
@@ -497,7 +540,7 @@ class TestChainSummary:
[_draft(
"用户继续迁移 app config;将加载路径改为 /etc/app/config.yaml,"
"配置加载测试通过。"
)], user_id="u1",
)], user_id="u1", session_id="test-session",
)
rec2 = WorkingFactReconciler(
@@ -509,7 +552,7 @@ class TestChainSummary:
[_draft(
"用户继续完善同一 config 迁移;改由环境变量 APP_CONFIG 指定路径,"
"回归测试通过。"
)], user_id="u1",
)], user_id="u1", session_id="test-session",
)
stored = await coding_store.get_fact(facts2[0].fact_id, user_id="u1")
@@ -526,7 +569,7 @@ class TestChainSummary:
[_draft(
"用户继续迁移 app config;将加载路径改为 /etc/app/config.yaml,"
"配置加载测试通过。"
)], user_id="u1",
)], user_id="u1", session_id="test-session",
)
assert len(llm.calls) == 1
@@ -564,7 +607,8 @@ class TestFailSafe:
async def test_llm_failure_still_inserts(self, coding_store, mock_embed):
"""LLM 挂了宁可多留一条过期事实,也不能因为一次调用失败丢结论。"""
await coding_store.insert_fact(WorkingFact(
fact_id="f_old", user_id="u1", statement="config 位于 /opt/app/config.yaml",
fact_id="f_old", user_id="u1", session_id="test-session",
statement="config 位于 /opt/app/config.yaml",
))
rec = WorkingFactReconciler(coding_store, mock_embed, FailingLLMProvider())
@@ -578,7 +622,8 @@ class TestFailSafe:
async def test_unparsable_llm_output(self, coding_store, mock_embed):
await coding_store.insert_fact(WorkingFact(
fact_id="f_old", user_id="u1", statement="config 位于 /opt/app/config.yaml",
fact_id="f_old", user_id="u1", session_id="test-session",
statement="config 位于 /opt/app/config.yaml",
))
rec = WorkingFactReconciler(
coding_store, mock_embed, MockLLMProvider("不是 JSON 的一段话")
@@ -592,7 +637,8 @@ class TestFailSafe:
async def test_markdown_fence_parsed(self, coding_store, mock_embed):
await coding_store.insert_fact(WorkingFact(
fact_id="f_old", user_id="u1", statement="config 位于 /opt/app/config.yaml",
fact_id="f_old", user_id="u1", session_id="test-session",
statement="config 位于 /opt/app/config.yaml",
))
llm = MockLLMProvider([
"```json\n" + _ops_resp([{"draft_idx": 0, "supersedes": ["f_old"]}]) + "\n```",
@@ -601,7 +647,9 @@ class TestFailSafe:
rec = WorkingFactReconciler(coding_store, mock_embed, llm)
_, ops = await rec.reconcile(
[_draft("config 位于 /etc/app/config.yaml")], user_id="u1",
[_draft("config 位于 /etc/app/config.yaml")],
user_id="u1",
session_id="test-session",
)
assert sum(1 for o in ops if o.action == "SUPERSEDE") == 1
@@ -618,7 +666,8 @@ class TestSubjectGateSwitch:
async def _seed(self, store, fact_id, subject, statement):
await store.insert_fact(WorkingFact(
fact_id=fact_id, user_id="u1", subject=subject, statement=statement,
fact_id=fact_id, user_id="u1", session_id="test-session",
subject=subject, statement=statement,
))
async def test_gate_on_blocks_cross_subject_supersede(self, coding_store, mock_embed):
@@ -632,6 +681,7 @@ class TestSubjectGateSwitch:
_, ops = await rec.reconcile(
[WorkingFactDraft(subject="Bart", statement="Bart 的三日行程已排定,预算待定。")],
user_id="u1",
session_id="test-session",
)
assert [o.action for o in ops] == ["INSERT"]
assert (await coding_store.get_fact("f_emma", user_id="u1")).status == "active"
@@ -647,6 +697,7 @@ class TestSubjectGateSwitch:
facts, ops = await rec.reconcile(
[WorkingFactDraft(subject="Bart", statement="Bart 的三日行程已排定,预算待定。")],
user_id="u1",
session_id="test-session",
)
assert [o.action for o in ops] == ["SUPERSEDE", "INSERT"]
old = await coding_store.get_fact("f_emma", user_id="u1")
@@ -667,7 +718,8 @@ class TestSubjectGateSwitch:
[WorkingFactDraft(subject="Bart", statement="Bart 的行程。")],
[[old]],
{0: ["f_emma"]},
user_id="u1", agent_id="a1", session_id=None, workspace_id=None,
user_id="u1", agent_id="default_agent",
session_id="test-session", workspace_id=None,
)
assert "SUPERSEDE" in [o.action for o in ops]
@@ -687,7 +739,8 @@ class TestSubjectGateSwitch:
[WorkingFactDraft(subject="deploy.sh", statement="新的部署脚本结论。")],
[[old]],
{0: [f"f_noneed_{gate}"]},
user_id="u1", agent_id="a1", session_id=None, workspace_id=None,
user_id="u1", agent_id="default_agent",
session_id="test-session", workspace_id=None,
)
assert "SUPERSEDE" in [o.action for o in ops], f"gate={gate} 时空 subject 被误拦"
@@ -714,6 +767,7 @@ class TestDetailsInheritance:
async def test_ancestor_details_copied_verbatim(self, coding_store, mock_embed):
await coding_store.insert_fact(WorkingFact(
fact_id="f_old", user_id="u1", subject="Emma",
session_id="test-session",
statement="Emma 的行程草案已出,预算待核。",
details="日均餐费 $32.4-$57.6\n酒店评分 3.2-3.8",
))
@@ -726,6 +780,7 @@ class TestDetailsInheritance:
[WorkingFactDraft(subject="Emma", statement="Emma 的行程已定稿。",
details="航班 CA1234")],
user_id="u1",
session_id="test-session",
)
stored = await coding_store.get_fact(facts[0].fact_id, user_id="u1")
# 本代的 details 在,祖先的具体取值也一字不差地在
@@ -742,6 +797,7 @@ class TestDetailsInheritance:
"""
await coding_store.insert_fact(WorkingFact(
fact_id="f_g1", user_id="u1", subject="T",
session_id="test-session",
statement="第一代结论。", details="V1=" + "a" * 300,
))
prev = "f_g1"
@@ -756,6 +812,7 @@ class TestDetailsInheritance:
[WorkingFactDraft(subject="T", statement=f"第{gen}代结论。",
details=f"V{gen}=" + "a" * 300)],
user_id="u1",
session_id="test-session",
)
prev = facts[0].fact_id
stored = await coding_store.get_fact(prev, user_id="u1")
@@ -772,6 +829,7 @@ class TestDetailsInheritance:
"""预算是硬的,但被丢掉的代数必须写出来,不静默截断。"""
await coding_store.insert_fact(WorkingFact(
fact_id="f_old", user_id="u1", subject="T",
session_id="test-session",
statement="旧结论。", details="X=" + "b" * 8000,
))
rec = WorkingFactReconciler(
@@ -782,6 +840,7 @@ class TestDetailsInheritance:
facts, _ = await rec.reconcile(
[WorkingFactDraft(subject="T", statement="新结论。", details="Y=1")],
user_id="u1",
session_id="test-session",
)
stored = await coding_store.get_fact(facts[0].fact_id, user_id="u1")
assert "…(truncated)" in stored.details
@@ -790,6 +849,7 @@ class TestDetailsInheritance:
async def test_switch_off_keeps_details_untouched(self, coding_store, mock_embed):
await coding_store.insert_fact(WorkingFact(
fact_id="f_old", user_id="u1", subject="T",
session_id="test-session",
statement="旧结论。", details="祖先取值 Z=9",
))
rec = WorkingFactReconciler(
@@ -800,6 +860,7 @@ class TestDetailsInheritance:
facts, ops = await rec.reconcile(
[WorkingFactDraft(subject="T", statement="新结论。", details="Y=1")],
user_id="u1",
session_id="test-session",
)
# 覆盖照样发生,只是不过继
assert "SUPERSEDE" in [o.action for o in ops]
@@ -814,6 +875,7 @@ class TestDetailsInheritance:
"""
await coding_store.insert_fact(WorkingFact(
fact_id="f_old", user_id="u1", subject="T",
session_id="test-session",
statement="旧结论。", details="无关噪声 " + "q" * 2000,
))
rec = WorkingFactReconciler(
@@ -824,6 +886,7 @@ class TestDetailsInheritance:
facts, _ = await rec.reconcile(
[WorkingFactDraft(subject="T", statement="新结论。", details="Y=1")],
user_id="u1",
session_id="test-session",
)
new_id = facts[0].fact_id
emb = await coding_store.embed_service.embed("T 新结论。")
+129 -5
View File
@@ -35,7 +35,8 @@ class TestReconcilerActions:
async def test_update(self, coding_store, mock_embed):
# 种入 m1
await coding_store.insert(CodingMemory(
experience_id="m1", user_id="u1", task="T1",
experience_id="m1", user_id="u1", agent_id="scope-day-1",
workspace_id="repo-a", session_id="conversation-1", task="T1",
search_keys=["old"], solution="old",
))
llm = MockLLMProvider(_llm_resp([
@@ -44,23 +45,146 @@ class TestReconcilerActions:
]))
rec = CodingMemoryReconciler(coding_store, mock_embed, llm)
d = CodingMemoryDraft(task="T1", search_keys=["new1"], solution="new",
user_id="u1")
user_id="u1", agent_id="scope-day-1",
workspace_id="repo-a", session_id="conversation-1")
ops = await rec.reconcile([d])
assert len(ops) == 1 and ops[0].action == "UPDATE"
got = await coding_store.get_by_id("m1", "u1")
assert got.solution == "new"
assert got.search_keys == ["new1"]
@pytest.mark.parametrize(
(
"old_agent",
"old_workspace",
"old_session",
"new_agent",
"new_workspace",
"new_session",
),
[
(
"scope:same-day",
"repo-a",
"conversation-a",
"scope:same-day",
"repo-a",
"conversation-b",
),
(
"scope:2026-10-18",
"repo-a",
"conversation-a",
"scope:2026-10-19",
"repo-a",
"conversation-a",
),
(
"shared-scope",
"repo-a",
"conversation-a",
"shared-scope",
"repo-b",
"conversation-a",
),
],
)
async def test_update_target_must_match_frozen_authority_scope(
self,
coding_store,
mock_embed,
old_agent,
old_workspace,
old_session,
new_agent,
new_workspace,
new_session,
):
"""Near duplicates cannot cross conversation, date, or workspace authority."""
await coding_store.insert(CodingMemory(
experience_id="old-target", user_id="u1", agent_id=old_agent,
workspace_id=old_workspace, session_id=old_session,
task="fix flaky release", solution="old solution",
))
llm = MockLLMProvider(_llm_resp([
{
"draft_idx": 0,
"action": "UPDATE",
"target_experience_id": "old-target",
"reason": "near duplicate",
},
]))
rec = CodingMemoryReconciler(coding_store, mock_embed, llm)
draft = CodingMemoryDraft(
task="fix flaky release",
solution="new solution",
user_id="u1",
agent_id=new_agent,
workspace_id=new_workspace,
session_id=new_session,
)
ops = await rec.reconcile([draft])
old = await coding_store.get_by_id("old-target", "u1")
assert old is not None
assert old.solution == "old solution"
assert old.agent_id == old_agent
assert old.workspace_id == old_workspace
assert old.session_id == old_session
assert len(ops) == 1
assert ops[0].action == "ADD"
assert ops[0].target_experience_id != "old-target"
@pytest.mark.parametrize("legacy_agent", ["", "default_agent", "scope:legacy"])
async def test_legacy_target_without_conversation_falls_back_to_add(
self, coding_store, mock_embed, legacy_agent
):
await coding_store.insert(CodingMemory(
experience_id="legacy-target",
user_id="u1",
agent_id=legacy_agent,
session_id=None,
workspace_id="repo-a",
task="upgrade dependency",
solution="legacy solution",
))
llm = MockLLMProvider(_llm_resp([
{
"draft_idx": 0,
"action": "UPDATE",
"target_experience_id": "legacy-target",
},
]))
rec = CodingMemoryReconciler(coding_store, mock_embed, llm)
draft = CodingMemoryDraft(
task="upgrade dependency",
solution="current solution",
user_id="u1",
agent_id=legacy_agent,
session_id="current-conversation",
workspace_id="repo-a",
)
ops = await rec.reconcile([draft])
legacy = await coding_store.get_by_id("legacy-target", "u1")
assert legacy is not None
assert legacy.solution == "legacy solution"
assert legacy.session_id is None
assert [op.action for op in ops] == ["ADD"]
async def test_skip(self, coding_store, mock_embed):
await coding_store.insert(CodingMemory(
experience_id="m1", user_id="u1", task="X", solution="Y",
session_id="conversation-1",
))
llm = MockLLMProvider(_llm_resp([
{"draft_idx": 0, "action": "SKIP", "reason": "duplicate"},
]))
rec = CodingMemoryReconciler(coding_store, mock_embed, llm)
d = CodingMemoryDraft(task="X", solution="Y",
user_id="u1")
user_id="u1", session_id="conversation-1")
ops = await rec.reconcile([d])
assert len(ops) == 1 and ops[0].action == "SKIP"
# store 不变
@@ -71,7 +195,7 @@ class TestReconcilerActions:
# 旧记忆要被 DELETE
await coding_store.insert(CodingMemory(
experience_id="m_dep", user_id="u1", task="use old.sh",
solution="old script",
solution="old script", session_id="conversation-1",
))
llm = MockLLMProvider(_llm_resp([
{"draft_idx": 0, "action": "ADD"},
@@ -80,7 +204,7 @@ class TestReconcilerActions:
]))
rec = CodingMemoryReconciler(coding_store, mock_embed, llm)
d = CodingMemoryDraft(task="use new CLI", solution="new way",
user_id="u1")
user_id="u1", session_id="conversation-1")
ops = await rec.reconcile([d])
assert sorted(op.action for op in ops) == ["ADD", "DELETE"]
# 旧的没了
+55 -1
View File
@@ -67,7 +67,8 @@ class TestStoreCRUD:
async def test_update_overrides(self, coding_store, mock_embed):
m = CodingMemory(experience_id="e1", user_id="u1", task="T",
search_keys=["old1", "old2"], solution="old")
search_keys=["old1", "old2"], solution="old",
session_id="conversation-1")
await coding_store.insert(m)
# 旧 aspect 应可命中
@@ -90,6 +91,59 @@ class TestStoreCRUD:
assert got.solution == "new"
assert got.search_keys == ["new1"]
async def test_update_rejects_authority_scope_change(self, coding_store):
await coding_store.insert(CodingMemory(
experience_id="e1", user_id="u1", agent_id="scope-a",
workspace_id="repo-a", session_id="conversation-1",
task="T", solution="old",
))
replacement = CodingMemory(
experience_id="e1", user_id="u1", agent_id="scope-b",
workspace_id="repo-a", session_id="conversation-1",
task="T", solution="new",
)
with pytest.raises(ValueError, match="coding_memory_update_authority_mismatch"):
await coding_store.update(replacement)
stored = await coding_store.get_by_id("e1", user_id="u1")
assert stored is not None
assert stored.agent_id == "scope-a"
assert stored.solution == "old"
async def test_update_rejects_conversation_change_and_legacy_empty_session(
self, coding_store
):
for experience_id, old_session in (
("different-conversation", "conversation-1"),
("legacy-empty-session", None),
):
await coding_store.insert(CodingMemory(
experience_id=experience_id,
user_id="u1",
agent_id="scope-a",
workspace_id="repo-a",
session_id=old_session,
task="T",
solution="old",
))
replacement = CodingMemory(
experience_id=experience_id,
user_id="u1",
agent_id="scope-a",
workspace_id="repo-a",
session_id="conversation-2",
task="T",
solution="new",
)
with pytest.raises(ValueError, match="coding_memory_update_authority_mismatch"):
await coding_store.update(replacement)
stored = await coding_store.get_by_id(experience_id, user_id="u1")
assert stored is not None
assert stored.solution == "old"
async def test_delete(self, coding_store):
m = CodingMemory(experience_id="e1", user_id="u1", task="T",
search_keys=["k"], solution="S")
+4 -2
View File
@@ -335,7 +335,7 @@ class TestDualLayerWrite:
from hy_memory.coding.types import CodingMemory
await coding_store.insert(CodingMemory(
experience_id="e1", user_id="u1", task="老 task", solution="旧 solution",
source_fact_ids=["f_before"],
source_fact_ids=["f_before"], session_id="test-session",
))
llm = MockLLMProvider([
_extract_resp(
@@ -348,7 +348,9 @@ class TestDualLayerWrite:
])
writer = self._writer(coding_store, mock_embed, llm)
resp = await writer.write(_coding_messages(), user_id="u1")
resp = await writer.write(
_coding_messages(), user_id="u1", session_id="test-session"
)
exp = await coding_store.get_by_id("e1", user_id="u1")
assert exp.solution == "新 solution"
assert exp.source_fact_ids == ["f_before", resp["fact_ids"][0]]
+59 -23
View File
@@ -12,8 +12,9 @@ L1_RAW 对外读路径不可见:闸门焊在向量库唯一查询入口。
hybrid_search_native / list_by_user 四个查询方法的共同必经处,
上层无法绕开,新增接口自动受保护。
按 ID 直取走主键查询、不经过 filter,故在 `client.async_get` 处单独挡
(GET /v1/memories/{id} 与 /v1/memories/batch 的共同必经处)。
按 ID 直取走主键查询、不经过 filter,故在 `client.async_get` 处默认挡住。
Python SDK 调用方可显式传 `include_raw=True` 读取;HTTP 接口不传该参数,
继续保持原有隐藏行为。
内部管线需读原始对话时显式 allow_raw_layer=True 放行:
- `writer._get_recent_dialogue_history`:提取前拼多轮对话历史喂 LLM
@@ -124,28 +125,63 @@ class TestListNoLongerFiltersInProcess:
"list 又出现进程内过滤 raw 的写法,应交由 vdb 侧 filter 完成"
class TestGetByIdPathBlocksRaw:
"""按 ID 直取:走主键查询不经 filter,须在 async_get 处挡住。"""
class TestGetByIdRawOptIn:
"""按 ID 直取默认隐藏 raw,只有 SDK 显式选择时才返回。"""
def test_async_get_drops_raw_node(self):
@staticmethod
def _client(node):
from hy_memory.client import HyMemoryClient
class Store:
async def get_by_id(self, memory_id):
return node if memory_id == node.node_id else None
async def no_op_log(**kwargs):
return None
client = HyMemoryClient.__new__(HyMemoryClient)
client._vector_store = Store()
client._graph_store = None
client._backend_identity = lambda: {}
client._maybe_store_op_log = no_op_log
return client
@staticmethod
def _raw_node():
return MemoryNode(
node_id="raw-1",
user_id="u1",
agent_id="a1",
session_id="s1",
content="原始对话",
layer=MemoryLayer.L1_RAW,
status=MemoryStatus.ACTIVE,
custom={"meta": [{"source": "test"}]},
)
def test_default_still_hides_raw(self):
from hy_memory.client import HyMemoryClient
client = self._client(self._raw_node())
raw_get = HyMemoryClient.async_get.__wrapped__
assert run(raw_get(client, "raw-1")) is None
def test_explicit_include_raw_returns_lite_memory(self):
from hy_memory.client import HyMemoryClient
client = self._client(self._raw_node())
raw_get = HyMemoryClient.async_get.__wrapped__
result = run(raw_get(client, "raw-1", include_raw=True))
assert result["memory_id"] == "raw-1"
assert result["content"] == "原始对话"
assert result["layer"] == MemoryLayer.L1_RAW.value
assert result["meta"] == [{"source": "test"}]
def test_public_get_exposes_keyword_only_opt_in(self):
import inspect
from hy_memory.client import HyMemoryClient
src = inspect.getsource(HyMemoryClient.async_get)
assert "MemoryLayer.L1_RAW" in src, \
"async_get 未挡 raw,batch 与按 ID 单取会漏出原始对话"
def test_block_happens_before_return(self):
"""挡的位置必须在两处 return 之前,否则等于没挡。"""
import inspect
from hy_memory.client import HyMemoryClient
lines = inspect.getsource(HyMemoryClient.async_get).splitlines()
block_at = next(
i for i, l in enumerate(lines)
if "MemoryLayer.L1_RAW" in l and "vdb_node" in l
)
first_return_at = next(
i for i, l in enumerate(lines)
if l.strip().startswith("return {")
)
assert block_at < first_return_at, \
f"挡 raw 在第 {block_at} 行、首个 return 在第 {first_return_at} 行,顺序错了"
param = inspect.signature(HyMemoryClient.get).parameters["include_raw"]
assert param.default is False
assert param.kind is inspect.Parameter.KEYWORD_ONLY
+136
View File
@@ -0,0 +1,136 @@
import asyncio
import threading
import pytest
from hy_memory.config import MemoryConfig
from hy_memory.data.cache_sqlite import SqliteCache
from hy_memory.data.history_store import HistoryStore
class _BlockingCursor:
def fetchone(self):
return (1,)
def fetchall(self):
return []
class _BlockingConnection:
"""Hold execute() open so a concurrent close() can be observed."""
def __init__(self):
self.entered = threading.Event()
self.release = threading.Event()
self.closed = threading.Event()
def execute(self, *_args, **_kwargs):
self.entered.set()
if not self.release.wait(timeout=2):
raise TimeoutError("test did not release the blocking query")
if self.closed.is_set():
raise AssertionError("connection was closed while execute() was active")
return _BlockingCursor()
def close(self):
self.closed.set()
def _config(tmp_path) -> MemoryConfig:
config = MemoryConfig()
config.history.db_path = str(tmp_path / "history.db")
config.cache.db_path = str(tmp_path / "cache.db")
return config
async def _wait_for(event: threading.Event) -> None:
assert await asyncio.wait_for(asyncio.to_thread(event.wait, 1), timeout=2)
@pytest.mark.asyncio
async def test_history_close_waits_for_active_query(tmp_path):
store = HistoryStore(_config(tmp_path))
connection = _BlockingConnection()
store._conn = connection
query = asyncio.create_task(store.count())
await _wait_for(connection.entered)
closing = asyncio.create_task(store.close())
await asyncio.sleep(0.05)
assert not connection.closed.is_set()
connection.release.set()
assert await query == 1
await closing
assert connection.closed.is_set()
@pytest.mark.asyncio
async def test_cache_close_waits_for_active_query(tmp_path):
store = SqliteCache(_config(tmp_path))
connection = _BlockingConnection()
store._conn = connection
query = asyncio.create_task(store.load_metrics_range("0", "9"))
await _wait_for(connection.entered)
closing = asyncio.create_task(store.close())
await asyncio.sleep(0.05)
assert not connection.closed.is_set()
connection.release.set()
assert await query == []
await closing
assert connection.closed.is_set()
@pytest.mark.asyncio
async def test_history_concurrent_reads_and_writes_are_serialized(tmp_path):
store = HistoryStore(_config(tmp_path))
await store.initialize()
try:
inserted = await asyncio.gather(
*(
store.record_add(
memory_id=f"memory-{index}",
content="content",
layer="l2_fact",
isolation_key="user",
)
for index in range(200)
)
)
assert all(row_id > 0 for row_id in inserted)
assert len(set(inserted)) == 200
counts = await asyncio.gather(*(store.count() for _ in range(100)))
assert counts == [200] * 100
finally:
await store.close()
@pytest.mark.asyncio
async def test_cache_concurrent_pipeline_log_reads_and_writes_are_serialized(tmp_path):
store = SqliteCache(_config(tmp_path))
await store.initialize()
try:
written = await asyncio.gather(
*(
store.store_pipeline_log(
request_id="request",
user_id="user",
agent_id="agent",
step="EXTRACT",
prompt=f"prompt-{index}",
response="response",
)
for index in range(200)
)
)
assert written == [True] * 200
pages = await asyncio.gather(
*(store.get_pipeline_logs(request_id="request", limit=200) for _ in range(50))
)
assert all(len(page) == 200 for page in pages)
finally:
await store.close()