Compare commits

..
Author SHA1 Message Date
alvinfeiandCursor 1821a31295 fix(core): drop native store write gate on HY search
MemHub expects concurrent native reads during precipitation. The
process-local read slot mostly existed for a Chroma/GIL fail-fast and
was never held by writers on main; activating it (as in #98) would only
push recall onto weaker fallbacks. Remove the lock module and the
hy_memory_store_busy path instead.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-21 12:07:37 +08:00
54 changed files with 875 additions and 1247 deletions

No files matched your search

-1
View File
@@ -86,7 +86,6 @@ MEMORY_VECTOR_STORE=chroma
# MEMORY_CODING_DB_PATH= # coding_memory.db 路径(默认 {DATA_DIR}/data/coding_memory.db) # MEMORY_CODING_DB_PATH= # coding_memory.db 路径(默认 {DATA_DIR}/data/coding_memory.db)
# MEMORY_CODING_TOOL_RESULT_MAX_BYTES=2048 # 单条 tool_result 截断字节上限 # MEMORY_CODING_TOOL_RESULT_MAX_BYTES=2048 # 单条 tool_result 截断字节上限
# MEMORY_CODING_VERBATIM_FALLBACK_ENABLED=false # 模型无结果时是否保存原文 fact;默认关,避免 tool trace 外泄 # MEMORY_CODING_VERBATIM_FALLBACK_ENABLED=false # 模型无结果时是否保存原文 fact;默认关,避免 tool trace 外泄
# MEMORY_CODING_CHAIN_BLOCK_ENABLED=false # search 是否注入版本演进链块(diff);默认关,省 token
# MEMORY_EXTRACT_HISTORY_TURNS=5 # 除本次 write 外,额外注入 extractor 的历史 user-turn 数 # MEMORY_EXTRACT_HISTORY_TURNS=5 # 除本次 write 外,额外注入 extractor 的历史 user-turn 数
# ── 写入流程开关 ── # ── 写入流程开关 ──
+7 -13
View File
@@ -122,25 +122,19 @@ jobs:
shell: bash shell: bash
run: | run: |
set -euo pipefail set -euo pipefail
shopt -s globstar nullglob
version=$(node -p 'require("./package.json").version') version=$(node -p 'require("./package.json").version')
node scripts/verify-release-version.mjs \ node scripts/verify-release-version.mjs \
--channel stable \ --channel stable \
--version "$version" \ --version "$version" \
--build-number "$(node -p 'require("./package.json").release.build_number')" \ --build-number "$(node -p 'require("./package.json").release.build_number')" \
--tag "$GITHUB_REF_NAME" --tag "$GITHUB_REF_NAME"
mkdir -p publish-assets
for arch in x64 arm64; do for arch in x64 arm64; do
name="memhub-runtime-v${version}-linux-${arch}.tar.gz" archive="release-assets/memhub-runtime-v${version}-linux-${arch}.tar.gz"
archives=(release-assets/**/"$name") test -f "$archive"
checksums=(release-assets/**/"$name.sha256") test -f "$archive.sha256"
test "${#archives[@]}" -eq 1
test "${#checksums[@]}" -eq 1
cp -- "${archives[0]}" "publish-assets/$name"
cp -- "${checksums[0]}" "publish-assets/$name.sha256"
( (
cd publish-assets cd release-assets
sha256sum -c "$name.sha256" sha256sum -c "$(basename "$archive.sha256")"
) )
done done
- name: Upload Linux assets to a draft GitHub Release - name: Upload Linux assets to a draft GitHub Release
@@ -166,7 +160,7 @@ jobs:
--title "$GITHUB_REF_NAME" --title "$GITHUB_REF_NAME"
fi fi
gh release upload "$GITHUB_REF_NAME" \ gh release upload "$GITHUB_REF_NAME" \
publish-assets/memhub-runtime-v*-linux-*.tar.gz \ release-assets/memhub-runtime-v*-linux-*.tar.gz \
publish-assets/memhub-runtime-v*-linux-*.tar.gz.sha256 \ release-assets/memhub-runtime-v*-linux-*.tar.gz.sha256 \
--repo "$GITHUB_REPOSITORY" \ --repo "$GITHUB_REPOSITORY" \
--clobber --clobber
-4
View File
@@ -14,10 +14,6 @@ Hy Memory and MemHub are versioned and released independently.
## Unreleased ## Unreleased
- Coding search no longer injects the version-evolution diff block
(`chain_block`) by default. Enable with
`MEMORY_CODING_CHAIN_BLOCK_ENABLED=true` or
`CodingConfig.chain_block_enabled=True` (write-side observe unchanged).
- Fixed governed-memory revisions and repeated evidence links failing when the - Fixed governed-memory revisions and repeated evidence links failing when the
visibility dirty-marker trigger encounters the same memory twice. Startup visibility dirty-marker trigger encounters the same memory twice. Startup
now replaces the affected legacy trigger without changing schema versions now replaces the affected legacy trigger without changing schema versions
+8 -8
View File
@@ -65,13 +65,13 @@ Welcome to the **Hy Memory Research Family**, covering cognitive chat memory, lo
### 1. Installation ### 1. Installation
```bash ```bash
pip install hy-memory==2.0.4 # core (includes ChromaDB and Kuzu) pip install hy-memory==2.0.3 # core (includes ChromaDB and Kuzu)
pip install "hy-memory[qdrant]==2.0.4" # + Qdrant pip install "hy-memory[qdrant]==2.0.3" # + Qdrant
pip install "hy-memory[faiss]==2.0.4" # + FAISS pip install "hy-memory[faiss]==2.0.3" # + FAISS
pip install "hy-memory[graph]==2.0.4" # + Neo4j pip install "hy-memory[graph]==2.0.3" # + Neo4j
pip install "hy-memory[mysql]==2.0.4" # + MySQL cache pip install "hy-memory[mysql]==2.0.3" # + MySQL cache
pip install "hy-memory[rerank]==2.0.4" # + local multilingual cross-encoder rerank pip install "hy-memory[rerank]==2.0.3" # + local multilingual cross-encoder rerank
pip install "hy-memory[all]==2.0.4" # everything pip install "hy-memory[all]==2.0.3" # everything
``` ```
### 2. Configuration ### 2. Configuration
@@ -153,7 +153,7 @@ result = client.search(
result["facts"] # fact layer result["facts"] # fact layer
result["experiences"] # experience layer result["experiences"] # experience layer
result["chain_block"] # coding only: version-evolution context (empty unless MEMORY_CODING_CHAIN_BLOCK_ENABLED) result["chain_block"] # coding only: version-evolution context
``` ```
### 5. Routing, Source Isolation & Advanced APIs ### 5. Routing, Source Isolation & Advanced APIs
+8 -8
View File
@@ -65,13 +65,13 @@
### 1. 安装 ### 1. 安装
```bash ```bash
pip install hy-memory==2.0.4 # 核心(含 ChromaDB 与 Kuzu) pip install hy-memory==2.0.3 # 核心(含 ChromaDB 与 Kuzu)
pip install "hy-memory[qdrant]==2.0.4" # + Qdrant pip install "hy-memory[qdrant]==2.0.3" # + Qdrant
pip install "hy-memory[faiss]==2.0.4" # + FAISS pip install "hy-memory[faiss]==2.0.3" # + FAISS
pip install "hy-memory[graph]==2.0.4" # + Neo4j pip install "hy-memory[graph]==2.0.3" # + Neo4j
pip install "hy-memory[mysql]==2.0.4" # + MySQL 缓存 pip install "hy-memory[mysql]==2.0.3" # + MySQL 缓存
pip install "hy-memory[rerank]==2.0.4" # + 本地多语言 cross-encoder rerank pip install "hy-memory[rerank]==2.0.3" # + 本地多语言 cross-encoder rerank
pip install "hy-memory[all]==2.0.4" # 全部 pip install "hy-memory[all]==2.0.3" # 全部
``` ```
### 2. 配置 ### 2. 配置
@@ -153,7 +153,7 @@ result = client.search(
result["facts"] # 事实层 result["facts"] # 事实层
result["experiences"] # 经验层 result["experiences"] # 经验层
result["chain_block"] # 仅 coding:版本演进上下文(默认空;需 MEMORY_CODING_CHAIN_BLOCK_ENABLED) result["chain_block"] # 仅 coding:版本演进上下文
``` ```
### 5. 自动路由、来源过滤与进阶 API ### 5. 自动路由、来源过滤与进阶 API
-1
View File
@@ -197,7 +197,6 @@ SDK 不设置 Rerank Search 并发上限。开启后模型在 `HyMemoryClient`
| `MEMORY_HISTORY_DB_PATH` | `{DATA_DIR}/data/history.db` | History SQLite 路径 | | `MEMORY_HISTORY_DB_PATH` | `{DATA_DIR}/data/history.db` | History SQLite 路径 |
| `MEMORY_HISTORY_RECORD_SEARCHES` | `false` | 是否记录 Search 操作 | | `MEMORY_HISTORY_RECORD_SEARCHES` | `false` | 是否记录 Search 操作 |
| `MEMORY_CODING_CHAIN_REPO_ROOT` | — | coding 版本链读取本地 Git diff 的仓库根目录 | | `MEMORY_CODING_CHAIN_REPO_ROOT` | — | coding 版本链读取本地 Git diff 的仓库根目录 |
| `MEMORY_CODING_CHAIN_BLOCK_ENABLED` | `false` | search(scene=coding) 是否注入版本演进链块(diff / `chain_block`);默认关 |
## HTTP Server ## HTTP Server
+2 -3
View File
@@ -78,8 +78,7 @@ default 多-query 超过 `MEMORY_QUERY_REWRITE_MAX_CHARS` 时不改写:Coding
Coding 关闭则直接使用最后一条 query。 Coding 关闭则直接使用最后一条 query。
路由到 `working` / `coding` 后,Search 读取独立生产力记忆库,返回 `facts` / 路由到 `working` / `coding` 后,Search 读取独立生产力记忆库,返回 `facts` /
`experiences`,不使用 chat 三通道结构。`coding` 还返回 `chain_block` / `chain_used` `experiences`,不使用 chat 三通道结构。`coding` 还返回 `chain_block` / `chain_used`。
(默认空;需 `MEMORY_CODING_CHAIN_BLOCK_ENABLED` 才注入 diff 块)。
## Chat 成功返回结构 ## Chat 成功返回结构
@@ -153,7 +152,7 @@ client = HyMemoryClient.from_config({
安装可选依赖: 安装可选依赖:
```bash ```bash
pip install 'hy-memory[rerank]==2.0.4' pip install 'hy-memory[rerank]==2.0.1'
``` ```
启用后,`HyMemoryClient(...)` 构造阶段会完成模型下载/读取、device 放置和一次 dummy 启用后,`HyMemoryClient(...)` 构造阶段会完成模型下载/读取、device 放置和一次 dummy
+3 -2
View File
@@ -37,7 +37,8 @@ do not belong in the public source tree.
- Root GitHub workflows run SDK and MemHub checks from their respective working - Root GitHub workflows run SDK and MemHub checks from their respective working
directories. Native lifecycle workflows produce reviewable Actions artifacts; directories. Native lifecycle workflows produce reviewable Actions artifacts;
tag releases remain fail-closed on public identity, signing and notarization. tag releases remain fail-closed on public identity, signing and notarization.
- Hy Memory is published as `hy-memory==2.0.4`. MemHub pins that public - Hy Memory is published as `hy-memory==2.0.3`. MemHub pins that public
distribution; its selected npm package is `memhub-cli` 1.0.5. distribution; its selected npm package is `memhub-cli`, which is not yet
published.
- Hy-Memory source uses Apache-2.0 at the repository root. Third-party - Hy-Memory source uses Apache-2.0 at the repository root. Third-party
components retain their own licenses and attribution notices. components retain their own licenses and attribution notices.
+2 -6
View File
@@ -168,11 +168,9 @@ working/coding 搜索返回:
`limit` 分别限制 facts 与 experiences,因此两层最多各返回 `limit` 条。 `limit` 分别限制 facts 与 experiences,因此两层最多各返回 `limit` 条。
`scene="coding"` 额外返回 `chain_block` 和 `chain_used` 字段(键始终存在;默认内容为空): `scene="coding"` 额外返回 `chain_block` 和 `chain_used`:
- 注入 diff / 版本演进块需显式开启 `MEMORY_CODING_CHAIN_BLOCK_ENABLED=true` - `chain_block` 是与已召回 fact 对齐的版本演进上下文;无法挂链时为 `""`。
(或 `CodingConfig.chain_block_enabled=True`)。默认关闭,以节约 prompt token。
- 开启后,`chain_block` 是与已召回 fact 对齐的版本演进上下文;无法挂链时为 `""`。
- `chain_used` 是已经进入 chain block 的 `facts` 下标。 - `chain_used` 是已经进入 chain block 的 `facts` 下标。
- SDK 不会从 `facts` 删除这些条目。调用方若要拼成“chain block + 剩余 facts”,需自行按 `chain_used` 过滤。 - SDK 不会从 `facts` 删除这些条目。调用方若要拼成“chain block + 剩余 facts”,需自行按 `chain_used` 过滤。
@@ -224,12 +222,10 @@ working/coding 搜索返回:
| `MEMORY_CODING_DB_PATH` | `{DATA_DIR}/data/coding_memory.db` | working 基础 SQLite 路径;coding/trace 路径由此派生 | | `MEMORY_CODING_DB_PATH` | `{DATA_DIR}/data/coding_memory.db` | working 基础 SQLite 路径;coding/trace 路径由此派生 |
| `MEMORY_CODING_TOOL_RESULT_MAX_BYTES` | `2048` | 单条 tool result 的字节上限 | | `MEMORY_CODING_TOOL_RESULT_MAX_BYTES` | `2048` | 单条 tool result 的字节上限 |
| `MEMORY_CODING_VERBATIM_FALLBACK_ENABLED` | `false` | 模型无结果时是否保存原文 fact;默认关闭,避免 assistant/tool trace 直接展示 | | `MEMORY_CODING_VERBATIM_FALLBACK_ENABLED` | `false` | 模型无结果时是否保存原文 fact;默认关闭,避免 assistant/tool trace 直接展示 |
| `MEMORY_CODING_CHAIN_BLOCK_ENABLED` | `false` | search 是否注入版本演进链块(`chain_block` / diff);默认关闭(opt-in) |
| `MEMORY_EXTRACT_HISTORY_TURNS` | `5` | 回读的历史 user-turn 数;当前 prompt 最终最多使用 5 | | `MEMORY_EXTRACT_HISTORY_TURNS` | `5` | 回读的历史 user-turn 数;当前 prompt 最终最多使用 5 |
| `MEMORY_CODING_CHAIN_REPO_ROOT` | 空 | coding 版本链读取本地 Git diff 的仓库路径 | | `MEMORY_CODING_CHAIN_REPO_ROOT` | 空 | coding 版本链读取本地 Git diff 的仓库路径 |
`HCM_*` 变量属于版本链的高级调优项,不是稳定的公共配置契约;普通接入不应依赖其默认值。 `HCM_*` 变量属于版本链的高级调优项,不是稳定的公共配置契约;普通接入不应依赖其默认值。
其中 `HCM_BLOCK_OFF=1` 可在已开启 `MEMORY_CODING_CHAIN_BLOCK_ENABLED` 时做消融(完全不出块)。
## 8. 定向验证 ## 8. 定向验证
+7 -8
View File
@@ -5,14 +5,14 @@
## 安装与快速开始 ## 安装与快速开始
```bash ```bash
pip install hy-memory==2.0.4 pip install hy-memory==2.0.3
# 可选后端 # 可选后端
pip install 'hy-memory[qdrant]==2.0.4' pip install 'hy-memory[qdrant]==2.0.1'
pip install 'hy-memory[faiss]==2.0.4' pip install 'hy-memory[faiss]==2.0.1'
pip install 'hy-memory[graph]==2.0.4' # Neo4j;Kuzu 已在核心依赖中 pip install 'hy-memory[graph]==2.0.1' # Neo4j;Kuzu 已在核心依赖中
pip install 'hy-memory[mysql]==2.0.4' pip install 'hy-memory[mysql]==2.0.1'
pip install 'hy-memory[rerank]==2.0.4' # 本地多语言 cross-encoder rerank pip install 'hy-memory[rerank]==2.0.1' # 本地多语言 cross-encoder rerank
``` ```
默认 LLM 和 Embedding 均使用 OpenAI-compatible 接口。使用 OpenAI 时可只设置: 默认 LLM 和 Embedding 均使用 OpenAI-compatible 接口。使用 OpenAI 时可只设置:
@@ -210,8 +210,7 @@ result = client.search(
``` ```
`working` / `coding` 路径返回 `scene="productivity"`、`facts`、`experiences`,不使用 `working` / `coding` 路径返回 `scene="productivity"`、`facts`、`experiences`,不使用
chat 的三桶结构;`coding` 还会返回 `chain_block` 和 `chain_used` 字段(默认空; chat 的三桶结构;`coding` 还会返回 `chain_block` 和 `chain_used`。
需 `MEMORY_CODING_CHAIN_BLOCK_ENABLED=true` 才注入 diff 块)。
#### `search_mode` #### `search_mode`
+1 -2
View File
@@ -22,8 +22,7 @@ logger = logging.getLogger(__name__)
# ================================================================ # ================================================================
# 数据结构 # 数据结构
# ================================================================ # ================================================================@dataclass
@dataclass
class SessionSummaryResult: class SessionSummaryResult:
"""Session 摘要结果 (V2)""" """Session 摘要结果 (V2)"""
success: bool success: bool
+3 -13
View File
@@ -39,7 +39,6 @@ import logging
import hashlib import hashlib
import json import json
import re import re
from datetime import datetime
from typing import Any, Dict, List, Optional from typing import Any, Dict, List, Optional
import uuid import uuid
@@ -253,17 +252,6 @@ def _series_episode_id(
return str(uuid.uuid5(uuid.NAMESPACE_URL, "hy-memory:series-episode:" + raw)) return str(uuid.uuid5(uuid.NAMESPACE_URL, "hy-memory:series-episode:" + raw))
def _series_memory_at(times: List[int]) -> Optional[datetime]:
"""Earliest member timestamp as the same local wall clock it came from.
``_node_timestamp`` produces ``datetime.timestamp()`` values. Reading them
back with ``utcfromtimestamp`` turns them into a naive *UTC* datetime, which
the vector payload later re-interprets as local time and shifts by the
machine's UTC offset.
"""
return datetime.fromtimestamp(min(times)) if times else None
def _node_timestamp(node: Any) -> Optional[int]: def _node_timestamp(node: Any) -> Optional[int]:
value = getattr(node, "memory_at", None) or getattr(node, "gmt_created", None) value = getattr(node, "memory_at", None) or getattr(node, "gmt_created", None)
if value is None: if value is None:
@@ -986,6 +974,8 @@ async def finalize_series_episode_containers(
Stored text is only a title plus a structural intro. Exact L2 bodies remain in Stored text is only a title plus a structural intro. Exact L2 bodies remain in
their canonical nodes and are referenced by ID from dated timeline segments. their canonical nodes and are referenced by ID from dated timeline segments.
""" """
from datetime import datetime
from ..models.memory import MemoryLayer, MemoryNode, MemoryStatus, SourceType from ..models.memory import MemoryLayer, MemoryNode, MemoryStatus, SourceType
from ..pipelines._retrieval.session_episode import ( from ..pipelines._retrieval.session_episode import (
add_parent_episode_membership, add_parent_episode_membership,
@@ -1071,7 +1061,7 @@ async def finalize_series_episode_containers(
segment["memory_at"] for segment in plan["timeline"] segment["memory_at"] for segment in plan["timeline"]
if segment.get("memory_at") is not None if segment.get("memory_at") is not None
] ]
memory_at = _series_memory_at(times) memory_at = datetime.utcfromtimestamp(min(times)) if times else None
content = f"{plan['title']}. {plan['intro']}".strip() content = f"{plan['title']}. {plan['intro']}".strip()
series = MemoryNode( series = MemoryNode(
node_id=series_id, user_id=user_id, node_id=series_id, user_id=user_id,
+68 -142
View File
@@ -30,7 +30,6 @@ import asyncio
import json import json
import logging import logging
import os import os
import threading
import time import time
import uuid import uuid
from datetime import datetime from datetime import datetime
@@ -378,14 +377,6 @@ class _LoopThread:
return await await_concurrent_future_guarded(future, self._loop) return await await_concurrent_future_guarded(future, self._loop)
def stop(self): def stop(self):
if self._loop.is_closed():
return
if threading.current_thread() is self._thread:
# Called from inside the loop: stopping and joining ourselves would
# deadlock. Just ask run_forever() to return.
self._loop.stop()
return
def _stop(): def _stop():
if self._lag_handle is not None: if self._lag_handle is not None:
self._lag_handle.cancel() self._lag_handle.cancel()
@@ -395,8 +386,6 @@ class _LoopThread:
self._loop.call_soon_threadsafe(_stop) self._loop.call_soon_threadsafe(_stop)
self._thread.join(timeout=5) self._thread.join(timeout=5)
if not self._thread.is_alive() and not self._loop.is_closed():
self._loop.close()
class HyMemoryClient: class HyMemoryClient:
@@ -1192,19 +1181,6 @@ class HyMemoryClient:
logger.warning("[coding] chain observe failed: %s", e) logger.warning("[coding] chain observe failed: %s", e)
return 0 return 0
def _chain_block_enabled(self) -> bool:
"""search 是否注入 coding 版本演进链块(diff / chain_block)。
公共开关:CodingConfig.chain_block_enabled /
MEMORY_CODING_CHAIN_BLOCK_ENABLED(默认 False,opt-in)。
无 _config 时按关闭处理。高级消融开关 HCM_BLOCK_OFF=1 仍在
chain_store.decorate 内生效(仅当本开关已开启时才有意义)。
"""
coding = getattr(getattr(self, "_config", None), "coding", None)
if coding is None:
return False
return bool(getattr(coding, "chain_block_enabled", False))
def _decorate_with_chain(self, facts, instruction: str = "", scope=None, def _decorate_with_chain(self, facts, instruction: str = "", scope=None,
user_id=None, workspace_id=None): user_id=None, workspace_id=None):
"""读取侧:给向量层召回的 facts 挂上版本演进链块。 """读取侧:给向量层召回的 facts 挂上版本演进链块。
@@ -1217,14 +1193,11 @@ class HyMemoryClient:
facts 顺序不重排 —— top-k 是向量层的事,这层只负责"每条 fact 后面跟什么"。 facts 顺序不重排 —— top-k 是向量层的事,这层只负责"每条 fact 后面跟什么"。
best-effort:失败返回 ("", []),退化成纯向量双层(= working 的行为)。 best-effort:失败返回 ("", []),退化成纯向量双层(= working 的行为)。
默认不注入(见 _chain_block_enabled);需显式开启 chain_block_enabled。
""" """
if not facts: if not facts:
return "", [] return "", []
# 走类而不是 self:测试常把本方法绑到裸 SimpleNamespace 上,self 上未必有 # 走类而不是 self:_chain_owner 是 staticmethod,而这两个方法会被测试绑到裸
# 同名 bound method;_chain_owner 同理。 # SimpleNamespace 上调(同 _scene_variant 那处),self 上没有这个属性。
if not HyMemoryClient._chain_block_enabled(self):
return "", []
store = self._get_chain_store( store = self._get_chain_store(
HyMemoryClient._chain_owner(user_id, workspace_id)) HyMemoryClient._chain_owner(user_id, workspace_id))
if store is None: if store is None:
@@ -2179,33 +2152,14 @@ class HyMemoryClient:
from .metrics import MetricsCollector from .metrics import MetricsCollector
return await MetricsCollector.get().get_snapshot(minutes=minutes) return await MetricsCollector.get().get_snapshot(minutes=minutes)
def _stop_owned_loops(self) -> None:
"""Stop every event-loop thread this client owns.
Solo clients create both a write loop and a search loop. Stopping only
the write loop left the search thread (and a self-wakeup timer every
50ms) alive for the process lifetime. Shared runtimes own their loops
and manage them through ``SharedRuntime.aclose()``.
"""
if not getattr(self, "_owns_runtime", False):
return
seen = set()
for loop_thread in (
getattr(self, "_loop_thread", None),
getattr(self, "_search_loop_thread", None),
):
if loop_thread is None or id(loop_thread) in seen:
continue
seen.add(id(loop_thread))
loop_thread.stop()
def close(self): def close(self):
"""关闭客户端,释放所有资源""" """关闭客户端,释放所有资源"""
try: if self._registry is not None:
if self._registry is not None: self._loop_thread.run(self._close_async())
self._loop_thread.run(self._close_async()) # solo 模式:本 client 拥有 loop_thread,可以安全 stop
finally: # shared 模式:loop_thread 由 runtime 管,留给 runtime.aclose() 关
self._stop_owned_loops() if hasattr(self, '_loop_thread') and self._owns_runtime:
self._loop_thread.stop()
def history(self, memory_id: str) -> List[Dict[str, Any]]: def history(self, memory_id: str) -> List[Dict[str, Any]]:
""" """
@@ -2908,81 +2862,6 @@ class HyMemoryClient:
return resp return resp
@staticmethod
def _public_search_views(
result_memories: List[Dict[str, Any]],
*,
wanted_layers: Optional[set],
lite_mode: bool,
) -> Tuple[List[Dict[str, Any]], List[Dict[str, Any]]]:
"""Build the public search items and the private dedup view together.
The dedup marker must come from the same source element as its public
item. Building the dedup view with a second ``zip(result_memories,
all_mems)`` is wrong as soon as one result is dropped by the layer
filter: every later kind shifts by one, and a canonical_state can enter
fuzzy dedup and be shadowed.
"""
from .models.memory import MemoryLayer
all_mems: List[Dict[str, Any]] = []
dedup_mems: List[Dict[str, Any]] = []
for mem in result_memories:
if wanted_layers is not None and mem.get("layer", "") not in wanted_layers:
continue
if lite_mode and mem.get("layer", "") != MemoryLayer.L1_RAW.value:
continue
score = mem.get("score", 0.0)
item = {
"memory_id": mem.get("memory_id", ""),
"content": mem.get("content", ""),
"score": round(score, 4),
"layer": mem.get("layer", ""),
# 多 user 检索时调用方需据此追溯归属(R1)
"user_id": mem.get("user_id", "") or "",
"agent_id": mem.get("agent_id", "") or "",
"source": mem.get("source", "default") or "default",
"retrieval_source": mem.get("retrieval_source", "") or "",
"owner": mem.get("owner"),
"speculate": mem.get("speculate"),
# 不透出 source_raw_memory_id:它指向 L1_RAW 节点,而 raw 对外
# 读路径一律不可见,该 id 到手也查不到东西(按 ID 取返回"不存在"),
# 留着只会让调用方以为能据此追溯原文。
# 节点上仍保留该字段,内部溯源与 clone 重映射不受影响。
"tags": mem.get("tags") or [],
"memory_at": mem.get("memory_at"),
"gmt_created": mem.get("gmt_created"),
# 调用方自定义 meta(透传负载)。由 reader 从 custom["meta"] 取出,
# 未实现该字段的 reader 下为空列表。
"meta": mem.get("meta") or [],
"_schema_channel": bool(mem.get("_schema_channel")),
}
if mem.get("schema_type"):
item["schema_type"] = mem["schema_type"]
if mem.get("workspace_id"):
item["workspace_id"] = mem["workspace_id"]
if mem.get("branch"):
item["branch"] = mem["branch"]
if mem.get("evolution_chain"):
item["evolution_chain"] = mem["evolution_chain"]
if mem.get("capsule_memories") is not None:
item["capsule_memories"] = mem["capsule_memories"]
if mem.get("episode_id"):
item["episode_id"] = mem["episode_id"]
if mem.get("member_ids") is not None:
item["member_ids"] = mem["member_ids"]
if mem.get("timeline") is not None:
item["timeline"] = mem["timeline"]
if mem.get("rerank_score") is not None:
item["rerank_score"] = mem["rerank_score"]
all_mems.append(item)
dedup_item = dict(item)
dedup_item["_memory_kind"] = mem.get("_memory_kind", "")
dedup_mems.append(dedup_item)
return all_mems, dedup_mems
def _spawn_search_dedup( def _spawn_search_dedup(
self, self,
all_mems: List[Dict[str, Any]], all_mems: List[Dict[str, Any]],
@@ -3618,11 +3497,62 @@ class HyMemoryClient:
if effective_search_layers else None if effective_search_layers else None
) )
_all_mems, _dedup_mems = self._public_search_views( _all_mems = []
result.memories, for mem in result.memories:
wanted_layers=_wanted_layers, if _wanted_layers is not None and mem.get("layer", "") not in _wanted_layers:
lite_mode=lite_mode, continue
) if lite_mode and mem.get("layer", "") != MemoryLayer.L1_RAW.value:
continue
score = mem.get("score", 0.0)
item = {
"memory_id": mem.get("memory_id", ""),
"content": mem.get("content", ""),
"score": round(score, 4),
"layer": mem.get("layer", ""),
# 多 user 检索时调用方需据此追溯归属(R1)
"user_id": mem.get("user_id", "") or "",
"agent_id": mem.get("agent_id", "") or "",
"source": mem.get("source", "default") or "default",
"retrieval_source": mem.get("retrieval_source", "") or "",
"owner": mem.get("owner"),
"speculate": mem.get("speculate"),
# 不透出 source_raw_memory_id:它指向 L1_RAW 节点,而 raw 对外
# 读路径一律不可见,该 id 到手也查不到东西(按 ID 取返回"不存在"),
# 留着只会让调用方以为能据此追溯原文。
# 节点上仍保留该字段,内部溯源与 clone 重映射不受影响。
"tags": mem.get("tags") or [],
"memory_at": mem.get("memory_at"),
"gmt_created": mem.get("gmt_created"),
# 调用方自定义 meta(透传负载)。由 reader 从 custom["meta"] 取出,
# 未实现该字段的 reader 下为空列表。
"meta": mem.get("meta") or [],
"_schema_channel": bool(mem.get("_schema_channel")),
}
if mem.get("schema_type"):
item["schema_type"] = mem["schema_type"]
if mem.get("workspace_id"):
item["workspace_id"] = mem["workspace_id"]
if mem.get("branch"):
item["branch"] = mem["branch"]
if mem.get("evolution_chain"):
item["evolution_chain"] = mem["evolution_chain"]
if mem.get("capsule_memories") is not None:
item["capsule_memories"] = mem["capsule_memories"]
if mem.get("episode_id"):
item["episode_id"] = mem["episode_id"]
if mem.get("member_ids") is not None:
item["member_ids"] = mem["member_ids"]
if mem.get("timeline") is not None:
item["timeline"] = mem["timeline"]
if mem.get("rerank_score") is not None:
item["rerank_score"] = mem["rerank_score"]
_all_mems.append(item)
_dedup_mems = []
for source, public in zip(result.memories, _all_mems):
copy = dict(public)
copy["_memory_kind"] = source.get("_memory_kind", "")
_dedup_mems.append(copy)
# search 链路去重:仅对 L2_FACT + L4_IDENTITY(重复重灾区)判重删库。 # search 链路去重:仅对 L2_FACT + L4_IDENTITY(重复重灾区)判重删库。
# L0 及其他层不参与。fire-and-forget,不阻塞本次返回(清理面向后续查询)。 # L0 及其他层不参与。fire-and-forget,不阻塞本次返回(清理面向后续查询)。
@@ -6173,10 +6103,9 @@ class HyMemoryClient:
effective_agent_ids = agent_ids if agent_ids is not None else [] effective_agent_ids = agent_ids if agent_ids is not None else []
if not effective_agent_ids: if not effective_agent_ids:
# 查所有 agent 的历史:isolation_key 前缀匹配。 # 查所有 agent 的历史:用 "user_id:" 前缀匹配
# 不能退回 "u:default" —— add() 的默认 agent 是 "default_agent", isolation_key = self._build_isolation_key(uid, "default")
# 默认用法下会查空。 return await self._history_store.get_recent(isolation_key, limit=limit)
return await self._history_store.get_recent_for_user(uid, limit=limit)
else: else:
# 查指定 agent 的历史 # 查指定 agent 的历史
all_results = [] all_results = []
@@ -6202,7 +6131,4 @@ class HyMemoryClient:
return self return self
async def __aexit__(self, *args): async def __aexit__(self, *args):
try: await self._close_async()
await self._close_async()
finally:
self._stop_owned_loops()
+1 -3
View File
@@ -647,9 +647,7 @@ class HyCodeMem:
top-k,但它们的 diff 对当前工作区不可能有用,因此挂链前必须做同仓过滤。 top-k,但它们的 diff 对当前工作区不可能有用,因此挂链前必须做同仓过滤。
""" """
facts = facts or [] facts = facts or []
# 高级消融开关: HCM_BLOCK_OFF=1 → 完全不出块, 也不从 bullet 删任何 fact。 # 开关: HCM_BLOCK_OFF=1 → 完全不出块, 也不从 bullet 删任何 fact。
# 公共 opt-in 见 CodingConfig.chain_block_enabled /
# MEMORY_CODING_CHAIN_BLOCK_ENABLED(默认关;客户端在调用 decorate 前拦截)。
if os.environ.get("HCM_BLOCK_OFF") == "1": if os.environ.get("HCM_BLOCK_OFF") == "1":
return "", [] return "", []
picked, used = [], [] picked, used = [], []
-7
View File
@@ -1136,9 +1136,6 @@ class CodingConfig:
# LLM 失败或没有抽出 fact 时,是否把对话原文存成 working fact。 # LLM 失败或没有抽出 fact 时,是否把对话原文存成 working fact。
# 默认关闭,避免 assistant/tool 轨迹以 statement 形式直接暴露给消费方。 # 默认关闭,避免 assistant/tool 轨迹以 statement 形式直接暴露给消费方。
verbatim_fallback_enabled: bool = None verbatim_fallback_enabled: bool = None
# search(scene="coding") 是否注入版本演进链块(diff / chain_block)。
# 默认关闭(opt-in):中重度 coding 用户可省下大量 prompt token;写入侧仍可 observe。
chain_block_enabled: bool = None
def __post_init__(self): def __post_init__(self):
if self.enable is None: if self.enable is None:
@@ -1156,10 +1153,6 @@ class CodingConfig:
self.verbatim_fallback_enabled = _get_env_bool( self.verbatim_fallback_enabled = _get_env_bool(
"MEMORY_CODING_VERBATIM_FALLBACK_ENABLED", False "MEMORY_CODING_VERBATIM_FALLBACK_ENABLED", False
) )
if self.chain_block_enabled is None:
self.chain_block_enabled = _get_env_bool(
"MEMORY_CODING_CHAIN_BLOCK_ENABLED", False
)
@dataclass @dataclass
+16 -59
View File
@@ -464,67 +464,24 @@ class MysqlCache(CacheBase):
# ================================================================ # ================================================================
async def store_metrics_minute(self, minute_ts: str, data: dict) -> None: async def store_metrics_minute(self, minute_ts: str, data: dict) -> None:
import aiomysql
try: try:
now = datetime.now().isoformat() now = datetime.now().isoformat()
# Read-merge-write must be one transaction: two instances flushing # 尝试 UPSERT
# the same minute used to overwrite each other's counters, and a existing = await self._fetchone(
# concurrent insert could lose the whole batch to a duplicate key. "SELECT data FROM system_metrics WHERE minute_ts = %s", (minute_ts,)
for attempt in range(2): )
try: if existing:
async with self._pool.acquire() as conn: old_data = json.loads(existing["data"])
await conn.begin() merged = self._merge_metric_buckets(old_data, data)
try: await self._execute(
async with conn.cursor(aiomysql.DictCursor) as cur: "UPDATE system_metrics SET data = %s, created_at = %s WHERE minute_ts = %s",
await cur.execute( (json.dumps(merged, default=str), now, minute_ts),
"SELECT data FROM system_metrics " )
"WHERE minute_ts = %s FOR UPDATE", else:
(minute_ts,), await self._execute(
) "INSERT INTO system_metrics (minute_ts, data, created_at) VALUES (%s, %s, %s)",
row = await cur.fetchone() (minute_ts, json.dumps(data, default=str), now),
if row is not None: )
old_raw = (
row["data"]
if isinstance(row, dict)
else row[0]
)
merged = self._merge_metric_buckets(
json.loads(old_raw), data,
)
await cur.execute(
"UPDATE system_metrics SET data = %s, "
"created_at = %s WHERE minute_ts = %s",
(
json.dumps(merged, default=str),
now,
minute_ts,
),
)
else:
await cur.execute(
"INSERT INTO system_metrics "
"(minute_ts, data, created_at) "
"VALUES (%s, %s, %s)",
(
minute_ts,
json.dumps(data, default=str),
now,
),
)
await conn.commit()
return
except Exception:
await conn.rollback()
raise
except (aiomysql.IntegrityError, aiomysql.OperationalError) as error:
code = error.args[0] if error.args else None
deadlock = code in (1205, 1213)
if attempt or (isinstance(error, aiomysql.OperationalError) and not deadlock):
raise
# A competing writer inserted the row, or InnoDB chose us
# as the deadlock victim. Retry so this batch merges.
continue
except Exception as e: except Exception as e:
logger.warning(f"store_metrics_minute failed: {e}") logger.warning(f"store_metrics_minute failed: {e}")
-33
View File
@@ -66,11 +66,6 @@ def _now_iso() -> str:
return datetime.now(timezone.utc).isoformat() return datetime.now(timezone.utc).isoformat()
def _escape_like(value: str) -> str:
"""Escape LIKE wildcards so a user id is matched literally."""
return value.replace("\\", "\\\\").replace("%", "\\%").replace("_", "\\_")
class HistoryStore: class HistoryStore:
""" """
SQLite 历史存储 - 记录所有内存操作的审计日志。 SQLite 历史存储 - 记录所有内存操作的审计日志。
@@ -266,34 +261,6 @@ class HistoryStore:
) )
return [dict(row) for row in cursor.fetchall()] return [dict(row) for row in cursor.fetchall()]
async def get_recent_for_user(
self,
user_id: str,
limit: int = 50,
) -> List[Dict[str, Any]]:
"""获取某 user 下所有 agent 的最近操作。
isolation_key 形如 ``"{user_id}:{agent_id}"``。user_id 里的 LIKE
通配符会被转义,保证只匹配这个 user,不会漏到 ``u1`` 之外。
"""
prefix = _escape_like(f"{user_id}:") + "%"
return await _run_in_sqlite_pool(
self._get_recent_prefix_sync, prefix, limit,
)
def _get_recent_prefix_sync(
self, pattern: str, limit: int,
) -> List[Dict[str, Any]]:
with self._conn_lock:
if self._conn is None:
return []
cursor = self._conn.execute(
"SELECT * FROM memory_history WHERE isolation_key LIKE ? ESCAPE '\\' "
"ORDER BY created_at DESC LIMIT ?",
(pattern, limit),
)
return [dict(row) for row in cursor.fetchall()]
async def count(self, isolation_key: str = "") -> int: async def count(self, isolation_key: str = "") -> int:
""" """
统计历史记录数量。 统计历史记录数量。
+8 -33
View File
@@ -462,39 +462,14 @@ class FaissVectorStore(VectorStoreBase):
return None return None
async def get_embeddings(self, node_ids: List[str]) -> Dict[str, List[float]]: async def get_embeddings(self, node_ids: List[str]) -> Dict[str, List[float]]:
"""批量取向量,供去重 / episode 摘要计算 cosine。 """批量取向量(in-memory _data 直接读 embedding),供去重算 cosine。"""
out: Dict[str, List[float]] = {}
``_data`` 只保存 payload 与 FAISS 内部下标;普通 upsert 不额外存一份 for nid in node_ids:
embedding,向量本体在 FAISS index 里。这里按当前有效下标 reconstruct, point_id = self._node_id_to_point_id(nid)
并用 ``_idx_to_id`` 校验,避免读到已被替换的旧下标。 entry = self._data.get(point_id)
""" if entry and entry.get("embedding"):
out[nid] = list(entry["embedding"])
def _get() -> Dict[str, List[float]]: return out
out: Dict[str, List[float]] = {}
with self._lock:
for nid in node_ids:
point_id = self._node_id_to_point_id(nid)
entry = self._data.get(point_id)
if entry is None:
continue
embedding = entry.get("embedding")
if embedding:
out[nid] = list(embedding)
continue
idx = entry.get("idx")
if (
idx is None
or self._index is None
or self._idx_to_id.get(int(idx)) != point_id
):
continue
try:
out[nid] = self._index.reconstruct(int(idx)).tolist()
except Exception:
continue
return out
return await _run_in_vdb_pool(_get)
# ================================================================ # ================================================================
# 删除 # 删除
+1 -1
View File
@@ -968,7 +968,7 @@ class MemoryWriter(WritePipeline):
async def _run(): async def _run():
try: try:
nid = await summarize_closed_episode( nid = await summarize_closed_episode(
vector_store=await self._get_vector_store(), vector_store=self.vector_store,
embed_service=self.embed_service, embed_service=self.embed_service,
llm_provider=self.llm_provider, llm_provider=self.llm_provider,
episode_id=episode_id, episode_id=episode_id,
+6 -19
View File
@@ -29,7 +29,7 @@ import logging
import os import os
import re import re
import signal import signal
import threading import sys
from datetime import datetime from datetime import datetime
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from typing import Any, Dict, Optional from typing import Any, Dict, Optional
@@ -495,16 +495,6 @@ class MemoryHTTPHandler(BaseHTTPRequestHandler):
_json_response(self, 200, result) _json_response(self, 200, result)
def _request_shutdown(server: ThreadingHTTPServer) -> None:
"""Ask ``serve_forever`` to stop without blocking the calling thread.
``BaseServer.shutdown()`` waits for the ``serve_forever`` loop to exit. A
signal handler runs on the very thread executing that loop, so calling it
there deadlocks forever. Hand the call to a helper thread instead.
"""
threading.Thread(target=server.shutdown, daemon=True).start()
def run_server(port: int = 19527, host: str = "127.0.0.1"): def run_server(port: int = 19527, host: str = "127.0.0.1"):
"""Start the HTTP server (blocking).""" """Start the HTTP server (blocking)."""
# Configure logging — basicConfig for root; hy_memory level set after client init # Configure logging — basicConfig for root; hy_memory level set after client init
@@ -533,12 +523,13 @@ def run_server(port: int = 19527, host: str = "127.0.0.1"):
logger.info(f"[server] Listening on http://{host}:{port}") logger.info(f"[server] Listening on http://{host}:{port}")
logger.info(f"[server] Health check: http://{host}:{port}/healthz") logger.info(f"[server] Health check: http://{host}:{port}/healthz")
# Graceful shutdown. The signal handler only asks a helper thread to stop # Graceful shutdown
# the loop; cleanup runs after serve_forever() returns so it cannot race
# the request threads or deadlock the serving thread.
def _shutdown(signum, frame): def _shutdown(signum, frame):
logger.info("[server] Shutting down...") logger.info("[server] Shutting down...")
_request_shutdown(server) server.shutdown()
if _client is not None:
_client.close()
sys.exit(0)
signal.signal(signal.SIGINT, _shutdown) signal.signal(signal.SIGINT, _shutdown)
signal.signal(signal.SIGTERM, _shutdown) signal.signal(signal.SIGTERM, _shutdown)
@@ -547,10 +538,6 @@ def run_server(port: int = 19527, host: str = "127.0.0.1"):
server.serve_forever() server.serve_forever()
except KeyboardInterrupt: except KeyboardInterrupt:
_shutdown(None, None) _shutdown(None, None)
finally:
server.server_close()
if _client is not None:
_client.close()
def main(): def main():
+4 -4
View File
@@ -18,13 +18,13 @@ If you run into a problem, please open an Issue with the version, platform,repro
## 发行与平台合同 ## 发行与平台合同
公共 Python SDK 为 `hy-memory==2.0.4`。MemHub 源码版本为 `1.0.4` 公共 Python SDK 为 `hy-memory==2.0.3`。MemHub 源码版本为 `1.0.3`
(build 13),GitHub 仓库为 (build 7),GitHub 仓库为
[`Tencent-Hunyuan/Hy-Memory`](https://github.com/Tencent-Hunyuan/Hy-Memory),npm 包名为 [`Tencent-Hunyuan/Hy-Memory`](https://github.com/Tencent-Hunyuan/Hy-Memory),npm 包名为
`memhub-cli`。当前代码架构见 `memhub-cli`。当前代码架构见
[`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md)。 [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md)。
`v1.0.4` 的发行目标是 Linux x64/arm64 Runtime 与 macOS arm64/x64 App 和 Runtime。 `v1.0.3` 的发行目标是 Linux x64/arm64 Runtime 与 macOS arm64/x64 App 和 Runtime。
Linux 制品由 GitHub Actions 在原生架构 runner 上构建并完成生命周期验收;macOS Linux 制品由 GitHub Actions 在原生架构 runner 上构建并完成生命周期验收;macOS
制品由发行流水线完成 Developer ID 签名、公证和验收后手动上传。Windows 不支持。 制品由发行流水线完成 Developer ID 签名、公证和验收后手动上传。Windows 不支持。
只有同时出现在 GitHub Release 与 `release-manifest.json` 中、且通过公开校验的制品才属于 只有同时出现在 GitHub Release 与 `release-manifest.json` 中、且通过公开校验的制品才属于
@@ -113,7 +113,7 @@ MemHub 的 SQLite 数据默认位于 `~/.one-memory`,不在 npm 包或 Runtime
## Hy Memory ## Hy Memory
当前开发依赖固定为公开 PyPI 包 `hy-memory==2.0.4`。SDK 配置、权限边界及 search 当前开发依赖固定为公开 PyPI 包 `hy-memory==2.0.3`。SDK 配置、权限边界及 search
返回格式见 [`MEMORY.md`](MEMORY.md)。HY search 返回的 `content` 已包含 SDK 统一生成的 返回格式见 [`MEMORY.md`](MEMORY.md)。HY search 返回的 `content` 已包含 SDK 统一生成的
逻辑时间和演化上下文;调用方不应再次转换时间戳或重复拼接 `When`。 逻辑时间和演化上下文;调用方不应再次转换时间戳或重复拼接 `When`。
@@ -76,7 +76,8 @@ def _memory(db: sqlite3.Connection, reference: CallReference) -> SourceRecordRef
if row["status"] == "deleted" or row["deleted_at"] is not None: if row["status"] == "deleted" or row["deleted_at"] is not None:
return _hidden(reference) return _hidden(reference)
# Same local revocation/ownership predicate as SourceReadAccess.memories. # Same local revocation/ownership predicate as SourceReadAccess.memories,
# evaluated BEFORE freshness (which can read support text), in this snapshot.
# Its public methods open their own connections, so cannot be used here. # Its public methods open their own connections, so cannot be used here.
access = db.execute( access = db.execute(
"SELECT EXISTS (SELECT 1 FROM governed_memory_evidence " "SELECT EXISTS (SELECT 1 FROM governed_memory_evidence "
@@ -120,8 +121,8 @@ def resolve_call_references(database_path: Path, event: CallEvent) -> list[Sourc
Local-owner only: event host attribution is NOT authorization. Explicit local Local-owner only: event host attribution is NOT authorization. Explicit local
source revocations deny access; absent grants retain legacy local semantics. source revocations deny access; absent grants retain legacy local semantics.
A memory needs at least one accessible active citation. Evidence freshness A memory needs at least one accessible active citation. No proof preserves
proofs are retired and do not affect availability. shared freshness compatibility, not a claim of verified historical freshness.
Any read/parse/budget failure raises a fixed whole-request error, never a Any read/parse/budget failure raises a fixed whole-request error, never a
partial result or per-item deletion. The route owner must map this separately partial result or per-item deletion. The route owner must map this separately
@@ -34,11 +34,10 @@ class ScanIssueCode(StrEnum):
SOURCE_BUSY = "source_busy" SOURCE_BUSY = "source_busy"
UNSUPPORTED_SCHEMA = "unsupported_schema" UNSUPPORTED_SCHEMA = "unsupported_schema"
BUDGET_EXCEEDED = "budget_exceeded" BUDGET_EXCEEDED = "budget_exceeded"
#: The file is bigger than a per-file ceiling: either the import's own #: The file is bigger than the import's own per-file ceiling. Skipped with
#: ceiling or the access gate's per-root `max_file_bytes`. Both are skipped #: a reason rather than read, because one oversized history file can hold
#: with this reason rather than read, because one oversized history file #: the import open long enough to take the process with it. Distinct from
#: can hold the import open long enough to take the process with it, and #: the access layer's `FILE_TOO_LARGE`, which is a per-root limit.
#: the user-facing cause is the same size limit.
FILE_TOO_LARGE = "file_too_large" FILE_TOO_LARGE = "file_too_large"
HISTORY_WINDOW_EXCLUDED = "history_window_excluded" HISTORY_WINDOW_EXCLUDED = "history_window_excluded"
@@ -1120,11 +1120,6 @@ def discovery_failure_code(
) -> ScanIssueCode: ) -> ScanIssueCode:
if failure.reason_code is SourceAccessErrorCode.BUDGET_EXCEEDED: if failure.reason_code is SourceAccessErrorCode.BUDGET_EXCEEDED:
return ScanIssueCode.BUDGET_EXCEEDED return ScanIssueCode.BUDGET_EXCEEDED
if failure.reason_code is SourceAccessErrorCode.FILE_TOO_LARGE:
# The file is readable and its format is fine; only its size is over
# the access gate. Reporting that as `unsupported_schema` sends the
# user looking for a parser problem that does not exist.
return ScanIssueCode.FILE_TOO_LARGE
return ScanIssueCode.UNSUPPORTED_SCHEMA return ScanIssueCode.UNSUPPORTED_SCHEMA
@@ -1132,8 +1127,6 @@ def _scan_failure_code(code: SourceAccessErrorCode) -> ScanIssueCode:
"""Map a per-file open failure onto a machine code, never onto free text.""" """Map a per-file open failure onto a machine code, never onto free text."""
if code is SourceAccessErrorCode.BUDGET_EXCEEDED: if code is SourceAccessErrorCode.BUDGET_EXCEEDED:
return ScanIssueCode.BUDGET_EXCEEDED return ScanIssueCode.BUDGET_EXCEEDED
if code is SourceAccessErrorCode.FILE_TOO_LARGE:
return ScanIssueCode.FILE_TOO_LARGE
if code is SourceAccessErrorCode.SYMLINK_NOT_ALLOWED: if code is SourceAccessErrorCode.SYMLINK_NOT_ALLOWED:
return ScanIssueCode.UNSUPPORTED_SCHEMA return ScanIssueCode.UNSUPPORTED_SCHEMA
if code in { if code in {
@@ -1,25 +1,36 @@
"""Memory evidence freshness checks are retired. """Read-only, exact-revision support checks inside the caller's SQLite snapshot.
Read paths used to re-validate ``memory_task_outputs.support_refs`` against No proof is legacy/unknown, not stale. One complete fresh proof suffices; all
live ``evidence`` rows via ``is_memory_evidence_current``. That design hid complete proofs stale means false. Malformed records raise. An exceeded budget
memories when evidence changed, but at high cost (library 503s, per-row SQL, means the memory is too large to verify, not that it is stale: it reads as
segment-key breakage). Call sites still import this module; registration is current, the same as having no proof, and is reported once in a while as a
now a no-op that always admits the row. content-free diagnostic. One such memory used to fail every COUNT over the
library, so the homepage returned 503 for as long as it existed (D76).
Proof rows may still be written by the ingest path; they are ignored on read. Set ONE_MEMORY_FRESHNESS_BUDGET_RAISES=1 to restore the old raising behavior.
See ``新鲜度_todo.md`` for the remaining teardown steps. The caller owns BEGIN/deadline/progress-handler and error-to-unavailable mapping.
""" """
from __future__ import annotations import json
import logging
import os
import sqlite3 import sqlite3
import threading
import time
from collections.abc import Iterator, Sequence from collections.abc import Iterator, Sequence
from contextlib import contextmanager from contextlib import contextmanager
from one_memory.database_access import register_function from one_memory.database_access import register_function
from one_memory.diagnostic_log import diagnostic_event
from one_memory.imports.models import EvidenceRecord
from one_memory.memory_tasks import MemoryTaskOutput, matches_evidence_ref
#: Historical budget error codes. Retained so diagnostics/env docs do not break _PROOFS = 64
#: mid-teardown; nothing raises them on the read path anymore. _REFERENCES = 128
_BODY_BYTES = 65_536
_EVIDENCE_BYTES = 8 * 1024 * 1024
_SCOPE_STEPS = 64
#: Budget refusals: the check could not finish, the data is not malformed.
BUDGET_ERRORS = frozenset( BUDGET_ERRORS = frozenset(
{ {
"memory_freshness_proof_budget", "memory_freshness_proof_budget",
@@ -28,23 +39,80 @@ BUDGET_ERRORS = frozenset(
"memory_freshness_scope_budget", "memory_freshness_scope_budget",
} }
) )
_REPORT_INTERVAL_SECONDS = 60.0
_reported: dict[str, tuple[float, int]] = {}
_reported_lock = threading.Lock()
def budget_errors_raise() -> bool: def budget_errors_raise() -> bool:
"""Former opt-in to raise on budget overrun. Always false after retirement.""" """Opt back into the pre-D76 behavior, where one large memory fails the query."""
return False return os.environ.get("ONE_MEMORY_FRESHNESS_BUDGET_RAISES", "").strip() == "1"
def has_proof_table(db: sqlite3.Connection) -> bool: def _report_budget(code: str) -> None:
"""Whether ``memory_task_outputs`` exists. Unused by the retired check.""" """Count every budget skip; write at most one line per reason per interval."""
return ( now = time.monotonic()
db.execute( with _reported_lock:
"SELECT 1 FROM sqlite_master WHERE type='table' AND name='memory_task_outputs'" last, pending = _reported.get(code, (float("-inf"), 0))
).fetchone() pending += 1
is not None if now - last < _REPORT_INTERVAL_SECONDS:
_reported[code] = (last, pending)
return
_reported[code] = (now, 0)
diagnostic_event(
level="warn",
component="memory.freshness",
event="memory_freshness.budget_exceeded",
outcome="skipped",
details={"reason": code, "count": pending},
) )
def _object(pairs: list[tuple[str, object]]) -> dict[str, object]:
result: dict[str, object] = {}
for key, value in pairs:
if key in result:
raise ValueError("memory_freshness_invalid_proof")
result[key] = value
return result
class _Prefetched:
"""Prefetched rows for the current batch, private to thread and connection.
Scoped, never global: a cache that outlived the transaction would answer
from rows a later write had already replaced.
"""
def __init__(self) -> None:
self._local = threading.local()
def get(self) -> dict[str, list[sqlite3.Row]] | None:
return getattr(self._local, "rows", None)
def evidence(self) -> dict[str, sqlite3.Row] | None:
return getattr(self._local, "evidence", None)
@contextmanager
def using(
self,
rows: dict[str, list[sqlite3.Row]],
evidence: dict[str, sqlite3.Row] | None = None,
) -> Iterator[None]:
previous = getattr(self._local, "rows", None)
previous_evidence = getattr(self._local, "evidence", None)
self._local.rows = rows
self._local.evidence = evidence
try:
yield
finally:
self._local.rows = previous
self._local.evidence = previous_evidence
_prefetched_proofs = _Prefetched()
@contextmanager @contextmanager
def freshness_proofs_prefetched( def freshness_proofs_prefetched(
db: sqlite3.Connection, db: sqlite3.Connection,
@@ -52,16 +120,328 @@ def freshness_proofs_prefetched(
*, *,
evidence_ids: Sequence[str] = (), evidence_ids: Sequence[str] = (),
) -> Iterator[None]: ) -> Iterator[None]:
"""No-op. Kept so batched read callers need not change in this PR.""" """Serve the freshness check's proof reads from one query.
del db, memory_ids, evidence_ids
yield The decisions are unchanged — the rows come from the same transaction —
but the per-memory reads collapse into the one in `proof_rows_for`.
"""
with _prefetched_proofs.using(
proof_rows_for(db, memory_ids),
evidence_rows_for(db, evidence_ids) if evidence_ids else None,
):
yield
def has_proof_table(db: sqlite3.Connection) -> bool:
"""Whether this database keeps proofs at all.
Without the table the registered UDF answers 1 unconditionally, so there
is nothing to prefetch and nothing to read.
"""
return (
db.execute(
"SELECT 1 FROM sqlite_master WHERE type='table' AND name='memory_task_outputs'"
).fetchone()
is not None
)
_EVIDENCE_COLUMNS = (
"evidence_id,user_id,source_id,workspace,project,conversation_id,"
"message_id,correlation_id,source_ordinal,content_kind,role,written_by,"
"created_at,verbatim,content_hash,revision,imported_at,updated_at"
)
#: The budget measure the single read applies, kept as one string so the
#: batched read cannot compute it differently.
_EVIDENCE_SIZE_SQL = (
"length(CAST(verbatim AS BLOB)) + "
"length(CAST(json_array(evidence_id,user_id,source_id,workspace,project,"
"conversation_id,message_id,correlation_id,source_ordinal,content_kind,"
"role,written_by,created_at,content_hash,revision,imported_at,updated_at) "
"AS BLOB))"
)
def evidence_rows_for(
db: sqlite3.Connection,
evidence_ids: Sequence[str],
) -> dict[str, sqlite3.Row]:
"""Read the cited evidence rows for a page in one query.
The check reads every citation twice — once for its size, against the
evidence budget, and once for the row it validates against the proof's
reference. Both are per citation, so both grow with the page.
The size comes from the same expression the single read uses, so the
budget arithmetic cannot drift between the two paths.
"""
if not evidence_ids:
return {}
cursor = db.cursor()
cursor.row_factory = sqlite3.Row
by_id: dict[str, sqlite3.Row] = {}
for start in range(0, len(evidence_ids), 900):
batch = list(dict.fromkeys(evidence_ids[start : start + 900]))
placeholders = ",".join("?" for _ in batch)
for row in cursor.execute(
f"SELECT {_EVIDENCE_COLUMNS},{_EVIDENCE_SIZE_SQL} AS measure "
f"FROM evidence WHERE evidence_id IN ({placeholders})",
batch,
):
by_id[str(row["evidence_id"])] = row
return by_id
def proof_rows_for(
db: sqlite3.Connection,
memory_ids: Sequence[str],
) -> dict[str, list[sqlite3.Row]]:
"""Read the proof rows for a whole page in one query.
The freshness check reads these per memory, and that is the read path's
whole cost: measured on the old-user library, 160 memories cost 0.69 s in
the UDF against 0.08 s for the citation subquery, and grew further while
the library was also being written to — enough to cross the five second
interrupt on `/api/v1/memories` and on the map.
Returns only ids that have rows; a caller falling back to a single read
for anything absent keeps the same answer.
"""
if not memory_ids:
return {}
if not has_proof_table(db):
return {}
cursor = db.cursor()
cursor.row_factory = sqlite3.Row
by_id: dict[str, list[sqlite3.Row]] = {}
for start in range(0, len(memory_ids), 400):
batch = list(memory_ids[start : start + 400])
placeholders = ",".join("?" for _ in batch)
for row in cursor.execute(
"SELECT task_id,memory_id,length(CAST(body AS BLOB)) AS size,"
"substr(CAST(body AS BLOB),1,?) AS body FROM memory_task_outputs "
f"WHERE memory_id IN ({placeholders})",
(_BODY_BYTES + 1, *batch),
):
by_id.setdefault(str(row["memory_id"]), []).append(row)
return by_id
def _proof_revision(
cursor: sqlite3.Cursor,
memory_id: str,
revision: int,
user_id: str,
available: set[int],
) -> int:
if revision in available or not available:
return revision
if (
cursor.execute(
"SELECT 1 FROM sqlite_master WHERE type='table' AND name='governed_memory_versions'"
).fetchone()
is None
):
return revision
original = revision
for step in range(_SCOPE_STEPS + 1):
if revision in available:
return revision
row = cursor.execute(
"SELECT v.action, typeof(v.content)='text' AND typeof(p.content)='text' "
"AND CAST(v.content AS BLOB)=CAST(p.content AS BLOB) AS same_content, "
"v.status=p.status AS same_status, "
"CAST(v.content AS BLOB)=CAST(m.content AS BLOB) AS current_content "
"FROM governed_memory_versions v "
"JOIN governed_memory_versions p ON p.memory_id=v.memory_id "
"AND p.revision=v.revision-1 "
"JOIN governed_memories m ON m.memory_id=v.memory_id "
"WHERE v.memory_id=? AND v.revision=? AND m.user_id=? AND m.revision=?",
(memory_id, revision, user_id, original),
).fetchone()
if row is None or row[0] != "set_scopes" or not all(row[index] == 1 for index in (1, 2, 3)):
return original
if step == _SCOPE_STEPS:
raise ValueError("memory_freshness_scope_budget")
revision -= 1
raise ValueError("memory_freshness_scope_budget")
def register_memory_freshness(db: sqlite3.Connection) -> str: def register_memory_freshness(db: sqlite3.Connection) -> str:
"""Register a always-true UDF; return SQL predicate ``1``. """Register UDF; return a predicate for alias m. Does not change row_factory.
Callers that embed the return value in ``WHERE`` keep working. Callers that Production initialization supplies memory_task_outputs(memory_id) index.
invoke ``is_memory_evidence_current`` by name also get current. Old no-index databases depend on the caller's SQL deadline. Per evaluation:
65 rows / 64KiB each, 129 active IDs, 8MiB total Evidence material. No library
bodies are loaded; Evidence objects live only for this evaluation. No cache
persists across transactions (safe when a caller reuses its connection).
""" """
register_function(db, "is_memory_evidence_current", 3, lambda *_: 1) if not has_proof_table(db):
return "1" register_function(db, "is_memory_evidence_current", 3, lambda *_: 1)
return "1"
raising = budget_errors_raise()
def current(memory_id: str, revision: int, user_id: str) -> int:
try:
return checked(memory_id, revision, user_id)
except ValueError as error:
code = str(error)
if raising or code not in BUDGET_ERRORS:
raise
_report_budget(code)
return 1
except sqlite3.Error as error:
# A freshness proof is an enhancement, never the reason a person
# cannot see their library. Under a concurrent history import the
# 5s read deadline interrupts this UDF mid-statement; the old
# behaviour let that escape, and the caller turned it into
# `memory_library_unavailable`, so the memory list stayed empty for
# as long as the import ran. Answer "current" instead and record
# why, which is the same trade the D76 budget skip already makes.
_report_budget("memory_freshness_sql_error")
logging.getLogger(__name__).warning(
"memory_freshness_udf_failed %s: %s", type(error).__name__, error
)
return 1
def checked(memory_id: str, revision: int, user_id: str) -> int:
if not isinstance(memory_id, str) or type(revision) is not int or revision < 1:
raise ValueError("memory_freshness_invalid_identity")
if not isinstance(user_id, str):
raise ValueError("memory_freshness_invalid_identity")
cursor = db.cursor()
cursor.row_factory = sqlite3.Row
prefetched = _prefetched_proofs.get()
if prefetched is not None:
rows = prefetched.get(memory_id, ())
else:
rows = cursor.execute(
"SELECT task_id,memory_id,length(CAST(body AS BLOB)) AS size,"
"substr(CAST(body AS BLOB),1,?) AS body FROM memory_task_outputs "
"WHERE memory_id=? LIMIT ?",
(_BODY_BYTES + 1, memory_id, _PROOFS + 1),
).fetchall()
rows = list(rows)
if len(rows) > _PROOFS:
raise ValueError("memory_freshness_proof_budget")
decoded = []
for row in rows:
if type(row["size"]) is not int or row["size"] > _BODY_BYTES:
raise ValueError("memory_freshness_proof_budget")
body = json.loads(row["body"], object_pairs_hook=_object)
if not isinstance(body, dict) or type(body.get("memory_revision")) is not int:
raise ValueError("memory_freshness_invalid_proof")
decoded.append((row, body))
revision = _proof_revision(
cursor,
memory_id,
revision,
user_id,
{body["memory_revision"] for _, body in decoded},
)
applicable = []
for row, body in decoded:
if body["memory_revision"] != revision:
continue
raw_refs = body.get("support_refs", [])
if not isinstance(raw_refs, list) or any(
not isinstance(ref, dict)
or set(ref) - {"evidence_id", "revision", "content_hash", "identity_hash"}
for ref in raw_refs
):
raise ValueError("memory_freshness_invalid_reference")
output = MemoryTaskOutput.model_validate_json(row["body"], strict=True)
if output.memory_id != memory_id or output.task_id != row["task_id"]:
raise ValueError("memory_freshness_proof_identity")
if len(output.support_refs) > _REFERENCES:
raise ValueError("memory_freshness_reference_budget")
for ref in output.support_refs:
if (
set(ref) - {"evidence_id", "revision", "content_hash", "identity_hash"}
or not ref["evidence_id"]
or ref["revision"] < 1
or any(
len(ref[key]) != 64
or any(char not in "0123456789abcdef" for char in ref[key])
for key in ("content_hash", "identity_hash")
if key in ref
)
):
raise ValueError("memory_freshness_invalid_reference")
proof_ids = [ref["evidence_id"] for ref in output.support_refs]
if len(proof_ids) != len(set(proof_ids)):
raise ValueError("memory_freshness_duplicate_reference")
applicable.append(output)
if not applicable:
return 1
active = cursor.execute(
"SELECT evidence_id FROM governed_memory_evidence "
"WHERE memory_id=? AND active=1 LIMIT ?",
(memory_id, _REFERENCES + 1),
).fetchall()
if len(active) > _REFERENCES:
raise ValueError("memory_freshness_reference_budget")
ids = {row[0] for row in active}
complete = [
output
for output in applicable
if output.support_refs and {ref["evidence_id"] for ref in output.support_refs} == ids
]
if not complete:
return 1
records: dict[str, EvidenceRecord | None] = {}
used = 0
prefetched_evidence = _prefetched_proofs.evidence()
for identifier in ids:
evidence_row = (
prefetched_evidence.get(identifier) if prefetched_evidence is not None else None
)
if evidence_row is not None:
size = evidence_row["measure"]
if type(size) is not int or size < 0:
raise ValueError("memory_freshness_invalid_evidence")
else:
measured = cursor.execute(
f"SELECT {_EVIDENCE_SIZE_SQL} FROM evidence WHERE evidence_id=?",
(identifier,),
).fetchone()
if measured is None:
records[identifier] = None
continue
if type(measured[0]) is not int or measured[0] < 0:
raise ValueError("memory_freshness_invalid_evidence")
size = measured[0]
used += size
if used > _EVIDENCE_BYTES:
raise ValueError("memory_freshness_evidence_budget")
if evidence_row is None:
evidence_row = cursor.execute(
f"SELECT {_EVIDENCE_COLUMNS} FROM evidence WHERE evidence_id=?",
(identifier,),
).fetchone()
if evidence_row is None:
records[identifier] = None
continue
# evidence_ref explicitly excludes origin_meta/imported_at/updated_at.
# `sqlite3.Row` iterates values, not column names, so pair the
# names with the values explicitly.
fields = {
name: value
for name, value in zip(evidence_row.keys(), tuple(evidence_row), strict=False)
if name != "measure"
}
records[identifier] = EvidenceRecord.model_validate({**fields, "origin_meta": {}})
for output in complete:
if all(
(record := records[ref["evidence_id"]]) is not None
and record.user_id == user_id
and matches_evidence_ref(record, ref)
for ref in output.support_refs
):
return 1
return 0
register_function(db, "is_memory_evidence_current", 3, current)
return "is_memory_evidence_current(m.memory_id,m.revision,m.user_id)"
+1 -1
View File
@@ -203,7 +203,7 @@
| `test_memory_correction_sync.py` | `integration` | R-27: a correction that only lands locally, while HY keeps the old text. | | `test_memory_correction_sync.py` | `integration` | R-27: a correction that only lands locally, while HY keeps the old text. |
| `test_memory_deletion_outbox.py` | `contract` | 删除发件箱:一条删除同一时刻只归一个执行者,结果只由持有领取凭证的那个写。 | | `test_memory_deletion_outbox.py` | `contract` | 删除发件箱:一条删除同一时刻只归一个执行者,结果只由持有领取凭证的那个写。 |
| `test_memory_dispatch_lifecycle.py` | `contract` | Exercise the automatic enqueue boundary without producing substitute memories. | | `test_memory_dispatch_lifecycle.py` | `contract` | Exercise the automatic enqueue boundary without producing substitute memories. |
| `test_memory_evidence_freshness.py` | `contract` | 新鲜度拆除后:读路径恒为 current;证据修订 / 畸形 proof / segment ref 不再藏记忆或 503。 | | `test_memory_evidence_freshness.py` | `contract` | 覆盖 fresh then revised and identity only change、any complete fresh proof wins over old proof、other revision and incomplete proof do not hide legacy or correction 等 21 个场景。 |
| `test_memory_freshness_integration_review.py` | `integration` | Independent synthetic read-path review; no lifespan, model or user database. | | `test_memory_freshness_integration_review.py` | `integration` | Independent synthetic read-path review; no lifespan, model or user database. |
| `test_memory_governance.py` | `contract` | 覆盖 correction keeps verbatim and appends history、mutations are revision checked and request idempotent、delete result survives restart and request replay 等 42 个场景。 | | `test_memory_governance.py` | `contract` | 覆盖 correction keeps verbatim and appends history、mutations are revision checked and request idempotent、delete result survives restart and request replay 等 42 个场景。 |
| `test_memory_governance_api.py` | `integration` | 覆盖 memory governance api preserves evidence and reports cleanup、memory validation does not echo content 场景。 | | `test_memory_governance_api.py` | `integration` | 覆盖 memory governance api preserves evidence and reports cleanup、memory validation does not echo content 场景。 |
@@ -355,9 +355,8 @@ def test_actual_evidence_proof_revision_and_corrected_memory(fresh):
assert all(item.available for item in resolve_call_references(path, event(raw, reference))) assert all(item.available for item in resolve_call_references(path, event(raw, reference)))
evidence.put(value.model_copy(update={"verbatim": "revised evidence"})) evidence.put(value.model_copy(update={"verbatim": "revised evidence"}))
result = resolve_call_references(path, event(raw, reference)) result = resolve_call_references(path, event(raw, reference))
# Raw refs still track evidence.revision; memory refs no longer use freshness.
assert_hidden(result[0], raw, "changed") assert_hidden(result[0], raw, "changed")
assert result[1].available assert_hidden(result[1], reference, "changed")
corrected = as_person(MemoryGovernanceStore(path)).correct( corrected = as_person(MemoryGovernanceStore(path)).correct(
memory.memory_id, memory.memory_id,
user_id="local", user_id="local",
@@ -370,7 +369,7 @@ def test_actual_evidence_proof_revision_and_corrected_memory(fresh):
assert resolve_call_references(path, event(current))[0].title == corrected.content assert resolve_call_references(path, event(current))[0].title == corrected.content
def test_scope_only_revision_keeps_memory_readable_when_evidence_changes(fresh): def test_scope_only_revision_still_checks_original_support(fresh):
path, evidence, value, record, memory, tasks, output = fresh path, evidence, value, record, memory, tasks, output = fresh
updated = as_person(MemoryGovernanceStore(path)).set_scopes( updated = as_person(MemoryGovernanceStore(path)).set_scopes(
memory.memory_id, memory.memory_id,
@@ -382,27 +381,25 @@ def test_scope_only_revision_keeps_memory_readable_when_evidence_changes(fresh):
reference = memory_ref(updated.memory_id, updated.revision) reference = memory_ref(updated.memory_id, updated.revision)
assert resolve_call_references(path, event(reference))[0].available assert resolve_call_references(path, event(reference))[0].available
evidence.put(value.model_copy(update={"verbatim": "revised evidence"})) evidence.put(value.model_copy(update={"verbatim": "revised evidence"}))
assert resolve_call_references(path, event(reference))[0].available assert_hidden(resolve_call_references(path, event(reference))[0], reference, "changed")
def test_evidence_identity_change_without_revision_keeps_memory_readable(fresh): def test_freshness_identity_change_without_revision_change(fresh):
path, evidence, value, record, memory, tasks, output = fresh path, evidence, value, record, memory, tasks, output = fresh
reference = memory_ref(memory.memory_id, memory.revision) reference = memory_ref(memory.memory_id, memory.revision)
assert resolve_call_references(path, event(reference))[0].available assert resolve_call_references(path, event(reference))[0].available
with closing(sqlite3.connect(path)) as db, db: with closing(sqlite3.connect(path)) as db, db:
db.execute("UPDATE evidence SET workspace='workspace:changed'") db.execute("UPDATE evidence SET workspace='workspace:changed'")
assert resolve_call_references(path, event(reference))[0].available assert_hidden(resolve_call_references(path, event(reference))[0], reference, "changed")
def test_invalid_proof_is_ignored_but_revocation_still_hides_memory(fresh): def test_invalid_freshness_proof_fails_whole_response_but_revoked_never_reads_it(fresh):
path, evidence, value, record, memory, tasks, output = fresh path, evidence, value, record, memory, tasks, output = fresh
reference = memory_ref(memory.memory_id, memory.revision) reference = memory_ref(memory.memory_id, memory.revision)
with closing(sqlite3.connect(path)) as db, db: with closing(sqlite3.connect(path)) as db, db:
db.execute("UPDATE memory_task_outputs SET body='invalid secret proof'") db.execute("UPDATE memory_task_outputs SET body='invalid secret proof'")
assert all( with pytest.raises(CallReferencesUnavailable, match="^call_references_unavailable$"):
item.available resolve_call_references(path, event(ref(key=record.evidence_id), reference))
for item in resolve_call_references(path, event(ref(key=record.evidence_id), reference))
)
with closing(sqlite3.connect(path)) as db, db: with closing(sqlite3.connect(path)) as db, db:
grant(db, "cursor") grant(db, "cursor")
result = resolve_call_references(path, event(reference)) result = resolve_call_references(path, event(reference))
@@ -458,13 +455,9 @@ def test_authorization_revision_freshness_and_titles_share_one_snapshot(fresh, m
assert [item.title for item in snapshot] == [memory.content, record.verbatim] assert [item.title for item in snapshot] == [memory.content, record.verbatim]
assert changed == [True] and reads and all(reads) assert changed == [True] and reads and all(reads)
current = resolve_call_references(path, event(reference, raw)) current = resolve_call_references(path, event(reference, raw))
# Memory refs no longer hide on evidence revision/identity via freshness. assert_hidden(
if change == "evidence": current[0], reference, "changed" if change in ("evidence", "memory") else "unavailable"
assert current[0].available )
else:
assert_hidden(
current[0], reference, "changed" if change == "memory" else "unavailable"
)
assert current[1].available == (change in ("memory", "link")) assert current[1].available == (change in ("memory", "link"))
@@ -8,11 +8,6 @@ import pytest
pytestmark = [pytest.mark.performance, pytest.mark.serial] pytestmark = [pytest.mark.performance, pytest.mark.serial]
SCRIPT = Path(__file__).resolve().parents[3] / "scripts" / "bench_context_raw_scale.py" SCRIPT = Path(__file__).resolve().parents[3] / "scripts" / "bench_context_raw_scale.py"
if not SCRIPT.exists():
pytest.skip(
f"benchmark script is not part of this checkout: {SCRIPT.name}",
allow_module_level=True,
)
spec = importlib.util.spec_from_file_location("bench_context_raw_scale", SCRIPT) spec = importlib.util.spec_from_file_location("bench_context_raw_scale", SCRIPT)
benchmark = importlib.util.module_from_spec(spec) benchmark = importlib.util.module_from_spec(spec)
sys.modules["bench_context_raw_scale"] = benchmark sys.modules["bench_context_raw_scale"] = benchmark
@@ -1,5 +1,3 @@
"""Freshness retirement: read path always admits, even when evidence changed."""
import json import json
import sqlite3 import sqlite3
@@ -7,11 +5,7 @@ import pytest
from one_memory.imports.models import EvidenceInput, EvidenceOrigin, OriginMetadata, WrittenBy from one_memory.imports.models import EvidenceInput, EvidenceOrigin, OriginMetadata, WrittenBy
from one_memory.imports.storage import EvidenceStore from one_memory.imports.storage import EvidenceStore
from one_memory.memory_evidence_freshness import ( from one_memory.memory_evidence_freshness import register_memory_freshness
budget_errors_raise,
freshness_proofs_prefetched,
register_memory_freshness,
)
from one_memory.memory_governance import MemoryDraft, MemoryGovernanceStore, MemoryKind from one_memory.memory_governance import MemoryDraft, MemoryGovernanceStore, MemoryKind
from one_memory.memory_tasks import MemoryTaskOutput, MemoryTaskStore, evidence_ref from one_memory.memory_tasks import MemoryTaskOutput, MemoryTaskStore, evidence_ref
from tests.person_door import as_person from tests.person_door import as_person
@@ -67,45 +61,46 @@ def check(path, memory):
).fetchone()[0] ).fetchone()[0]
def test_register_returns_constant_true_predicate(case): def test_fresh_then_revised_and_identity_only_change(case):
path, evidence, value, record, memory, tasks, output = case
with sqlite3.connect(path) as db:
db.execute("BEGIN")
assert register_memory_freshness(db) == "1"
assert db.execute("SELECT is_memory_evidence_current(?,?,?)", ("x", 1, "local")).fetchone() == (
1,
)
def test_revised_evidence_no_longer_hides_memory(case):
path, evidence, value, record, memory, tasks, output = case path, evidence, value, record, memory, tasks, output = case
assert check(path, memory) == 1 assert check(path, memory) == 1
evidence.put(value.model_copy(update={"verbatim": "new statement"})) evidence.put(value.model_copy(update={"verbatim": "new statement"}))
assert check(path, memory) == 0
def test_any_complete_fresh_proof_wins_over_old_proof(case):
path, evidence, value, record, memory, tasks, output = case
revised = evidence.put(value.model_copy(update={"verbatim": "new statement"})).record
tasks.save_output(
output.model_copy(update={"task_id": "task2", "support_refs": (evidence_ref(revised),)})
)
assert check(path, memory) == 1 assert check(path, memory) == 1
def test_segment_support_refs_do_not_fail_reads(case): def test_other_revision_and_incomplete_proof_do_not_hide_legacy_or_correction(case):
path, evidence, value, record, memory, tasks, output = case
evidence.put(value.model_copy(update={"verbatim": "new statement"}))
tasks.save_output(output.model_copy(update={"memory_revision": memory.revision + 1}))
assert check(path, memory) == 1
tasks.save_output(output.model_copy(update={"support_refs": ()}))
assert check(path, memory) == 1
@pytest.mark.parametrize("broken", ["json", "identity", "ref"])
def test_bad_proof_raises_not_false_empty(case, broken):
path, evidence, value, record, memory, tasks, output = case path, evidence, value, record, memory, tasks, output = case
body = output.model_dump(mode="json") body = output.model_dump(mode="json")
body["support_refs"][0].update( if broken == "identity":
segment_index=0, body["memory_id"] = "other"
segment_start=0, if broken == "ref":
segment_end=len(record.verbatim), body["support_refs"][0]["revision"] = "1"
segment_hash="0" * 64, payload = json.dumps(body)
) if broken == "json":
payload = "{"
with sqlite3.connect(path) as db: with sqlite3.connect(path) as db:
db.execute("UPDATE memory_task_outputs SET body=?", (json.dumps(body),)) db.execute("UPDATE memory_task_outputs SET body=?", (payload,))
assert check(path, memory) == 1 with pytest.raises(sqlite3.OperationalError):
check(path, memory)
def test_malformed_proof_does_not_fail_reads(case):
path, evidence, value, record, memory, tasks, output = case
with sqlite3.connect(path) as db:
db.execute(
"UPDATE memory_task_outputs SET body=? WHERE memory_id=?",
("{", memory.memory_id),
)
assert check(path, memory) == 1
def test_missing_table_compatibility_is_readonly(): def test_missing_table_compatibility_is_readonly():
@@ -116,19 +111,239 @@ def test_missing_table_compatibility_is_readonly():
assert db.execute("SELECT name FROM sqlite_master").fetchall() == [] assert db.execute("SELECT name FROM sqlite_master").fetchall() == []
def test_prefetch_is_noop(case): def test_table_presence_is_checked_once_at_registration(case):
path, evidence, value, record, memory, tasks, output = case path, evidence, value, record, memory, tasks, output = case
with sqlite3.connect(path) as db: with sqlite3.connect(path) as db:
db.execute("BEGIN") db.execute("BEGIN")
with freshness_proofs_prefetched(db, [memory.memory_id], evidence_ids=[record.evidence_id]): statements = []
assert register_memory_freshness(db) == "1" db.set_trace_callback(statements.append)
assert check(path, memory) == 1 predicate = register_memory_freshness(db)
for _ in range(3):
assert db.execute(f"SELECT {predicate} FROM governed_memories m").fetchone() == (1,)
assert sum("sqlite_master" in statement for statement in statements) == 1
def test_budget_raise_env_is_inert(case, monkeypatch): def test_proof_budget_reads_current_instead_of_failing(case):
path, evidence, value, record, memory, tasks, output = case path, evidence, value, record, memory, tasks, output = case
monkeypatch.setenv("ONE_MEMORY_FRESHNESS_BUDGET_RAISES", "1")
assert budget_errors_raise() is False
for index in range(65): for index in range(65):
tasks.save_output(output.model_copy(update={"task_id": f"task-{index}"})) tasks.save_output(output.model_copy(update={"task_id": f"task-{index}"}))
assert check(path, memory) == 1 assert check(path, memory) == 1
def test_proof_budget_still_raises_when_opted_back(case, monkeypatch):
path, evidence, value, record, memory, tasks, output = case
monkeypatch.setenv("ONE_MEMORY_FRESHNESS_BUDGET_RAISES", "1")
for index in range(65):
tasks.save_output(output.model_copy(update={"task_id": f"task-{index}"}))
with pytest.raises(sqlite3.OperationalError):
check(path, memory)
def test_snapshot_cache_does_not_hide_changes_across_transactions(case):
path, evidence, value, record, memory, tasks, output = case
with sqlite3.connect(path) as db:
db.execute("BEGIN")
predicate = register_memory_freshness(db)
query = f"SELECT {predicate} FROM governed_memories m"
assert db.execute(query).fetchone() == (1,)
db.commit()
evidence.put(value.model_copy(update={"verbatim": "new statement"}))
db.execute("BEGIN")
register_memory_freshness(db)
assert db.execute(query).fetchone() == (0,)
def test_identity_hash_detects_metadata_change_without_revision_or_content_change(case):
path, evidence, value, record, memory, tasks, output = case
with sqlite3.connect(path) as db:
db.execute("UPDATE evidence SET workspace='workspace:changed'")
assert check(path, memory) == 0
def test_incomplete_fresh_proof_cannot_override_complete_stale(case):
path, evidence, value, record, memory, tasks, output = case
evidence.put(value.model_copy(update={"verbatim": "new statement"}))
tasks.save_output(output.model_copy(update={"task_id": "partial", "support_refs": ()}))
assert check(path, memory) == 0
def test_sql_query_only_and_missing_proof_compatibility(case):
path, evidence, value, record, memory, tasks, output = case
with sqlite3.connect(path) as db:
db.execute("DELETE FROM memory_task_outputs")
with sqlite3.connect(path) as db:
db.execute("PRAGMA query_only=ON")
db.execute("BEGIN")
before = db.total_changes
predicate = register_memory_freshness(db)
assert db.execute(f"SELECT {predicate} FROM governed_memories m").fetchone() == (1,)
assert db.total_changes == before
def test_oversized_evidence_reads_current_without_loading_body(case):
path, evidence, value, record, memory, tasks, output = case
with sqlite3.connect(path) as db:
db.execute("UPDATE evidence SET verbatim=zeroblob(?)", (8 * 1024 * 1024 + 1,))
assert check(path, memory) == 1
@pytest.mark.parametrize("revise_first", [False, True])
def test_scope_only_chain_preserves_old_support_guard(case, revise_first):
path, evidence, value, record, memory, tasks, output = case
governance = MemoryGovernanceStore(path)
if revise_first:
evidence.put(value.model_copy(update={"verbatim": "new statement"}))
for index in range(3):
memory = as_person(governance).set_scopes(
memory.memory_id,
user_id="local",
request_id=f"scope-{index}",
expected_revision=memory.revision,
scopes=(f"workspace:{index}",),
)
if not revise_first:
assert check(path, memory) == 1
evidence.put(value.model_copy(update={"verbatim": "new statement"}))
assert check(path, memory) == 0
@pytest.mark.parametrize("action", ["correct", "undo", "hy_reconcile", "unknown"])
def test_scope_chain_does_not_cross_non_scope_action(case, action):
path, evidence, value, record, memory, tasks, output = case
governance = MemoryGovernanceStore(path)
memory = as_person(governance).correct(
memory.memory_id,
user_id="local",
request_id="correct",
expected_revision=memory.revision,
content=memory.content,
)
with sqlite3.connect(path) as db:
db.execute(
"UPDATE governed_memory_versions SET action=? WHERE revision=?",
(action, memory.revision),
)
memory = as_person(governance).set_scopes(
memory.memory_id,
user_id="local",
request_id="scope",
expected_revision=memory.revision,
scopes=("workspace:changed",),
)
evidence.put(value.model_copy(update={"verbatim": "new statement"}))
assert check(path, memory) == 1
def test_scope_chain_budget_reads_current(case):
path, evidence, value, record, memory, tasks, output = case
governance = MemoryGovernanceStore(path)
for index in range(65):
memory = as_person(governance).set_scopes(
memory.memory_id,
user_id="local",
request_id=f"scope-{index}",
expected_revision=memory.revision,
scopes=(f"workspace:{index}",),
)
assert check(path, memory) == 1
def test_changed_content_in_scope_history_is_not_inherited(case):
path, evidence, value, record, memory, tasks, output = case
governance = MemoryGovernanceStore(path)
memory = as_person(governance).set_scopes(
memory.memory_id,
user_id="local",
request_id="scope",
expected_revision=memory.revision,
scopes=("workspace:changed",),
)
with sqlite3.connect(path) as db:
db.execute(
"UPDATE governed_memory_versions SET content='different' WHERE revision=?",
(output.memory_revision,),
)
evidence.put(value.model_copy(update={"verbatim": "new statement"}))
assert check(path, memory) == 1
def test_scope_chain_keeps_large_governed_content_inside_sql(case):
path, evidence, value, record, memory, tasks, output = case
with sqlite3.connect(path) as db:
content = "x" * 1_000_000
db.execute("UPDATE governed_memories SET content=?", (content,))
db.execute("UPDATE governed_memory_versions SET content=?", (content,))
governance = MemoryGovernanceStore(path)
memory = as_person(governance).set_scopes(
memory.memory_id,
user_id="local",
request_id="scope",
expected_revision=memory.revision,
scopes=("workspace:changed",),
)
evidence.put(value.model_copy(update={"verbatim": "new statement"}))
class BoundedCursor(sqlite3.Cursor):
def fetchone(self):
row = super().fetchone()
if row is not None:
assert all(not isinstance(item, str) or len(item) < 65_536 for item in row)
return row
class BoundedConnection(sqlite3.Connection):
def cursor(self, factory=BoundedCursor):
return super().cursor(factory)
with sqlite3.connect(path, factory=BoundedConnection) as db:
db.execute("BEGIN")
predicate = register_memory_freshness(db)
assert db.execute(f"SELECT {predicate} FROM governed_memories m").fetchone() == (0,)
def _confirmed_with_oversized_proof(case):
path, evidence, value, record, memory, tasks, output = case
with sqlite3.connect(path) as db:
db.execute(
"UPDATE memory_task_outputs SET body=? WHERE memory_id=?",
(
json.dumps({"memory_revision": memory.revision, "pad": "x" * 70_000}),
memory.memory_id,
),
)
return path, memory
def test_one_oversized_memory_no_longer_fails_the_library_count(case):
"""D76: one memory whose proof is too large used to 503 the homepage forever."""
path, memory = _confirmed_with_oversized_proof(case)
with sqlite3.connect(path) as db:
db.execute("BEGIN")
predicate = register_memory_freshness(db)
count = db.execute(
f"SELECT COUNT(*) FROM governed_memories m WHERE {predicate}"
).fetchone()[0]
assert count == 1
def test_malformed_proof_still_raises(case):
path, evidence, value, record, memory, tasks, output = case
with sqlite3.connect(path) as db:
db.execute(
"UPDATE memory_task_outputs SET body=? WHERE memory_id=?", ("{}", memory.memory_id)
)
with pytest.raises(sqlite3.OperationalError):
check(path, memory)
def test_budget_skip_is_reported_without_content(case, monkeypatch):
import one_memory.memory_evidence_freshness as freshness
events = []
monkeypatch.setattr(freshness, "diagnostic_event", lambda **values: events.append(values))
monkeypatch.setattr(freshness, "_reported", {})
path, memory = _confirmed_with_oversized_proof(case)
for _ in range(3):
assert check(path, memory) == 1
assert len(events) == 1
assert events[0]["details"] == {"reason": "memory_freshness_proof_budget", "count": 1}
assert "pad" not in json.dumps(events[0])
@@ -102,11 +102,17 @@ def revise(case):
).record ).record
def test_revised_evidence_remains_readable_after_freshness_retirement(case): def test_complete_stale_proof_hides_all_four_reads_without_mutation(case):
"""Evidence revision no longer hides library / context / source reads.""" # D22: the app-facing totals (library total, homepage count) count with
# the cheap permission filters only, so a stale proof no longer removes a
# row from the totals; the freshness gate still hides it from every read
# that returns rows (context, source read access).
assert visible(case) == (1, 1, 1, 1) assert visible(case) == (1, 1, 1, 1)
revise(case) revise(case)
assert visible(case) == (1, 1, 1, 1) total, home, context_items, source_items = visible(case)
# D22: home total counts cheaply and may keep the stale row; the reads
# that return rows still hide it.
assert (home, context_items, source_items) == (1, 0, 0)
assert case.governance.get(case.memory.memory_id, user_id="local") == case.memory assert case.governance.get(case.memory.memory_id, user_id="local") == case.memory
assert case.tasks.outputs(case.output.task_id) == (case.output,) assert case.tasks.outputs(case.output.task_id) == (case.output,)
@@ -114,10 +120,11 @@ def test_revised_evidence_remains_readable_after_freshness_retirement(case):
@pytest.mark.parametrize( @pytest.mark.parametrize(
"revise_first", [False, True], ids=["scope-before-revision", "scope-after-revision"] "revise_first", [False, True], ids=["scope-before-revision", "scope-after-revision"]
) )
def test_scope_only_revision_keeps_memory_readable_after_evidence_change(case, revise_first): def test_scope_only_revision_must_not_resurrect_stale_derived_content(case, revise_first):
if revise_first: if revise_first:
revise(case) revise(case)
assert visible(case) == (1, 1, 1, 1) _, home, context_items, source_items = visible(case)
assert (context_items, source_items) == (0, 0)
changed = as_person(case.governance).set_scopes( changed = as_person(case.governance).set_scopes(
case.memory.memory_id, case.memory.memory_id,
user_id="local", user_id="local",
@@ -129,7 +136,8 @@ def test_scope_only_revision_keeps_memory_readable_after_evidence_change(case, r
if not revise_first: if not revise_first:
assert visible(case, changed) == (1, 1, 1, 1) assert visible(case, changed) == (1, 1, 1, 1)
revise(case) revise(case)
assert visible(case, changed) == (1, 1, 1, 1) _, home, context_items, source_items = visible(case, changed)
assert (context_items, source_items) == (0, 0)
def test_explicit_correct_remains_readable_but_never_overrides_revocation(case): def test_explicit_correct_remains_readable_but_never_overrides_revocation(case):
@@ -144,10 +152,7 @@ def test_explicit_correct_remains_readable_but_never_overrides_revocation(case):
assert visible(case, corrected) == (1, 1, 1, 1) assert visible(case, corrected) == (1, 1, 1, 1)
with closing(sqlite3.connect(case.path)) as db, db: with closing(sqlite3.connect(case.path)) as db, db:
db.execute( db.execute(
"""INSERT INTO source_grants "INSERT INTO source_grants VALUES ('local','cursor','/fixture','default',?,?)",
(user_id,source_id,root_path,requested_path,root_identity,
candidate_kind,granted_at,revoked_at,revision)
VALUES ('local','cursor','/fixture','/fixture','fixture','default',?,?,1)""",
("2026-09-20T00:00:00Z", "2026-09-20T00:00:01Z"), ("2026-09-20T00:00:00Z", "2026-09-20T00:00:01Z"),
) )
assert visible(case, corrected) == (0, 0, 0, 0) assert visible(case, corrected) == (0, 0, 0, 0)
@@ -176,10 +181,7 @@ def test_revocation_before_freshness_snapshot_cannot_release_memory(case, monkey
# Revocation commits before any SELECT pins the freshness transaction. # Revocation commits before any SELECT pins the freshness transaction.
with closing(sqlite3.connect(case.path)) as writer, writer: with closing(sqlite3.connect(case.path)) as writer, writer:
writer.execute( writer.execute(
"""INSERT INTO source_grants "INSERT INTO source_grants VALUES ('local','cursor','/fixture','default',?,?)",
(user_id,source_id,root_path,requested_path,root_identity,
candidate_kind,granted_at,revoked_at,revision)
VALUES ('local','cursor','/fixture','/fixture','fixture','default',?,?,1)""",
("2026-09-20T00:00:00Z", "2026-09-20T00:00:01Z"), ("2026-09-20T00:00:00Z", "2026-09-20T00:00:01Z"),
) )
changes.append(True) changes.append(True)
@@ -192,7 +194,7 @@ def test_revocation_before_freshness_snapshot_cannot_release_memory(case, monkey
assert returned == [] assert returned == []
def test_udf_stays_current_across_evidence_revision(case): def test_udf_uses_caller_snapshot_then_observes_revision_in_next_transaction(case):
with closing(sqlite3.connect(case.path)) as reader: with closing(sqlite3.connect(case.path)) as reader:
reader.execute("PRAGMA journal_mode=WAL") reader.execute("PRAGMA journal_mode=WAL")
reader.execute("BEGIN") reader.execute("BEGIN")
@@ -204,7 +206,7 @@ def test_udf_stays_current_across_evidence_revision(case):
reader.commit() reader.commit()
reader.execute("BEGIN") reader.execute("BEGIN")
register_memory_freshness(reader) register_memory_freshness(reader)
assert reader.execute(sql).fetchone() == (1,) assert reader.execute(sql).fetchone() == (0,)
def _oversize(case, bad): def _oversize(case, bad):
@@ -225,16 +227,20 @@ def test_material_too_large_to_verify_reads_as_current(case, bad):
@pytest.mark.parametrize("bad", ["duplicate-key"]) @pytest.mark.parametrize("bad", ["duplicate-key"])
def test_bad_proof_material_no_longer_makes_reads_unavailable(case, bad): def test_bad_material_is_unavailable_not_successful_empty(case, bad):
with closing(sqlite3.connect(case.path)) as db, db: with closing(sqlite3.connect(case.path)) as db, db:
db.execute( db.execute(
"UPDATE memory_task_outputs SET body=?", "UPDATE memory_task_outputs SET body=?",
('{"memory_revision":2,"memory_revision":2}',), ('{"memory_revision":2,"memory_revision":2}',),
) )
assert visible(case) == (1, 1, 1, 1)
result = context(case) result = context(case)
assert result.status != "unavailable" assert result.status == "unavailable" and result.reason == "read_unavailable"
assert len(result.items) == 1 assert result.items == ()
for read in (read_memory_library, read_home_summary):
with pytest.raises(sqlite3.Error):
read(case.path)
with pytest.raises(sqlite3.Error):
SourceReadAccess(case.path).memories([case.memory])
def test_route_serves_memories_whose_proof_is_too_large(case, tmp_path): def test_route_serves_memories_whose_proof_is_too_large(case, tmp_path):
@@ -73,9 +73,8 @@ def put(
def grant(db, source, *, user="local", revoked=NOW): def grant(db, source, *, user="local", revoked=NOW):
db.execute( db.execute(
"""INSERT OR REPLACE INTO source_grants """INSERT OR REPLACE INTO source_grants
(user_id,source_id,root_path,requested_path,root_identity, (user_id,source_id,root_path,candidate_kind,granted_at,revoked_at)
candidate_kind,granted_at,revoked_at,revision) VALUES (?,?, '/synthetic', 'default', ?, ?)""",
VALUES (?,?, '/synthetic', '/synthetic', 'synthetic', 'default', ?, ?, 1)""",
(user, source, NOW, revoked), (user, source, NOW, revoked),
) )
@@ -8,11 +8,8 @@ from uuid import UUID
import pytest import pytest
from pydantic import ValidationError from pydantic import ValidationError
requests_module = pytest.importorskip( import one_memory.processing_requests as requests_module
"one_memory.processing_requests", from one_memory.processing_requests import (
reason="processing-request store is not part of the public Core snapshot",
)
from one_memory.processing_requests import ( # noqa: E402 - importorskip must run first
ProcessingRequest, ProcessingRequest,
ProcessingRequestConflictError, ProcessingRequestConflictError,
ProcessingRequestState, ProcessingRequestState,
@@ -7,11 +7,6 @@ import pytest
pytestmark = [pytest.mark.performance, pytest.mark.serial] pytestmark = [pytest.mark.performance, pytest.mark.serial]
SCRIPT = Path(__file__).resolve().parents[3] / "scripts" / "bench_raw_vector_scale.py" SCRIPT = Path(__file__).resolve().parents[3] / "scripts" / "bench_raw_vector_scale.py"
if not SCRIPT.exists():
pytest.skip(
f"benchmark script is not part of this checkout: {SCRIPT.name}",
allow_module_level=True,
)
spec = importlib.util.spec_from_file_location("bench_raw_vector_scale", SCRIPT) spec = importlib.util.spec_from_file_location("bench_raw_vector_scale", SCRIPT)
benchmark = importlib.util.module_from_spec(spec) benchmark = importlib.util.module_from_spec(spec)
sys.modules["bench_raw_vector_scale"] = benchmark sys.modules["bench_raw_vector_scale"] = benchmark
@@ -8,11 +8,6 @@ import pytest
pytestmark = [pytest.mark.performance, pytest.mark.serial] pytestmark = [pytest.mark.performance, pytest.mark.serial]
SCRIPT = Path(__file__).resolve().parents[3] / "scripts" / "bench_retrieval_eligibility_scale.py" SCRIPT = Path(__file__).resolve().parents[3] / "scripts" / "bench_retrieval_eligibility_scale.py"
if not SCRIPT.exists():
pytest.skip(
f"benchmark script is not part of this checkout: {SCRIPT.name}",
allow_module_level=True,
)
spec = importlib.util.spec_from_file_location("bench_retrieval_eligibility_scale", SCRIPT) spec = importlib.util.spec_from_file_location("bench_retrieval_eligibility_scale", SCRIPT)
benchmark = importlib.util.module_from_spec(spec) benchmark = importlib.util.module_from_spec(spec)
sys.modules["bench_retrieval_eligibility_scale"] = benchmark sys.modules["bench_retrieval_eligibility_scale"] = benchmark
@@ -224,11 +224,6 @@ def test_live_adapter_keeps_symlink_refusal_in_existing_issue_taxonomy() -> None
) )
def test_live_adapter_reports_an_oversized_file_as_file_too_large() -> None:
"""The open path and the discovery path must agree on the size reason."""
assert _scan_failure_code(SourceAccessErrorCode.FILE_TOO_LARGE) is ScanIssueCode.FILE_TOO_LARGE
def test_live_adapter_runner_persists_symlink_refusal_as_unsupported_schema( def test_live_adapter_runner_persists_symlink_refusal_as_unsupported_schema(
tmp_path, monkeypatch tmp_path, monkeypatch
) -> None: ) -> None:
@@ -3,12 +3,10 @@ from one_memory.imports.access import (
AuthorizedSourceRoot, AuthorizedSourceRoot,
SourceAccessErrorCode, SourceAccessErrorCode,
) )
from one_memory.imports.adapters import ScanIssueCode
from one_memory.imports.discovery import ( from one_memory.imports.discovery import (
SourceFileDiscoveryFailure, SourceFileDiscoveryFailure,
discover_source_files, discover_source_files,
) )
from one_memory.imports.sources.common import discovery_failure_code
def grant(root, home): def grant(root, home):
@@ -62,35 +60,3 @@ def test_discovery_does_not_follow_directory_or_file_symlinks(tmp_path) -> None:
assert len(events) == 1 assert len(events) == 1
assert isinstance(events[0], SourceFileDiscoveryFailure) assert isinstance(events[0], SourceFileDiscoveryFailure)
assert "private" not in events[0].file_key assert "private" not in events[0].file_key
def test_discovery_reports_an_oversized_file_as_file_too_large(tmp_path) -> None:
"""A size-gate refusal is a size problem, not a schema problem.
The Codex report in #102 shows the access gate refusing a file over the
per-root limit while the UI said `unsupported_schema`, which sends the user
looking for a parser that is working fine.
"""
root = tmp_path / "source"
root.mkdir()
(root / "big.jsonl").write_text('{"content":"too big"}\n')
granted = AuthorizedSourceRoot.grant(
source_id="fixture",
root=root,
user_confirmed=True,
home=tmp_path / "home",
max_file_bytes=8,
)
events = list(
discover_source_files(
granted,
suffixes=frozenset({".jsonl"}),
max_targets=10,
)
)
assert len(events) == 1
assert isinstance(events[0], SourceFileDiscoveryFailure)
assert events[0].reason_code is SourceAccessErrorCode.FILE_TOO_LARGE
assert discovery_failure_code(events[0]) is ScanIssueCode.FILE_TOO_LARGE
+1 -1
View File
@@ -363,7 +363,7 @@ Support is declared per immutable GitHub Release manifest:
- macOS arm64 and x64 App/Runtime artifacts must carry Developer ID signatures, - macOS arm64 and x64 App/Runtime artifacts must carry Developer ID signatures,
notarization tickets, and native-platform acceptance receipts. notarization tickets, and native-platform acceptance receipts.
- Windows is not supported. - Windows is not supported.
- `hy-memory==2.0.4` is the public Python SDK dependency. - `hy-memory==2.0.3` is the public Python SDK dependency.
See [`INSTALLATION.md`](INSTALLATION.md) for the release matrix. A source test, See [`INSTALLATION.md`](INSTALLATION.md) for the release matrix. A source test,
fake hosting test, or cross-build does not substitute for a native-platform fake hosting test, or cross-build does not substitute for a native-platform
+1 -1
View File
@@ -1,6 +1,6 @@
# MemHub 安装与发行 # MemHub 安装与发行
MemHub `v1.0.4`(build 13)的发行目标如下: MemHub `v1.0.3`(build 7)的发行目标如下:
| 平台 | 发行物 | 构建与验收 | | 平台 | 发行物 | 构建与验收 |
| --- | --- | --- | | --- | --- | --- |
@@ -1,66 +0,0 @@
import type { ChildProcess, SpawnOptions } from "node:child_process";
import { EventEmitter } from "node:events";
import { describe, expect, it, vi } from "vitest";
import { CoreSupervisor } from "./core-supervisor.js";
function fakeChild(): ChildProcess {
const child = new EventEmitter() as ChildProcess;
child.stderr = new EventEmitter() as ChildProcess["stderr"];
child.stdout = new EventEmitter() as ChildProcess["stdout"];
child.kill = vi.fn(() => {
setImmediate(() => child.emit("exit", 0, null));
return true;
});
return child;
}
function fakeReaderChild(): ChildProcess {
return fakeChild();
}
describe("CoreSupervisor reader startup", () => {
it("starts the reader after spawning its own Core", async () => {
const core = fakeChild();
const reader = fakeReaderChild();
const spawned: string[][] = [];
const spawnProcess = vi.fn(
(_command: string, args: readonly string[], _options: SpawnOptions) => {
spawned.push([...args]);
if (args.includes("--reader")) {
// The Reader attaches its stderr listener synchronously after this
// call returns, so publish the port on the next tick.
setImmediate(() =>
reader.stderr?.emit("data", Buffer.from("reader_port:43210\n")),
);
return reader;
}
return core;
},
);
const supervisor = new CoreSupervisor({
launch: { command: "one-memory", args: [] },
dataDir: "/tmp/one-memory-reader-start-test",
instanceId: "owned",
port: 9600,
attempts: 1,
delayMs: 0,
readerEnabled: true,
spawnProcess,
probeHealth: async () => ({
status: "ok",
service: "one-memory-core",
version: "0.0.0",
instance_id: "owned",
}),
});
await supervisor.start();
expect(spawnProcess).toHaveBeenCalledTimes(2);
expect(spawned[1]).toContain("--reader");
expect(supervisor.readerPort).toBe(43210);
await supervisor.stop();
});
});
@@ -339,11 +339,7 @@ export class CoreSupervisor {
} }
this.useOwnedIdentity(); this.useOwnedIdentity();
try { try {
const health = await this.attempt(); return await this.attempt();
if (this.options.readerEnabled) {
await this.startReader();
}
return health;
} catch (error) { } catch (error) {
// A standalone Runtime may have won the race after our initial // A standalone Runtime may have won the race after our initial
// discovery read but before our child acquired the data lock. Verify it // discovery read but before our child acquired the data lock. Verify it
@@ -351,12 +347,7 @@ export class CoreSupervisor {
let raced: HealthResponse | null = null; let raced: HealthResponse | null = null;
try { try {
raced = await this.attachExisting(); raced = await this.attachExisting();
if (raced !== null) { if (raced !== null) return raced;
if (this.options.readerEnabled) {
await this.startReader();
}
return raced;
}
} catch (attachError) { } catch (attachError) {
// A stale receipt can fail identity validation after the owned Core // A stale receipt can fail identity validation after the owned Core
// exited. Treat that as a cleanup candidate, not as a new startup // exited. Treat that as a cleanup candidate, not as a new startup
@@ -396,11 +387,7 @@ export class CoreSupervisor {
this.exitSignal = null; this.exitSignal = null;
this.spawnError = null; this.spawnError = null;
try { try {
const health = await this.attempt(); return await this.attempt();
if (this.options.readerEnabled) {
await this.startReader();
}
return health;
} catch (retryError) { } catch (retryError) {
await this.logStartFailure(retryError); await this.logStartFailure(retryError);
throw retryError; throw retryError;
@@ -709,9 +696,6 @@ export class CoreSupervisor {
await delay(delayMs); await delay(delayMs);
if (this.stopping) return; if (this.stopping) return;
try { try {
// The old Reader pointed at the Core that just died. Stop it so
// the relaunched Core gets a Reader with the current token/port.
await this.stopReader();
// Keep the configured port stable across relaunches. The packaged // Keep the configured port stable across relaunches. The packaged
// renderer and discovery receipts point at this port, so moving to // renderer and discovery receipts point at this port, so moving to
// an ephemeral port would make a recovered Core unreachable. // an ephemeral port would make a recovered Core unreachable.
@@ -722,9 +706,6 @@ export class CoreSupervisor {
this.exited = false; this.exited = false;
this.spawnError = null; this.spawnError = null;
const health = await this.attempt(); const health = await this.attempt();
if (this.options.readerEnabled) {
await this.startReader();
}
await this.options.diagnostics?.log({ await this.options.diagnostics?.log({
level: "info", level: "info",
component: "runtime.core", component: "runtime.core",
@@ -843,14 +824,6 @@ export class CoreSupervisor {
cwd: this.options.launch.cwd, cwd: this.options.launch.cwd,
}); });
this.readerChild = child; this.readerChild = child;
// A Reader exits when its Core goes away. Clear the handle so the restart
// path can spawn a fresh one instead of returning early forever.
child.on("exit", () => {
if (this.readerChild === child) {
this.readerChild = null;
this.activeReaderPort = null;
}
});
// Wait briefly for the reader to publish its port on stderr. // Wait briefly for the reader to publish its port on stderr.
this.activeReaderPort = await new Promise<number | null>((resolve) => { this.activeReaderPort = await new Promise<number | null>((resolve) => {
let settled = false; let settled = false;
+2 -2
View File
@@ -12,8 +12,8 @@ test("installation docs disclose publication and lifecycle boundaries", async ()
"utf8", "utf8",
); );
assert.match(readme, /公共 Python SDK 为 `hy-memory==2\.0\.4`/); assert.match(readme, /公共 Python SDK 为 `hy-memory==2\.0\.3`/);
assert.match(readme, /`v1\.0\.4` 的发行目标是 Linux x64\/arm64 Runtime/); assert.match(readme, /`v1\.0\.3` 的发行目标是 Linux x64\/arm64 Runtime/);
assert.match(readme, /Linux 制品由 GitHub Actions/); assert.match(readme, /Linux 制品由 GitHub Actions/);
assert.match(readme, /macOS.*Developer ID 签名、公证/s); assert.match(readme, /macOS.*Developer ID 签名、公证/s);
assert.match(readme, /release-manifest\.json/); assert.match(readme, /release-manifest\.json/);
@@ -117,19 +117,9 @@ describe("Linux runtime scripts", () => {
workflow, workflow,
/verify-release-version\.mjs[\s\S]*--channel stable/, /verify-release-version\.mjs[\s\S]*--channel stable/,
); );
assert.match(workflow, /shopt -s globstar nullglob/);
assert.match(workflow, /archives=\(release-assets\/\*\*\/"\$name"\)/);
assert.match(
workflow,
/cp -- "\$\{archives\[0\]\}" "publish-assets\/\$name"/,
);
assert.match(workflow, /sha256sum -c/); assert.match(workflow, /sha256sum -c/);
assert.match(workflow, /--json isDraft[\s\S]*--jq \.isDraft/); assert.match(workflow, /--json isDraft[\s\S]*--jq \.isDraft/);
assert.match(workflow, /gh release create[\s\S]*--draft/); assert.match(workflow, /gh release create[\s\S]*--draft/);
assert.match(workflow, /gh release upload[\s\S]*--clobber/); assert.match(workflow, /gh release upload[\s\S]*--clobber/);
assert.match(
workflow,
/publish-assets\/memhub-runtime-v\*-linux-\*\.tar\.gz/,
);
}); });
}); });
-19
View File
@@ -1,19 +0,0 @@
from hy_memory.config import MemoryConfig
def test_chain_block_defaults_disabled(monkeypatch):
monkeypatch.delenv("MEMORY_CODING_CHAIN_BLOCK_ENABLED", raising=False)
assert MemoryConfig().coding.chain_block_enabled is False
def test_chain_block_env_can_enable(monkeypatch):
monkeypatch.setenv("MEMORY_CODING_CHAIN_BLOCK_ENABLED", "true")
assert MemoryConfig().coding.chain_block_enabled is True
def test_chain_block_from_dict_can_enable(monkeypatch):
monkeypatch.delenv("MEMORY_CODING_CHAIN_BLOCK_ENABLED", raising=False)
config = MemoryConfig.from_dict({
"coding": {"chain_block_enabled": True},
})
assert config.coding.chain_block_enabled is True
+1 -28
View File
@@ -62,21 +62,7 @@ def _seed_chain(tmpdir, branch="feat/q4", path=_PATH):
return H return H
def _enable_chain_block(ctx): def _bind_chain(ctx, store):
"""search 侧 chain_block 默认关;测注入路径时显式 opt-in。"""
from hy_memory.config import MemoryConfig
cfg = getattr(ctx, "_config", None)
if cfg is None:
cfg = MemoryConfig()
ctx._config = cfg
cfg.coding.chain_block_enabled = True
ctx._chain_block_enabled = HyMemoryClient._chain_block_enabled.__get__(
ctx, type(ctx))
return ctx
def _bind_chain(ctx, store, *, enable_block=True):
"""把链层挂到 search_ctx 这个 SimpleNamespace 上。 """把链层挂到 search_ctx 这个 SimpleNamespace 上。
search_ctx 是裸 namespace,不是真 client,所以 _decorate_with_chain 这个 search_ctx 是裸 namespace,不是真 client,所以 _decorate_with_chain 这个
@@ -87,8 +73,6 @@ def _bind_chain(ctx, store, *, enable_block=True):
ctx._get_chain_store = lambda owner="": store ctx._get_chain_store = lambda owner="": store
ctx._decorate_with_chain = HyMemoryClient._decorate_with_chain.__get__( ctx._decorate_with_chain = HyMemoryClient._decorate_with_chain.__get__(
ctx, type(ctx)) ctx, type(ctx))
if enable_block:
_enable_chain_block(ctx)
return ctx return ctx
@@ -145,17 +129,6 @@ class TestSearchCodingVariantEndToEnd:
assert "第1版" in blk and "第2版" in blk assert "第1版" in blk and "第2版" in blk
assert "255.0" in blk and "360.0" in blk assert "255.0" in blk and "360.0" in blk
async def test_coding_variant_chain_block_default_off(self, search_ctx, chain_dir):
"""默认不注入 chain_block(省 token);库里有链也返回空块。"""
await self._seed_fact(search_ctx)
# 挂上真链层,但不开启 chain_block_enabled
_bind_chain(search_ctx, _seed_chain(chain_dir), enable_block=False)
resp = await self._search(search_ctx, _STATEMENT, variant="coding")
assert len(resp["facts"]) == 1
assert resp["chain_block"] == ""
assert resp["chain_used"] == []
async def test_facts_not_deduped(self, search_ctx, chain_dir): async def test_facts_not_deduped(self, search_ctx, chain_dir):
"""★ 挂上链的 fact 仍留在 facts 里 —— 刻意不去重。 """★ 挂上链的 fact 仍留在 facts 里 —— 刻意不去重。
+3 -24
View File
@@ -326,34 +326,12 @@ class TestChainDecorate:
os.environ.pop("HCM_BLOCK_OFF", None) os.environ.pop("HCM_BLOCK_OFF", None)
def _ns_with_chain_block(**kwargs):
"""胶水测需要显式 opt-in:默认 chain_block_enabled=False 会在进 store 前短路。"""
import types
from hy_memory.config import MemoryConfig
cfg = MemoryConfig()
cfg.coding.chain_block_enabled = True
ns = types.SimpleNamespace(_config=cfg, **kwargs)
ns._chain_block_enabled = HyMemoryClient._chain_block_enabled.__get__(
ns, type(ns))
return ns
class TestDecorateGlue: class TestDecorateGlue:
"""client._decorate_with_chain —— 胶水层的 fail-safe。""" """client._decorate_with_chain —— 胶水层的 fail-safe。"""
def test_returns_empty_when_disabled_by_default(self): def test_returns_empty_when_chain_unavailable(self):
import types import types
ns = types.SimpleNamespace( ns = types.SimpleNamespace(
_chain_store=None, _chain_store_failed=False,
_get_chain_store=lambda owner="": object(),
)
ns._chain_block_enabled = HyMemoryClient._chain_block_enabled.__get__(
ns, type(ns))
assert HyMemoryClient._decorate_with_chain(ns, [dict(_FACT)]) == ("", [])
def test_returns_empty_when_chain_unavailable(self):
ns = _ns_with_chain_block(
_chain_store=None, _chain_store_failed=True, _chain_store=None, _chain_store_failed=True,
_get_chain_store=lambda owner="": None, _get_chain_store=lambda owner="": None,
) )
@@ -361,12 +339,13 @@ class TestDecorateGlue:
def test_store_exception_degrades_not_raises(self): def test_store_exception_degrades_not_raises(self):
# 这层是增量, 挂了要退回纯向量双层而不是把整次 search 拖失败。 # 这层是增量, 挂了要退回纯向量双层而不是把整次 search 拖失败。
import types
class _Boom: class _Boom:
def decorate(self, *a, **k): def decorate(self, *a, **k):
raise RuntimeError("boom") raise RuntimeError("boom")
ns = _ns_with_chain_block(_get_chain_store=lambda owner="": _Boom()) ns = types.SimpleNamespace(_get_chain_store=lambda owner="": _Boom())
assert HyMemoryClient._decorate_with_chain(ns, [dict(_FACT)]) == ("", []) assert HyMemoryClient._decorate_with_chain(ns, [dict(_FACT)]) == ("", [])
-13
View File
@@ -1,13 +0,0 @@
# -*- coding: utf-8 -*-
"""Public Abstractor result types must remain constructible dataclasses."""
from hy_memory.agent.abstractor import SessionSummaryResult
def test_session_summary_result_accepts_fields():
result = SessionSummaryResult(success=True, summary="session summary")
assert result.success is True
assert result.summary == "session summary"
assert result.key_topics == []
assert result.error is None
-45
View File
@@ -1,45 +0,0 @@
# -*- coding: utf-8 -*-
"""Owned event-loop threads must be stopped and closed with the client."""
from hy_memory.client import HyMemoryClient, _LoopThread
def test_loop_thread_stop_closes_the_loop_and_is_idempotent():
loop_thread = _LoopThread()
loop = loop_thread._loop
loop_thread.stop()
assert not loop_thread._thread.is_alive()
assert loop.is_closed()
# A second stop must not touch a closed loop.
loop_thread.stop()
def test_close_stops_both_loops_owned_by_a_solo_client():
client = HyMemoryClient.__new__(HyMemoryClient)
client._registry = None
client._owns_runtime = True
client._loop_thread = _LoopThread()
client._search_loop_thread = _LoopThread()
loops = [client._loop_thread._loop, client._search_loop_thread._loop]
client.close()
assert all(loop.is_closed() for loop in loops)
def test_shared_runtime_loops_are_left_to_the_runtime():
client = HyMemoryClient.__new__(HyMemoryClient)
client._registry = None
client._owns_runtime = False
client._loop_thread = _LoopThread()
client._search_loop_thread = _LoopThread()
loops = [client._loop_thread._loop, client._search_loop_thread._loop]
try:
client.close()
assert all(not loop.is_closed() for loop in loops)
finally:
client._loop_thread.stop()
client._search_loop_thread.stop()
-52
View File
@@ -173,41 +173,6 @@ class TestEpisodeCloseTriggersSummary:
await w._resolve_episode_id(_req(t0 + timedelta(hours=3)), 3) await w._resolve_episode_id(_req(t0 + timedelta(hours=3)), 3)
assert scheduled == [ep1], "只在关闭时对刚关闭的那个 episode 触发一次" assert scheduled == [ep1], "只在关闭时对刚关闭的那个 episode 触发一次"
@pytest.mark.asyncio
async def test_summary_uses_the_writer_vector_store(self, monkeypatch):
"""后台摘要必须拿到 writer 已初始化的 vector store。
旧实现访问了不存在的 ``self.vector_store``,AttributeError 被
except 吞掉,L3 episode 摘要从不生成。
"""
from types import SimpleNamespace
import hy_memory.agent.episode_summary as episode_summary
captured = {}
async def fake_summarize(**kwargs):
captured.update(kwargs)
return "episode-node"
monkeypatch.setattr(
episode_summary, "summarize_closed_episode", fake_summarize,
)
w = _writer(_FakeCache())
w.config.recall.episode_summary_enabled = True
w._mem_agent = SimpleNamespace(llm_provider=object())
w._embed_service = object()
vector_store = object()
w._external_vector_store = vector_store
w._vector_store = vector_store
w._vector_store_initialized = True
w._schedule_episode_summary(_req(datetime(2026, 9, 13)), "ep-1")
await asyncio.sleep(0.05)
assert captured["vector_store"] is vector_store
# ====================================================================== # ======================================================================
# Episode 摘要(写侧,省 token) # Episode 摘要(写侧,省 token)
@@ -710,20 +675,3 @@ class TestInstrumentation:
assert logged[1]["count"] == 5, "累计条数要落下来" assert logged[1]["count"] == 5, "累计条数要落下来"
assert logged[2]["reused"] is False assert logged[2]["reused"] is False
assert all(r["gap"] == 1800.0 for r in logged) assert all(r["gap"] == 1800.0 for r in logged)
def test_series_memory_at_round_trips_local_wall_clock(monkeypatch):
"""Series timestamps must not shift by the machine's UTC offset."""
import time as _time
from hy_memory.agent.episode_summary import _series_memory_at
monkeypatch.setenv("TZ", "Asia/Shanghai")
_time.tzset()
try:
expected = datetime(2026, 1, 1, 12, 0)
assert _series_memory_at([int(expected.timestamp())]) == expected
assert _series_memory_at([]) is None
finally:
monkeypatch.delenv("TZ", raising=False)
_time.tzset()
-57
View File
@@ -1,57 +0,0 @@
# -*- coding: utf-8 -*-
"""FAISS 后端的 get_embeddings 必须能取回普通 upsert 写入的向量。
``_data`` 只保存 payload 与内部下标,向量本体在 FAISS index 里。去重和
episode 摘要依赖 get_embeddings,取不到向量会让这些能力静默失效。
"""
import asyncio
import pytest
pytest.importorskip("faiss")
from hy_memory.config import MemoryConfig # noqa: E402
from hy_memory.data.vector_store_faiss import FaissVectorStore # noqa: E402
from hy_memory.models.memory import MemoryLayer, MemoryNode # noqa: E402
def _config(tmp_path):
return MemoryConfig.from_dict({
"vector_store": {
"provider": "faiss",
"collection_name": "get_embeddings_test",
"persist_directory": str(tmp_path / "faiss"),
"embedding_dims": 4,
},
})
def test_get_embeddings_reconstructs_upserted_and_updated_vectors(tmp_path):
async def _run():
store = FaissVectorStore(_config(tmp_path))
await store.initialize()
try:
node = MemoryNode(
user_id="u1",
agent_id="a1",
session_id="s1",
source="default",
layer=MemoryLayer.L1_RAW,
content="hello",
embedding=[1.0, 0.0, 0.0, 0.0],
)
await store.upsert(node)
got = await store.get_embeddings([node.node_id])
assert got[node.node_id] == pytest.approx([1.0, 0.0, 0.0, 0.0])
assert await store.update_embedding(node.node_id, [0.0, 1.0, 0.0, 0.0])
got = await store.get_embeddings([node.node_id])
assert got[node.node_id] == pytest.approx([0.0, 1.0, 0.0, 0.0])
assert await store.get_embeddings(["missing"]) == {}
finally:
await store.close()
asyncio.run(_run())
-60
View File
@@ -1,60 +0,0 @@
# -*- coding: utf-8 -*-
"""History queries without agent_ids must cover every agent of the user."""
import asyncio
from hy_memory.config import MemoryConfig
from hy_memory.data.history_store import HistoryStore
def _store(tmp_path):
config = MemoryConfig()
config.history.db_path = str(tmp_path / "history.db")
return HistoryStore(config)
def test_get_recent_for_user_matches_all_agents(tmp_path):
async def _run():
store = _store(tmp_path)
await store.initialize()
try:
keys = [
"u1:default_agent",
"u1:agent_b",
"u10:default_agent",
"u2:default_agent",
]
for index, key in enumerate(keys):
await store.record_add(
memory_id=f"m{index}",
content="x",
layer="l1_raw",
isolation_key=key,
)
rows = await store.get_recent_for_user("u1", limit=10)
finally:
await store.close()
return [row["isolation_key"] for row in rows]
assert set(asyncio.run(_run())) == {"u1:default_agent", "u1:agent_b"}
def test_get_recent_for_user_escapes_like_wildcards(tmp_path):
async def _run():
store = _store(tmp_path)
await store.initialize()
try:
await store.record_add(
memory_id="m1", content="x", layer="l1_raw",
isolation_key="u_1:default_agent",
)
await store.record_add(
memory_id="m2", content="x", layer="l1_raw",
isolation_key="uX1:default_agent",
)
rows = await store.get_recent_for_user("u_1", limit=10)
finally:
await store.close()
return [row["isolation_key"] for row in rows]
assert asyncio.run(_run()) == ["u_1:default_agent"]
-111
View File
@@ -1,111 +0,0 @@
# -*- coding: utf-8 -*-
"""MySQL minute metrics must merge under one row lock."""
import asyncio
import json
import pytest
pytest.importorskip("aiomysql")
from hy_memory.config import MemoryConfig # noqa: E402
from hy_memory.data.cache_mysql import MysqlCache # noqa: E402
class FakeCursor:
def __init__(self, row):
self.row = row
self.executed = []
async def __aenter__(self):
return self
async def __aexit__(self, *args):
return None
async def execute(self, sql, args=None):
self.executed.append((sql, args))
async def fetchone(self):
return self.row
class FakeConnection:
def __init__(self, row):
self.cursor_obj = FakeCursor(row)
self.begun = False
self.committed = False
self.rolled_back = False
async def begin(self):
self.begun = True
async def commit(self):
self.committed = True
async def rollback(self):
self.rolled_back = True
def cursor(self, *args, **kwargs):
return self.cursor_obj
class _Acquire:
def __init__(self, connection):
self.connection = connection
async def __aenter__(self):
return self.connection
async def __aexit__(self, *args):
return None
class FakePool:
def __init__(self, connection):
self.connection = connection
def acquire(self):
return _Acquire(self.connection)
def _cache_with(row):
cache = MysqlCache(MemoryConfig())
connection = FakeConnection(row)
cache._pool = FakePool(connection)
return cache, connection
def test_metrics_insert_takes_the_row_lock_and_commits():
cache, connection = _cache_with(None)
asyncio.run(
cache.store_metrics_minute("2026-01-01T00:00", {"sys1_started": 2})
)
statements = [sql for sql, _ in connection.cursor_obj.executed]
assert any("FOR UPDATE" in sql for sql in statements)
assert any(sql.startswith("INSERT") for sql in statements)
assert connection.begun and connection.committed
assert not connection.rolled_back
def test_metrics_update_merges_existing_buckets():
old = {"sys1_started": 1, "sys1_timing_sums": {"a": 1}}
cache, connection = _cache_with({"data": json.dumps(old)})
asyncio.run(
cache.store_metrics_minute(
"2026-01-01T00:00",
{"sys1_started": 2, "sys1_timing_sums": {"a": 3}},
)
)
update = next(
args for sql, args in connection.cursor_obj.executed
if sql.startswith("UPDATE")
)
merged = json.loads(update[0])
assert merged["sys1_started"] == 3
assert merged["sys1_timing_sums"]["a"] == 4
assert connection.committed
-22
View File
@@ -77,28 +77,6 @@ class TestReadRequestWiring:
HyMemoryClient._normalize_layers(["l3_identity"]) HyMemoryClient._normalize_layers(["l3_identity"])
class TestDedupKindPairing:
"""dedup 的 _memory_kind 必须与它的 public item 来自同一条 reader 结果。"""
def test_memory_kind_follows_source_after_layer_filter(self):
memories = [
{"memory_id": "raw", "layer": "l1_raw", "content": "raw"},
{
"memory_id": "canon",
"layer": "l2_fact",
"content": "canon",
"_memory_kind": "canonical_state",
},
]
all_mems, dedup_mems = HyMemoryClient._public_search_views(
memories,
wanted_layers={"l2_fact"},
lite_mode=False,
)
assert [m["memory_id"] for m in all_mems] == ["canon"]
assert dedup_mems[0]["_memory_kind"] == "canonical_state"
class TestHybridDirectedRecall: class TestHybridDirectedRecall:
"""共享 reader 主流程必须把 request.layers 下推到召回阶段。""" """共享 reader 主流程必须把 request.layers 下推到召回阶段。"""
-76
View File
@@ -1,76 +0,0 @@
# -*- coding: utf-8 -*-
"""HTTP server signal shutdown must not deadlock the serving thread."""
import subprocess
import sys
import textwrap
import threading
import time
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from hy_memory.server import _request_shutdown
class _Handler(BaseHTTPRequestHandler):
def do_GET(self): # pragma: no cover - the test only needs a live server
self.send_response(200)
self.end_headers()
def log_message(self, *args): # pragma: no cover - silence test output
pass
def test_request_shutdown_returns_promptly():
server = ThreadingHTTPServer(("127.0.0.1", 0), _Handler)
server.daemon_threads = True
thread = threading.Thread(target=server.serve_forever)
thread.start()
try:
started = time.monotonic()
_request_shutdown(server)
assert time.monotonic() - started < 1.0
thread.join(timeout=2.0)
assert not thread.is_alive()
finally:
server.server_close()
def test_signal_handler_does_not_deadlock_the_serving_thread():
"""A signal handler calling shutdown() directly hangs forever.
Run the serving loop on the main thread (as run_server does), fire an
alarm, and require the process to exit on its own.
"""
script = textwrap.dedent(
"""
import signal
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from hy_memory.server import _request_shutdown
class H(BaseHTTPRequestHandler):
def do_GET(self):
self.send_response(200)
self.end_headers()
def log_message(self, *args):
pass
server = ThreadingHTTPServer(("127.0.0.1", 0), H)
def _handler(signum, frame):
_request_shutdown(server)
signal.signal(signal.SIGALRM, _handler)
signal.alarm(1)
server.serve_forever()
server.server_close()
print("exited")
"""
)
result = subprocess.run(
[sys.executable, "-c", script],
capture_output=True,
text=True,
timeout=10,
)
assert result.returncode == 0, result.stderr
assert "exited" in result.stdout
-70
View File
@@ -1,70 +0,0 @@
# 新鲜度(memory evidence freshness)拆除 TODO
状态:PR1 已开(读路径短路)——分支 `chore/retire-memory-freshness`。#86 不合并。
## 结论
- 新鲜度 = 写记忆时记下 `support_refs` 指纹,读时在 MemHub SQLite 里逐条核对 evidence 是否仍匹配;对不上则滤掉该 memory。
- 这是过度设计:读路径成本高、失败面大(曾导致记忆库整页 503),收益仅是「证据被改过就藏记忆」。
- 业务层为喂 SDK 再做长证据分段(`segment_*`),与 SDK 自身阶段/窗口处理重复;#86 只是继续兼容这套 proof,不应合入。
## 范围(MemHub Core,非 SDK)
权威库:`~/.one-memory/one-memory.sqlite3`
不涉及 SDK `hy-memory/**` 内部 store。
### 读路径(优先拆)
| 位置 | 作用 |
|---|---|
| `memhub/apps/core/src/one_memory/memory_evidence_freshness.py` | UDF `is_memory_evidence_current`、proof prefetch、预算跳过 |
| `source_read_access.py` | 列表/过滤时 per-memory freshness |
| `context_memory.py` | search/context 候选行 `_memory_row_is_fresh` |
| `memory_library.py` / `home_summary.py` | 记忆库 / 首页可读谓词 |
| `memory_visibility.py` | 可见性缓存与 freshness 注册 |
| `call_event_references.py` | 引用解析时的 freshness |
### 写路径 / 数据(后拆,可分 PR)
| 位置 | 作用 |
|---|---|
| `memory_tasks.py` → `MemoryTaskOutput.support_refs` / `TaskEvidenceRef` | 抽取输出里的证明 |
| `memory_task_outputs` 表 | proof 持久化 |
| `memory_processing.py` 分段(`segment_evidence_record` / `_semantic_segment_spans`) | 为 HY 窗口切段;与 freshness 耦合但不全是 freshness——分段是否保留需单独拍板 |
### 测试(随拆除改/删)
- `tests/test_memory_evidence_freshness.py`
- `tests/test_memory_freshness_integration_review.py`
- `tests/test_read_path_freshness_batch.py`
- 以及 mock `register_memory_freshness` 的 library / context / visibility / home_summary / call_event 测试
## 建议落地步骤
1. **读路径短路(最小 PR)** — 本 PR
`register_memory_freshness` 恒返回 `"1"`(UDF 恒 `1`),`freshness_proofs_prefetched` 为空操作。行为:证据改过也不再藏 memory;畸形/segment proof 不再 503。
同步改「期望 stale=0」的单测为「始终可读」。
2. **拆调用点**
去掉 `AND is_memory_evidence_current(...)`、`freshness_proofs_prefetched`、`_memory_row_is_fresh` 等死代码,避免假接口长期残留。
3. **停写 / 弃用 proof(可选第二 PR)**
新任务不再依赖 `support_refs` 做读决策;表与历史行可先保留只读,再决定是否 migration 清理。
4. **分段策略另议**
长证据切段是 ingest 窗口问题,不是新鲜度问题。拆除 freshness 后单独决定:继续由 MemHub 切 messages,还是交给 SDK。
5. **对外沟通**
- #86:已留言不合并、将拆除设计。
- 相关 issue(如 #74 里 freshness 503)关闭或改指向「拆除」而非「修 segment」。
## 验收
- 记忆库 / search / context 在存在「证据已修订 + 旧 proof」时仍返回该 memory,且不再因 malformed/segment ref 503。
- 无 `ONE_MEMORY_FRESHNESS_*` 行为依赖;文档/env 若有提及一并删。
- Core 相关测试绿。
## 非目标
- 不改 SDK 内部抽取/存储。
- 不在本轮做「用别的一致性模型替换 freshness」;先删,再按需重设计。