Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
1821a31295 |
No files matched your search
@@ -86,7 +86,6 @@ MEMORY_VECTOR_STORE=chroma
|
|||||||
# MEMORY_CODING_DB_PATH= # coding_memory.db 路径(默认 {DATA_DIR}/data/coding_memory.db)
|
# MEMORY_CODING_DB_PATH= # coding_memory.db 路径(默认 {DATA_DIR}/data/coding_memory.db)
|
||||||
# MEMORY_CODING_TOOL_RESULT_MAX_BYTES=2048 # 单条 tool_result 截断字节上限
|
# MEMORY_CODING_TOOL_RESULT_MAX_BYTES=2048 # 单条 tool_result 截断字节上限
|
||||||
# MEMORY_CODING_VERBATIM_FALLBACK_ENABLED=false # 模型无结果时是否保存原文 fact;默认关,避免 tool trace 外泄
|
# MEMORY_CODING_VERBATIM_FALLBACK_ENABLED=false # 模型无结果时是否保存原文 fact;默认关,避免 tool trace 外泄
|
||||||
# MEMORY_CODING_CHAIN_BLOCK_ENABLED=false # search 是否注入版本演进链块(diff);默认关,省 token
|
|
||||||
# MEMORY_EXTRACT_HISTORY_TURNS=5 # 除本次 write 外,额外注入 extractor 的历史 user-turn 数
|
# MEMORY_EXTRACT_HISTORY_TURNS=5 # 除本次 write 外,额外注入 extractor 的历史 user-turn 数
|
||||||
|
|
||||||
# ── 写入流程开关 ──
|
# ── 写入流程开关 ──
|
||||||
|
|||||||
@@ -122,25 +122,19 @@ jobs:
|
|||||||
shell: bash
|
shell: bash
|
||||||
run: |
|
run: |
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
shopt -s globstar nullglob
|
|
||||||
version=$(node -p 'require("./package.json").version')
|
version=$(node -p 'require("./package.json").version')
|
||||||
node scripts/verify-release-version.mjs \
|
node scripts/verify-release-version.mjs \
|
||||||
--channel stable \
|
--channel stable \
|
||||||
--version "$version" \
|
--version "$version" \
|
||||||
--build-number "$(node -p 'require("./package.json").release.build_number')" \
|
--build-number "$(node -p 'require("./package.json").release.build_number')" \
|
||||||
--tag "$GITHUB_REF_NAME"
|
--tag "$GITHUB_REF_NAME"
|
||||||
mkdir -p publish-assets
|
|
||||||
for arch in x64 arm64; do
|
for arch in x64 arm64; do
|
||||||
name="memhub-runtime-v${version}-linux-${arch}.tar.gz"
|
archive="release-assets/memhub-runtime-v${version}-linux-${arch}.tar.gz"
|
||||||
archives=(release-assets/**/"$name")
|
test -f "$archive"
|
||||||
checksums=(release-assets/**/"$name.sha256")
|
test -f "$archive.sha256"
|
||||||
test "${#archives[@]}" -eq 1
|
|
||||||
test "${#checksums[@]}" -eq 1
|
|
||||||
cp -- "${archives[0]}" "publish-assets/$name"
|
|
||||||
cp -- "${checksums[0]}" "publish-assets/$name.sha256"
|
|
||||||
(
|
(
|
||||||
cd publish-assets
|
cd release-assets
|
||||||
sha256sum -c "$name.sha256"
|
sha256sum -c "$(basename "$archive.sha256")"
|
||||||
)
|
)
|
||||||
done
|
done
|
||||||
- name: Upload Linux assets to a draft GitHub Release
|
- name: Upload Linux assets to a draft GitHub Release
|
||||||
@@ -166,7 +160,7 @@ jobs:
|
|||||||
--title "$GITHUB_REF_NAME"
|
--title "$GITHUB_REF_NAME"
|
||||||
fi
|
fi
|
||||||
gh release upload "$GITHUB_REF_NAME" \
|
gh release upload "$GITHUB_REF_NAME" \
|
||||||
publish-assets/memhub-runtime-v*-linux-*.tar.gz \
|
release-assets/memhub-runtime-v*-linux-*.tar.gz \
|
||||||
publish-assets/memhub-runtime-v*-linux-*.tar.gz.sha256 \
|
release-assets/memhub-runtime-v*-linux-*.tar.gz.sha256 \
|
||||||
--repo "$GITHUB_REPOSITORY" \
|
--repo "$GITHUB_REPOSITORY" \
|
||||||
--clobber
|
--clobber
|
||||||
@@ -14,10 +14,6 @@ Hy Memory and MemHub are versioned and released independently.
|
|||||||
|
|
||||||
## Unreleased
|
## Unreleased
|
||||||
|
|
||||||
- Coding search no longer injects the version-evolution diff block
|
|
||||||
(`chain_block`) by default. Enable with
|
|
||||||
`MEMORY_CODING_CHAIN_BLOCK_ENABLED=true` or
|
|
||||||
`CodingConfig.chain_block_enabled=True` (write-side observe unchanged).
|
|
||||||
- Fixed governed-memory revisions and repeated evidence links failing when the
|
- Fixed governed-memory revisions and repeated evidence links failing when the
|
||||||
visibility dirty-marker trigger encounters the same memory twice. Startup
|
visibility dirty-marker trigger encounters the same memory twice. Startup
|
||||||
now replaces the affected legacy trigger without changing schema versions
|
now replaces the affected legacy trigger without changing schema versions
|
||||||
|
|||||||
@@ -65,13 +65,13 @@ Welcome to the **Hy Memory Research Family**, covering cognitive chat memory, lo
|
|||||||
### 1. Installation
|
### 1. Installation
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
pip install hy-memory==2.0.4 # core (includes ChromaDB and Kuzu)
|
pip install hy-memory==2.0.3 # core (includes ChromaDB and Kuzu)
|
||||||
pip install "hy-memory[qdrant]==2.0.4" # + Qdrant
|
pip install "hy-memory[qdrant]==2.0.3" # + Qdrant
|
||||||
pip install "hy-memory[faiss]==2.0.4" # + FAISS
|
pip install "hy-memory[faiss]==2.0.3" # + FAISS
|
||||||
pip install "hy-memory[graph]==2.0.4" # + Neo4j
|
pip install "hy-memory[graph]==2.0.3" # + Neo4j
|
||||||
pip install "hy-memory[mysql]==2.0.4" # + MySQL cache
|
pip install "hy-memory[mysql]==2.0.3" # + MySQL cache
|
||||||
pip install "hy-memory[rerank]==2.0.4" # + local multilingual cross-encoder rerank
|
pip install "hy-memory[rerank]==2.0.3" # + local multilingual cross-encoder rerank
|
||||||
pip install "hy-memory[all]==2.0.4" # everything
|
pip install "hy-memory[all]==2.0.3" # everything
|
||||||
```
|
```
|
||||||
|
|
||||||
### 2. Configuration
|
### 2. Configuration
|
||||||
@@ -153,7 +153,7 @@ result = client.search(
|
|||||||
|
|
||||||
result["facts"] # fact layer
|
result["facts"] # fact layer
|
||||||
result["experiences"] # experience layer
|
result["experiences"] # experience layer
|
||||||
result["chain_block"] # coding only: version-evolution context (empty unless MEMORY_CODING_CHAIN_BLOCK_ENABLED)
|
result["chain_block"] # coding only: version-evolution context
|
||||||
```
|
```
|
||||||
|
|
||||||
### 5. Routing, Source Isolation & Advanced APIs
|
### 5. Routing, Source Isolation & Advanced APIs
|
||||||
|
|||||||
+8
-8
@@ -65,13 +65,13 @@
|
|||||||
### 1. 安装
|
### 1. 安装
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
pip install hy-memory==2.0.4 # 核心(含 ChromaDB 与 Kuzu)
|
pip install hy-memory==2.0.3 # 核心(含 ChromaDB 与 Kuzu)
|
||||||
pip install "hy-memory[qdrant]==2.0.4" # + Qdrant
|
pip install "hy-memory[qdrant]==2.0.3" # + Qdrant
|
||||||
pip install "hy-memory[faiss]==2.0.4" # + FAISS
|
pip install "hy-memory[faiss]==2.0.3" # + FAISS
|
||||||
pip install "hy-memory[graph]==2.0.4" # + Neo4j
|
pip install "hy-memory[graph]==2.0.3" # + Neo4j
|
||||||
pip install "hy-memory[mysql]==2.0.4" # + MySQL 缓存
|
pip install "hy-memory[mysql]==2.0.3" # + MySQL 缓存
|
||||||
pip install "hy-memory[rerank]==2.0.4" # + 本地多语言 cross-encoder rerank
|
pip install "hy-memory[rerank]==2.0.3" # + 本地多语言 cross-encoder rerank
|
||||||
pip install "hy-memory[all]==2.0.4" # 全部
|
pip install "hy-memory[all]==2.0.3" # 全部
|
||||||
```
|
```
|
||||||
|
|
||||||
### 2. 配置
|
### 2. 配置
|
||||||
@@ -153,7 +153,7 @@ result = client.search(
|
|||||||
|
|
||||||
result["facts"] # 事实层
|
result["facts"] # 事实层
|
||||||
result["experiences"] # 经验层
|
result["experiences"] # 经验层
|
||||||
result["chain_block"] # 仅 coding:版本演进上下文(默认空;需 MEMORY_CODING_CHAIN_BLOCK_ENABLED)
|
result["chain_block"] # 仅 coding:版本演进上下文
|
||||||
```
|
```
|
||||||
|
|
||||||
### 5. 自动路由、来源过滤与进阶 API
|
### 5. 自动路由、来源过滤与进阶 API
|
||||||
|
|||||||
@@ -197,7 +197,6 @@ SDK 不设置 Rerank Search 并发上限。开启后模型在 `HyMemoryClient`
|
|||||||
| `MEMORY_HISTORY_DB_PATH` | `{DATA_DIR}/data/history.db` | History SQLite 路径 |
|
| `MEMORY_HISTORY_DB_PATH` | `{DATA_DIR}/data/history.db` | History SQLite 路径 |
|
||||||
| `MEMORY_HISTORY_RECORD_SEARCHES` | `false` | 是否记录 Search 操作 |
|
| `MEMORY_HISTORY_RECORD_SEARCHES` | `false` | 是否记录 Search 操作 |
|
||||||
| `MEMORY_CODING_CHAIN_REPO_ROOT` | — | coding 版本链读取本地 Git diff 的仓库根目录 |
|
| `MEMORY_CODING_CHAIN_REPO_ROOT` | — | coding 版本链读取本地 Git diff 的仓库根目录 |
|
||||||
| `MEMORY_CODING_CHAIN_BLOCK_ENABLED` | `false` | search(scene=coding) 是否注入版本演进链块(diff / `chain_block`);默认关 |
|
|
||||||
|
|
||||||
## HTTP Server
|
## HTTP Server
|
||||||
|
|
||||||
|
|||||||
@@ -78,8 +78,7 @@ default 多-query 超过 `MEMORY_QUERY_REWRITE_MAX_CHARS` 时不改写:Coding
|
|||||||
Coding 关闭则直接使用最后一条 query。
|
Coding 关闭则直接使用最后一条 query。
|
||||||
|
|
||||||
路由到 `working` / `coding` 后,Search 读取独立生产力记忆库,返回 `facts` /
|
路由到 `working` / `coding` 后,Search 读取独立生产力记忆库,返回 `facts` /
|
||||||
`experiences`,不使用 chat 三通道结构。`coding` 还返回 `chain_block` / `chain_used`
|
`experiences`,不使用 chat 三通道结构。`coding` 还返回 `chain_block` / `chain_used`。
|
||||||
(默认空;需 `MEMORY_CODING_CHAIN_BLOCK_ENABLED` 才注入 diff 块)。
|
|
||||||
|
|
||||||
## Chat 成功返回结构
|
## Chat 成功返回结构
|
||||||
|
|
||||||
@@ -153,7 +152,7 @@ client = HyMemoryClient.from_config({
|
|||||||
安装可选依赖:
|
安装可选依赖:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
pip install 'hy-memory[rerank]==2.0.4'
|
pip install 'hy-memory[rerank]==2.0.1'
|
||||||
```
|
```
|
||||||
|
|
||||||
启用后,`HyMemoryClient(...)` 构造阶段会完成模型下载/读取、device 放置和一次 dummy
|
启用后,`HyMemoryClient(...)` 构造阶段会完成模型下载/读取、device 放置和一次 dummy
|
||||||
|
|||||||
+3
-2
@@ -37,7 +37,8 @@ do not belong in the public source tree.
|
|||||||
- Root GitHub workflows run SDK and MemHub checks from their respective working
|
- Root GitHub workflows run SDK and MemHub checks from their respective working
|
||||||
directories. Native lifecycle workflows produce reviewable Actions artifacts;
|
directories. Native lifecycle workflows produce reviewable Actions artifacts;
|
||||||
tag releases remain fail-closed on public identity, signing and notarization.
|
tag releases remain fail-closed on public identity, signing and notarization.
|
||||||
- Hy Memory is published as `hy-memory==2.0.4`. MemHub pins that public
|
- Hy Memory is published as `hy-memory==2.0.3`. MemHub pins that public
|
||||||
distribution; its selected npm package is `memhub-cli` 1.0.5.
|
distribution; its selected npm package is `memhub-cli`, which is not yet
|
||||||
|
published.
|
||||||
- Hy-Memory source uses Apache-2.0 at the repository root. Third-party
|
- Hy-Memory source uses Apache-2.0 at the repository root. Third-party
|
||||||
components retain their own licenses and attribution notices.
|
components retain their own licenses and attribution notices.
|
||||||
@@ -168,11 +168,9 @@ working/coding 搜索返回:
|
|||||||
|
|
||||||
`limit` 分别限制 facts 与 experiences,因此两层最多各返回 `limit` 条。
|
`limit` 分别限制 facts 与 experiences,因此两层最多各返回 `limit` 条。
|
||||||
|
|
||||||
`scene="coding"` 额外返回 `chain_block` 和 `chain_used` 字段(键始终存在;默认内容为空):
|
`scene="coding"` 额外返回 `chain_block` 和 `chain_used`:
|
||||||
|
|
||||||
- 注入 diff / 版本演进块需显式开启 `MEMORY_CODING_CHAIN_BLOCK_ENABLED=true`
|
- `chain_block` 是与已召回 fact 对齐的版本演进上下文;无法挂链时为 `""`。
|
||||||
(或 `CodingConfig.chain_block_enabled=True`)。默认关闭,以节约 prompt token。
|
|
||||||
- 开启后,`chain_block` 是与已召回 fact 对齐的版本演进上下文;无法挂链时为 `""`。
|
|
||||||
- `chain_used` 是已经进入 chain block 的 `facts` 下标。
|
- `chain_used` 是已经进入 chain block 的 `facts` 下标。
|
||||||
- SDK 不会从 `facts` 删除这些条目。调用方若要拼成“chain block + 剩余 facts”,需自行按 `chain_used` 过滤。
|
- SDK 不会从 `facts` 删除这些条目。调用方若要拼成“chain block + 剩余 facts”,需自行按 `chain_used` 过滤。
|
||||||
|
|
||||||
@@ -224,12 +222,10 @@ working/coding 搜索返回:
|
|||||||
| `MEMORY_CODING_DB_PATH` | `{DATA_DIR}/data/coding_memory.db` | working 基础 SQLite 路径;coding/trace 路径由此派生 |
|
| `MEMORY_CODING_DB_PATH` | `{DATA_DIR}/data/coding_memory.db` | working 基础 SQLite 路径;coding/trace 路径由此派生 |
|
||||||
| `MEMORY_CODING_TOOL_RESULT_MAX_BYTES` | `2048` | 单条 tool result 的字节上限 |
|
| `MEMORY_CODING_TOOL_RESULT_MAX_BYTES` | `2048` | 单条 tool result 的字节上限 |
|
||||||
| `MEMORY_CODING_VERBATIM_FALLBACK_ENABLED` | `false` | 模型无结果时是否保存原文 fact;默认关闭,避免 assistant/tool trace 直接展示 |
|
| `MEMORY_CODING_VERBATIM_FALLBACK_ENABLED` | `false` | 模型无结果时是否保存原文 fact;默认关闭,避免 assistant/tool trace 直接展示 |
|
||||||
| `MEMORY_CODING_CHAIN_BLOCK_ENABLED` | `false` | search 是否注入版本演进链块(`chain_block` / diff);默认关闭(opt-in) |
|
|
||||||
| `MEMORY_EXTRACT_HISTORY_TURNS` | `5` | 回读的历史 user-turn 数;当前 prompt 最终最多使用 5 |
|
| `MEMORY_EXTRACT_HISTORY_TURNS` | `5` | 回读的历史 user-turn 数;当前 prompt 最终最多使用 5 |
|
||||||
| `MEMORY_CODING_CHAIN_REPO_ROOT` | 空 | coding 版本链读取本地 Git diff 的仓库路径 |
|
| `MEMORY_CODING_CHAIN_REPO_ROOT` | 空 | coding 版本链读取本地 Git diff 的仓库路径 |
|
||||||
|
|
||||||
`HCM_*` 变量属于版本链的高级调优项,不是稳定的公共配置契约;普通接入不应依赖其默认值。
|
`HCM_*` 变量属于版本链的高级调优项,不是稳定的公共配置契约;普通接入不应依赖其默认值。
|
||||||
其中 `HCM_BLOCK_OFF=1` 可在已开启 `MEMORY_CODING_CHAIN_BLOCK_ENABLED` 时做消融(完全不出块)。
|
|
||||||
|
|
||||||
## 8. 定向验证
|
## 8. 定向验证
|
||||||
|
|
||||||
|
|||||||
+7
-8
@@ -5,14 +5,14 @@
|
|||||||
## 安装与快速开始
|
## 安装与快速开始
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
pip install hy-memory==2.0.4
|
pip install hy-memory==2.0.3
|
||||||
|
|
||||||
# 可选后端
|
# 可选后端
|
||||||
pip install 'hy-memory[qdrant]==2.0.4'
|
pip install 'hy-memory[qdrant]==2.0.1'
|
||||||
pip install 'hy-memory[faiss]==2.0.4'
|
pip install 'hy-memory[faiss]==2.0.1'
|
||||||
pip install 'hy-memory[graph]==2.0.4' # Neo4j;Kuzu 已在核心依赖中
|
pip install 'hy-memory[graph]==2.0.1' # Neo4j;Kuzu 已在核心依赖中
|
||||||
pip install 'hy-memory[mysql]==2.0.4'
|
pip install 'hy-memory[mysql]==2.0.1'
|
||||||
pip install 'hy-memory[rerank]==2.0.4' # 本地多语言 cross-encoder rerank
|
pip install 'hy-memory[rerank]==2.0.1' # 本地多语言 cross-encoder rerank
|
||||||
```
|
```
|
||||||
|
|
||||||
默认 LLM 和 Embedding 均使用 OpenAI-compatible 接口。使用 OpenAI 时可只设置:
|
默认 LLM 和 Embedding 均使用 OpenAI-compatible 接口。使用 OpenAI 时可只设置:
|
||||||
@@ -210,8 +210,7 @@ result = client.search(
|
|||||||
```
|
```
|
||||||
|
|
||||||
`working` / `coding` 路径返回 `scene="productivity"`、`facts`、`experiences`,不使用
|
`working` / `coding` 路径返回 `scene="productivity"`、`facts`、`experiences`,不使用
|
||||||
chat 的三桶结构;`coding` 还会返回 `chain_block` 和 `chain_used` 字段(默认空;
|
chat 的三桶结构;`coding` 还会返回 `chain_block` 和 `chain_used`。
|
||||||
需 `MEMORY_CODING_CHAIN_BLOCK_ENABLED=true` 才注入 diff 块)。
|
|
||||||
|
|
||||||
#### `search_mode`
|
#### `search_mode`
|
||||||
|
|
||||||
|
|||||||
@@ -22,8 +22,7 @@ logger = logging.getLogger(__name__)
|
|||||||
|
|
||||||
# ================================================================
|
# ================================================================
|
||||||
# 数据结构
|
# 数据结构
|
||||||
# ================================================================
|
# ================================================================@dataclass
|
||||||
@dataclass
|
|
||||||
class SessionSummaryResult:
|
class SessionSummaryResult:
|
||||||
"""Session 摘要结果 (V2)"""
|
"""Session 摘要结果 (V2)"""
|
||||||
success: bool
|
success: bool
|
||||||
|
|||||||
@@ -39,7 +39,6 @@ import logging
|
|||||||
import hashlib
|
import hashlib
|
||||||
import json
|
import json
|
||||||
import re
|
import re
|
||||||
from datetime import datetime
|
|
||||||
from typing import Any, Dict, List, Optional
|
from typing import Any, Dict, List, Optional
|
||||||
import uuid
|
import uuid
|
||||||
|
|
||||||
@@ -253,17 +252,6 @@ def _series_episode_id(
|
|||||||
return str(uuid.uuid5(uuid.NAMESPACE_URL, "hy-memory:series-episode:" + raw))
|
return str(uuid.uuid5(uuid.NAMESPACE_URL, "hy-memory:series-episode:" + raw))
|
||||||
|
|
||||||
|
|
||||||
def _series_memory_at(times: List[int]) -> Optional[datetime]:
|
|
||||||
"""Earliest member timestamp as the same local wall clock it came from.
|
|
||||||
|
|
||||||
``_node_timestamp`` produces ``datetime.timestamp()`` values. Reading them
|
|
||||||
back with ``utcfromtimestamp`` turns them into a naive *UTC* datetime, which
|
|
||||||
the vector payload later re-interprets as local time and shifts by the
|
|
||||||
machine's UTC offset.
|
|
||||||
"""
|
|
||||||
return datetime.fromtimestamp(min(times)) if times else None
|
|
||||||
|
|
||||||
|
|
||||||
def _node_timestamp(node: Any) -> Optional[int]:
|
def _node_timestamp(node: Any) -> Optional[int]:
|
||||||
value = getattr(node, "memory_at", None) or getattr(node, "gmt_created", None)
|
value = getattr(node, "memory_at", None) or getattr(node, "gmt_created", None)
|
||||||
if value is None:
|
if value is None:
|
||||||
@@ -986,6 +974,8 @@ async def finalize_series_episode_containers(
|
|||||||
Stored text is only a title plus a structural intro. Exact L2 bodies remain in
|
Stored text is only a title plus a structural intro. Exact L2 bodies remain in
|
||||||
their canonical nodes and are referenced by ID from dated timeline segments.
|
their canonical nodes and are referenced by ID from dated timeline segments.
|
||||||
"""
|
"""
|
||||||
|
from datetime import datetime
|
||||||
|
|
||||||
from ..models.memory import MemoryLayer, MemoryNode, MemoryStatus, SourceType
|
from ..models.memory import MemoryLayer, MemoryNode, MemoryStatus, SourceType
|
||||||
from ..pipelines._retrieval.session_episode import (
|
from ..pipelines._retrieval.session_episode import (
|
||||||
add_parent_episode_membership,
|
add_parent_episode_membership,
|
||||||
@@ -1071,7 +1061,7 @@ async def finalize_series_episode_containers(
|
|||||||
segment["memory_at"] for segment in plan["timeline"]
|
segment["memory_at"] for segment in plan["timeline"]
|
||||||
if segment.get("memory_at") is not None
|
if segment.get("memory_at") is not None
|
||||||
]
|
]
|
||||||
memory_at = _series_memory_at(times)
|
memory_at = datetime.utcfromtimestamp(min(times)) if times else None
|
||||||
content = f"{plan['title']}. {plan['intro']}".strip()
|
content = f"{plan['title']}. {plan['intro']}".strip()
|
||||||
series = MemoryNode(
|
series = MemoryNode(
|
||||||
node_id=series_id, user_id=user_id,
|
node_id=series_id, user_id=user_id,
|
||||||
|
|||||||
+68
-142
@@ -30,7 +30,6 @@ import asyncio
|
|||||||
import json
|
import json
|
||||||
import logging
|
import logging
|
||||||
import os
|
import os
|
||||||
import threading
|
|
||||||
import time
|
import time
|
||||||
import uuid
|
import uuid
|
||||||
from datetime import datetime
|
from datetime import datetime
|
||||||
@@ -378,14 +377,6 @@ class _LoopThread:
|
|||||||
return await await_concurrent_future_guarded(future, self._loop)
|
return await await_concurrent_future_guarded(future, self._loop)
|
||||||
|
|
||||||
def stop(self):
|
def stop(self):
|
||||||
if self._loop.is_closed():
|
|
||||||
return
|
|
||||||
if threading.current_thread() is self._thread:
|
|
||||||
# Called from inside the loop: stopping and joining ourselves would
|
|
||||||
# deadlock. Just ask run_forever() to return.
|
|
||||||
self._loop.stop()
|
|
||||||
return
|
|
||||||
|
|
||||||
def _stop():
|
def _stop():
|
||||||
if self._lag_handle is not None:
|
if self._lag_handle is not None:
|
||||||
self._lag_handle.cancel()
|
self._lag_handle.cancel()
|
||||||
@@ -395,8 +386,6 @@ class _LoopThread:
|
|||||||
|
|
||||||
self._loop.call_soon_threadsafe(_stop)
|
self._loop.call_soon_threadsafe(_stop)
|
||||||
self._thread.join(timeout=5)
|
self._thread.join(timeout=5)
|
||||||
if not self._thread.is_alive() and not self._loop.is_closed():
|
|
||||||
self._loop.close()
|
|
||||||
|
|
||||||
|
|
||||||
class HyMemoryClient:
|
class HyMemoryClient:
|
||||||
@@ -1192,19 +1181,6 @@ class HyMemoryClient:
|
|||||||
logger.warning("[coding] chain observe failed: %s", e)
|
logger.warning("[coding] chain observe failed: %s", e)
|
||||||
return 0
|
return 0
|
||||||
|
|
||||||
def _chain_block_enabled(self) -> bool:
|
|
||||||
"""search 是否注入 coding 版本演进链块(diff / chain_block)。
|
|
||||||
|
|
||||||
公共开关:CodingConfig.chain_block_enabled /
|
|
||||||
MEMORY_CODING_CHAIN_BLOCK_ENABLED(默认 False,opt-in)。
|
|
||||||
无 _config 时按关闭处理。高级消融开关 HCM_BLOCK_OFF=1 仍在
|
|
||||||
chain_store.decorate 内生效(仅当本开关已开启时才有意义)。
|
|
||||||
"""
|
|
||||||
coding = getattr(getattr(self, "_config", None), "coding", None)
|
|
||||||
if coding is None:
|
|
||||||
return False
|
|
||||||
return bool(getattr(coding, "chain_block_enabled", False))
|
|
||||||
|
|
||||||
def _decorate_with_chain(self, facts, instruction: str = "", scope=None,
|
def _decorate_with_chain(self, facts, instruction: str = "", scope=None,
|
||||||
user_id=None, workspace_id=None):
|
user_id=None, workspace_id=None):
|
||||||
"""读取侧:给向量层召回的 facts 挂上版本演进链块。
|
"""读取侧:给向量层召回的 facts 挂上版本演进链块。
|
||||||
@@ -1217,14 +1193,11 @@ class HyMemoryClient:
|
|||||||
facts 顺序不重排 —— top-k 是向量层的事,这层只负责"每条 fact 后面跟什么"。
|
facts 顺序不重排 —— top-k 是向量层的事,这层只负责"每条 fact 后面跟什么"。
|
||||||
|
|
||||||
best-effort:失败返回 ("", []),退化成纯向量双层(= working 的行为)。
|
best-effort:失败返回 ("", []),退化成纯向量双层(= working 的行为)。
|
||||||
默认不注入(见 _chain_block_enabled);需显式开启 chain_block_enabled。
|
|
||||||
"""
|
"""
|
||||||
if not facts:
|
if not facts:
|
||||||
return "", []
|
return "", []
|
||||||
# 走类而不是 self:测试常把本方法绑到裸 SimpleNamespace 上,self 上未必有
|
# 走类而不是 self:_chain_owner 是 staticmethod,而这两个方法会被测试绑到裸
|
||||||
# 同名 bound method;_chain_owner 同理。
|
# SimpleNamespace 上调(同 _scene_variant 那处),self 上没有这个属性。
|
||||||
if not HyMemoryClient._chain_block_enabled(self):
|
|
||||||
return "", []
|
|
||||||
store = self._get_chain_store(
|
store = self._get_chain_store(
|
||||||
HyMemoryClient._chain_owner(user_id, workspace_id))
|
HyMemoryClient._chain_owner(user_id, workspace_id))
|
||||||
if store is None:
|
if store is None:
|
||||||
@@ -2179,33 +2152,14 @@ class HyMemoryClient:
|
|||||||
from .metrics import MetricsCollector
|
from .metrics import MetricsCollector
|
||||||
return await MetricsCollector.get().get_snapshot(minutes=minutes)
|
return await MetricsCollector.get().get_snapshot(minutes=minutes)
|
||||||
|
|
||||||
def _stop_owned_loops(self) -> None:
|
|
||||||
"""Stop every event-loop thread this client owns.
|
|
||||||
|
|
||||||
Solo clients create both a write loop and a search loop. Stopping only
|
|
||||||
the write loop left the search thread (and a self-wakeup timer every
|
|
||||||
50ms) alive for the process lifetime. Shared runtimes own their loops
|
|
||||||
and manage them through ``SharedRuntime.aclose()``.
|
|
||||||
"""
|
|
||||||
if not getattr(self, "_owns_runtime", False):
|
|
||||||
return
|
|
||||||
seen = set()
|
|
||||||
for loop_thread in (
|
|
||||||
getattr(self, "_loop_thread", None),
|
|
||||||
getattr(self, "_search_loop_thread", None),
|
|
||||||
):
|
|
||||||
if loop_thread is None or id(loop_thread) in seen:
|
|
||||||
continue
|
|
||||||
seen.add(id(loop_thread))
|
|
||||||
loop_thread.stop()
|
|
||||||
|
|
||||||
def close(self):
|
def close(self):
|
||||||
"""关闭客户端,释放所有资源"""
|
"""关闭客户端,释放所有资源"""
|
||||||
try:
|
if self._registry is not None:
|
||||||
if self._registry is not None:
|
self._loop_thread.run(self._close_async())
|
||||||
self._loop_thread.run(self._close_async())
|
# solo 模式:本 client 拥有 loop_thread,可以安全 stop
|
||||||
finally:
|
# shared 模式:loop_thread 由 runtime 管,留给 runtime.aclose() 关
|
||||||
self._stop_owned_loops()
|
if hasattr(self, '_loop_thread') and self._owns_runtime:
|
||||||
|
self._loop_thread.stop()
|
||||||
|
|
||||||
def history(self, memory_id: str) -> List[Dict[str, Any]]:
|
def history(self, memory_id: str) -> List[Dict[str, Any]]:
|
||||||
"""
|
"""
|
||||||
@@ -2908,81 +2862,6 @@ class HyMemoryClient:
|
|||||||
|
|
||||||
return resp
|
return resp
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
def _public_search_views(
|
|
||||||
result_memories: List[Dict[str, Any]],
|
|
||||||
*,
|
|
||||||
wanted_layers: Optional[set],
|
|
||||||
lite_mode: bool,
|
|
||||||
) -> Tuple[List[Dict[str, Any]], List[Dict[str, Any]]]:
|
|
||||||
"""Build the public search items and the private dedup view together.
|
|
||||||
|
|
||||||
The dedup marker must come from the same source element as its public
|
|
||||||
item. Building the dedup view with a second ``zip(result_memories,
|
|
||||||
all_mems)`` is wrong as soon as one result is dropped by the layer
|
|
||||||
filter: every later kind shifts by one, and a canonical_state can enter
|
|
||||||
fuzzy dedup and be shadowed.
|
|
||||||
"""
|
|
||||||
from .models.memory import MemoryLayer
|
|
||||||
|
|
||||||
all_mems: List[Dict[str, Any]] = []
|
|
||||||
dedup_mems: List[Dict[str, Any]] = []
|
|
||||||
for mem in result_memories:
|
|
||||||
if wanted_layers is not None and mem.get("layer", "") not in wanted_layers:
|
|
||||||
continue
|
|
||||||
if lite_mode and mem.get("layer", "") != MemoryLayer.L1_RAW.value:
|
|
||||||
continue
|
|
||||||
score = mem.get("score", 0.0)
|
|
||||||
item = {
|
|
||||||
"memory_id": mem.get("memory_id", ""),
|
|
||||||
"content": mem.get("content", ""),
|
|
||||||
"score": round(score, 4),
|
|
||||||
"layer": mem.get("layer", ""),
|
|
||||||
# 多 user 检索时调用方需据此追溯归属(R1)
|
|
||||||
"user_id": mem.get("user_id", "") or "",
|
|
||||||
"agent_id": mem.get("agent_id", "") or "",
|
|
||||||
"source": mem.get("source", "default") or "default",
|
|
||||||
"retrieval_source": mem.get("retrieval_source", "") or "",
|
|
||||||
"owner": mem.get("owner"),
|
|
||||||
"speculate": mem.get("speculate"),
|
|
||||||
# 不透出 source_raw_memory_id:它指向 L1_RAW 节点,而 raw 对外
|
|
||||||
# 读路径一律不可见,该 id 到手也查不到东西(按 ID 取返回"不存在"),
|
|
||||||
# 留着只会让调用方以为能据此追溯原文。
|
|
||||||
# 节点上仍保留该字段,内部溯源与 clone 重映射不受影响。
|
|
||||||
"tags": mem.get("tags") or [],
|
|
||||||
"memory_at": mem.get("memory_at"),
|
|
||||||
"gmt_created": mem.get("gmt_created"),
|
|
||||||
# 调用方自定义 meta(透传负载)。由 reader 从 custom["meta"] 取出,
|
|
||||||
# 未实现该字段的 reader 下为空列表。
|
|
||||||
"meta": mem.get("meta") or [],
|
|
||||||
"_schema_channel": bool(mem.get("_schema_channel")),
|
|
||||||
}
|
|
||||||
if mem.get("schema_type"):
|
|
||||||
item["schema_type"] = mem["schema_type"]
|
|
||||||
if mem.get("workspace_id"):
|
|
||||||
item["workspace_id"] = mem["workspace_id"]
|
|
||||||
if mem.get("branch"):
|
|
||||||
item["branch"] = mem["branch"]
|
|
||||||
if mem.get("evolution_chain"):
|
|
||||||
item["evolution_chain"] = mem["evolution_chain"]
|
|
||||||
if mem.get("capsule_memories") is not None:
|
|
||||||
item["capsule_memories"] = mem["capsule_memories"]
|
|
||||||
if mem.get("episode_id"):
|
|
||||||
item["episode_id"] = mem["episode_id"]
|
|
||||||
if mem.get("member_ids") is not None:
|
|
||||||
item["member_ids"] = mem["member_ids"]
|
|
||||||
if mem.get("timeline") is not None:
|
|
||||||
item["timeline"] = mem["timeline"]
|
|
||||||
if mem.get("rerank_score") is not None:
|
|
||||||
item["rerank_score"] = mem["rerank_score"]
|
|
||||||
all_mems.append(item)
|
|
||||||
|
|
||||||
dedup_item = dict(item)
|
|
||||||
dedup_item["_memory_kind"] = mem.get("_memory_kind", "")
|
|
||||||
dedup_mems.append(dedup_item)
|
|
||||||
|
|
||||||
return all_mems, dedup_mems
|
|
||||||
|
|
||||||
def _spawn_search_dedup(
|
def _spawn_search_dedup(
|
||||||
self,
|
self,
|
||||||
all_mems: List[Dict[str, Any]],
|
all_mems: List[Dict[str, Any]],
|
||||||
@@ -3618,11 +3497,62 @@ class HyMemoryClient:
|
|||||||
if effective_search_layers else None
|
if effective_search_layers else None
|
||||||
)
|
)
|
||||||
|
|
||||||
_all_mems, _dedup_mems = self._public_search_views(
|
_all_mems = []
|
||||||
result.memories,
|
for mem in result.memories:
|
||||||
wanted_layers=_wanted_layers,
|
if _wanted_layers is not None and mem.get("layer", "") not in _wanted_layers:
|
||||||
lite_mode=lite_mode,
|
continue
|
||||||
)
|
if lite_mode and mem.get("layer", "") != MemoryLayer.L1_RAW.value:
|
||||||
|
continue
|
||||||
|
score = mem.get("score", 0.0)
|
||||||
|
item = {
|
||||||
|
"memory_id": mem.get("memory_id", ""),
|
||||||
|
"content": mem.get("content", ""),
|
||||||
|
"score": round(score, 4),
|
||||||
|
"layer": mem.get("layer", ""),
|
||||||
|
# 多 user 检索时调用方需据此追溯归属(R1)
|
||||||
|
"user_id": mem.get("user_id", "") or "",
|
||||||
|
"agent_id": mem.get("agent_id", "") or "",
|
||||||
|
"source": mem.get("source", "default") or "default",
|
||||||
|
"retrieval_source": mem.get("retrieval_source", "") or "",
|
||||||
|
"owner": mem.get("owner"),
|
||||||
|
"speculate": mem.get("speculate"),
|
||||||
|
# 不透出 source_raw_memory_id:它指向 L1_RAW 节点,而 raw 对外
|
||||||
|
# 读路径一律不可见,该 id 到手也查不到东西(按 ID 取返回"不存在"),
|
||||||
|
# 留着只会让调用方以为能据此追溯原文。
|
||||||
|
# 节点上仍保留该字段,内部溯源与 clone 重映射不受影响。
|
||||||
|
"tags": mem.get("tags") or [],
|
||||||
|
"memory_at": mem.get("memory_at"),
|
||||||
|
"gmt_created": mem.get("gmt_created"),
|
||||||
|
# 调用方自定义 meta(透传负载)。由 reader 从 custom["meta"] 取出,
|
||||||
|
# 未实现该字段的 reader 下为空列表。
|
||||||
|
"meta": mem.get("meta") or [],
|
||||||
|
"_schema_channel": bool(mem.get("_schema_channel")),
|
||||||
|
}
|
||||||
|
if mem.get("schema_type"):
|
||||||
|
item["schema_type"] = mem["schema_type"]
|
||||||
|
if mem.get("workspace_id"):
|
||||||
|
item["workspace_id"] = mem["workspace_id"]
|
||||||
|
if mem.get("branch"):
|
||||||
|
item["branch"] = mem["branch"]
|
||||||
|
if mem.get("evolution_chain"):
|
||||||
|
item["evolution_chain"] = mem["evolution_chain"]
|
||||||
|
if mem.get("capsule_memories") is not None:
|
||||||
|
item["capsule_memories"] = mem["capsule_memories"]
|
||||||
|
if mem.get("episode_id"):
|
||||||
|
item["episode_id"] = mem["episode_id"]
|
||||||
|
if mem.get("member_ids") is not None:
|
||||||
|
item["member_ids"] = mem["member_ids"]
|
||||||
|
if mem.get("timeline") is not None:
|
||||||
|
item["timeline"] = mem["timeline"]
|
||||||
|
if mem.get("rerank_score") is not None:
|
||||||
|
item["rerank_score"] = mem["rerank_score"]
|
||||||
|
_all_mems.append(item)
|
||||||
|
|
||||||
|
_dedup_mems = []
|
||||||
|
for source, public in zip(result.memories, _all_mems):
|
||||||
|
copy = dict(public)
|
||||||
|
copy["_memory_kind"] = source.get("_memory_kind", "")
|
||||||
|
_dedup_mems.append(copy)
|
||||||
|
|
||||||
# search 链路去重:仅对 L2_FACT + L4_IDENTITY(重复重灾区)判重删库。
|
# search 链路去重:仅对 L2_FACT + L4_IDENTITY(重复重灾区)判重删库。
|
||||||
# L0 及其他层不参与。fire-and-forget,不阻塞本次返回(清理面向后续查询)。
|
# L0 及其他层不参与。fire-and-forget,不阻塞本次返回(清理面向后续查询)。
|
||||||
@@ -6173,10 +6103,9 @@ class HyMemoryClient:
|
|||||||
effective_agent_ids = agent_ids if agent_ids is not None else []
|
effective_agent_ids = agent_ids if agent_ids is not None else []
|
||||||
|
|
||||||
if not effective_agent_ids:
|
if not effective_agent_ids:
|
||||||
# 查所有 agent 的历史:isolation_key 前缀匹配。
|
# 查所有 agent 的历史:用 "user_id:" 前缀匹配
|
||||||
# 不能退回 "u:default" —— add() 的默认 agent 是 "default_agent",
|
isolation_key = self._build_isolation_key(uid, "default")
|
||||||
# 默认用法下会查空。
|
return await self._history_store.get_recent(isolation_key, limit=limit)
|
||||||
return await self._history_store.get_recent_for_user(uid, limit=limit)
|
|
||||||
else:
|
else:
|
||||||
# 查指定 agent 的历史
|
# 查指定 agent 的历史
|
||||||
all_results = []
|
all_results = []
|
||||||
@@ -6202,7 +6131,4 @@ class HyMemoryClient:
|
|||||||
return self
|
return self
|
||||||
|
|
||||||
async def __aexit__(self, *args):
|
async def __aexit__(self, *args):
|
||||||
try:
|
await self._close_async()
|
||||||
await self._close_async()
|
|
||||||
finally:
|
|
||||||
self._stop_owned_loops()
|
|
||||||
@@ -647,9 +647,7 @@ class HyCodeMem:
|
|||||||
top-k,但它们的 diff 对当前工作区不可能有用,因此挂链前必须做同仓过滤。
|
top-k,但它们的 diff 对当前工作区不可能有用,因此挂链前必须做同仓过滤。
|
||||||
"""
|
"""
|
||||||
facts = facts or []
|
facts = facts or []
|
||||||
# 高级消融开关: HCM_BLOCK_OFF=1 → 完全不出块, 也不从 bullet 删任何 fact。
|
# 开关: HCM_BLOCK_OFF=1 → 完全不出块, 也不从 bullet 删任何 fact。
|
||||||
# 公共 opt-in 见 CodingConfig.chain_block_enabled /
|
|
||||||
# MEMORY_CODING_CHAIN_BLOCK_ENABLED(默认关;客户端在调用 decorate 前拦截)。
|
|
||||||
if os.environ.get("HCM_BLOCK_OFF") == "1":
|
if os.environ.get("HCM_BLOCK_OFF") == "1":
|
||||||
return "", []
|
return "", []
|
||||||
picked, used = [], []
|
picked, used = [], []
|
||||||
|
|||||||
@@ -1136,9 +1136,6 @@ class CodingConfig:
|
|||||||
# LLM 失败或没有抽出 fact 时,是否把对话原文存成 working fact。
|
# LLM 失败或没有抽出 fact 时,是否把对话原文存成 working fact。
|
||||||
# 默认关闭,避免 assistant/tool 轨迹以 statement 形式直接暴露给消费方。
|
# 默认关闭,避免 assistant/tool 轨迹以 statement 形式直接暴露给消费方。
|
||||||
verbatim_fallback_enabled: bool = None
|
verbatim_fallback_enabled: bool = None
|
||||||
# search(scene="coding") 是否注入版本演进链块(diff / chain_block)。
|
|
||||||
# 默认关闭(opt-in):中重度 coding 用户可省下大量 prompt token;写入侧仍可 observe。
|
|
||||||
chain_block_enabled: bool = None
|
|
||||||
|
|
||||||
def __post_init__(self):
|
def __post_init__(self):
|
||||||
if self.enable is None:
|
if self.enable is None:
|
||||||
@@ -1156,10 +1153,6 @@ class CodingConfig:
|
|||||||
self.verbatim_fallback_enabled = _get_env_bool(
|
self.verbatim_fallback_enabled = _get_env_bool(
|
||||||
"MEMORY_CODING_VERBATIM_FALLBACK_ENABLED", False
|
"MEMORY_CODING_VERBATIM_FALLBACK_ENABLED", False
|
||||||
)
|
)
|
||||||
if self.chain_block_enabled is None:
|
|
||||||
self.chain_block_enabled = _get_env_bool(
|
|
||||||
"MEMORY_CODING_CHAIN_BLOCK_ENABLED", False
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
@dataclass
|
||||||
|
|||||||
@@ -464,67 +464,24 @@ class MysqlCache(CacheBase):
|
|||||||
# ================================================================
|
# ================================================================
|
||||||
|
|
||||||
async def store_metrics_minute(self, minute_ts: str, data: dict) -> None:
|
async def store_metrics_minute(self, minute_ts: str, data: dict) -> None:
|
||||||
import aiomysql
|
|
||||||
|
|
||||||
try:
|
try:
|
||||||
now = datetime.now().isoformat()
|
now = datetime.now().isoformat()
|
||||||
# Read-merge-write must be one transaction: two instances flushing
|
# 尝试 UPSERT
|
||||||
# the same minute used to overwrite each other's counters, and a
|
existing = await self._fetchone(
|
||||||
# concurrent insert could lose the whole batch to a duplicate key.
|
"SELECT data FROM system_metrics WHERE minute_ts = %s", (minute_ts,)
|
||||||
for attempt in range(2):
|
)
|
||||||
try:
|
if existing:
|
||||||
async with self._pool.acquire() as conn:
|
old_data = json.loads(existing["data"])
|
||||||
await conn.begin()
|
merged = self._merge_metric_buckets(old_data, data)
|
||||||
try:
|
await self._execute(
|
||||||
async with conn.cursor(aiomysql.DictCursor) as cur:
|
"UPDATE system_metrics SET data = %s, created_at = %s WHERE minute_ts = %s",
|
||||||
await cur.execute(
|
(json.dumps(merged, default=str), now, minute_ts),
|
||||||
"SELECT data FROM system_metrics "
|
)
|
||||||
"WHERE minute_ts = %s FOR UPDATE",
|
else:
|
||||||
(minute_ts,),
|
await self._execute(
|
||||||
)
|
"INSERT INTO system_metrics (minute_ts, data, created_at) VALUES (%s, %s, %s)",
|
||||||
row = await cur.fetchone()
|
(minute_ts, json.dumps(data, default=str), now),
|
||||||
if row is not None:
|
)
|
||||||
old_raw = (
|
|
||||||
row["data"]
|
|
||||||
if isinstance(row, dict)
|
|
||||||
else row[0]
|
|
||||||
)
|
|
||||||
merged = self._merge_metric_buckets(
|
|
||||||
json.loads(old_raw), data,
|
|
||||||
)
|
|
||||||
await cur.execute(
|
|
||||||
"UPDATE system_metrics SET data = %s, "
|
|
||||||
"created_at = %s WHERE minute_ts = %s",
|
|
||||||
(
|
|
||||||
json.dumps(merged, default=str),
|
|
||||||
now,
|
|
||||||
minute_ts,
|
|
||||||
),
|
|
||||||
)
|
|
||||||
else:
|
|
||||||
await cur.execute(
|
|
||||||
"INSERT INTO system_metrics "
|
|
||||||
"(minute_ts, data, created_at) "
|
|
||||||
"VALUES (%s, %s, %s)",
|
|
||||||
(
|
|
||||||
minute_ts,
|
|
||||||
json.dumps(data, default=str),
|
|
||||||
now,
|
|
||||||
),
|
|
||||||
)
|
|
||||||
await conn.commit()
|
|
||||||
return
|
|
||||||
except Exception:
|
|
||||||
await conn.rollback()
|
|
||||||
raise
|
|
||||||
except (aiomysql.IntegrityError, aiomysql.OperationalError) as error:
|
|
||||||
code = error.args[0] if error.args else None
|
|
||||||
deadlock = code in (1205, 1213)
|
|
||||||
if attempt or (isinstance(error, aiomysql.OperationalError) and not deadlock):
|
|
||||||
raise
|
|
||||||
# A competing writer inserted the row, or InnoDB chose us
|
|
||||||
# as the deadlock victim. Retry so this batch merges.
|
|
||||||
continue
|
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
logger.warning(f"store_metrics_minute failed: {e}")
|
logger.warning(f"store_metrics_minute failed: {e}")
|
||||||
|
|
||||||
|
|||||||
@@ -66,11 +66,6 @@ def _now_iso() -> str:
|
|||||||
return datetime.now(timezone.utc).isoformat()
|
return datetime.now(timezone.utc).isoformat()
|
||||||
|
|
||||||
|
|
||||||
def _escape_like(value: str) -> str:
|
|
||||||
"""Escape LIKE wildcards so a user id is matched literally."""
|
|
||||||
return value.replace("\\", "\\\\").replace("%", "\\%").replace("_", "\\_")
|
|
||||||
|
|
||||||
|
|
||||||
class HistoryStore:
|
class HistoryStore:
|
||||||
"""
|
"""
|
||||||
SQLite 历史存储 - 记录所有内存操作的审计日志。
|
SQLite 历史存储 - 记录所有内存操作的审计日志。
|
||||||
@@ -266,34 +261,6 @@ class HistoryStore:
|
|||||||
)
|
)
|
||||||
return [dict(row) for row in cursor.fetchall()]
|
return [dict(row) for row in cursor.fetchall()]
|
||||||
|
|
||||||
async def get_recent_for_user(
|
|
||||||
self,
|
|
||||||
user_id: str,
|
|
||||||
limit: int = 50,
|
|
||||||
) -> List[Dict[str, Any]]:
|
|
||||||
"""获取某 user 下所有 agent 的最近操作。
|
|
||||||
|
|
||||||
isolation_key 形如 ``"{user_id}:{agent_id}"``。user_id 里的 LIKE
|
|
||||||
通配符会被转义,保证只匹配这个 user,不会漏到 ``u1`` 之外。
|
|
||||||
"""
|
|
||||||
prefix = _escape_like(f"{user_id}:") + "%"
|
|
||||||
return await _run_in_sqlite_pool(
|
|
||||||
self._get_recent_prefix_sync, prefix, limit,
|
|
||||||
)
|
|
||||||
|
|
||||||
def _get_recent_prefix_sync(
|
|
||||||
self, pattern: str, limit: int,
|
|
||||||
) -> List[Dict[str, Any]]:
|
|
||||||
with self._conn_lock:
|
|
||||||
if self._conn is None:
|
|
||||||
return []
|
|
||||||
cursor = self._conn.execute(
|
|
||||||
"SELECT * FROM memory_history WHERE isolation_key LIKE ? ESCAPE '\\' "
|
|
||||||
"ORDER BY created_at DESC LIMIT ?",
|
|
||||||
(pattern, limit),
|
|
||||||
)
|
|
||||||
return [dict(row) for row in cursor.fetchall()]
|
|
||||||
|
|
||||||
async def count(self, isolation_key: str = "") -> int:
|
async def count(self, isolation_key: str = "") -> int:
|
||||||
"""
|
"""
|
||||||
统计历史记录数量。
|
统计历史记录数量。
|
||||||
|
|||||||
@@ -462,39 +462,14 @@ class FaissVectorStore(VectorStoreBase):
|
|||||||
return None
|
return None
|
||||||
|
|
||||||
async def get_embeddings(self, node_ids: List[str]) -> Dict[str, List[float]]:
|
async def get_embeddings(self, node_ids: List[str]) -> Dict[str, List[float]]:
|
||||||
"""批量取向量,供去重 / episode 摘要计算 cosine。
|
"""批量取向量(in-memory _data 直接读 embedding),供去重算 cosine。"""
|
||||||
|
out: Dict[str, List[float]] = {}
|
||||||
``_data`` 只保存 payload 与 FAISS 内部下标;普通 upsert 不额外存一份
|
for nid in node_ids:
|
||||||
embedding,向量本体在 FAISS index 里。这里按当前有效下标 reconstruct,
|
point_id = self._node_id_to_point_id(nid)
|
||||||
并用 ``_idx_to_id`` 校验,避免读到已被替换的旧下标。
|
entry = self._data.get(point_id)
|
||||||
"""
|
if entry and entry.get("embedding"):
|
||||||
|
out[nid] = list(entry["embedding"])
|
||||||
def _get() -> Dict[str, List[float]]:
|
return out
|
||||||
out: Dict[str, List[float]] = {}
|
|
||||||
with self._lock:
|
|
||||||
for nid in node_ids:
|
|
||||||
point_id = self._node_id_to_point_id(nid)
|
|
||||||
entry = self._data.get(point_id)
|
|
||||||
if entry is None:
|
|
||||||
continue
|
|
||||||
embedding = entry.get("embedding")
|
|
||||||
if embedding:
|
|
||||||
out[nid] = list(embedding)
|
|
||||||
continue
|
|
||||||
idx = entry.get("idx")
|
|
||||||
if (
|
|
||||||
idx is None
|
|
||||||
or self._index is None
|
|
||||||
or self._idx_to_id.get(int(idx)) != point_id
|
|
||||||
):
|
|
||||||
continue
|
|
||||||
try:
|
|
||||||
out[nid] = self._index.reconstruct(int(idx)).tolist()
|
|
||||||
except Exception:
|
|
||||||
continue
|
|
||||||
return out
|
|
||||||
|
|
||||||
return await _run_in_vdb_pool(_get)
|
|
||||||
|
|
||||||
# ================================================================
|
# ================================================================
|
||||||
# 删除
|
# 删除
|
||||||
|
|||||||
@@ -968,7 +968,7 @@ class MemoryWriter(WritePipeline):
|
|||||||
async def _run():
|
async def _run():
|
||||||
try:
|
try:
|
||||||
nid = await summarize_closed_episode(
|
nid = await summarize_closed_episode(
|
||||||
vector_store=await self._get_vector_store(),
|
vector_store=self.vector_store,
|
||||||
embed_service=self.embed_service,
|
embed_service=self.embed_service,
|
||||||
llm_provider=self.llm_provider,
|
llm_provider=self.llm_provider,
|
||||||
episode_id=episode_id,
|
episode_id=episode_id,
|
||||||
|
|||||||
+6
-19
@@ -29,7 +29,7 @@ import logging
|
|||||||
import os
|
import os
|
||||||
import re
|
import re
|
||||||
import signal
|
import signal
|
||||||
import threading
|
import sys
|
||||||
from datetime import datetime
|
from datetime import datetime
|
||||||
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
||||||
from typing import Any, Dict, Optional
|
from typing import Any, Dict, Optional
|
||||||
@@ -495,16 +495,6 @@ class MemoryHTTPHandler(BaseHTTPRequestHandler):
|
|||||||
_json_response(self, 200, result)
|
_json_response(self, 200, result)
|
||||||
|
|
||||||
|
|
||||||
def _request_shutdown(server: ThreadingHTTPServer) -> None:
|
|
||||||
"""Ask ``serve_forever`` to stop without blocking the calling thread.
|
|
||||||
|
|
||||||
``BaseServer.shutdown()`` waits for the ``serve_forever`` loop to exit. A
|
|
||||||
signal handler runs on the very thread executing that loop, so calling it
|
|
||||||
there deadlocks forever. Hand the call to a helper thread instead.
|
|
||||||
"""
|
|
||||||
threading.Thread(target=server.shutdown, daemon=True).start()
|
|
||||||
|
|
||||||
|
|
||||||
def run_server(port: int = 19527, host: str = "127.0.0.1"):
|
def run_server(port: int = 19527, host: str = "127.0.0.1"):
|
||||||
"""Start the HTTP server (blocking)."""
|
"""Start the HTTP server (blocking)."""
|
||||||
# Configure logging — basicConfig for root; hy_memory level set after client init
|
# Configure logging — basicConfig for root; hy_memory level set after client init
|
||||||
@@ -533,12 +523,13 @@ def run_server(port: int = 19527, host: str = "127.0.0.1"):
|
|||||||
logger.info(f"[server] Listening on http://{host}:{port}")
|
logger.info(f"[server] Listening on http://{host}:{port}")
|
||||||
logger.info(f"[server] Health check: http://{host}:{port}/healthz")
|
logger.info(f"[server] Health check: http://{host}:{port}/healthz")
|
||||||
|
|
||||||
# Graceful shutdown. The signal handler only asks a helper thread to stop
|
# Graceful shutdown
|
||||||
# the loop; cleanup runs after serve_forever() returns so it cannot race
|
|
||||||
# the request threads or deadlock the serving thread.
|
|
||||||
def _shutdown(signum, frame):
|
def _shutdown(signum, frame):
|
||||||
logger.info("[server] Shutting down...")
|
logger.info("[server] Shutting down...")
|
||||||
_request_shutdown(server)
|
server.shutdown()
|
||||||
|
if _client is not None:
|
||||||
|
_client.close()
|
||||||
|
sys.exit(0)
|
||||||
|
|
||||||
signal.signal(signal.SIGINT, _shutdown)
|
signal.signal(signal.SIGINT, _shutdown)
|
||||||
signal.signal(signal.SIGTERM, _shutdown)
|
signal.signal(signal.SIGTERM, _shutdown)
|
||||||
@@ -547,10 +538,6 @@ def run_server(port: int = 19527, host: str = "127.0.0.1"):
|
|||||||
server.serve_forever()
|
server.serve_forever()
|
||||||
except KeyboardInterrupt:
|
except KeyboardInterrupt:
|
||||||
_shutdown(None, None)
|
_shutdown(None, None)
|
||||||
finally:
|
|
||||||
server.server_close()
|
|
||||||
if _client is not None:
|
|
||||||
_client.close()
|
|
||||||
|
|
||||||
|
|
||||||
def main():
|
def main():
|
||||||
|
|||||||
+4
-4
@@ -18,13 +18,13 @@ If you run into a problem, please open an Issue with the version, platform,repro
|
|||||||
|
|
||||||
## 发行与平台合同
|
## 发行与平台合同
|
||||||
|
|
||||||
公共 Python SDK 为 `hy-memory==2.0.4`。MemHub 源码版本为 `1.0.4`
|
公共 Python SDK 为 `hy-memory==2.0.3`。MemHub 源码版本为 `1.0.3`
|
||||||
(build 13),GitHub 仓库为
|
(build 7),GitHub 仓库为
|
||||||
[`Tencent-Hunyuan/Hy-Memory`](https://github.com/Tencent-Hunyuan/Hy-Memory),npm 包名为
|
[`Tencent-Hunyuan/Hy-Memory`](https://github.com/Tencent-Hunyuan/Hy-Memory),npm 包名为
|
||||||
`memhub-cli`。当前代码架构见
|
`memhub-cli`。当前代码架构见
|
||||||
[`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md)。
|
[`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md)。
|
||||||
|
|
||||||
`v1.0.4` 的发行目标是 Linux x64/arm64 Runtime 与 macOS arm64/x64 App 和 Runtime。
|
`v1.0.3` 的发行目标是 Linux x64/arm64 Runtime 与 macOS arm64/x64 App 和 Runtime。
|
||||||
Linux 制品由 GitHub Actions 在原生架构 runner 上构建并完成生命周期验收;macOS
|
Linux 制品由 GitHub Actions 在原生架构 runner 上构建并完成生命周期验收;macOS
|
||||||
制品由发行流水线完成 Developer ID 签名、公证和验收后手动上传。Windows 不支持。
|
制品由发行流水线完成 Developer ID 签名、公证和验收后手动上传。Windows 不支持。
|
||||||
只有同时出现在 GitHub Release 与 `release-manifest.json` 中、且通过公开校验的制品才属于
|
只有同时出现在 GitHub Release 与 `release-manifest.json` 中、且通过公开校验的制品才属于
|
||||||
@@ -113,7 +113,7 @@ MemHub 的 SQLite 数据默认位于 `~/.one-memory`,不在 npm 包或 Runtime
|
|||||||
|
|
||||||
## Hy Memory
|
## Hy Memory
|
||||||
|
|
||||||
当前开发依赖固定为公开 PyPI 包 `hy-memory==2.0.4`。SDK 配置、权限边界及 search
|
当前开发依赖固定为公开 PyPI 包 `hy-memory==2.0.3`。SDK 配置、权限边界及 search
|
||||||
返回格式见 [`MEMORY.md`](MEMORY.md)。HY search 返回的 `content` 已包含 SDK 统一生成的
|
返回格式见 [`MEMORY.md`](MEMORY.md)。HY search 返回的 `content` 已包含 SDK 统一生成的
|
||||||
逻辑时间和演化上下文;调用方不应再次转换时间戳或重复拼接 `When`。
|
逻辑时间和演化上下文;调用方不应再次转换时间戳或重复拼接 `When`。
|
||||||
|
|
||||||
|
|||||||
@@ -76,7 +76,8 @@ def _memory(db: sqlite3.Connection, reference: CallReference) -> SourceRecordRef
|
|||||||
if row["status"] == "deleted" or row["deleted_at"] is not None:
|
if row["status"] == "deleted" or row["deleted_at"] is not None:
|
||||||
return _hidden(reference)
|
return _hidden(reference)
|
||||||
|
|
||||||
# Same local revocation/ownership predicate as SourceReadAccess.memories.
|
# Same local revocation/ownership predicate as SourceReadAccess.memories,
|
||||||
|
# evaluated BEFORE freshness (which can read support text), in this snapshot.
|
||||||
# Its public methods open their own connections, so cannot be used here.
|
# Its public methods open their own connections, so cannot be used here.
|
||||||
access = db.execute(
|
access = db.execute(
|
||||||
"SELECT EXISTS (SELECT 1 FROM governed_memory_evidence "
|
"SELECT EXISTS (SELECT 1 FROM governed_memory_evidence "
|
||||||
@@ -120,8 +121,8 @@ def resolve_call_references(database_path: Path, event: CallEvent) -> list[Sourc
|
|||||||
|
|
||||||
Local-owner only: event host attribution is NOT authorization. Explicit local
|
Local-owner only: event host attribution is NOT authorization. Explicit local
|
||||||
source revocations deny access; absent grants retain legacy local semantics.
|
source revocations deny access; absent grants retain legacy local semantics.
|
||||||
A memory needs at least one accessible active citation. Evidence freshness
|
A memory needs at least one accessible active citation. No proof preserves
|
||||||
proofs are retired and do not affect availability.
|
shared freshness compatibility, not a claim of verified historical freshness.
|
||||||
|
|
||||||
Any read/parse/budget failure raises a fixed whole-request error, never a
|
Any read/parse/budget failure raises a fixed whole-request error, never a
|
||||||
partial result or per-item deletion. The route owner must map this separately
|
partial result or per-item deletion. The route owner must map this separately
|
||||||
|
|||||||
@@ -34,11 +34,10 @@ class ScanIssueCode(StrEnum):
|
|||||||
SOURCE_BUSY = "source_busy"
|
SOURCE_BUSY = "source_busy"
|
||||||
UNSUPPORTED_SCHEMA = "unsupported_schema"
|
UNSUPPORTED_SCHEMA = "unsupported_schema"
|
||||||
BUDGET_EXCEEDED = "budget_exceeded"
|
BUDGET_EXCEEDED = "budget_exceeded"
|
||||||
#: The file is bigger than a per-file ceiling: either the import's own
|
#: The file is bigger than the import's own per-file ceiling. Skipped with
|
||||||
#: ceiling or the access gate's per-root `max_file_bytes`. Both are skipped
|
#: a reason rather than read, because one oversized history file can hold
|
||||||
#: with this reason rather than read, because one oversized history file
|
#: the import open long enough to take the process with it. Distinct from
|
||||||
#: can hold the import open long enough to take the process with it, and
|
#: the access layer's `FILE_TOO_LARGE`, which is a per-root limit.
|
||||||
#: the user-facing cause is the same size limit.
|
|
||||||
FILE_TOO_LARGE = "file_too_large"
|
FILE_TOO_LARGE = "file_too_large"
|
||||||
HISTORY_WINDOW_EXCLUDED = "history_window_excluded"
|
HISTORY_WINDOW_EXCLUDED = "history_window_excluded"
|
||||||
|
|
||||||
|
|||||||
@@ -1120,11 +1120,6 @@ def discovery_failure_code(
|
|||||||
) -> ScanIssueCode:
|
) -> ScanIssueCode:
|
||||||
if failure.reason_code is SourceAccessErrorCode.BUDGET_EXCEEDED:
|
if failure.reason_code is SourceAccessErrorCode.BUDGET_EXCEEDED:
|
||||||
return ScanIssueCode.BUDGET_EXCEEDED
|
return ScanIssueCode.BUDGET_EXCEEDED
|
||||||
if failure.reason_code is SourceAccessErrorCode.FILE_TOO_LARGE:
|
|
||||||
# The file is readable and its format is fine; only its size is over
|
|
||||||
# the access gate. Reporting that as `unsupported_schema` sends the
|
|
||||||
# user looking for a parser problem that does not exist.
|
|
||||||
return ScanIssueCode.FILE_TOO_LARGE
|
|
||||||
return ScanIssueCode.UNSUPPORTED_SCHEMA
|
return ScanIssueCode.UNSUPPORTED_SCHEMA
|
||||||
|
|
||||||
|
|
||||||
@@ -1132,8 +1127,6 @@ def _scan_failure_code(code: SourceAccessErrorCode) -> ScanIssueCode:
|
|||||||
"""Map a per-file open failure onto a machine code, never onto free text."""
|
"""Map a per-file open failure onto a machine code, never onto free text."""
|
||||||
if code is SourceAccessErrorCode.BUDGET_EXCEEDED:
|
if code is SourceAccessErrorCode.BUDGET_EXCEEDED:
|
||||||
return ScanIssueCode.BUDGET_EXCEEDED
|
return ScanIssueCode.BUDGET_EXCEEDED
|
||||||
if code is SourceAccessErrorCode.FILE_TOO_LARGE:
|
|
||||||
return ScanIssueCode.FILE_TOO_LARGE
|
|
||||||
if code is SourceAccessErrorCode.SYMLINK_NOT_ALLOWED:
|
if code is SourceAccessErrorCode.SYMLINK_NOT_ALLOWED:
|
||||||
return ScanIssueCode.UNSUPPORTED_SCHEMA
|
return ScanIssueCode.UNSUPPORTED_SCHEMA
|
||||||
if code in {
|
if code in {
|
||||||
|
|||||||
@@ -1,25 +1,36 @@
|
|||||||
"""Memory evidence freshness checks are retired.
|
"""Read-only, exact-revision support checks inside the caller's SQLite snapshot.
|
||||||
|
|
||||||
Read paths used to re-validate ``memory_task_outputs.support_refs`` against
|
No proof is legacy/unknown, not stale. One complete fresh proof suffices; all
|
||||||
live ``evidence`` rows via ``is_memory_evidence_current``. That design hid
|
complete proofs stale means false. Malformed records raise. An exceeded budget
|
||||||
memories when evidence changed, but at high cost (library 503s, per-row SQL,
|
means the memory is too large to verify, not that it is stale: it reads as
|
||||||
segment-key breakage). Call sites still import this module; registration is
|
current, the same as having no proof, and is reported once in a while as a
|
||||||
now a no-op that always admits the row.
|
content-free diagnostic. One such memory used to fail every COUNT over the
|
||||||
|
library, so the homepage returned 503 for as long as it existed (D76).
|
||||||
Proof rows may still be written by the ingest path; they are ignored on read.
|
Set ONE_MEMORY_FRESHNESS_BUDGET_RAISES=1 to restore the old raising behavior.
|
||||||
See ``新鲜度_todo.md`` for the remaining teardown steps.
|
The caller owns BEGIN/deadline/progress-handler and error-to-unavailable mapping.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
from __future__ import annotations
|
import json
|
||||||
|
import logging
|
||||||
|
import os
|
||||||
import sqlite3
|
import sqlite3
|
||||||
|
import threading
|
||||||
|
import time
|
||||||
from collections.abc import Iterator, Sequence
|
from collections.abc import Iterator, Sequence
|
||||||
from contextlib import contextmanager
|
from contextlib import contextmanager
|
||||||
|
|
||||||
from one_memory.database_access import register_function
|
from one_memory.database_access import register_function
|
||||||
|
from one_memory.diagnostic_log import diagnostic_event
|
||||||
|
from one_memory.imports.models import EvidenceRecord
|
||||||
|
from one_memory.memory_tasks import MemoryTaskOutput, matches_evidence_ref
|
||||||
|
|
||||||
#: Historical budget error codes. Retained so diagnostics/env docs do not break
|
_PROOFS = 64
|
||||||
#: mid-teardown; nothing raises them on the read path anymore.
|
_REFERENCES = 128
|
||||||
|
_BODY_BYTES = 65_536
|
||||||
|
_EVIDENCE_BYTES = 8 * 1024 * 1024
|
||||||
|
_SCOPE_STEPS = 64
|
||||||
|
|
||||||
|
#: Budget refusals: the check could not finish, the data is not malformed.
|
||||||
BUDGET_ERRORS = frozenset(
|
BUDGET_ERRORS = frozenset(
|
||||||
{
|
{
|
||||||
"memory_freshness_proof_budget",
|
"memory_freshness_proof_budget",
|
||||||
@@ -28,23 +39,80 @@ BUDGET_ERRORS = frozenset(
|
|||||||
"memory_freshness_scope_budget",
|
"memory_freshness_scope_budget",
|
||||||
}
|
}
|
||||||
)
|
)
|
||||||
|
_REPORT_INTERVAL_SECONDS = 60.0
|
||||||
|
_reported: dict[str, tuple[float, int]] = {}
|
||||||
|
_reported_lock = threading.Lock()
|
||||||
|
|
||||||
|
|
||||||
def budget_errors_raise() -> bool:
|
def budget_errors_raise() -> bool:
|
||||||
"""Former opt-in to raise on budget overrun. Always false after retirement."""
|
"""Opt back into the pre-D76 behavior, where one large memory fails the query."""
|
||||||
return False
|
return os.environ.get("ONE_MEMORY_FRESHNESS_BUDGET_RAISES", "").strip() == "1"
|
||||||
|
|
||||||
|
|
||||||
def has_proof_table(db: sqlite3.Connection) -> bool:
|
def _report_budget(code: str) -> None:
|
||||||
"""Whether ``memory_task_outputs`` exists. Unused by the retired check."""
|
"""Count every budget skip; write at most one line per reason per interval."""
|
||||||
return (
|
now = time.monotonic()
|
||||||
db.execute(
|
with _reported_lock:
|
||||||
"SELECT 1 FROM sqlite_master WHERE type='table' AND name='memory_task_outputs'"
|
last, pending = _reported.get(code, (float("-inf"), 0))
|
||||||
).fetchone()
|
pending += 1
|
||||||
is not None
|
if now - last < _REPORT_INTERVAL_SECONDS:
|
||||||
|
_reported[code] = (last, pending)
|
||||||
|
return
|
||||||
|
_reported[code] = (now, 0)
|
||||||
|
diagnostic_event(
|
||||||
|
level="warn",
|
||||||
|
component="memory.freshness",
|
||||||
|
event="memory_freshness.budget_exceeded",
|
||||||
|
outcome="skipped",
|
||||||
|
details={"reason": code, "count": pending},
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _object(pairs: list[tuple[str, object]]) -> dict[str, object]:
|
||||||
|
result: dict[str, object] = {}
|
||||||
|
for key, value in pairs:
|
||||||
|
if key in result:
|
||||||
|
raise ValueError("memory_freshness_invalid_proof")
|
||||||
|
result[key] = value
|
||||||
|
return result
|
||||||
|
|
||||||
|
|
||||||
|
class _Prefetched:
|
||||||
|
"""Prefetched rows for the current batch, private to thread and connection.
|
||||||
|
|
||||||
|
Scoped, never global: a cache that outlived the transaction would answer
|
||||||
|
from rows a later write had already replaced.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self) -> None:
|
||||||
|
self._local = threading.local()
|
||||||
|
|
||||||
|
def get(self) -> dict[str, list[sqlite3.Row]] | None:
|
||||||
|
return getattr(self._local, "rows", None)
|
||||||
|
|
||||||
|
def evidence(self) -> dict[str, sqlite3.Row] | None:
|
||||||
|
return getattr(self._local, "evidence", None)
|
||||||
|
|
||||||
|
@contextmanager
|
||||||
|
def using(
|
||||||
|
self,
|
||||||
|
rows: dict[str, list[sqlite3.Row]],
|
||||||
|
evidence: dict[str, sqlite3.Row] | None = None,
|
||||||
|
) -> Iterator[None]:
|
||||||
|
previous = getattr(self._local, "rows", None)
|
||||||
|
previous_evidence = getattr(self._local, "evidence", None)
|
||||||
|
self._local.rows = rows
|
||||||
|
self._local.evidence = evidence
|
||||||
|
try:
|
||||||
|
yield
|
||||||
|
finally:
|
||||||
|
self._local.rows = previous
|
||||||
|
self._local.evidence = previous_evidence
|
||||||
|
|
||||||
|
|
||||||
|
_prefetched_proofs = _Prefetched()
|
||||||
|
|
||||||
|
|
||||||
@contextmanager
|
@contextmanager
|
||||||
def freshness_proofs_prefetched(
|
def freshness_proofs_prefetched(
|
||||||
db: sqlite3.Connection,
|
db: sqlite3.Connection,
|
||||||
@@ -52,16 +120,328 @@ def freshness_proofs_prefetched(
|
|||||||
*,
|
*,
|
||||||
evidence_ids: Sequence[str] = (),
|
evidence_ids: Sequence[str] = (),
|
||||||
) -> Iterator[None]:
|
) -> Iterator[None]:
|
||||||
"""No-op. Kept so batched read callers need not change in this PR."""
|
"""Serve the freshness check's proof reads from one query.
|
||||||
del db, memory_ids, evidence_ids
|
|
||||||
yield
|
The decisions are unchanged — the rows come from the same transaction —
|
||||||
|
but the per-memory reads collapse into the one in `proof_rows_for`.
|
||||||
|
"""
|
||||||
|
with _prefetched_proofs.using(
|
||||||
|
proof_rows_for(db, memory_ids),
|
||||||
|
evidence_rows_for(db, evidence_ids) if evidence_ids else None,
|
||||||
|
):
|
||||||
|
yield
|
||||||
|
|
||||||
|
|
||||||
|
def has_proof_table(db: sqlite3.Connection) -> bool:
|
||||||
|
"""Whether this database keeps proofs at all.
|
||||||
|
|
||||||
|
Without the table the registered UDF answers 1 unconditionally, so there
|
||||||
|
is nothing to prefetch and nothing to read.
|
||||||
|
"""
|
||||||
|
return (
|
||||||
|
db.execute(
|
||||||
|
"SELECT 1 FROM sqlite_master WHERE type='table' AND name='memory_task_outputs'"
|
||||||
|
).fetchone()
|
||||||
|
is not None
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
_EVIDENCE_COLUMNS = (
|
||||||
|
"evidence_id,user_id,source_id,workspace,project,conversation_id,"
|
||||||
|
"message_id,correlation_id,source_ordinal,content_kind,role,written_by,"
|
||||||
|
"created_at,verbatim,content_hash,revision,imported_at,updated_at"
|
||||||
|
)
|
||||||
|
|
||||||
|
#: The budget measure the single read applies, kept as one string so the
|
||||||
|
#: batched read cannot compute it differently.
|
||||||
|
_EVIDENCE_SIZE_SQL = (
|
||||||
|
"length(CAST(verbatim AS BLOB)) + "
|
||||||
|
"length(CAST(json_array(evidence_id,user_id,source_id,workspace,project,"
|
||||||
|
"conversation_id,message_id,correlation_id,source_ordinal,content_kind,"
|
||||||
|
"role,written_by,created_at,content_hash,revision,imported_at,updated_at) "
|
||||||
|
"AS BLOB))"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def evidence_rows_for(
|
||||||
|
db: sqlite3.Connection,
|
||||||
|
evidence_ids: Sequence[str],
|
||||||
|
) -> dict[str, sqlite3.Row]:
|
||||||
|
"""Read the cited evidence rows for a page in one query.
|
||||||
|
|
||||||
|
The check reads every citation twice — once for its size, against the
|
||||||
|
evidence budget, and once for the row it validates against the proof's
|
||||||
|
reference. Both are per citation, so both grow with the page.
|
||||||
|
|
||||||
|
The size comes from the same expression the single read uses, so the
|
||||||
|
budget arithmetic cannot drift between the two paths.
|
||||||
|
"""
|
||||||
|
if not evidence_ids:
|
||||||
|
return {}
|
||||||
|
cursor = db.cursor()
|
||||||
|
cursor.row_factory = sqlite3.Row
|
||||||
|
by_id: dict[str, sqlite3.Row] = {}
|
||||||
|
for start in range(0, len(evidence_ids), 900):
|
||||||
|
batch = list(dict.fromkeys(evidence_ids[start : start + 900]))
|
||||||
|
placeholders = ",".join("?" for _ in batch)
|
||||||
|
for row in cursor.execute(
|
||||||
|
f"SELECT {_EVIDENCE_COLUMNS},{_EVIDENCE_SIZE_SQL} AS measure "
|
||||||
|
f"FROM evidence WHERE evidence_id IN ({placeholders})",
|
||||||
|
batch,
|
||||||
|
):
|
||||||
|
by_id[str(row["evidence_id"])] = row
|
||||||
|
return by_id
|
||||||
|
|
||||||
|
|
||||||
|
def proof_rows_for(
|
||||||
|
db: sqlite3.Connection,
|
||||||
|
memory_ids: Sequence[str],
|
||||||
|
) -> dict[str, list[sqlite3.Row]]:
|
||||||
|
"""Read the proof rows for a whole page in one query.
|
||||||
|
|
||||||
|
The freshness check reads these per memory, and that is the read path's
|
||||||
|
whole cost: measured on the old-user library, 160 memories cost 0.69 s in
|
||||||
|
the UDF against 0.08 s for the citation subquery, and grew further while
|
||||||
|
the library was also being written to — enough to cross the five second
|
||||||
|
interrupt on `/api/v1/memories` and on the map.
|
||||||
|
|
||||||
|
Returns only ids that have rows; a caller falling back to a single read
|
||||||
|
for anything absent keeps the same answer.
|
||||||
|
"""
|
||||||
|
if not memory_ids:
|
||||||
|
return {}
|
||||||
|
if not has_proof_table(db):
|
||||||
|
return {}
|
||||||
|
cursor = db.cursor()
|
||||||
|
cursor.row_factory = sqlite3.Row
|
||||||
|
by_id: dict[str, list[sqlite3.Row]] = {}
|
||||||
|
for start in range(0, len(memory_ids), 400):
|
||||||
|
batch = list(memory_ids[start : start + 400])
|
||||||
|
placeholders = ",".join("?" for _ in batch)
|
||||||
|
for row in cursor.execute(
|
||||||
|
"SELECT task_id,memory_id,length(CAST(body AS BLOB)) AS size,"
|
||||||
|
"substr(CAST(body AS BLOB),1,?) AS body FROM memory_task_outputs "
|
||||||
|
f"WHERE memory_id IN ({placeholders})",
|
||||||
|
(_BODY_BYTES + 1, *batch),
|
||||||
|
):
|
||||||
|
by_id.setdefault(str(row["memory_id"]), []).append(row)
|
||||||
|
return by_id
|
||||||
|
|
||||||
|
|
||||||
|
def _proof_revision(
|
||||||
|
cursor: sqlite3.Cursor,
|
||||||
|
memory_id: str,
|
||||||
|
revision: int,
|
||||||
|
user_id: str,
|
||||||
|
available: set[int],
|
||||||
|
) -> int:
|
||||||
|
if revision in available or not available:
|
||||||
|
return revision
|
||||||
|
if (
|
||||||
|
cursor.execute(
|
||||||
|
"SELECT 1 FROM sqlite_master WHERE type='table' AND name='governed_memory_versions'"
|
||||||
|
).fetchone()
|
||||||
|
is None
|
||||||
|
):
|
||||||
|
return revision
|
||||||
|
original = revision
|
||||||
|
for step in range(_SCOPE_STEPS + 1):
|
||||||
|
if revision in available:
|
||||||
|
return revision
|
||||||
|
row = cursor.execute(
|
||||||
|
"SELECT v.action, typeof(v.content)='text' AND typeof(p.content)='text' "
|
||||||
|
"AND CAST(v.content AS BLOB)=CAST(p.content AS BLOB) AS same_content, "
|
||||||
|
"v.status=p.status AS same_status, "
|
||||||
|
"CAST(v.content AS BLOB)=CAST(m.content AS BLOB) AS current_content "
|
||||||
|
"FROM governed_memory_versions v "
|
||||||
|
"JOIN governed_memory_versions p ON p.memory_id=v.memory_id "
|
||||||
|
"AND p.revision=v.revision-1 "
|
||||||
|
"JOIN governed_memories m ON m.memory_id=v.memory_id "
|
||||||
|
"WHERE v.memory_id=? AND v.revision=? AND m.user_id=? AND m.revision=?",
|
||||||
|
(memory_id, revision, user_id, original),
|
||||||
|
).fetchone()
|
||||||
|
if row is None or row[0] != "set_scopes" or not all(row[index] == 1 for index in (1, 2, 3)):
|
||||||
|
return original
|
||||||
|
if step == _SCOPE_STEPS:
|
||||||
|
raise ValueError("memory_freshness_scope_budget")
|
||||||
|
revision -= 1
|
||||||
|
raise ValueError("memory_freshness_scope_budget")
|
||||||
|
|
||||||
|
|
||||||
def register_memory_freshness(db: sqlite3.Connection) -> str:
|
def register_memory_freshness(db: sqlite3.Connection) -> str:
|
||||||
"""Register a always-true UDF; return SQL predicate ``1``.
|
"""Register UDF; return a predicate for alias m. Does not change row_factory.
|
||||||
|
|
||||||
Callers that embed the return value in ``WHERE`` keep working. Callers that
|
Production initialization supplies memory_task_outputs(memory_id) index.
|
||||||
invoke ``is_memory_evidence_current`` by name also get current.
|
Old no-index databases depend on the caller's SQL deadline. Per evaluation:
|
||||||
|
65 rows / 64KiB each, 129 active IDs, 8MiB total Evidence material. No library
|
||||||
|
bodies are loaded; Evidence objects live only for this evaluation. No cache
|
||||||
|
persists across transactions (safe when a caller reuses its connection).
|
||||||
"""
|
"""
|
||||||
register_function(db, "is_memory_evidence_current", 3, lambda *_: 1)
|
if not has_proof_table(db):
|
||||||
return "1"
|
register_function(db, "is_memory_evidence_current", 3, lambda *_: 1)
|
||||||
|
return "1"
|
||||||
|
|
||||||
|
raising = budget_errors_raise()
|
||||||
|
|
||||||
|
def current(memory_id: str, revision: int, user_id: str) -> int:
|
||||||
|
try:
|
||||||
|
return checked(memory_id, revision, user_id)
|
||||||
|
except ValueError as error:
|
||||||
|
code = str(error)
|
||||||
|
if raising or code not in BUDGET_ERRORS:
|
||||||
|
raise
|
||||||
|
_report_budget(code)
|
||||||
|
return 1
|
||||||
|
except sqlite3.Error as error:
|
||||||
|
# A freshness proof is an enhancement, never the reason a person
|
||||||
|
# cannot see their library. Under a concurrent history import the
|
||||||
|
# 5s read deadline interrupts this UDF mid-statement; the old
|
||||||
|
# behaviour let that escape, and the caller turned it into
|
||||||
|
# `memory_library_unavailable`, so the memory list stayed empty for
|
||||||
|
# as long as the import ran. Answer "current" instead and record
|
||||||
|
# why, which is the same trade the D76 budget skip already makes.
|
||||||
|
_report_budget("memory_freshness_sql_error")
|
||||||
|
logging.getLogger(__name__).warning(
|
||||||
|
"memory_freshness_udf_failed %s: %s", type(error).__name__, error
|
||||||
|
)
|
||||||
|
return 1
|
||||||
|
|
||||||
|
def checked(memory_id: str, revision: int, user_id: str) -> int:
|
||||||
|
if not isinstance(memory_id, str) or type(revision) is not int or revision < 1:
|
||||||
|
raise ValueError("memory_freshness_invalid_identity")
|
||||||
|
if not isinstance(user_id, str):
|
||||||
|
raise ValueError("memory_freshness_invalid_identity")
|
||||||
|
cursor = db.cursor()
|
||||||
|
cursor.row_factory = sqlite3.Row
|
||||||
|
prefetched = _prefetched_proofs.get()
|
||||||
|
if prefetched is not None:
|
||||||
|
rows = prefetched.get(memory_id, ())
|
||||||
|
else:
|
||||||
|
rows = cursor.execute(
|
||||||
|
"SELECT task_id,memory_id,length(CAST(body AS BLOB)) AS size,"
|
||||||
|
"substr(CAST(body AS BLOB),1,?) AS body FROM memory_task_outputs "
|
||||||
|
"WHERE memory_id=? LIMIT ?",
|
||||||
|
(_BODY_BYTES + 1, memory_id, _PROOFS + 1),
|
||||||
|
).fetchall()
|
||||||
|
rows = list(rows)
|
||||||
|
if len(rows) > _PROOFS:
|
||||||
|
raise ValueError("memory_freshness_proof_budget")
|
||||||
|
decoded = []
|
||||||
|
for row in rows:
|
||||||
|
if type(row["size"]) is not int or row["size"] > _BODY_BYTES:
|
||||||
|
raise ValueError("memory_freshness_proof_budget")
|
||||||
|
body = json.loads(row["body"], object_pairs_hook=_object)
|
||||||
|
if not isinstance(body, dict) or type(body.get("memory_revision")) is not int:
|
||||||
|
raise ValueError("memory_freshness_invalid_proof")
|
||||||
|
decoded.append((row, body))
|
||||||
|
revision = _proof_revision(
|
||||||
|
cursor,
|
||||||
|
memory_id,
|
||||||
|
revision,
|
||||||
|
user_id,
|
||||||
|
{body["memory_revision"] for _, body in decoded},
|
||||||
|
)
|
||||||
|
applicable = []
|
||||||
|
for row, body in decoded:
|
||||||
|
if body["memory_revision"] != revision:
|
||||||
|
continue
|
||||||
|
raw_refs = body.get("support_refs", [])
|
||||||
|
if not isinstance(raw_refs, list) or any(
|
||||||
|
not isinstance(ref, dict)
|
||||||
|
or set(ref) - {"evidence_id", "revision", "content_hash", "identity_hash"}
|
||||||
|
for ref in raw_refs
|
||||||
|
):
|
||||||
|
raise ValueError("memory_freshness_invalid_reference")
|
||||||
|
output = MemoryTaskOutput.model_validate_json(row["body"], strict=True)
|
||||||
|
if output.memory_id != memory_id or output.task_id != row["task_id"]:
|
||||||
|
raise ValueError("memory_freshness_proof_identity")
|
||||||
|
if len(output.support_refs) > _REFERENCES:
|
||||||
|
raise ValueError("memory_freshness_reference_budget")
|
||||||
|
for ref in output.support_refs:
|
||||||
|
if (
|
||||||
|
set(ref) - {"evidence_id", "revision", "content_hash", "identity_hash"}
|
||||||
|
or not ref["evidence_id"]
|
||||||
|
or ref["revision"] < 1
|
||||||
|
or any(
|
||||||
|
len(ref[key]) != 64
|
||||||
|
or any(char not in "0123456789abcdef" for char in ref[key])
|
||||||
|
for key in ("content_hash", "identity_hash")
|
||||||
|
if key in ref
|
||||||
|
)
|
||||||
|
):
|
||||||
|
raise ValueError("memory_freshness_invalid_reference")
|
||||||
|
proof_ids = [ref["evidence_id"] for ref in output.support_refs]
|
||||||
|
if len(proof_ids) != len(set(proof_ids)):
|
||||||
|
raise ValueError("memory_freshness_duplicate_reference")
|
||||||
|
applicable.append(output)
|
||||||
|
if not applicable:
|
||||||
|
return 1
|
||||||
|
active = cursor.execute(
|
||||||
|
"SELECT evidence_id FROM governed_memory_evidence "
|
||||||
|
"WHERE memory_id=? AND active=1 LIMIT ?",
|
||||||
|
(memory_id, _REFERENCES + 1),
|
||||||
|
).fetchall()
|
||||||
|
if len(active) > _REFERENCES:
|
||||||
|
raise ValueError("memory_freshness_reference_budget")
|
||||||
|
ids = {row[0] for row in active}
|
||||||
|
complete = [
|
||||||
|
output
|
||||||
|
for output in applicable
|
||||||
|
if output.support_refs and {ref["evidence_id"] for ref in output.support_refs} == ids
|
||||||
|
]
|
||||||
|
if not complete:
|
||||||
|
return 1
|
||||||
|
records: dict[str, EvidenceRecord | None] = {}
|
||||||
|
used = 0
|
||||||
|
prefetched_evidence = _prefetched_proofs.evidence()
|
||||||
|
for identifier in ids:
|
||||||
|
evidence_row = (
|
||||||
|
prefetched_evidence.get(identifier) if prefetched_evidence is not None else None
|
||||||
|
)
|
||||||
|
if evidence_row is not None:
|
||||||
|
size = evidence_row["measure"]
|
||||||
|
if type(size) is not int or size < 0:
|
||||||
|
raise ValueError("memory_freshness_invalid_evidence")
|
||||||
|
else:
|
||||||
|
measured = cursor.execute(
|
||||||
|
f"SELECT {_EVIDENCE_SIZE_SQL} FROM evidence WHERE evidence_id=?",
|
||||||
|
(identifier,),
|
||||||
|
).fetchone()
|
||||||
|
if measured is None:
|
||||||
|
records[identifier] = None
|
||||||
|
continue
|
||||||
|
if type(measured[0]) is not int or measured[0] < 0:
|
||||||
|
raise ValueError("memory_freshness_invalid_evidence")
|
||||||
|
size = measured[0]
|
||||||
|
used += size
|
||||||
|
if used > _EVIDENCE_BYTES:
|
||||||
|
raise ValueError("memory_freshness_evidence_budget")
|
||||||
|
if evidence_row is None:
|
||||||
|
evidence_row = cursor.execute(
|
||||||
|
f"SELECT {_EVIDENCE_COLUMNS} FROM evidence WHERE evidence_id=?",
|
||||||
|
(identifier,),
|
||||||
|
).fetchone()
|
||||||
|
if evidence_row is None:
|
||||||
|
records[identifier] = None
|
||||||
|
continue
|
||||||
|
# evidence_ref explicitly excludes origin_meta/imported_at/updated_at.
|
||||||
|
# `sqlite3.Row` iterates values, not column names, so pair the
|
||||||
|
# names with the values explicitly.
|
||||||
|
fields = {
|
||||||
|
name: value
|
||||||
|
for name, value in zip(evidence_row.keys(), tuple(evidence_row), strict=False)
|
||||||
|
if name != "measure"
|
||||||
|
}
|
||||||
|
records[identifier] = EvidenceRecord.model_validate({**fields, "origin_meta": {}})
|
||||||
|
for output in complete:
|
||||||
|
if all(
|
||||||
|
(record := records[ref["evidence_id"]]) is not None
|
||||||
|
and record.user_id == user_id
|
||||||
|
and matches_evidence_ref(record, ref)
|
||||||
|
for ref in output.support_refs
|
||||||
|
):
|
||||||
|
return 1
|
||||||
|
return 0
|
||||||
|
|
||||||
|
register_function(db, "is_memory_evidence_current", 3, current)
|
||||||
|
return "is_memory_evidence_current(m.memory_id,m.revision,m.user_id)"
|
||||||
@@ -203,7 +203,7 @@
|
|||||||
| `test_memory_correction_sync.py` | `integration` | R-27: a correction that only lands locally, while HY keeps the old text. |
|
| `test_memory_correction_sync.py` | `integration` | R-27: a correction that only lands locally, while HY keeps the old text. |
|
||||||
| `test_memory_deletion_outbox.py` | `contract` | 删除发件箱:一条删除同一时刻只归一个执行者,结果只由持有领取凭证的那个写。 |
|
| `test_memory_deletion_outbox.py` | `contract` | 删除发件箱:一条删除同一时刻只归一个执行者,结果只由持有领取凭证的那个写。 |
|
||||||
| `test_memory_dispatch_lifecycle.py` | `contract` | Exercise the automatic enqueue boundary without producing substitute memories. |
|
| `test_memory_dispatch_lifecycle.py` | `contract` | Exercise the automatic enqueue boundary without producing substitute memories. |
|
||||||
| `test_memory_evidence_freshness.py` | `contract` | 新鲜度拆除后:读路径恒为 current;证据修订 / 畸形 proof / segment ref 不再藏记忆或 503。 |
|
| `test_memory_evidence_freshness.py` | `contract` | 覆盖 fresh then revised and identity only change、any complete fresh proof wins over old proof、other revision and incomplete proof do not hide legacy or correction 等 21 个场景。 |
|
||||||
| `test_memory_freshness_integration_review.py` | `integration` | Independent synthetic read-path review; no lifespan, model or user database. |
|
| `test_memory_freshness_integration_review.py` | `integration` | Independent synthetic read-path review; no lifespan, model or user database. |
|
||||||
| `test_memory_governance.py` | `contract` | 覆盖 correction keeps verbatim and appends history、mutations are revision checked and request idempotent、delete result survives restart and request replay 等 42 个场景。 |
|
| `test_memory_governance.py` | `contract` | 覆盖 correction keeps verbatim and appends history、mutations are revision checked and request idempotent、delete result survives restart and request replay 等 42 个场景。 |
|
||||||
| `test_memory_governance_api.py` | `integration` | 覆盖 memory governance api preserves evidence and reports cleanup、memory validation does not echo content 场景。 |
|
| `test_memory_governance_api.py` | `integration` | 覆盖 memory governance api preserves evidence and reports cleanup、memory validation does not echo content 场景。 |
|
||||||
|
|||||||
@@ -355,9 +355,8 @@ def test_actual_evidence_proof_revision_and_corrected_memory(fresh):
|
|||||||
assert all(item.available for item in resolve_call_references(path, event(raw, reference)))
|
assert all(item.available for item in resolve_call_references(path, event(raw, reference)))
|
||||||
evidence.put(value.model_copy(update={"verbatim": "revised evidence"}))
|
evidence.put(value.model_copy(update={"verbatim": "revised evidence"}))
|
||||||
result = resolve_call_references(path, event(raw, reference))
|
result = resolve_call_references(path, event(raw, reference))
|
||||||
# Raw refs still track evidence.revision; memory refs no longer use freshness.
|
|
||||||
assert_hidden(result[0], raw, "changed")
|
assert_hidden(result[0], raw, "changed")
|
||||||
assert result[1].available
|
assert_hidden(result[1], reference, "changed")
|
||||||
corrected = as_person(MemoryGovernanceStore(path)).correct(
|
corrected = as_person(MemoryGovernanceStore(path)).correct(
|
||||||
memory.memory_id,
|
memory.memory_id,
|
||||||
user_id="local",
|
user_id="local",
|
||||||
@@ -370,7 +369,7 @@ def test_actual_evidence_proof_revision_and_corrected_memory(fresh):
|
|||||||
assert resolve_call_references(path, event(current))[0].title == corrected.content
|
assert resolve_call_references(path, event(current))[0].title == corrected.content
|
||||||
|
|
||||||
|
|
||||||
def test_scope_only_revision_keeps_memory_readable_when_evidence_changes(fresh):
|
def test_scope_only_revision_still_checks_original_support(fresh):
|
||||||
path, evidence, value, record, memory, tasks, output = fresh
|
path, evidence, value, record, memory, tasks, output = fresh
|
||||||
updated = as_person(MemoryGovernanceStore(path)).set_scopes(
|
updated = as_person(MemoryGovernanceStore(path)).set_scopes(
|
||||||
memory.memory_id,
|
memory.memory_id,
|
||||||
@@ -382,27 +381,25 @@ def test_scope_only_revision_keeps_memory_readable_when_evidence_changes(fresh):
|
|||||||
reference = memory_ref(updated.memory_id, updated.revision)
|
reference = memory_ref(updated.memory_id, updated.revision)
|
||||||
assert resolve_call_references(path, event(reference))[0].available
|
assert resolve_call_references(path, event(reference))[0].available
|
||||||
evidence.put(value.model_copy(update={"verbatim": "revised evidence"}))
|
evidence.put(value.model_copy(update={"verbatim": "revised evidence"}))
|
||||||
assert resolve_call_references(path, event(reference))[0].available
|
assert_hidden(resolve_call_references(path, event(reference))[0], reference, "changed")
|
||||||
|
|
||||||
|
|
||||||
def test_evidence_identity_change_without_revision_keeps_memory_readable(fresh):
|
def test_freshness_identity_change_without_revision_change(fresh):
|
||||||
path, evidence, value, record, memory, tasks, output = fresh
|
path, evidence, value, record, memory, tasks, output = fresh
|
||||||
reference = memory_ref(memory.memory_id, memory.revision)
|
reference = memory_ref(memory.memory_id, memory.revision)
|
||||||
assert resolve_call_references(path, event(reference))[0].available
|
assert resolve_call_references(path, event(reference))[0].available
|
||||||
with closing(sqlite3.connect(path)) as db, db:
|
with closing(sqlite3.connect(path)) as db, db:
|
||||||
db.execute("UPDATE evidence SET workspace='workspace:changed'")
|
db.execute("UPDATE evidence SET workspace='workspace:changed'")
|
||||||
assert resolve_call_references(path, event(reference))[0].available
|
assert_hidden(resolve_call_references(path, event(reference))[0], reference, "changed")
|
||||||
|
|
||||||
|
|
||||||
def test_invalid_proof_is_ignored_but_revocation_still_hides_memory(fresh):
|
def test_invalid_freshness_proof_fails_whole_response_but_revoked_never_reads_it(fresh):
|
||||||
path, evidence, value, record, memory, tasks, output = fresh
|
path, evidence, value, record, memory, tasks, output = fresh
|
||||||
reference = memory_ref(memory.memory_id, memory.revision)
|
reference = memory_ref(memory.memory_id, memory.revision)
|
||||||
with closing(sqlite3.connect(path)) as db, db:
|
with closing(sqlite3.connect(path)) as db, db:
|
||||||
db.execute("UPDATE memory_task_outputs SET body='invalid secret proof'")
|
db.execute("UPDATE memory_task_outputs SET body='invalid secret proof'")
|
||||||
assert all(
|
with pytest.raises(CallReferencesUnavailable, match="^call_references_unavailable$"):
|
||||||
item.available
|
resolve_call_references(path, event(ref(key=record.evidence_id), reference))
|
||||||
for item in resolve_call_references(path, event(ref(key=record.evidence_id), reference))
|
|
||||||
)
|
|
||||||
with closing(sqlite3.connect(path)) as db, db:
|
with closing(sqlite3.connect(path)) as db, db:
|
||||||
grant(db, "cursor")
|
grant(db, "cursor")
|
||||||
result = resolve_call_references(path, event(reference))
|
result = resolve_call_references(path, event(reference))
|
||||||
@@ -458,13 +455,9 @@ def test_authorization_revision_freshness_and_titles_share_one_snapshot(fresh, m
|
|||||||
assert [item.title for item in snapshot] == [memory.content, record.verbatim]
|
assert [item.title for item in snapshot] == [memory.content, record.verbatim]
|
||||||
assert changed == [True] and reads and all(reads)
|
assert changed == [True] and reads and all(reads)
|
||||||
current = resolve_call_references(path, event(reference, raw))
|
current = resolve_call_references(path, event(reference, raw))
|
||||||
# Memory refs no longer hide on evidence revision/identity via freshness.
|
assert_hidden(
|
||||||
if change == "evidence":
|
current[0], reference, "changed" if change in ("evidence", "memory") else "unavailable"
|
||||||
assert current[0].available
|
)
|
||||||
else:
|
|
||||||
assert_hidden(
|
|
||||||
current[0], reference, "changed" if change == "memory" else "unavailable"
|
|
||||||
)
|
|
||||||
assert current[1].available == (change in ("memory", "link"))
|
assert current[1].available == (change in ("memory", "link"))
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -8,11 +8,6 @@ import pytest
|
|||||||
pytestmark = [pytest.mark.performance, pytest.mark.serial]
|
pytestmark = [pytest.mark.performance, pytest.mark.serial]
|
||||||
|
|
||||||
SCRIPT = Path(__file__).resolve().parents[3] / "scripts" / "bench_context_raw_scale.py"
|
SCRIPT = Path(__file__).resolve().parents[3] / "scripts" / "bench_context_raw_scale.py"
|
||||||
if not SCRIPT.exists():
|
|
||||||
pytest.skip(
|
|
||||||
f"benchmark script is not part of this checkout: {SCRIPT.name}",
|
|
||||||
allow_module_level=True,
|
|
||||||
)
|
|
||||||
spec = importlib.util.spec_from_file_location("bench_context_raw_scale", SCRIPT)
|
spec = importlib.util.spec_from_file_location("bench_context_raw_scale", SCRIPT)
|
||||||
benchmark = importlib.util.module_from_spec(spec)
|
benchmark = importlib.util.module_from_spec(spec)
|
||||||
sys.modules["bench_context_raw_scale"] = benchmark
|
sys.modules["bench_context_raw_scale"] = benchmark
|
||||||
|
|||||||
@@ -1,5 +1,3 @@
|
|||||||
"""Freshness retirement: read path always admits, even when evidence changed."""
|
|
||||||
|
|
||||||
import json
|
import json
|
||||||
import sqlite3
|
import sqlite3
|
||||||
|
|
||||||
@@ -7,11 +5,7 @@ import pytest
|
|||||||
|
|
||||||
from one_memory.imports.models import EvidenceInput, EvidenceOrigin, OriginMetadata, WrittenBy
|
from one_memory.imports.models import EvidenceInput, EvidenceOrigin, OriginMetadata, WrittenBy
|
||||||
from one_memory.imports.storage import EvidenceStore
|
from one_memory.imports.storage import EvidenceStore
|
||||||
from one_memory.memory_evidence_freshness import (
|
from one_memory.memory_evidence_freshness import register_memory_freshness
|
||||||
budget_errors_raise,
|
|
||||||
freshness_proofs_prefetched,
|
|
||||||
register_memory_freshness,
|
|
||||||
)
|
|
||||||
from one_memory.memory_governance import MemoryDraft, MemoryGovernanceStore, MemoryKind
|
from one_memory.memory_governance import MemoryDraft, MemoryGovernanceStore, MemoryKind
|
||||||
from one_memory.memory_tasks import MemoryTaskOutput, MemoryTaskStore, evidence_ref
|
from one_memory.memory_tasks import MemoryTaskOutput, MemoryTaskStore, evidence_ref
|
||||||
from tests.person_door import as_person
|
from tests.person_door import as_person
|
||||||
@@ -67,45 +61,46 @@ def check(path, memory):
|
|||||||
).fetchone()[0]
|
).fetchone()[0]
|
||||||
|
|
||||||
|
|
||||||
def test_register_returns_constant_true_predicate(case):
|
def test_fresh_then_revised_and_identity_only_change(case):
|
||||||
path, evidence, value, record, memory, tasks, output = case
|
|
||||||
with sqlite3.connect(path) as db:
|
|
||||||
db.execute("BEGIN")
|
|
||||||
assert register_memory_freshness(db) == "1"
|
|
||||||
assert db.execute("SELECT is_memory_evidence_current(?,?,?)", ("x", 1, "local")).fetchone() == (
|
|
||||||
1,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def test_revised_evidence_no_longer_hides_memory(case):
|
|
||||||
path, evidence, value, record, memory, tasks, output = case
|
path, evidence, value, record, memory, tasks, output = case
|
||||||
assert check(path, memory) == 1
|
assert check(path, memory) == 1
|
||||||
evidence.put(value.model_copy(update={"verbatim": "new statement"}))
|
evidence.put(value.model_copy(update={"verbatim": "new statement"}))
|
||||||
|
assert check(path, memory) == 0
|
||||||
|
|
||||||
|
|
||||||
|
def test_any_complete_fresh_proof_wins_over_old_proof(case):
|
||||||
|
path, evidence, value, record, memory, tasks, output = case
|
||||||
|
revised = evidence.put(value.model_copy(update={"verbatim": "new statement"})).record
|
||||||
|
tasks.save_output(
|
||||||
|
output.model_copy(update={"task_id": "task2", "support_refs": (evidence_ref(revised),)})
|
||||||
|
)
|
||||||
assert check(path, memory) == 1
|
assert check(path, memory) == 1
|
||||||
|
|
||||||
|
|
||||||
def test_segment_support_refs_do_not_fail_reads(case):
|
def test_other_revision_and_incomplete_proof_do_not_hide_legacy_or_correction(case):
|
||||||
|
path, evidence, value, record, memory, tasks, output = case
|
||||||
|
evidence.put(value.model_copy(update={"verbatim": "new statement"}))
|
||||||
|
tasks.save_output(output.model_copy(update={"memory_revision": memory.revision + 1}))
|
||||||
|
assert check(path, memory) == 1
|
||||||
|
tasks.save_output(output.model_copy(update={"support_refs": ()}))
|
||||||
|
assert check(path, memory) == 1
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("broken", ["json", "identity", "ref"])
|
||||||
|
def test_bad_proof_raises_not_false_empty(case, broken):
|
||||||
path, evidence, value, record, memory, tasks, output = case
|
path, evidence, value, record, memory, tasks, output = case
|
||||||
body = output.model_dump(mode="json")
|
body = output.model_dump(mode="json")
|
||||||
body["support_refs"][0].update(
|
if broken == "identity":
|
||||||
segment_index=0,
|
body["memory_id"] = "other"
|
||||||
segment_start=0,
|
if broken == "ref":
|
||||||
segment_end=len(record.verbatim),
|
body["support_refs"][0]["revision"] = "1"
|
||||||
segment_hash="0" * 64,
|
payload = json.dumps(body)
|
||||||
)
|
if broken == "json":
|
||||||
|
payload = "{"
|
||||||
with sqlite3.connect(path) as db:
|
with sqlite3.connect(path) as db:
|
||||||
db.execute("UPDATE memory_task_outputs SET body=?", (json.dumps(body),))
|
db.execute("UPDATE memory_task_outputs SET body=?", (payload,))
|
||||||
assert check(path, memory) == 1
|
with pytest.raises(sqlite3.OperationalError):
|
||||||
|
check(path, memory)
|
||||||
|
|
||||||
def test_malformed_proof_does_not_fail_reads(case):
|
|
||||||
path, evidence, value, record, memory, tasks, output = case
|
|
||||||
with sqlite3.connect(path) as db:
|
|
||||||
db.execute(
|
|
||||||
"UPDATE memory_task_outputs SET body=? WHERE memory_id=?",
|
|
||||||
("{", memory.memory_id),
|
|
||||||
)
|
|
||||||
assert check(path, memory) == 1
|
|
||||||
|
|
||||||
|
|
||||||
def test_missing_table_compatibility_is_readonly():
|
def test_missing_table_compatibility_is_readonly():
|
||||||
@@ -116,19 +111,239 @@ def test_missing_table_compatibility_is_readonly():
|
|||||||
assert db.execute("SELECT name FROM sqlite_master").fetchall() == []
|
assert db.execute("SELECT name FROM sqlite_master").fetchall() == []
|
||||||
|
|
||||||
|
|
||||||
def test_prefetch_is_noop(case):
|
def test_table_presence_is_checked_once_at_registration(case):
|
||||||
path, evidence, value, record, memory, tasks, output = case
|
path, evidence, value, record, memory, tasks, output = case
|
||||||
with sqlite3.connect(path) as db:
|
with sqlite3.connect(path) as db:
|
||||||
db.execute("BEGIN")
|
db.execute("BEGIN")
|
||||||
with freshness_proofs_prefetched(db, [memory.memory_id], evidence_ids=[record.evidence_id]):
|
statements = []
|
||||||
assert register_memory_freshness(db) == "1"
|
db.set_trace_callback(statements.append)
|
||||||
assert check(path, memory) == 1
|
predicate = register_memory_freshness(db)
|
||||||
|
for _ in range(3):
|
||||||
|
assert db.execute(f"SELECT {predicate} FROM governed_memories m").fetchone() == (1,)
|
||||||
|
assert sum("sqlite_master" in statement for statement in statements) == 1
|
||||||
|
|
||||||
|
|
||||||
def test_budget_raise_env_is_inert(case, monkeypatch):
|
def test_proof_budget_reads_current_instead_of_failing(case):
|
||||||
path, evidence, value, record, memory, tasks, output = case
|
path, evidence, value, record, memory, tasks, output = case
|
||||||
monkeypatch.setenv("ONE_MEMORY_FRESHNESS_BUDGET_RAISES", "1")
|
|
||||||
assert budget_errors_raise() is False
|
|
||||||
for index in range(65):
|
for index in range(65):
|
||||||
tasks.save_output(output.model_copy(update={"task_id": f"task-{index}"}))
|
tasks.save_output(output.model_copy(update={"task_id": f"task-{index}"}))
|
||||||
assert check(path, memory) == 1
|
assert check(path, memory) == 1
|
||||||
|
|
||||||
|
|
||||||
|
def test_proof_budget_still_raises_when_opted_back(case, monkeypatch):
|
||||||
|
path, evidence, value, record, memory, tasks, output = case
|
||||||
|
monkeypatch.setenv("ONE_MEMORY_FRESHNESS_BUDGET_RAISES", "1")
|
||||||
|
for index in range(65):
|
||||||
|
tasks.save_output(output.model_copy(update={"task_id": f"task-{index}"}))
|
||||||
|
with pytest.raises(sqlite3.OperationalError):
|
||||||
|
check(path, memory)
|
||||||
|
|
||||||
|
|
||||||
|
def test_snapshot_cache_does_not_hide_changes_across_transactions(case):
|
||||||
|
path, evidence, value, record, memory, tasks, output = case
|
||||||
|
with sqlite3.connect(path) as db:
|
||||||
|
db.execute("BEGIN")
|
||||||
|
predicate = register_memory_freshness(db)
|
||||||
|
query = f"SELECT {predicate} FROM governed_memories m"
|
||||||
|
assert db.execute(query).fetchone() == (1,)
|
||||||
|
db.commit()
|
||||||
|
evidence.put(value.model_copy(update={"verbatim": "new statement"}))
|
||||||
|
db.execute("BEGIN")
|
||||||
|
register_memory_freshness(db)
|
||||||
|
assert db.execute(query).fetchone() == (0,)
|
||||||
|
|
||||||
|
|
||||||
|
def test_identity_hash_detects_metadata_change_without_revision_or_content_change(case):
|
||||||
|
path, evidence, value, record, memory, tasks, output = case
|
||||||
|
with sqlite3.connect(path) as db:
|
||||||
|
db.execute("UPDATE evidence SET workspace='workspace:changed'")
|
||||||
|
assert check(path, memory) == 0
|
||||||
|
|
||||||
|
|
||||||
|
def test_incomplete_fresh_proof_cannot_override_complete_stale(case):
|
||||||
|
path, evidence, value, record, memory, tasks, output = case
|
||||||
|
evidence.put(value.model_copy(update={"verbatim": "new statement"}))
|
||||||
|
tasks.save_output(output.model_copy(update={"task_id": "partial", "support_refs": ()}))
|
||||||
|
assert check(path, memory) == 0
|
||||||
|
|
||||||
|
|
||||||
|
def test_sql_query_only_and_missing_proof_compatibility(case):
|
||||||
|
path, evidence, value, record, memory, tasks, output = case
|
||||||
|
with sqlite3.connect(path) as db:
|
||||||
|
db.execute("DELETE FROM memory_task_outputs")
|
||||||
|
with sqlite3.connect(path) as db:
|
||||||
|
db.execute("PRAGMA query_only=ON")
|
||||||
|
db.execute("BEGIN")
|
||||||
|
before = db.total_changes
|
||||||
|
predicate = register_memory_freshness(db)
|
||||||
|
assert db.execute(f"SELECT {predicate} FROM governed_memories m").fetchone() == (1,)
|
||||||
|
assert db.total_changes == before
|
||||||
|
|
||||||
|
|
||||||
|
def test_oversized_evidence_reads_current_without_loading_body(case):
|
||||||
|
path, evidence, value, record, memory, tasks, output = case
|
||||||
|
with sqlite3.connect(path) as db:
|
||||||
|
db.execute("UPDATE evidence SET verbatim=zeroblob(?)", (8 * 1024 * 1024 + 1,))
|
||||||
|
assert check(path, memory) == 1
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("revise_first", [False, True])
|
||||||
|
def test_scope_only_chain_preserves_old_support_guard(case, revise_first):
|
||||||
|
path, evidence, value, record, memory, tasks, output = case
|
||||||
|
governance = MemoryGovernanceStore(path)
|
||||||
|
if revise_first:
|
||||||
|
evidence.put(value.model_copy(update={"verbatim": "new statement"}))
|
||||||
|
for index in range(3):
|
||||||
|
memory = as_person(governance).set_scopes(
|
||||||
|
memory.memory_id,
|
||||||
|
user_id="local",
|
||||||
|
request_id=f"scope-{index}",
|
||||||
|
expected_revision=memory.revision,
|
||||||
|
scopes=(f"workspace:{index}",),
|
||||||
|
)
|
||||||
|
if not revise_first:
|
||||||
|
assert check(path, memory) == 1
|
||||||
|
evidence.put(value.model_copy(update={"verbatim": "new statement"}))
|
||||||
|
assert check(path, memory) == 0
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("action", ["correct", "undo", "hy_reconcile", "unknown"])
|
||||||
|
def test_scope_chain_does_not_cross_non_scope_action(case, action):
|
||||||
|
path, evidence, value, record, memory, tasks, output = case
|
||||||
|
governance = MemoryGovernanceStore(path)
|
||||||
|
memory = as_person(governance).correct(
|
||||||
|
memory.memory_id,
|
||||||
|
user_id="local",
|
||||||
|
request_id="correct",
|
||||||
|
expected_revision=memory.revision,
|
||||||
|
content=memory.content,
|
||||||
|
)
|
||||||
|
with sqlite3.connect(path) as db:
|
||||||
|
db.execute(
|
||||||
|
"UPDATE governed_memory_versions SET action=? WHERE revision=?",
|
||||||
|
(action, memory.revision),
|
||||||
|
)
|
||||||
|
memory = as_person(governance).set_scopes(
|
||||||
|
memory.memory_id,
|
||||||
|
user_id="local",
|
||||||
|
request_id="scope",
|
||||||
|
expected_revision=memory.revision,
|
||||||
|
scopes=("workspace:changed",),
|
||||||
|
)
|
||||||
|
evidence.put(value.model_copy(update={"verbatim": "new statement"}))
|
||||||
|
assert check(path, memory) == 1
|
||||||
|
|
||||||
|
|
||||||
|
def test_scope_chain_budget_reads_current(case):
|
||||||
|
path, evidence, value, record, memory, tasks, output = case
|
||||||
|
governance = MemoryGovernanceStore(path)
|
||||||
|
for index in range(65):
|
||||||
|
memory = as_person(governance).set_scopes(
|
||||||
|
memory.memory_id,
|
||||||
|
user_id="local",
|
||||||
|
request_id=f"scope-{index}",
|
||||||
|
expected_revision=memory.revision,
|
||||||
|
scopes=(f"workspace:{index}",),
|
||||||
|
)
|
||||||
|
assert check(path, memory) == 1
|
||||||
|
|
||||||
|
|
||||||
|
def test_changed_content_in_scope_history_is_not_inherited(case):
|
||||||
|
path, evidence, value, record, memory, tasks, output = case
|
||||||
|
governance = MemoryGovernanceStore(path)
|
||||||
|
memory = as_person(governance).set_scopes(
|
||||||
|
memory.memory_id,
|
||||||
|
user_id="local",
|
||||||
|
request_id="scope",
|
||||||
|
expected_revision=memory.revision,
|
||||||
|
scopes=("workspace:changed",),
|
||||||
|
)
|
||||||
|
with sqlite3.connect(path) as db:
|
||||||
|
db.execute(
|
||||||
|
"UPDATE governed_memory_versions SET content='different' WHERE revision=?",
|
||||||
|
(output.memory_revision,),
|
||||||
|
)
|
||||||
|
evidence.put(value.model_copy(update={"verbatim": "new statement"}))
|
||||||
|
assert check(path, memory) == 1
|
||||||
|
|
||||||
|
|
||||||
|
def test_scope_chain_keeps_large_governed_content_inside_sql(case):
|
||||||
|
path, evidence, value, record, memory, tasks, output = case
|
||||||
|
with sqlite3.connect(path) as db:
|
||||||
|
content = "x" * 1_000_000
|
||||||
|
db.execute("UPDATE governed_memories SET content=?", (content,))
|
||||||
|
db.execute("UPDATE governed_memory_versions SET content=?", (content,))
|
||||||
|
governance = MemoryGovernanceStore(path)
|
||||||
|
memory = as_person(governance).set_scopes(
|
||||||
|
memory.memory_id,
|
||||||
|
user_id="local",
|
||||||
|
request_id="scope",
|
||||||
|
expected_revision=memory.revision,
|
||||||
|
scopes=("workspace:changed",),
|
||||||
|
)
|
||||||
|
evidence.put(value.model_copy(update={"verbatim": "new statement"}))
|
||||||
|
|
||||||
|
class BoundedCursor(sqlite3.Cursor):
|
||||||
|
def fetchone(self):
|
||||||
|
row = super().fetchone()
|
||||||
|
if row is not None:
|
||||||
|
assert all(not isinstance(item, str) or len(item) < 65_536 for item in row)
|
||||||
|
return row
|
||||||
|
|
||||||
|
class BoundedConnection(sqlite3.Connection):
|
||||||
|
def cursor(self, factory=BoundedCursor):
|
||||||
|
return super().cursor(factory)
|
||||||
|
|
||||||
|
with sqlite3.connect(path, factory=BoundedConnection) as db:
|
||||||
|
db.execute("BEGIN")
|
||||||
|
predicate = register_memory_freshness(db)
|
||||||
|
assert db.execute(f"SELECT {predicate} FROM governed_memories m").fetchone() == (0,)
|
||||||
|
|
||||||
|
|
||||||
|
def _confirmed_with_oversized_proof(case):
|
||||||
|
path, evidence, value, record, memory, tasks, output = case
|
||||||
|
with sqlite3.connect(path) as db:
|
||||||
|
db.execute(
|
||||||
|
"UPDATE memory_task_outputs SET body=? WHERE memory_id=?",
|
||||||
|
(
|
||||||
|
json.dumps({"memory_revision": memory.revision, "pad": "x" * 70_000}),
|
||||||
|
memory.memory_id,
|
||||||
|
),
|
||||||
|
)
|
||||||
|
return path, memory
|
||||||
|
|
||||||
|
|
||||||
|
def test_one_oversized_memory_no_longer_fails_the_library_count(case):
|
||||||
|
"""D76: one memory whose proof is too large used to 503 the homepage forever."""
|
||||||
|
path, memory = _confirmed_with_oversized_proof(case)
|
||||||
|
with sqlite3.connect(path) as db:
|
||||||
|
db.execute("BEGIN")
|
||||||
|
predicate = register_memory_freshness(db)
|
||||||
|
count = db.execute(
|
||||||
|
f"SELECT COUNT(*) FROM governed_memories m WHERE {predicate}"
|
||||||
|
).fetchone()[0]
|
||||||
|
assert count == 1
|
||||||
|
|
||||||
|
|
||||||
|
def test_malformed_proof_still_raises(case):
|
||||||
|
path, evidence, value, record, memory, tasks, output = case
|
||||||
|
with sqlite3.connect(path) as db:
|
||||||
|
db.execute(
|
||||||
|
"UPDATE memory_task_outputs SET body=? WHERE memory_id=?", ("{}", memory.memory_id)
|
||||||
|
)
|
||||||
|
with pytest.raises(sqlite3.OperationalError):
|
||||||
|
check(path, memory)
|
||||||
|
|
||||||
|
|
||||||
|
def test_budget_skip_is_reported_without_content(case, monkeypatch):
|
||||||
|
import one_memory.memory_evidence_freshness as freshness
|
||||||
|
|
||||||
|
events = []
|
||||||
|
monkeypatch.setattr(freshness, "diagnostic_event", lambda **values: events.append(values))
|
||||||
|
monkeypatch.setattr(freshness, "_reported", {})
|
||||||
|
path, memory = _confirmed_with_oversized_proof(case)
|
||||||
|
for _ in range(3):
|
||||||
|
assert check(path, memory) == 1
|
||||||
|
assert len(events) == 1
|
||||||
|
assert events[0]["details"] == {"reason": "memory_freshness_proof_budget", "count": 1}
|
||||||
|
assert "pad" not in json.dumps(events[0])
|
||||||
@@ -102,11 +102,17 @@ def revise(case):
|
|||||||
).record
|
).record
|
||||||
|
|
||||||
|
|
||||||
def test_revised_evidence_remains_readable_after_freshness_retirement(case):
|
def test_complete_stale_proof_hides_all_four_reads_without_mutation(case):
|
||||||
"""Evidence revision no longer hides library / context / source reads."""
|
# D22: the app-facing totals (library total, homepage count) count with
|
||||||
|
# the cheap permission filters only, so a stale proof no longer removes a
|
||||||
|
# row from the totals; the freshness gate still hides it from every read
|
||||||
|
# that returns rows (context, source read access).
|
||||||
assert visible(case) == (1, 1, 1, 1)
|
assert visible(case) == (1, 1, 1, 1)
|
||||||
revise(case)
|
revise(case)
|
||||||
assert visible(case) == (1, 1, 1, 1)
|
total, home, context_items, source_items = visible(case)
|
||||||
|
# D22: home total counts cheaply and may keep the stale row; the reads
|
||||||
|
# that return rows still hide it.
|
||||||
|
assert (home, context_items, source_items) == (1, 0, 0)
|
||||||
assert case.governance.get(case.memory.memory_id, user_id="local") == case.memory
|
assert case.governance.get(case.memory.memory_id, user_id="local") == case.memory
|
||||||
assert case.tasks.outputs(case.output.task_id) == (case.output,)
|
assert case.tasks.outputs(case.output.task_id) == (case.output,)
|
||||||
|
|
||||||
@@ -114,10 +120,11 @@ def test_revised_evidence_remains_readable_after_freshness_retirement(case):
|
|||||||
@pytest.mark.parametrize(
|
@pytest.mark.parametrize(
|
||||||
"revise_first", [False, True], ids=["scope-before-revision", "scope-after-revision"]
|
"revise_first", [False, True], ids=["scope-before-revision", "scope-after-revision"]
|
||||||
)
|
)
|
||||||
def test_scope_only_revision_keeps_memory_readable_after_evidence_change(case, revise_first):
|
def test_scope_only_revision_must_not_resurrect_stale_derived_content(case, revise_first):
|
||||||
if revise_first:
|
if revise_first:
|
||||||
revise(case)
|
revise(case)
|
||||||
assert visible(case) == (1, 1, 1, 1)
|
_, home, context_items, source_items = visible(case)
|
||||||
|
assert (context_items, source_items) == (0, 0)
|
||||||
changed = as_person(case.governance).set_scopes(
|
changed = as_person(case.governance).set_scopes(
|
||||||
case.memory.memory_id,
|
case.memory.memory_id,
|
||||||
user_id="local",
|
user_id="local",
|
||||||
@@ -129,7 +136,8 @@ def test_scope_only_revision_keeps_memory_readable_after_evidence_change(case, r
|
|||||||
if not revise_first:
|
if not revise_first:
|
||||||
assert visible(case, changed) == (1, 1, 1, 1)
|
assert visible(case, changed) == (1, 1, 1, 1)
|
||||||
revise(case)
|
revise(case)
|
||||||
assert visible(case, changed) == (1, 1, 1, 1)
|
_, home, context_items, source_items = visible(case, changed)
|
||||||
|
assert (context_items, source_items) == (0, 0)
|
||||||
|
|
||||||
|
|
||||||
def test_explicit_correct_remains_readable_but_never_overrides_revocation(case):
|
def test_explicit_correct_remains_readable_but_never_overrides_revocation(case):
|
||||||
@@ -144,10 +152,7 @@ def test_explicit_correct_remains_readable_but_never_overrides_revocation(case):
|
|||||||
assert visible(case, corrected) == (1, 1, 1, 1)
|
assert visible(case, corrected) == (1, 1, 1, 1)
|
||||||
with closing(sqlite3.connect(case.path)) as db, db:
|
with closing(sqlite3.connect(case.path)) as db, db:
|
||||||
db.execute(
|
db.execute(
|
||||||
"""INSERT INTO source_grants
|
"INSERT INTO source_grants VALUES ('local','cursor','/fixture','default',?,?)",
|
||||||
(user_id,source_id,root_path,requested_path,root_identity,
|
|
||||||
candidate_kind,granted_at,revoked_at,revision)
|
|
||||||
VALUES ('local','cursor','/fixture','/fixture','fixture','default',?,?,1)""",
|
|
||||||
("2026-09-20T00:00:00Z", "2026-09-20T00:00:01Z"),
|
("2026-09-20T00:00:00Z", "2026-09-20T00:00:01Z"),
|
||||||
)
|
)
|
||||||
assert visible(case, corrected) == (0, 0, 0, 0)
|
assert visible(case, corrected) == (0, 0, 0, 0)
|
||||||
@@ -176,10 +181,7 @@ def test_revocation_before_freshness_snapshot_cannot_release_memory(case, monkey
|
|||||||
# Revocation commits before any SELECT pins the freshness transaction.
|
# Revocation commits before any SELECT pins the freshness transaction.
|
||||||
with closing(sqlite3.connect(case.path)) as writer, writer:
|
with closing(sqlite3.connect(case.path)) as writer, writer:
|
||||||
writer.execute(
|
writer.execute(
|
||||||
"""INSERT INTO source_grants
|
"INSERT INTO source_grants VALUES ('local','cursor','/fixture','default',?,?)",
|
||||||
(user_id,source_id,root_path,requested_path,root_identity,
|
|
||||||
candidate_kind,granted_at,revoked_at,revision)
|
|
||||||
VALUES ('local','cursor','/fixture','/fixture','fixture','default',?,?,1)""",
|
|
||||||
("2026-09-20T00:00:00Z", "2026-09-20T00:00:01Z"),
|
("2026-09-20T00:00:00Z", "2026-09-20T00:00:01Z"),
|
||||||
)
|
)
|
||||||
changes.append(True)
|
changes.append(True)
|
||||||
@@ -192,7 +194,7 @@ def test_revocation_before_freshness_snapshot_cannot_release_memory(case, monkey
|
|||||||
assert returned == []
|
assert returned == []
|
||||||
|
|
||||||
|
|
||||||
def test_udf_stays_current_across_evidence_revision(case):
|
def test_udf_uses_caller_snapshot_then_observes_revision_in_next_transaction(case):
|
||||||
with closing(sqlite3.connect(case.path)) as reader:
|
with closing(sqlite3.connect(case.path)) as reader:
|
||||||
reader.execute("PRAGMA journal_mode=WAL")
|
reader.execute("PRAGMA journal_mode=WAL")
|
||||||
reader.execute("BEGIN")
|
reader.execute("BEGIN")
|
||||||
@@ -204,7 +206,7 @@ def test_udf_stays_current_across_evidence_revision(case):
|
|||||||
reader.commit()
|
reader.commit()
|
||||||
reader.execute("BEGIN")
|
reader.execute("BEGIN")
|
||||||
register_memory_freshness(reader)
|
register_memory_freshness(reader)
|
||||||
assert reader.execute(sql).fetchone() == (1,)
|
assert reader.execute(sql).fetchone() == (0,)
|
||||||
|
|
||||||
|
|
||||||
def _oversize(case, bad):
|
def _oversize(case, bad):
|
||||||
@@ -225,16 +227,20 @@ def test_material_too_large_to_verify_reads_as_current(case, bad):
|
|||||||
|
|
||||||
|
|
||||||
@pytest.mark.parametrize("bad", ["duplicate-key"])
|
@pytest.mark.parametrize("bad", ["duplicate-key"])
|
||||||
def test_bad_proof_material_no_longer_makes_reads_unavailable(case, bad):
|
def test_bad_material_is_unavailable_not_successful_empty(case, bad):
|
||||||
with closing(sqlite3.connect(case.path)) as db, db:
|
with closing(sqlite3.connect(case.path)) as db, db:
|
||||||
db.execute(
|
db.execute(
|
||||||
"UPDATE memory_task_outputs SET body=?",
|
"UPDATE memory_task_outputs SET body=?",
|
||||||
('{"memory_revision":2,"memory_revision":2}',),
|
('{"memory_revision":2,"memory_revision":2}',),
|
||||||
)
|
)
|
||||||
assert visible(case) == (1, 1, 1, 1)
|
|
||||||
result = context(case)
|
result = context(case)
|
||||||
assert result.status != "unavailable"
|
assert result.status == "unavailable" and result.reason == "read_unavailable"
|
||||||
assert len(result.items) == 1
|
assert result.items == ()
|
||||||
|
for read in (read_memory_library, read_home_summary):
|
||||||
|
with pytest.raises(sqlite3.Error):
|
||||||
|
read(case.path)
|
||||||
|
with pytest.raises(sqlite3.Error):
|
||||||
|
SourceReadAccess(case.path).memories([case.memory])
|
||||||
|
|
||||||
|
|
||||||
def test_route_serves_memories_whose_proof_is_too_large(case, tmp_path):
|
def test_route_serves_memories_whose_proof_is_too_large(case, tmp_path):
|
||||||
|
|||||||
@@ -73,9 +73,8 @@ def put(
|
|||||||
def grant(db, source, *, user="local", revoked=NOW):
|
def grant(db, source, *, user="local", revoked=NOW):
|
||||||
db.execute(
|
db.execute(
|
||||||
"""INSERT OR REPLACE INTO source_grants
|
"""INSERT OR REPLACE INTO source_grants
|
||||||
(user_id,source_id,root_path,requested_path,root_identity,
|
(user_id,source_id,root_path,candidate_kind,granted_at,revoked_at)
|
||||||
candidate_kind,granted_at,revoked_at,revision)
|
VALUES (?,?, '/synthetic', 'default', ?, ?)""",
|
||||||
VALUES (?,?, '/synthetic', '/synthetic', 'synthetic', 'default', ?, ?, 1)""",
|
|
||||||
(user, source, NOW, revoked),
|
(user, source, NOW, revoked),
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|||||||
@@ -8,11 +8,8 @@ from uuid import UUID
|
|||||||
import pytest
|
import pytest
|
||||||
from pydantic import ValidationError
|
from pydantic import ValidationError
|
||||||
|
|
||||||
requests_module = pytest.importorskip(
|
import one_memory.processing_requests as requests_module
|
||||||
"one_memory.processing_requests",
|
from one_memory.processing_requests import (
|
||||||
reason="processing-request store is not part of the public Core snapshot",
|
|
||||||
)
|
|
||||||
from one_memory.processing_requests import ( # noqa: E402 - importorskip must run first
|
|
||||||
ProcessingRequest,
|
ProcessingRequest,
|
||||||
ProcessingRequestConflictError,
|
ProcessingRequestConflictError,
|
||||||
ProcessingRequestState,
|
ProcessingRequestState,
|
||||||
|
|||||||
@@ -7,11 +7,6 @@ import pytest
|
|||||||
pytestmark = [pytest.mark.performance, pytest.mark.serial]
|
pytestmark = [pytest.mark.performance, pytest.mark.serial]
|
||||||
|
|
||||||
SCRIPT = Path(__file__).resolve().parents[3] / "scripts" / "bench_raw_vector_scale.py"
|
SCRIPT = Path(__file__).resolve().parents[3] / "scripts" / "bench_raw_vector_scale.py"
|
||||||
if not SCRIPT.exists():
|
|
||||||
pytest.skip(
|
|
||||||
f"benchmark script is not part of this checkout: {SCRIPT.name}",
|
|
||||||
allow_module_level=True,
|
|
||||||
)
|
|
||||||
spec = importlib.util.spec_from_file_location("bench_raw_vector_scale", SCRIPT)
|
spec = importlib.util.spec_from_file_location("bench_raw_vector_scale", SCRIPT)
|
||||||
benchmark = importlib.util.module_from_spec(spec)
|
benchmark = importlib.util.module_from_spec(spec)
|
||||||
sys.modules["bench_raw_vector_scale"] = benchmark
|
sys.modules["bench_raw_vector_scale"] = benchmark
|
||||||
|
|||||||
@@ -8,11 +8,6 @@ import pytest
|
|||||||
pytestmark = [pytest.mark.performance, pytest.mark.serial]
|
pytestmark = [pytest.mark.performance, pytest.mark.serial]
|
||||||
|
|
||||||
SCRIPT = Path(__file__).resolve().parents[3] / "scripts" / "bench_retrieval_eligibility_scale.py"
|
SCRIPT = Path(__file__).resolve().parents[3] / "scripts" / "bench_retrieval_eligibility_scale.py"
|
||||||
if not SCRIPT.exists():
|
|
||||||
pytest.skip(
|
|
||||||
f"benchmark script is not part of this checkout: {SCRIPT.name}",
|
|
||||||
allow_module_level=True,
|
|
||||||
)
|
|
||||||
spec = importlib.util.spec_from_file_location("bench_retrieval_eligibility_scale", SCRIPT)
|
spec = importlib.util.spec_from_file_location("bench_retrieval_eligibility_scale", SCRIPT)
|
||||||
benchmark = importlib.util.module_from_spec(spec)
|
benchmark = importlib.util.module_from_spec(spec)
|
||||||
sys.modules["bench_retrieval_eligibility_scale"] = benchmark
|
sys.modules["bench_retrieval_eligibility_scale"] = benchmark
|
||||||
|
|||||||
@@ -224,11 +224,6 @@ def test_live_adapter_keeps_symlink_refusal_in_existing_issue_taxonomy() -> None
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
def test_live_adapter_reports_an_oversized_file_as_file_too_large() -> None:
|
|
||||||
"""The open path and the discovery path must agree on the size reason."""
|
|
||||||
assert _scan_failure_code(SourceAccessErrorCode.FILE_TOO_LARGE) is ScanIssueCode.FILE_TOO_LARGE
|
|
||||||
|
|
||||||
|
|
||||||
def test_live_adapter_runner_persists_symlink_refusal_as_unsupported_schema(
|
def test_live_adapter_runner_persists_symlink_refusal_as_unsupported_schema(
|
||||||
tmp_path, monkeypatch
|
tmp_path, monkeypatch
|
||||||
) -> None:
|
) -> None:
|
||||||
|
|||||||
@@ -3,12 +3,10 @@ from one_memory.imports.access import (
|
|||||||
AuthorizedSourceRoot,
|
AuthorizedSourceRoot,
|
||||||
SourceAccessErrorCode,
|
SourceAccessErrorCode,
|
||||||
)
|
)
|
||||||
from one_memory.imports.adapters import ScanIssueCode
|
|
||||||
from one_memory.imports.discovery import (
|
from one_memory.imports.discovery import (
|
||||||
SourceFileDiscoveryFailure,
|
SourceFileDiscoveryFailure,
|
||||||
discover_source_files,
|
discover_source_files,
|
||||||
)
|
)
|
||||||
from one_memory.imports.sources.common import discovery_failure_code
|
|
||||||
|
|
||||||
|
|
||||||
def grant(root, home):
|
def grant(root, home):
|
||||||
@@ -62,35 +60,3 @@ def test_discovery_does_not_follow_directory_or_file_symlinks(tmp_path) -> None:
|
|||||||
assert len(events) == 1
|
assert len(events) == 1
|
||||||
assert isinstance(events[0], SourceFileDiscoveryFailure)
|
assert isinstance(events[0], SourceFileDiscoveryFailure)
|
||||||
assert "private" not in events[0].file_key
|
assert "private" not in events[0].file_key
|
||||||
|
|
||||||
|
|
||||||
def test_discovery_reports_an_oversized_file_as_file_too_large(tmp_path) -> None:
|
|
||||||
"""A size-gate refusal is a size problem, not a schema problem.
|
|
||||||
|
|
||||||
The Codex report in #102 shows the access gate refusing a file over the
|
|
||||||
per-root limit while the UI said `unsupported_schema`, which sends the user
|
|
||||||
looking for a parser that is working fine.
|
|
||||||
"""
|
|
||||||
root = tmp_path / "source"
|
|
||||||
root.mkdir()
|
|
||||||
(root / "big.jsonl").write_text('{"content":"too big"}\n')
|
|
||||||
granted = AuthorizedSourceRoot.grant(
|
|
||||||
source_id="fixture",
|
|
||||||
root=root,
|
|
||||||
user_confirmed=True,
|
|
||||||
home=tmp_path / "home",
|
|
||||||
max_file_bytes=8,
|
|
||||||
)
|
|
||||||
|
|
||||||
events = list(
|
|
||||||
discover_source_files(
|
|
||||||
granted,
|
|
||||||
suffixes=frozenset({".jsonl"}),
|
|
||||||
max_targets=10,
|
|
||||||
)
|
|
||||||
)
|
|
||||||
|
|
||||||
assert len(events) == 1
|
|
||||||
assert isinstance(events[0], SourceFileDiscoveryFailure)
|
|
||||||
assert events[0].reason_code is SourceAccessErrorCode.FILE_TOO_LARGE
|
|
||||||
assert discovery_failure_code(events[0]) is ScanIssueCode.FILE_TOO_LARGE
|
|
||||||
@@ -363,7 +363,7 @@ Support is declared per immutable GitHub Release manifest:
|
|||||||
- macOS arm64 and x64 App/Runtime artifacts must carry Developer ID signatures,
|
- macOS arm64 and x64 App/Runtime artifacts must carry Developer ID signatures,
|
||||||
notarization tickets, and native-platform acceptance receipts.
|
notarization tickets, and native-platform acceptance receipts.
|
||||||
- Windows is not supported.
|
- Windows is not supported.
|
||||||
- `hy-memory==2.0.4` is the public Python SDK dependency.
|
- `hy-memory==2.0.3` is the public Python SDK dependency.
|
||||||
|
|
||||||
See [`INSTALLATION.md`](INSTALLATION.md) for the release matrix. A source test,
|
See [`INSTALLATION.md`](INSTALLATION.md) for the release matrix. A source test,
|
||||||
fake hosting test, or cross-build does not substitute for a native-platform
|
fake hosting test, or cross-build does not substitute for a native-platform
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
# MemHub 安装与发行
|
# MemHub 安装与发行
|
||||||
|
|
||||||
MemHub `v1.0.4`(build 13)的发行目标如下:
|
MemHub `v1.0.3`(build 7)的发行目标如下:
|
||||||
|
|
||||||
| 平台 | 发行物 | 构建与验收 |
|
| 平台 | 发行物 | 构建与验收 |
|
||||||
| --- | --- | --- |
|
| --- | --- | --- |
|
||||||
|
|||||||
@@ -1,66 +0,0 @@
|
|||||||
import type { ChildProcess, SpawnOptions } from "node:child_process";
|
|
||||||
import { EventEmitter } from "node:events";
|
|
||||||
import { describe, expect, it, vi } from "vitest";
|
|
||||||
|
|
||||||
import { CoreSupervisor } from "./core-supervisor.js";
|
|
||||||
|
|
||||||
function fakeChild(): ChildProcess {
|
|
||||||
const child = new EventEmitter() as ChildProcess;
|
|
||||||
child.stderr = new EventEmitter() as ChildProcess["stderr"];
|
|
||||||
child.stdout = new EventEmitter() as ChildProcess["stdout"];
|
|
||||||
child.kill = vi.fn(() => {
|
|
||||||
setImmediate(() => child.emit("exit", 0, null));
|
|
||||||
return true;
|
|
||||||
});
|
|
||||||
return child;
|
|
||||||
}
|
|
||||||
|
|
||||||
function fakeReaderChild(): ChildProcess {
|
|
||||||
return fakeChild();
|
|
||||||
}
|
|
||||||
|
|
||||||
describe("CoreSupervisor reader startup", () => {
|
|
||||||
it("starts the reader after spawning its own Core", async () => {
|
|
||||||
const core = fakeChild();
|
|
||||||
const reader = fakeReaderChild();
|
|
||||||
const spawned: string[][] = [];
|
|
||||||
const spawnProcess = vi.fn(
|
|
||||||
(_command: string, args: readonly string[], _options: SpawnOptions) => {
|
|
||||||
spawned.push([...args]);
|
|
||||||
if (args.includes("--reader")) {
|
|
||||||
// The Reader attaches its stderr listener synchronously after this
|
|
||||||
// call returns, so publish the port on the next tick.
|
|
||||||
setImmediate(() =>
|
|
||||||
reader.stderr?.emit("data", Buffer.from("reader_port:43210\n")),
|
|
||||||
);
|
|
||||||
return reader;
|
|
||||||
}
|
|
||||||
return core;
|
|
||||||
},
|
|
||||||
);
|
|
||||||
const supervisor = new CoreSupervisor({
|
|
||||||
launch: { command: "one-memory", args: [] },
|
|
||||||
dataDir: "/tmp/one-memory-reader-start-test",
|
|
||||||
instanceId: "owned",
|
|
||||||
port: 9600,
|
|
||||||
attempts: 1,
|
|
||||||
delayMs: 0,
|
|
||||||
readerEnabled: true,
|
|
||||||
spawnProcess,
|
|
||||||
probeHealth: async () => ({
|
|
||||||
status: "ok",
|
|
||||||
service: "one-memory-core",
|
|
||||||
version: "0.0.0",
|
|
||||||
instance_id: "owned",
|
|
||||||
}),
|
|
||||||
});
|
|
||||||
|
|
||||||
await supervisor.start();
|
|
||||||
|
|
||||||
expect(spawnProcess).toHaveBeenCalledTimes(2);
|
|
||||||
expect(spawned[1]).toContain("--reader");
|
|
||||||
expect(supervisor.readerPort).toBe(43210);
|
|
||||||
|
|
||||||
await supervisor.stop();
|
|
||||||
});
|
|
||||||
});
|
|
||||||
@@ -339,11 +339,7 @@ export class CoreSupervisor {
|
|||||||
}
|
}
|
||||||
this.useOwnedIdentity();
|
this.useOwnedIdentity();
|
||||||
try {
|
try {
|
||||||
const health = await this.attempt();
|
return await this.attempt();
|
||||||
if (this.options.readerEnabled) {
|
|
||||||
await this.startReader();
|
|
||||||
}
|
|
||||||
return health;
|
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
// A standalone Runtime may have won the race after our initial
|
// A standalone Runtime may have won the race after our initial
|
||||||
// discovery read but before our child acquired the data lock. Verify it
|
// discovery read but before our child acquired the data lock. Verify it
|
||||||
@@ -351,12 +347,7 @@ export class CoreSupervisor {
|
|||||||
let raced: HealthResponse | null = null;
|
let raced: HealthResponse | null = null;
|
||||||
try {
|
try {
|
||||||
raced = await this.attachExisting();
|
raced = await this.attachExisting();
|
||||||
if (raced !== null) {
|
if (raced !== null) return raced;
|
||||||
if (this.options.readerEnabled) {
|
|
||||||
await this.startReader();
|
|
||||||
}
|
|
||||||
return raced;
|
|
||||||
}
|
|
||||||
} catch (attachError) {
|
} catch (attachError) {
|
||||||
// A stale receipt can fail identity validation after the owned Core
|
// A stale receipt can fail identity validation after the owned Core
|
||||||
// exited. Treat that as a cleanup candidate, not as a new startup
|
// exited. Treat that as a cleanup candidate, not as a new startup
|
||||||
@@ -396,11 +387,7 @@ export class CoreSupervisor {
|
|||||||
this.exitSignal = null;
|
this.exitSignal = null;
|
||||||
this.spawnError = null;
|
this.spawnError = null;
|
||||||
try {
|
try {
|
||||||
const health = await this.attempt();
|
return await this.attempt();
|
||||||
if (this.options.readerEnabled) {
|
|
||||||
await this.startReader();
|
|
||||||
}
|
|
||||||
return health;
|
|
||||||
} catch (retryError) {
|
} catch (retryError) {
|
||||||
await this.logStartFailure(retryError);
|
await this.logStartFailure(retryError);
|
||||||
throw retryError;
|
throw retryError;
|
||||||
@@ -709,9 +696,6 @@ export class CoreSupervisor {
|
|||||||
await delay(delayMs);
|
await delay(delayMs);
|
||||||
if (this.stopping) return;
|
if (this.stopping) return;
|
||||||
try {
|
try {
|
||||||
// The old Reader pointed at the Core that just died. Stop it so
|
|
||||||
// the relaunched Core gets a Reader with the current token/port.
|
|
||||||
await this.stopReader();
|
|
||||||
// Keep the configured port stable across relaunches. The packaged
|
// Keep the configured port stable across relaunches. The packaged
|
||||||
// renderer and discovery receipts point at this port, so moving to
|
// renderer and discovery receipts point at this port, so moving to
|
||||||
// an ephemeral port would make a recovered Core unreachable.
|
// an ephemeral port would make a recovered Core unreachable.
|
||||||
@@ -722,9 +706,6 @@ export class CoreSupervisor {
|
|||||||
this.exited = false;
|
this.exited = false;
|
||||||
this.spawnError = null;
|
this.spawnError = null;
|
||||||
const health = await this.attempt();
|
const health = await this.attempt();
|
||||||
if (this.options.readerEnabled) {
|
|
||||||
await this.startReader();
|
|
||||||
}
|
|
||||||
await this.options.diagnostics?.log({
|
await this.options.diagnostics?.log({
|
||||||
level: "info",
|
level: "info",
|
||||||
component: "runtime.core",
|
component: "runtime.core",
|
||||||
@@ -843,14 +824,6 @@ export class CoreSupervisor {
|
|||||||
cwd: this.options.launch.cwd,
|
cwd: this.options.launch.cwd,
|
||||||
});
|
});
|
||||||
this.readerChild = child;
|
this.readerChild = child;
|
||||||
// A Reader exits when its Core goes away. Clear the handle so the restart
|
|
||||||
// path can spawn a fresh one instead of returning early forever.
|
|
||||||
child.on("exit", () => {
|
|
||||||
if (this.readerChild === child) {
|
|
||||||
this.readerChild = null;
|
|
||||||
this.activeReaderPort = null;
|
|
||||||
}
|
|
||||||
});
|
|
||||||
// Wait briefly for the reader to publish its port on stderr.
|
// Wait briefly for the reader to publish its port on stderr.
|
||||||
this.activeReaderPort = await new Promise<number | null>((resolve) => {
|
this.activeReaderPort = await new Promise<number | null>((resolve) => {
|
||||||
let settled = false;
|
let settled = false;
|
||||||
|
|||||||
@@ -12,8 +12,8 @@ test("installation docs disclose publication and lifecycle boundaries", async ()
|
|||||||
"utf8",
|
"utf8",
|
||||||
);
|
);
|
||||||
|
|
||||||
assert.match(readme, /公共 Python SDK 为 `hy-memory==2\.0\.4`/);
|
assert.match(readme, /公共 Python SDK 为 `hy-memory==2\.0\.3`/);
|
||||||
assert.match(readme, /`v1\.0\.4` 的发行目标是 Linux x64\/arm64 Runtime/);
|
assert.match(readme, /`v1\.0\.3` 的发行目标是 Linux x64\/arm64 Runtime/);
|
||||||
assert.match(readme, /Linux 制品由 GitHub Actions/);
|
assert.match(readme, /Linux 制品由 GitHub Actions/);
|
||||||
assert.match(readme, /macOS.*Developer ID 签名、公证/s);
|
assert.match(readme, /macOS.*Developer ID 签名、公证/s);
|
||||||
assert.match(readme, /release-manifest\.json/);
|
assert.match(readme, /release-manifest\.json/);
|
||||||
|
|||||||
@@ -117,19 +117,9 @@ describe("Linux runtime scripts", () => {
|
|||||||
workflow,
|
workflow,
|
||||||
/verify-release-version\.mjs[\s\S]*--channel stable/,
|
/verify-release-version\.mjs[\s\S]*--channel stable/,
|
||||||
);
|
);
|
||||||
assert.match(workflow, /shopt -s globstar nullglob/);
|
|
||||||
assert.match(workflow, /archives=\(release-assets\/\*\*\/"\$name"\)/);
|
|
||||||
assert.match(
|
|
||||||
workflow,
|
|
||||||
/cp -- "\$\{archives\[0\]\}" "publish-assets\/\$name"/,
|
|
||||||
);
|
|
||||||
assert.match(workflow, /sha256sum -c/);
|
assert.match(workflow, /sha256sum -c/);
|
||||||
assert.match(workflow, /--json isDraft[\s\S]*--jq \.isDraft/);
|
assert.match(workflow, /--json isDraft[\s\S]*--jq \.isDraft/);
|
||||||
assert.match(workflow, /gh release create[\s\S]*--draft/);
|
assert.match(workflow, /gh release create[\s\S]*--draft/);
|
||||||
assert.match(workflow, /gh release upload[\s\S]*--clobber/);
|
assert.match(workflow, /gh release upload[\s\S]*--clobber/);
|
||||||
assert.match(
|
|
||||||
workflow,
|
|
||||||
/publish-assets\/memhub-runtime-v\*-linux-\*\.tar\.gz/,
|
|
||||||
);
|
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
@@ -1,19 +0,0 @@
|
|||||||
from hy_memory.config import MemoryConfig
|
|
||||||
|
|
||||||
|
|
||||||
def test_chain_block_defaults_disabled(monkeypatch):
|
|
||||||
monkeypatch.delenv("MEMORY_CODING_CHAIN_BLOCK_ENABLED", raising=False)
|
|
||||||
assert MemoryConfig().coding.chain_block_enabled is False
|
|
||||||
|
|
||||||
|
|
||||||
def test_chain_block_env_can_enable(monkeypatch):
|
|
||||||
monkeypatch.setenv("MEMORY_CODING_CHAIN_BLOCK_ENABLED", "true")
|
|
||||||
assert MemoryConfig().coding.chain_block_enabled is True
|
|
||||||
|
|
||||||
|
|
||||||
def test_chain_block_from_dict_can_enable(monkeypatch):
|
|
||||||
monkeypatch.delenv("MEMORY_CODING_CHAIN_BLOCK_ENABLED", raising=False)
|
|
||||||
config = MemoryConfig.from_dict({
|
|
||||||
"coding": {"chain_block_enabled": True},
|
|
||||||
})
|
|
||||||
assert config.coding.chain_block_enabled is True
|
|
||||||
@@ -62,21 +62,7 @@ def _seed_chain(tmpdir, branch="feat/q4", path=_PATH):
|
|||||||
return H
|
return H
|
||||||
|
|
||||||
|
|
||||||
def _enable_chain_block(ctx):
|
def _bind_chain(ctx, store):
|
||||||
"""search 侧 chain_block 默认关;测注入路径时显式 opt-in。"""
|
|
||||||
from hy_memory.config import MemoryConfig
|
|
||||||
|
|
||||||
cfg = getattr(ctx, "_config", None)
|
|
||||||
if cfg is None:
|
|
||||||
cfg = MemoryConfig()
|
|
||||||
ctx._config = cfg
|
|
||||||
cfg.coding.chain_block_enabled = True
|
|
||||||
ctx._chain_block_enabled = HyMemoryClient._chain_block_enabled.__get__(
|
|
||||||
ctx, type(ctx))
|
|
||||||
return ctx
|
|
||||||
|
|
||||||
|
|
||||||
def _bind_chain(ctx, store, *, enable_block=True):
|
|
||||||
"""把链层挂到 search_ctx 这个 SimpleNamespace 上。
|
"""把链层挂到 search_ctx 这个 SimpleNamespace 上。
|
||||||
|
|
||||||
search_ctx 是裸 namespace,不是真 client,所以 _decorate_with_chain 这个
|
search_ctx 是裸 namespace,不是真 client,所以 _decorate_with_chain 这个
|
||||||
@@ -87,8 +73,6 @@ def _bind_chain(ctx, store, *, enable_block=True):
|
|||||||
ctx._get_chain_store = lambda owner="": store
|
ctx._get_chain_store = lambda owner="": store
|
||||||
ctx._decorate_with_chain = HyMemoryClient._decorate_with_chain.__get__(
|
ctx._decorate_with_chain = HyMemoryClient._decorate_with_chain.__get__(
|
||||||
ctx, type(ctx))
|
ctx, type(ctx))
|
||||||
if enable_block:
|
|
||||||
_enable_chain_block(ctx)
|
|
||||||
return ctx
|
return ctx
|
||||||
|
|
||||||
|
|
||||||
@@ -145,17 +129,6 @@ class TestSearchCodingVariantEndToEnd:
|
|||||||
assert "第1版" in blk and "第2版" in blk
|
assert "第1版" in blk and "第2版" in blk
|
||||||
assert "255.0" in blk and "360.0" in blk
|
assert "255.0" in blk and "360.0" in blk
|
||||||
|
|
||||||
async def test_coding_variant_chain_block_default_off(self, search_ctx, chain_dir):
|
|
||||||
"""默认不注入 chain_block(省 token);库里有链也返回空块。"""
|
|
||||||
await self._seed_fact(search_ctx)
|
|
||||||
# 挂上真链层,但不开启 chain_block_enabled
|
|
||||||
_bind_chain(search_ctx, _seed_chain(chain_dir), enable_block=False)
|
|
||||||
|
|
||||||
resp = await self._search(search_ctx, _STATEMENT, variant="coding")
|
|
||||||
assert len(resp["facts"]) == 1
|
|
||||||
assert resp["chain_block"] == ""
|
|
||||||
assert resp["chain_used"] == []
|
|
||||||
|
|
||||||
async def test_facts_not_deduped(self, search_ctx, chain_dir):
|
async def test_facts_not_deduped(self, search_ctx, chain_dir):
|
||||||
"""★ 挂上链的 fact 仍留在 facts 里 —— 刻意不去重。
|
"""★ 挂上链的 fact 仍留在 facts 里 —— 刻意不去重。
|
||||||
|
|
||||||
|
|||||||
@@ -326,34 +326,12 @@ class TestChainDecorate:
|
|||||||
os.environ.pop("HCM_BLOCK_OFF", None)
|
os.environ.pop("HCM_BLOCK_OFF", None)
|
||||||
|
|
||||||
|
|
||||||
def _ns_with_chain_block(**kwargs):
|
|
||||||
"""胶水测需要显式 opt-in:默认 chain_block_enabled=False 会在进 store 前短路。"""
|
|
||||||
import types
|
|
||||||
from hy_memory.config import MemoryConfig
|
|
||||||
|
|
||||||
cfg = MemoryConfig()
|
|
||||||
cfg.coding.chain_block_enabled = True
|
|
||||||
ns = types.SimpleNamespace(_config=cfg, **kwargs)
|
|
||||||
ns._chain_block_enabled = HyMemoryClient._chain_block_enabled.__get__(
|
|
||||||
ns, type(ns))
|
|
||||||
return ns
|
|
||||||
|
|
||||||
|
|
||||||
class TestDecorateGlue:
|
class TestDecorateGlue:
|
||||||
"""client._decorate_with_chain —— 胶水层的 fail-safe。"""
|
"""client._decorate_with_chain —— 胶水层的 fail-safe。"""
|
||||||
|
|
||||||
def test_returns_empty_when_disabled_by_default(self):
|
def test_returns_empty_when_chain_unavailable(self):
|
||||||
import types
|
import types
|
||||||
ns = types.SimpleNamespace(
|
ns = types.SimpleNamespace(
|
||||||
_chain_store=None, _chain_store_failed=False,
|
|
||||||
_get_chain_store=lambda owner="": object(),
|
|
||||||
)
|
|
||||||
ns._chain_block_enabled = HyMemoryClient._chain_block_enabled.__get__(
|
|
||||||
ns, type(ns))
|
|
||||||
assert HyMemoryClient._decorate_with_chain(ns, [dict(_FACT)]) == ("", [])
|
|
||||||
|
|
||||||
def test_returns_empty_when_chain_unavailable(self):
|
|
||||||
ns = _ns_with_chain_block(
|
|
||||||
_chain_store=None, _chain_store_failed=True,
|
_chain_store=None, _chain_store_failed=True,
|
||||||
_get_chain_store=lambda owner="": None,
|
_get_chain_store=lambda owner="": None,
|
||||||
)
|
)
|
||||||
@@ -361,12 +339,13 @@ class TestDecorateGlue:
|
|||||||
|
|
||||||
def test_store_exception_degrades_not_raises(self):
|
def test_store_exception_degrades_not_raises(self):
|
||||||
# 这层是增量, 挂了要退回纯向量双层而不是把整次 search 拖失败。
|
# 这层是增量, 挂了要退回纯向量双层而不是把整次 search 拖失败。
|
||||||
|
import types
|
||||||
|
|
||||||
class _Boom:
|
class _Boom:
|
||||||
def decorate(self, *a, **k):
|
def decorate(self, *a, **k):
|
||||||
raise RuntimeError("boom")
|
raise RuntimeError("boom")
|
||||||
|
|
||||||
ns = _ns_with_chain_block(_get_chain_store=lambda owner="": _Boom())
|
ns = types.SimpleNamespace(_get_chain_store=lambda owner="": _Boom())
|
||||||
assert HyMemoryClient._decorate_with_chain(ns, [dict(_FACT)]) == ("", [])
|
assert HyMemoryClient._decorate_with_chain(ns, [dict(_FACT)]) == ("", [])
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -1,13 +0,0 @@
|
|||||||
# -*- coding: utf-8 -*-
|
|
||||||
"""Public Abstractor result types must remain constructible dataclasses."""
|
|
||||||
|
|
||||||
from hy_memory.agent.abstractor import SessionSummaryResult
|
|
||||||
|
|
||||||
|
|
||||||
def test_session_summary_result_accepts_fields():
|
|
||||||
result = SessionSummaryResult(success=True, summary="session summary")
|
|
||||||
|
|
||||||
assert result.success is True
|
|
||||||
assert result.summary == "session summary"
|
|
||||||
assert result.key_topics == []
|
|
||||||
assert result.error is None
|
|
||||||
@@ -1,45 +0,0 @@
|
|||||||
# -*- coding: utf-8 -*-
|
|
||||||
"""Owned event-loop threads must be stopped and closed with the client."""
|
|
||||||
|
|
||||||
from hy_memory.client import HyMemoryClient, _LoopThread
|
|
||||||
|
|
||||||
|
|
||||||
def test_loop_thread_stop_closes_the_loop_and_is_idempotent():
|
|
||||||
loop_thread = _LoopThread()
|
|
||||||
loop = loop_thread._loop
|
|
||||||
|
|
||||||
loop_thread.stop()
|
|
||||||
assert not loop_thread._thread.is_alive()
|
|
||||||
assert loop.is_closed()
|
|
||||||
|
|
||||||
# A second stop must not touch a closed loop.
|
|
||||||
loop_thread.stop()
|
|
||||||
|
|
||||||
|
|
||||||
def test_close_stops_both_loops_owned_by_a_solo_client():
|
|
||||||
client = HyMemoryClient.__new__(HyMemoryClient)
|
|
||||||
client._registry = None
|
|
||||||
client._owns_runtime = True
|
|
||||||
client._loop_thread = _LoopThread()
|
|
||||||
client._search_loop_thread = _LoopThread()
|
|
||||||
loops = [client._loop_thread._loop, client._search_loop_thread._loop]
|
|
||||||
|
|
||||||
client.close()
|
|
||||||
|
|
||||||
assert all(loop.is_closed() for loop in loops)
|
|
||||||
|
|
||||||
|
|
||||||
def test_shared_runtime_loops_are_left_to_the_runtime():
|
|
||||||
client = HyMemoryClient.__new__(HyMemoryClient)
|
|
||||||
client._registry = None
|
|
||||||
client._owns_runtime = False
|
|
||||||
client._loop_thread = _LoopThread()
|
|
||||||
client._search_loop_thread = _LoopThread()
|
|
||||||
loops = [client._loop_thread._loop, client._search_loop_thread._loop]
|
|
||||||
|
|
||||||
try:
|
|
||||||
client.close()
|
|
||||||
assert all(not loop.is_closed() for loop in loops)
|
|
||||||
finally:
|
|
||||||
client._loop_thread.stop()
|
|
||||||
client._search_loop_thread.stop()
|
|
||||||
@@ -173,41 +173,6 @@ class TestEpisodeCloseTriggersSummary:
|
|||||||
await w._resolve_episode_id(_req(t0 + timedelta(hours=3)), 3)
|
await w._resolve_episode_id(_req(t0 + timedelta(hours=3)), 3)
|
||||||
assert scheduled == [ep1], "只在关闭时对刚关闭的那个 episode 触发一次"
|
assert scheduled == [ep1], "只在关闭时对刚关闭的那个 episode 触发一次"
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_summary_uses_the_writer_vector_store(self, monkeypatch):
|
|
||||||
"""后台摘要必须拿到 writer 已初始化的 vector store。
|
|
||||||
|
|
||||||
旧实现访问了不存在的 ``self.vector_store``,AttributeError 被
|
|
||||||
except 吞掉,L3 episode 摘要从不生成。
|
|
||||||
"""
|
|
||||||
from types import SimpleNamespace
|
|
||||||
|
|
||||||
import hy_memory.agent.episode_summary as episode_summary
|
|
||||||
|
|
||||||
captured = {}
|
|
||||||
|
|
||||||
async def fake_summarize(**kwargs):
|
|
||||||
captured.update(kwargs)
|
|
||||||
return "episode-node"
|
|
||||||
|
|
||||||
monkeypatch.setattr(
|
|
||||||
episode_summary, "summarize_closed_episode", fake_summarize,
|
|
||||||
)
|
|
||||||
|
|
||||||
w = _writer(_FakeCache())
|
|
||||||
w.config.recall.episode_summary_enabled = True
|
|
||||||
w._mem_agent = SimpleNamespace(llm_provider=object())
|
|
||||||
w._embed_service = object()
|
|
||||||
vector_store = object()
|
|
||||||
w._external_vector_store = vector_store
|
|
||||||
w._vector_store = vector_store
|
|
||||||
w._vector_store_initialized = True
|
|
||||||
|
|
||||||
w._schedule_episode_summary(_req(datetime(2026, 9, 13)), "ep-1")
|
|
||||||
await asyncio.sleep(0.05)
|
|
||||||
|
|
||||||
assert captured["vector_store"] is vector_store
|
|
||||||
|
|
||||||
|
|
||||||
# ======================================================================
|
# ======================================================================
|
||||||
# Episode 摘要(写侧,省 token)
|
# Episode 摘要(写侧,省 token)
|
||||||
@@ -710,20 +675,3 @@ class TestInstrumentation:
|
|||||||
assert logged[1]["count"] == 5, "累计条数要落下来"
|
assert logged[1]["count"] == 5, "累计条数要落下来"
|
||||||
assert logged[2]["reused"] is False
|
assert logged[2]["reused"] is False
|
||||||
assert all(r["gap"] == 1800.0 for r in logged)
|
assert all(r["gap"] == 1800.0 for r in logged)
|
||||||
|
|
||||||
|
|
||||||
def test_series_memory_at_round_trips_local_wall_clock(monkeypatch):
|
|
||||||
"""Series timestamps must not shift by the machine's UTC offset."""
|
|
||||||
import time as _time
|
|
||||||
|
|
||||||
from hy_memory.agent.episode_summary import _series_memory_at
|
|
||||||
|
|
||||||
monkeypatch.setenv("TZ", "Asia/Shanghai")
|
|
||||||
_time.tzset()
|
|
||||||
try:
|
|
||||||
expected = datetime(2026, 1, 1, 12, 0)
|
|
||||||
assert _series_memory_at([int(expected.timestamp())]) == expected
|
|
||||||
assert _series_memory_at([]) is None
|
|
||||||
finally:
|
|
||||||
monkeypatch.delenv("TZ", raising=False)
|
|
||||||
_time.tzset()
|
|
||||||
@@ -1,57 +0,0 @@
|
|||||||
# -*- coding: utf-8 -*-
|
|
||||||
"""FAISS 后端的 get_embeddings 必须能取回普通 upsert 写入的向量。
|
|
||||||
|
|
||||||
``_data`` 只保存 payload 与内部下标,向量本体在 FAISS index 里。去重和
|
|
||||||
episode 摘要依赖 get_embeddings,取不到向量会让这些能力静默失效。
|
|
||||||
"""
|
|
||||||
|
|
||||||
import asyncio
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
|
|
||||||
pytest.importorskip("faiss")
|
|
||||||
|
|
||||||
from hy_memory.config import MemoryConfig # noqa: E402
|
|
||||||
from hy_memory.data.vector_store_faiss import FaissVectorStore # noqa: E402
|
|
||||||
from hy_memory.models.memory import MemoryLayer, MemoryNode # noqa: E402
|
|
||||||
|
|
||||||
|
|
||||||
def _config(tmp_path):
|
|
||||||
return MemoryConfig.from_dict({
|
|
||||||
"vector_store": {
|
|
||||||
"provider": "faiss",
|
|
||||||
"collection_name": "get_embeddings_test",
|
|
||||||
"persist_directory": str(tmp_path / "faiss"),
|
|
||||||
"embedding_dims": 4,
|
|
||||||
},
|
|
||||||
})
|
|
||||||
|
|
||||||
|
|
||||||
def test_get_embeddings_reconstructs_upserted_and_updated_vectors(tmp_path):
|
|
||||||
async def _run():
|
|
||||||
store = FaissVectorStore(_config(tmp_path))
|
|
||||||
await store.initialize()
|
|
||||||
try:
|
|
||||||
node = MemoryNode(
|
|
||||||
user_id="u1",
|
|
||||||
agent_id="a1",
|
|
||||||
session_id="s1",
|
|
||||||
source="default",
|
|
||||||
layer=MemoryLayer.L1_RAW,
|
|
||||||
content="hello",
|
|
||||||
embedding=[1.0, 0.0, 0.0, 0.0],
|
|
||||||
)
|
|
||||||
await store.upsert(node)
|
|
||||||
|
|
||||||
got = await store.get_embeddings([node.node_id])
|
|
||||||
assert got[node.node_id] == pytest.approx([1.0, 0.0, 0.0, 0.0])
|
|
||||||
|
|
||||||
assert await store.update_embedding(node.node_id, [0.0, 1.0, 0.0, 0.0])
|
|
||||||
got = await store.get_embeddings([node.node_id])
|
|
||||||
assert got[node.node_id] == pytest.approx([0.0, 1.0, 0.0, 0.0])
|
|
||||||
|
|
||||||
assert await store.get_embeddings(["missing"]) == {}
|
|
||||||
finally:
|
|
||||||
await store.close()
|
|
||||||
|
|
||||||
asyncio.run(_run())
|
|
||||||
@@ -1,60 +0,0 @@
|
|||||||
# -*- coding: utf-8 -*-
|
|
||||||
"""History queries without agent_ids must cover every agent of the user."""
|
|
||||||
|
|
||||||
import asyncio
|
|
||||||
|
|
||||||
from hy_memory.config import MemoryConfig
|
|
||||||
from hy_memory.data.history_store import HistoryStore
|
|
||||||
|
|
||||||
|
|
||||||
def _store(tmp_path):
|
|
||||||
config = MemoryConfig()
|
|
||||||
config.history.db_path = str(tmp_path / "history.db")
|
|
||||||
return HistoryStore(config)
|
|
||||||
|
|
||||||
|
|
||||||
def test_get_recent_for_user_matches_all_agents(tmp_path):
|
|
||||||
async def _run():
|
|
||||||
store = _store(tmp_path)
|
|
||||||
await store.initialize()
|
|
||||||
try:
|
|
||||||
keys = [
|
|
||||||
"u1:default_agent",
|
|
||||||
"u1:agent_b",
|
|
||||||
"u10:default_agent",
|
|
||||||
"u2:default_agent",
|
|
||||||
]
|
|
||||||
for index, key in enumerate(keys):
|
|
||||||
await store.record_add(
|
|
||||||
memory_id=f"m{index}",
|
|
||||||
content="x",
|
|
||||||
layer="l1_raw",
|
|
||||||
isolation_key=key,
|
|
||||||
)
|
|
||||||
rows = await store.get_recent_for_user("u1", limit=10)
|
|
||||||
finally:
|
|
||||||
await store.close()
|
|
||||||
return [row["isolation_key"] for row in rows]
|
|
||||||
|
|
||||||
assert set(asyncio.run(_run())) == {"u1:default_agent", "u1:agent_b"}
|
|
||||||
|
|
||||||
|
|
||||||
def test_get_recent_for_user_escapes_like_wildcards(tmp_path):
|
|
||||||
async def _run():
|
|
||||||
store = _store(tmp_path)
|
|
||||||
await store.initialize()
|
|
||||||
try:
|
|
||||||
await store.record_add(
|
|
||||||
memory_id="m1", content="x", layer="l1_raw",
|
|
||||||
isolation_key="u_1:default_agent",
|
|
||||||
)
|
|
||||||
await store.record_add(
|
|
||||||
memory_id="m2", content="x", layer="l1_raw",
|
|
||||||
isolation_key="uX1:default_agent",
|
|
||||||
)
|
|
||||||
rows = await store.get_recent_for_user("u_1", limit=10)
|
|
||||||
finally:
|
|
||||||
await store.close()
|
|
||||||
return [row["isolation_key"] for row in rows]
|
|
||||||
|
|
||||||
assert asyncio.run(_run()) == ["u_1:default_agent"]
|
|
||||||
@@ -1,111 +0,0 @@
|
|||||||
# -*- coding: utf-8 -*-
|
|
||||||
"""MySQL minute metrics must merge under one row lock."""
|
|
||||||
|
|
||||||
import asyncio
|
|
||||||
import json
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
|
|
||||||
pytest.importorskip("aiomysql")
|
|
||||||
|
|
||||||
from hy_memory.config import MemoryConfig # noqa: E402
|
|
||||||
from hy_memory.data.cache_mysql import MysqlCache # noqa: E402
|
|
||||||
|
|
||||||
|
|
||||||
class FakeCursor:
|
|
||||||
def __init__(self, row):
|
|
||||||
self.row = row
|
|
||||||
self.executed = []
|
|
||||||
|
|
||||||
async def __aenter__(self):
|
|
||||||
return self
|
|
||||||
|
|
||||||
async def __aexit__(self, *args):
|
|
||||||
return None
|
|
||||||
|
|
||||||
async def execute(self, sql, args=None):
|
|
||||||
self.executed.append((sql, args))
|
|
||||||
|
|
||||||
async def fetchone(self):
|
|
||||||
return self.row
|
|
||||||
|
|
||||||
|
|
||||||
class FakeConnection:
|
|
||||||
def __init__(self, row):
|
|
||||||
self.cursor_obj = FakeCursor(row)
|
|
||||||
self.begun = False
|
|
||||||
self.committed = False
|
|
||||||
self.rolled_back = False
|
|
||||||
|
|
||||||
async def begin(self):
|
|
||||||
self.begun = True
|
|
||||||
|
|
||||||
async def commit(self):
|
|
||||||
self.committed = True
|
|
||||||
|
|
||||||
async def rollback(self):
|
|
||||||
self.rolled_back = True
|
|
||||||
|
|
||||||
def cursor(self, *args, **kwargs):
|
|
||||||
return self.cursor_obj
|
|
||||||
|
|
||||||
|
|
||||||
class _Acquire:
|
|
||||||
def __init__(self, connection):
|
|
||||||
self.connection = connection
|
|
||||||
|
|
||||||
async def __aenter__(self):
|
|
||||||
return self.connection
|
|
||||||
|
|
||||||
async def __aexit__(self, *args):
|
|
||||||
return None
|
|
||||||
|
|
||||||
|
|
||||||
class FakePool:
|
|
||||||
def __init__(self, connection):
|
|
||||||
self.connection = connection
|
|
||||||
|
|
||||||
def acquire(self):
|
|
||||||
return _Acquire(self.connection)
|
|
||||||
|
|
||||||
|
|
||||||
def _cache_with(row):
|
|
||||||
cache = MysqlCache(MemoryConfig())
|
|
||||||
connection = FakeConnection(row)
|
|
||||||
cache._pool = FakePool(connection)
|
|
||||||
return cache, connection
|
|
||||||
|
|
||||||
|
|
||||||
def test_metrics_insert_takes_the_row_lock_and_commits():
|
|
||||||
cache, connection = _cache_with(None)
|
|
||||||
|
|
||||||
asyncio.run(
|
|
||||||
cache.store_metrics_minute("2026-01-01T00:00", {"sys1_started": 2})
|
|
||||||
)
|
|
||||||
|
|
||||||
statements = [sql for sql, _ in connection.cursor_obj.executed]
|
|
||||||
assert any("FOR UPDATE" in sql for sql in statements)
|
|
||||||
assert any(sql.startswith("INSERT") for sql in statements)
|
|
||||||
assert connection.begun and connection.committed
|
|
||||||
assert not connection.rolled_back
|
|
||||||
|
|
||||||
|
|
||||||
def test_metrics_update_merges_existing_buckets():
|
|
||||||
old = {"sys1_started": 1, "sys1_timing_sums": {"a": 1}}
|
|
||||||
cache, connection = _cache_with({"data": json.dumps(old)})
|
|
||||||
|
|
||||||
asyncio.run(
|
|
||||||
cache.store_metrics_minute(
|
|
||||||
"2026-01-01T00:00",
|
|
||||||
{"sys1_started": 2, "sys1_timing_sums": {"a": 3}},
|
|
||||||
)
|
|
||||||
)
|
|
||||||
|
|
||||||
update = next(
|
|
||||||
args for sql, args in connection.cursor_obj.executed
|
|
||||||
if sql.startswith("UPDATE")
|
|
||||||
)
|
|
||||||
merged = json.loads(update[0])
|
|
||||||
assert merged["sys1_started"] == 3
|
|
||||||
assert merged["sys1_timing_sums"]["a"] == 4
|
|
||||||
assert connection.committed
|
|
||||||
@@ -77,28 +77,6 @@ class TestReadRequestWiring:
|
|||||||
HyMemoryClient._normalize_layers(["l3_identity"])
|
HyMemoryClient._normalize_layers(["l3_identity"])
|
||||||
|
|
||||||
|
|
||||||
class TestDedupKindPairing:
|
|
||||||
"""dedup 的 _memory_kind 必须与它的 public item 来自同一条 reader 结果。"""
|
|
||||||
|
|
||||||
def test_memory_kind_follows_source_after_layer_filter(self):
|
|
||||||
memories = [
|
|
||||||
{"memory_id": "raw", "layer": "l1_raw", "content": "raw"},
|
|
||||||
{
|
|
||||||
"memory_id": "canon",
|
|
||||||
"layer": "l2_fact",
|
|
||||||
"content": "canon",
|
|
||||||
"_memory_kind": "canonical_state",
|
|
||||||
},
|
|
||||||
]
|
|
||||||
all_mems, dedup_mems = HyMemoryClient._public_search_views(
|
|
||||||
memories,
|
|
||||||
wanted_layers={"l2_fact"},
|
|
||||||
lite_mode=False,
|
|
||||||
)
|
|
||||||
assert [m["memory_id"] for m in all_mems] == ["canon"]
|
|
||||||
assert dedup_mems[0]["_memory_kind"] == "canonical_state"
|
|
||||||
|
|
||||||
|
|
||||||
class TestHybridDirectedRecall:
|
class TestHybridDirectedRecall:
|
||||||
"""共享 reader 主流程必须把 request.layers 下推到召回阶段。"""
|
"""共享 reader 主流程必须把 request.layers 下推到召回阶段。"""
|
||||||
|
|
||||||
|
|||||||
@@ -1,76 +0,0 @@
|
|||||||
# -*- coding: utf-8 -*-
|
|
||||||
"""HTTP server signal shutdown must not deadlock the serving thread."""
|
|
||||||
|
|
||||||
import subprocess
|
|
||||||
import sys
|
|
||||||
import textwrap
|
|
||||||
import threading
|
|
||||||
import time
|
|
||||||
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
|
||||||
|
|
||||||
from hy_memory.server import _request_shutdown
|
|
||||||
|
|
||||||
|
|
||||||
class _Handler(BaseHTTPRequestHandler):
|
|
||||||
def do_GET(self): # pragma: no cover - the test only needs a live server
|
|
||||||
self.send_response(200)
|
|
||||||
self.end_headers()
|
|
||||||
|
|
||||||
def log_message(self, *args): # pragma: no cover - silence test output
|
|
||||||
pass
|
|
||||||
|
|
||||||
|
|
||||||
def test_request_shutdown_returns_promptly():
|
|
||||||
server = ThreadingHTTPServer(("127.0.0.1", 0), _Handler)
|
|
||||||
server.daemon_threads = True
|
|
||||||
thread = threading.Thread(target=server.serve_forever)
|
|
||||||
thread.start()
|
|
||||||
try:
|
|
||||||
started = time.monotonic()
|
|
||||||
_request_shutdown(server)
|
|
||||||
assert time.monotonic() - started < 1.0
|
|
||||||
thread.join(timeout=2.0)
|
|
||||||
assert not thread.is_alive()
|
|
||||||
finally:
|
|
||||||
server.server_close()
|
|
||||||
|
|
||||||
|
|
||||||
def test_signal_handler_does_not_deadlock_the_serving_thread():
|
|
||||||
"""A signal handler calling shutdown() directly hangs forever.
|
|
||||||
|
|
||||||
Run the serving loop on the main thread (as run_server does), fire an
|
|
||||||
alarm, and require the process to exit on its own.
|
|
||||||
"""
|
|
||||||
script = textwrap.dedent(
|
|
||||||
"""
|
|
||||||
import signal
|
|
||||||
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
|
||||||
from hy_memory.server import _request_shutdown
|
|
||||||
|
|
||||||
class H(BaseHTTPRequestHandler):
|
|
||||||
def do_GET(self):
|
|
||||||
self.send_response(200)
|
|
||||||
self.end_headers()
|
|
||||||
def log_message(self, *args):
|
|
||||||
pass
|
|
||||||
|
|
||||||
server = ThreadingHTTPServer(("127.0.0.1", 0), H)
|
|
||||||
|
|
||||||
def _handler(signum, frame):
|
|
||||||
_request_shutdown(server)
|
|
||||||
|
|
||||||
signal.signal(signal.SIGALRM, _handler)
|
|
||||||
signal.alarm(1)
|
|
||||||
server.serve_forever()
|
|
||||||
server.server_close()
|
|
||||||
print("exited")
|
|
||||||
"""
|
|
||||||
)
|
|
||||||
result = subprocess.run(
|
|
||||||
[sys.executable, "-c", script],
|
|
||||||
capture_output=True,
|
|
||||||
text=True,
|
|
||||||
timeout=10,
|
|
||||||
)
|
|
||||||
assert result.returncode == 0, result.stderr
|
|
||||||
assert "exited" in result.stdout
|
|
||||||
-70
@@ -1,70 +0,0 @@
|
|||||||
# 新鲜度(memory evidence freshness)拆除 TODO
|
|
||||||
|
|
||||||
状态:PR1 已开(读路径短路)——分支 `chore/retire-memory-freshness`。#86 不合并。
|
|
||||||
|
|
||||||
## 结论
|
|
||||||
|
|
||||||
- 新鲜度 = 写记忆时记下 `support_refs` 指纹,读时在 MemHub SQLite 里逐条核对 evidence 是否仍匹配;对不上则滤掉该 memory。
|
|
||||||
- 这是过度设计:读路径成本高、失败面大(曾导致记忆库整页 503),收益仅是「证据被改过就藏记忆」。
|
|
||||||
- 业务层为喂 SDK 再做长证据分段(`segment_*`),与 SDK 自身阶段/窗口处理重复;#86 只是继续兼容这套 proof,不应合入。
|
|
||||||
|
|
||||||
## 范围(MemHub Core,非 SDK)
|
|
||||||
|
|
||||||
权威库:`~/.one-memory/one-memory.sqlite3`
|
|
||||||
不涉及 SDK `hy-memory/**` 内部 store。
|
|
||||||
|
|
||||||
### 读路径(优先拆)
|
|
||||||
|
|
||||||
| 位置 | 作用 |
|
|
||||||
|---|---|
|
|
||||||
| `memhub/apps/core/src/one_memory/memory_evidence_freshness.py` | UDF `is_memory_evidence_current`、proof prefetch、预算跳过 |
|
|
||||||
| `source_read_access.py` | 列表/过滤时 per-memory freshness |
|
|
||||||
| `context_memory.py` | search/context 候选行 `_memory_row_is_fresh` |
|
|
||||||
| `memory_library.py` / `home_summary.py` | 记忆库 / 首页可读谓词 |
|
|
||||||
| `memory_visibility.py` | 可见性缓存与 freshness 注册 |
|
|
||||||
| `call_event_references.py` | 引用解析时的 freshness |
|
|
||||||
|
|
||||||
### 写路径 / 数据(后拆,可分 PR)
|
|
||||||
|
|
||||||
| 位置 | 作用 |
|
|
||||||
|---|---|
|
|
||||||
| `memory_tasks.py` → `MemoryTaskOutput.support_refs` / `TaskEvidenceRef` | 抽取输出里的证明 |
|
|
||||||
| `memory_task_outputs` 表 | proof 持久化 |
|
|
||||||
| `memory_processing.py` 分段(`segment_evidence_record` / `_semantic_segment_spans`) | 为 HY 窗口切段;与 freshness 耦合但不全是 freshness——分段是否保留需单独拍板 |
|
|
||||||
|
|
||||||
### 测试(随拆除改/删)
|
|
||||||
|
|
||||||
- `tests/test_memory_evidence_freshness.py`
|
|
||||||
- `tests/test_memory_freshness_integration_review.py`
|
|
||||||
- `tests/test_read_path_freshness_batch.py`
|
|
||||||
- 以及 mock `register_memory_freshness` 的 library / context / visibility / home_summary / call_event 测试
|
|
||||||
|
|
||||||
## 建议落地步骤
|
|
||||||
|
|
||||||
1. **读路径短路(最小 PR)** — 本 PR
|
|
||||||
`register_memory_freshness` 恒返回 `"1"`(UDF 恒 `1`),`freshness_proofs_prefetched` 为空操作。行为:证据改过也不再藏 memory;畸形/segment proof 不再 503。
|
|
||||||
同步改「期望 stale=0」的单测为「始终可读」。
|
|
||||||
|
|
||||||
2. **拆调用点**
|
|
||||||
去掉 `AND is_memory_evidence_current(...)`、`freshness_proofs_prefetched`、`_memory_row_is_fresh` 等死代码,避免假接口长期残留。
|
|
||||||
|
|
||||||
3. **停写 / 弃用 proof(可选第二 PR)**
|
|
||||||
新任务不再依赖 `support_refs` 做读决策;表与历史行可先保留只读,再决定是否 migration 清理。
|
|
||||||
|
|
||||||
4. **分段策略另议**
|
|
||||||
长证据切段是 ingest 窗口问题,不是新鲜度问题。拆除 freshness 后单独决定:继续由 MemHub 切 messages,还是交给 SDK。
|
|
||||||
|
|
||||||
5. **对外沟通**
|
|
||||||
- #86:已留言不合并、将拆除设计。
|
|
||||||
- 相关 issue(如 #74 里 freshness 503)关闭或改指向「拆除」而非「修 segment」。
|
|
||||||
|
|
||||||
## 验收
|
|
||||||
|
|
||||||
- 记忆库 / search / context 在存在「证据已修订 + 旧 proof」时仍返回该 memory,且不再因 malformed/segment ref 503。
|
|
||||||
- 无 `ONE_MEMORY_FRESHNESS_*` 行为依赖;文档/env 若有提及一并删。
|
|
||||||
- Core 相关测试绿。
|
|
||||||
|
|
||||||
## 非目标
|
|
||||||
|
|
||||||
- 不改 SDK 内部抽取/存储。
|
|
||||||
- 不在本轮做「用别的一致性模型替换 freshness」;先删,再按需重设计。
|
|
||||||
Reference in new issue
Block a user