Compare commits
8
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
f0f3d81ca8 | ||
|
|
405f49bfb7 | ||
|
|
2a4dc2ffa2 | ||
|
|
61b5ef30a9 | ||
|
|
f79770cdb0 | ||
|
|
9ba602ffd2 | ||
|
|
842fbe6044 | ||
|
|
268ad16d26 |
No files matched your search
@@ -102,3 +102,65 @@ jobs:
|
||||
if-no-files-found: error
|
||||
compression-level: 0
|
||||
retention-days: 14
|
||||
|
||||
publish-draft-release:
|
||||
if: ${{ inputs.run_lifecycle && startsWith(github.ref, 'refs/tags/v') }}
|
||||
needs: matrix
|
||||
runs-on: ubuntu-24.04
|
||||
permissions:
|
||||
contents: write
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262
|
||||
with:
|
||||
persist-credentials: false
|
||||
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093
|
||||
with:
|
||||
pattern: memhub-runtime-linux-*
|
||||
path: memhub/release-assets
|
||||
merge-multiple: true
|
||||
- name: Verify tag and Linux release assets
|
||||
shell: bash
|
||||
run: |
|
||||
set -euo pipefail
|
||||
version=$(node -p 'require("./package.json").version')
|
||||
node scripts/verify-release-version.mjs \
|
||||
--channel stable \
|
||||
--version "$version" \
|
||||
--build-number "$(node -p 'require("./package.json").release.build_number')" \
|
||||
--tag "$GITHUB_REF_NAME"
|
||||
for arch in x64 arm64; do
|
||||
archive="release-assets/memhub-runtime-v${version}-linux-${arch}.tar.gz"
|
||||
test -f "$archive"
|
||||
test -f "$archive.sha256"
|
||||
(
|
||||
cd release-assets
|
||||
sha256sum -c "$(basename "$archive.sha256")"
|
||||
)
|
||||
done
|
||||
- name: Upload Linux assets to a draft GitHub Release
|
||||
env:
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
shell: bash
|
||||
run: |
|
||||
set -euo pipefail
|
||||
if gh release view "$GITHUB_REF_NAME" \
|
||||
--repo "$GITHUB_REPOSITORY" >/dev/null 2>&1; then
|
||||
test "$(
|
||||
gh release view "$GITHUB_REF_NAME" \
|
||||
--repo "$GITHUB_REPOSITORY" \
|
||||
--json isDraft \
|
||||
--jq .isDraft
|
||||
)" = "true"
|
||||
else
|
||||
gh release create "$GITHUB_REF_NAME" \
|
||||
--repo "$GITHUB_REPOSITORY" \
|
||||
--verify-tag \
|
||||
--draft \
|
||||
--generate-notes \
|
||||
--title "$GITHUB_REF_NAME"
|
||||
fi
|
||||
gh release upload "$GITHUB_REF_NAME" \
|
||||
release-assets/memhub-runtime-v*-linux-*.tar.gz \
|
||||
release-assets/memhub-runtime-v*-linux-*.tar.gz.sha256 \
|
||||
--repo "$GITHUB_REPOSITORY" \
|
||||
--clobber
|
||||
+33
-8
@@ -2,6 +2,22 @@
|
||||
|
||||
Hy Memory and MemHub are versioned and released independently.
|
||||
|
||||
## Unreleased
|
||||
|
||||
- Removed MemHub's task-scoped HyPermission store wrappers and global
|
||||
model-content consent switch. The trusted embedded Hy Memory SDK now uses
|
||||
its native local stores directly.
|
||||
- Existing installations ignore the retired config flag, readmit previously
|
||||
parked pre-invocation work, and drop only the obsolete
|
||||
`hy_node_permissions` proof table. Evidence, governed memories, model
|
||||
profiles, SDK stores, and source/host grants are preserved.
|
||||
|
||||
## Hy Memory 2.0.2 — 2026-10-17
|
||||
|
||||
- Published
|
||||
[`hy-memory==2.0.2`](https://pypi.org/project/hy-memory/2.0.2/) with the
|
||||
current SDK fixes and Apache-2.0 package metadata.
|
||||
|
||||
## Hy Memory 2.0.1 — 2026-10-16
|
||||
|
||||
- Published the Python distribution as
|
||||
@@ -10,13 +26,22 @@ Hy Memory and MemHub are versioned and released independently.
|
||||
Because PyPI release metadata is immutable, the next SDK patch release must
|
||||
carry the updated license metadata; `2.0.1` retains its published metadata.
|
||||
|
||||
## MemHub
|
||||
## MemHub 1.0.2 — Unreleased
|
||||
|
||||
MemHub has no public release yet. The source version is `1.0.0` (build 4), the
|
||||
selected npm package is `memhub-cli`, and both npm publication and GitHub
|
||||
Release artifacts are still pending. Internal release-candidate builds are not
|
||||
public releases.
|
||||
- Updated the source version to `1.0.2` (build 6).
|
||||
- Pinned the public SDK to `hy-memory==2.0.2`.
|
||||
- Pinned `onnxruntime==1.23.2` to retain macOS x64 packaging support.
|
||||
- Fixed relaunch and upgrade failures when a prior managed Runtime exists.
|
||||
- Embedded canonical build identity when an external packaging pipeline omits
|
||||
release environment fields.
|
||||
- Added automatic Linux Runtime upload to tagged Draft Releases.
|
||||
|
||||
No compatibility or deprecation policy has been adopted beyond the explicit
|
||||
CLI/Runtime protocol ranges stored in the release manifests. Future public
|
||||
releases must record user-visible changes and supported platforms here.
|
||||
## MemHub 1.0.1 — Superseded before release
|
||||
|
||||
- The candidate tag was not published as a GitHub Release; `1.0.2` replaces it
|
||||
after the managed-Runtime relaunch fix.
|
||||
|
||||
## MemHub 1.0.0
|
||||
|
||||
- Published the initial GitHub Release with macOS arm64 and Linux Runtime
|
||||
artifacts.
|
||||
@@ -32,7 +32,7 @@ English | [中文](README.zh.md)
|
||||
|
||||
## Latest News
|
||||
|
||||
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-10]**: 🚀 **Released** **Hy Memory 2.0 (`hy-memory==2.0.1`)** on [PyPI](https://pypi.org/project/hy-memory/2.0.1/) and open-sourced the **Hy-Memory** monorepo together with **[MemHub](memhub/README.md)**!
|
||||
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-10]**: 🚀 **Released** **Hy Memory 2.0 (`hy-memory==2.0.2`)** on [PyPI](https://pypi.org/project/hy-memory/2.0.2/) and open-sourced the **Hy-Memory** monorepo together with **[MemHub](memhub/README.md)**!
|
||||
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-10]**: 🚀 **Open-sourced** **[Hy-Memory-30B-A3B](https://huggingface.co/tencent/Hy-Memory-30B-A3B)** on Hugging Face along with its [technical report](https://arxiv.org/abs/2610.22196) on attributed reinforcement learning for long-term memory writing!
|
||||
* *<mark>[new]</mark>* 🔥🔥 **[2026-09]**: 📊 **Released** **[CMB: A Coding Memory Benchmark for Long-Horizon and Interleaved Coding Agents](docs/papers/CMB_A_Coding_Memory_Benchmark_for_Long_Horizon_and_Interleaved_Coding_Agents.pdf)** (submitted to **ICLR 2027**).
|
||||
* *<mark>[new]</mark>* 🔥🔥 **[2026-08]**: 🎉 **Accepted** at **EMNLP 2026**: **[Memory Beyond Recall: A Dual-Process Cognitive Memory System for Self-Evolving LLM Agents](https://arxiv.org/abs/2606.09483)**!
|
||||
@@ -63,13 +63,13 @@ Welcome to the **Hy Memory Research Family**, covering cognitive chat memory, lo
|
||||
### 1. Installation
|
||||
|
||||
```bash
|
||||
pip install hy-memory==2.0.1 # core (includes ChromaDB and Kuzu)
|
||||
pip install "hy-memory[qdrant]==2.0.1" # + Qdrant
|
||||
pip install "hy-memory[faiss]==2.0.1" # + FAISS
|
||||
pip install "hy-memory[graph]==2.0.1" # + Neo4j
|
||||
pip install "hy-memory[mysql]==2.0.1" # + MySQL cache
|
||||
pip install "hy-memory[rerank]==2.0.1" # + local multilingual cross-encoder rerank
|
||||
pip install "hy-memory[all]==2.0.1" # everything
|
||||
pip install hy-memory==2.0.2 # core (includes ChromaDB and Kuzu)
|
||||
pip install "hy-memory[qdrant]==2.0.2" # + Qdrant
|
||||
pip install "hy-memory[faiss]==2.0.2" # + FAISS
|
||||
pip install "hy-memory[graph]==2.0.2" # + Neo4j
|
||||
pip install "hy-memory[mysql]==2.0.2" # + MySQL cache
|
||||
pip install "hy-memory[rerank]==2.0.2" # + local multilingual cross-encoder rerank
|
||||
pip install "hy-memory[all]==2.0.2" # everything
|
||||
```
|
||||
|
||||
### 2. Configuration
|
||||
|
||||
+8
-8
@@ -32,7 +32,7 @@
|
||||
|
||||
## 最新动态
|
||||
|
||||
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-10]**: 🚀 **正式发布** **Hy Memory 2.0 (`hy-memory==2.0.1`)**(已上线 [PyPI](https://pypi.org/project/hy-memory/2.0.1/)),并开源 **Hy-Memory** monorepo 与 **[MemHub](memhub/README.md)**!
|
||||
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-10]**: 🚀 **正式发布** **Hy Memory 2.0 (`hy-memory==2.0.2`)**(已上线 [PyPI](https://pypi.org/project/hy-memory/2.0.2/)),并开源 **Hy-Memory** monorepo 与 **[MemHub](memhub/README.md)**!
|
||||
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-10]**: 🚀 **模型开源** 在 Hugging Face 发布 **[Hy-Memory-30B-A3B](https://huggingface.co/tencent/Hy-Memory-30B-A3B)** 长期记忆写入模型及[技术报告](https://arxiv.org/abs/2610.22196)!
|
||||
* *<mark>[new]</mark>* 🔥🔥 **[2026-09]**: 📊 **基准与方法发布** 推出长周期与交错式编程记忆评测基准 **[CMB: A Coding Memory Benchmark for Long-Horizon and Interleaved Coding Agents](docs/papers/CMB_A_Coding_Memory_Benchmark_for_Long_Horizon_and_Interleaved_Coding_Agents.pdf)**(已投稿 **ICLR 2027**)。
|
||||
* *<mark>[new]</mark>* 🔥🔥 **[2026-08]**: 🎉 **论文录用** 双系统认知记忆架构论文 **[Memory Beyond Recall: A Dual-Process Cognitive Memory System for Self-Evolving LLM Agents](https://arxiv.org/abs/2606.09483)** 被 **EMNLP 2026** 接收!
|
||||
@@ -63,13 +63,13 @@
|
||||
### 1. 安装
|
||||
|
||||
```bash
|
||||
pip install hy-memory==2.0.1 # 核心(含 ChromaDB 与 Kuzu)
|
||||
pip install "hy-memory[qdrant]==2.0.1" # + Qdrant
|
||||
pip install "hy-memory[faiss]==2.0.1" # + FAISS
|
||||
pip install "hy-memory[graph]==2.0.1" # + Neo4j
|
||||
pip install "hy-memory[mysql]==2.0.1" # + MySQL 缓存
|
||||
pip install "hy-memory[rerank]==2.0.1" # + 本地多语言 cross-encoder rerank
|
||||
pip install "hy-memory[all]==2.0.1" # 全部
|
||||
pip install hy-memory==2.0.2 # 核心(含 ChromaDB 与 Kuzu)
|
||||
pip install "hy-memory[qdrant]==2.0.2" # + Qdrant
|
||||
pip install "hy-memory[faiss]==2.0.2" # + FAISS
|
||||
pip install "hy-memory[graph]==2.0.2" # + Neo4j
|
||||
pip install "hy-memory[mysql]==2.0.2" # + MySQL 缓存
|
||||
pip install "hy-memory[rerank]==2.0.2" # + 本地多语言 cross-encoder rerank
|
||||
pip install "hy-memory[all]==2.0.2" # 全部
|
||||
```
|
||||
|
||||
### 2. 配置
|
||||
|
||||
+1
-1
@@ -37,7 +37,7 @@ do not belong in the public source tree.
|
||||
- Root GitHub workflows run SDK and MemHub checks from their respective working
|
||||
directories. Native lifecycle workflows produce reviewable Actions artifacts;
|
||||
tag releases remain fail-closed on public identity, signing and notarization.
|
||||
- Hy Memory is published as `hy-memory==2.0.1`. MemHub pins that public
|
||||
- Hy Memory is published as `hy-memory==2.0.2`. MemHub pins that public
|
||||
distribution; its selected npm package is `memhub-cli`, which is not yet
|
||||
published.
|
||||
- Hy-Memory source uses Apache-2.0 at the repository root. Third-party
|
||||
|
||||
+1
-1
@@ -5,7 +5,7 @@
|
||||
## 安装与快速开始
|
||||
|
||||
```bash
|
||||
pip install hy-memory==2.0.1
|
||||
pip install hy-memory==2.0.2
|
||||
|
||||
# 可选后端
|
||||
pip install 'hy-memory[qdrant]==2.0.1'
|
||||
|
||||
@@ -496,11 +496,21 @@ class LLMProvider:
|
||||
clients = self._clients.setdefault(loop, {})
|
||||
client = clients.get(client_key)
|
||||
if client is None:
|
||||
client_options: Dict[str, Any] = {
|
||||
"api_key": base_api_key,
|
||||
"base_url": base_url,
|
||||
"timeout": self._llm_config.timeout,
|
||||
"max_retries": 0,
|
||||
}
|
||||
transport_factory = getattr(self.config, "http_transport_factory", None)
|
||||
if callable(transport_factory):
|
||||
import httpx
|
||||
|
||||
client_options["http_client"] = httpx.AsyncClient(
|
||||
transport=transport_factory()
|
||||
)
|
||||
client = AsyncOpenAI(
|
||||
api_key=base_api_key,
|
||||
base_url=base_url,
|
||||
timeout=self._llm_config.timeout,
|
||||
max_retries=0,
|
||||
**client_options,
|
||||
)
|
||||
clients[client_key] = client
|
||||
|
||||
|
||||
+8
-1
@@ -6,7 +6,7 @@ Agent Memory - 配置管理
|
||||
|
||||
import os
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Optional, Dict, Any, List
|
||||
from typing import Optional, Dict, Any, Callable, List
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
@@ -1252,6 +1252,13 @@ class MemoryConfig:
|
||||
enable_agent: bool = False
|
||||
debug: bool = False
|
||||
metrics_enabled: bool = True
|
||||
# Runtime-only HTTP transport factory. MemHub uses this for request-cost
|
||||
# budgeting without restricting the embedded SDK's local store access.
|
||||
http_transport_factory: Optional[Callable[[], Any]] = field(
|
||||
default=None,
|
||||
repr=False,
|
||||
compare=False,
|
||||
)
|
||||
|
||||
def __post_init__(self):
|
||||
# enable_graph 跟随 mode 走:mode=ultra 时自动启用,无需独立环境变量
|
||||
|
||||
@@ -356,11 +356,21 @@ class EmbedService:
|
||||
key = self._client_cache_key(base_url, api_key, timeout)
|
||||
client = self._client_cache.get(key)
|
||||
if client is None:
|
||||
client_options = {
|
||||
"api_key": api_key,
|
||||
"base_url": base_url or None,
|
||||
"timeout": timeout,
|
||||
"max_retries": 0,
|
||||
}
|
||||
transport_factory = getattr(self.config, "http_transport_factory", None)
|
||||
if callable(transport_factory):
|
||||
import httpx
|
||||
|
||||
client_options["http_client"] = httpx.AsyncClient(
|
||||
transport=transport_factory()
|
||||
)
|
||||
client = AsyncOpenAI(
|
||||
api_key=api_key,
|
||||
base_url=base_url or None,
|
||||
timeout=timeout,
|
||||
max_retries=0,
|
||||
**client_options,
|
||||
)
|
||||
self._client_cache[key] = client
|
||||
return client
|
||||
|
||||
+44
-7
@@ -15,12 +15,12 @@ write、search,并把搜索结果格式化为可交给模型的 memory text ch
|
||||
|
||||
```text
|
||||
distribution: hy-memory
|
||||
version: 2.0.1
|
||||
version: 2.0.2
|
||||
mode: 按运行时配置选择
|
||||
reader: hybrid
|
||||
```
|
||||
|
||||
仓库 pin、lockfile 和运行时版本守卫均固定到公开的 `hy-memory==2.0.1`。
|
||||
仓库 pin、lockfile 和运行时版本守卫均固定到公开的 `hy-memory==2.0.2`。
|
||||
MemHub 在本地 Runtime 内嵌入 SDK,但不绕过自身的来源授权、模型授权、Evidence、
|
||||
Governance 和撤权边界。
|
||||
|
||||
@@ -37,6 +37,40 @@ MemHub 对 SDK 使用以下公开、可复现的本地策略:
|
||||
- API Key 进入系统安全存储,不写入仓库、安装包、日志或 HY metadata;
|
||||
- 服务端数据库、租户队列及宿主 shell 中的 ambient 凭据不会进入 SDK config。
|
||||
|
||||
发布包固定应用以下 SDK 行为配置:
|
||||
|
||||
| 环境变量 | 值 |
|
||||
|---|---|
|
||||
| `MEMORY_MODE` | `ultra` |
|
||||
| `MEMORY_ENABLE_GRAPH` | `true` |
|
||||
| `MEMORY_GRAPH_RERANK_ENABLED` | `false` |
|
||||
| `MEMORY_GRAPH_TEMPORAL_SPINE` | `true` |
|
||||
| `MEMORY_TOPIC_SUMMARY_ENABLED` | `false` |
|
||||
| `MEMORY_EPISODE_SUMMARY_ENABLED` | `false` |
|
||||
| `MEMORY_EPISODE_RECALL_ENABLED` | `true` |
|
||||
| `MEMORY_DYNAMIC_EPISODE_ENABLED` | `true` |
|
||||
| `MEMORY_SESSION_EPISODE_ENABLED` | `false` |
|
||||
| `MEMORY_SESSION_EPISODE_CONTEXT_ENABLED` | `false` |
|
||||
| `MEMORY_EPISODE_CAPSULE_CONTEXT_ENABLED` | `true` |
|
||||
| `MEMORY_EPISODE_CAPSULE_QUERY_MIN` | `0.65` |
|
||||
| `MEMORY_EPISODE_CAPSULE_ASSIGNMENT_MIN` | `0.60` |
|
||||
| `MEMORY_EPISODE_CAPSULE_FIRST_RANK_MAX` | `3` |
|
||||
| `MEMORY_EPISODE_CAPSULE_MEMBER_LIMIT` | `3` |
|
||||
| `MEMORY_ENABLE_INTENTION` | `false` |
|
||||
| `MEMORY_ENABLE_QUERY_REWRITE` | `false` |
|
||||
| `MEMORY_FEW_SHOT_ENABLED` | `true` |
|
||||
| `MEMORY_HYBRID_V2_QUERY_EXPAND` | `1` |
|
||||
| `MEMORY_HYBRID_V2_GEO_INTENT_PRIOR` | `0.6` |
|
||||
| `MEMORY_HYBRID_V2_DIVERSITY` | `1` |
|
||||
| `MEMORY_HYBRID_V2_TEMPORAL_ADMIT_PRIOR` | `0.55` |
|
||||
| `MEMORY_HYBRID_V2_TEMPORAL_BOOST` | `0.18` |
|
||||
| `MEMORY_CROSS_DOMAIN_CON_THRESHOLD` | `0.5` |
|
||||
| `MEMORY_READER_ENABLE_SUMMARY` | `false` |
|
||||
| `MEMORY_SUMMARY_ENABLED_IN_SYS2` | `false` |
|
||||
| `MEMORY_LOCAL_RERANK_ENABLED` | `false` |
|
||||
| `RECONCILE_ENABLED` | `true` |
|
||||
| `HY_MEMORY_READER` | `hybrid` |
|
||||
|
||||
这些默认值集中在
|
||||
`apps/core/src/one_memory/hy_runtime_config.py`,并由发布测试和 lockfile 共同约束。
|
||||
独立脚本若绕过 MemHub 直接导入 SDK,应自行显式配置存储目录和模型端点。
|
||||
@@ -271,9 +305,9 @@ Runtime 应长期复用 client,不为每次请求重新初始化数据库和
|
||||
|
||||
```text
|
||||
Evidence
|
||||
-> ModelProcessingPermissionLease
|
||||
-> HyPermissionTransport
|
||||
-> HyMemoryBridge.ingest / ingest_extracted / search_ids
|
||||
-> HyMemoryBridge
|
||||
-> embedded Hy Memory SDK
|
||||
-> native Vector / Coding / Graph stores
|
||||
-> write result verification
|
||||
-> Governance
|
||||
-> Recall Broker / UI
|
||||
@@ -282,8 +316,11 @@ Evidence
|
||||
不要在新业务模块里直接创建 `HyMemoryClient`,否则会绕过:
|
||||
|
||||
- 来源读取授权;
|
||||
- 模型内容处理授权;
|
||||
- 发送前与响应后的撤权复核;
|
||||
- 共享 Runtime、模型配置、API Key 安全存储与请求预算;
|
||||
- Evidence 引用与作者归属;
|
||||
- 写后回读验证;
|
||||
- Governance 状态、版本、更正和删除级联。
|
||||
|
||||
Hy Memory SDK 是 MemHub 的内置可信组件,可自由访问和管理本机的全部
|
||||
Hy Memory 数据。来源授权仍决定哪些宿主数据可以进入本机;宿主 Recall Grant
|
||||
仍决定哪些外部 AI 应用可以读取治理后的记忆。这两类边界不限制内置 SDK 的本地 CRUD。
|
||||
+6
-4
@@ -6,13 +6,13 @@ SDK 和记忆引擎。One Memory / `one-memory` 是历史模块、数据目录
|
||||
|
||||
## 发行与平台合同
|
||||
|
||||
公共 Python SDK 为 `hy-memory==2.0.1`。MemHub 源码版本为 `1.0.0`
|
||||
(build 4),GitHub 仓库为
|
||||
公共 Python SDK 为 `hy-memory==2.0.2`。MemHub 源码版本为 `1.0.2`
|
||||
(build 6),GitHub 仓库为
|
||||
[`Tencent-Hunyuan/Hy-Memory`](https://github.com/Tencent-Hunyuan/Hy-Memory),npm 包名为
|
||||
`memhub-cli`。当前代码架构见
|
||||
[`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md)。
|
||||
|
||||
`v1.0.0` 的发行目标是 Linux x64/arm64 Runtime 与 macOS arm64/x64 App 和 Runtime。
|
||||
`v1.0.2` 的发行目标是 Linux x64/arm64 Runtime 与 macOS arm64/x64 App 和 Runtime。
|
||||
Linux 制品由 GitHub Actions 在原生架构 runner 上构建并完成生命周期验收;macOS
|
||||
制品由发行流水线完成 Developer ID 签名、公证和验收后手动上传。Windows 不支持。
|
||||
只有同时出现在 GitHub Release 与 `release-manifest.json` 中、且通过公开校验的制品才属于
|
||||
@@ -42,6 +42,8 @@ Linux 制品由 GitHub Actions 在原生架构 runner 上构建并完成生命
|
||||
|
||||
架构详情见 [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md) 和
|
||||
[`docs/contracts/runtime-unification.md`](docs/contracts/runtime-unification.md)。
|
||||
产品、测试和一线支持排查导入与记忆沉淀时,使用
|
||||
[`docs/PRODUCT_LOG_TROUBLESHOOTING.md`](docs/PRODUCT_LOG_TROUBLESHOOTING.md)。
|
||||
|
||||
## 开发环境
|
||||
|
||||
@@ -99,7 +101,7 @@ MemHub 的 SQLite 数据默认位于 `~/.one-memory`,不在 npm 包或 Runtime
|
||||
|
||||
## Hy Memory
|
||||
|
||||
当前开发依赖固定为公开 PyPI 包 `hy-memory==2.0.1`。SDK 配置、权限边界及 search
|
||||
当前开发依赖固定为公开 PyPI 包 `hy-memory==2.0.2`。SDK 配置、权限边界及 search
|
||||
返回格式见 [`MEMORY.md`](MEMORY.md)。HY search 返回的 `content` 已包含 SDK 统一生成的
|
||||
逻辑时间和演化上下文;调用方不应再次转换时间戳或重复拼接 `When`。
|
||||
|
||||
|
||||
+94
-6
@@ -479,6 +479,23 @@
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"SPDXID": "SPDXRef-Package-pypi-coloredlogs",
|
||||
"name": "coloredlogs",
|
||||
"versionInfo": "15.0.1",
|
||||
"downloadLocation": "NOASSERTION",
|
||||
"filesAnalyzed": false,
|
||||
"licenseConcluded": "NOASSERTION",
|
||||
"licenseDeclared": "NOASSERTION",
|
||||
"copyrightText": "NOASSERTION",
|
||||
"externalRefs": [
|
||||
{
|
||||
"referenceCategory": "PACKAGE-MANAGER",
|
||||
"referenceType": "purl",
|
||||
"referenceLocator": "pkg:pypi/coloredlogs@15.0.1"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"SPDXID": "SPDXRef-Package-pypi-distro",
|
||||
"name": "distro",
|
||||
@@ -751,6 +768,23 @@
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"SPDXID": "SPDXRef-Package-pypi-humanfriendly",
|
||||
"name": "humanfriendly",
|
||||
"versionInfo": "10.0",
|
||||
"downloadLocation": "NOASSERTION",
|
||||
"filesAnalyzed": false,
|
||||
"licenseConcluded": "NOASSERTION",
|
||||
"licenseDeclared": "NOASSERTION",
|
||||
"copyrightText": "NOASSERTION",
|
||||
"externalRefs": [
|
||||
{
|
||||
"referenceCategory": "PACKAGE-MANAGER",
|
||||
"referenceType": "purl",
|
||||
"referenceLocator": "pkg:pypi/humanfriendly@10.0"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"SPDXID": "SPDXRef-Package-pypi-idna",
|
||||
"name": "idna",
|
||||
@@ -1125,6 +1159,23 @@
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"SPDXID": "SPDXRef-Package-pypi-mpmath",
|
||||
"name": "mpmath",
|
||||
"versionInfo": "1.3.0",
|
||||
"downloadLocation": "NOASSERTION",
|
||||
"filesAnalyzed": false,
|
||||
"licenseConcluded": "NOASSERTION",
|
||||
"licenseDeclared": "NOASSERTION",
|
||||
"copyrightText": "NOASSERTION",
|
||||
"externalRefs": [
|
||||
{
|
||||
"referenceCategory": "PACKAGE-MANAGER",
|
||||
"referenceType": "purl",
|
||||
"referenceLocator": "pkg:pypi/mpmath@1.3.0"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"SPDXID": "SPDXRef-Package-pypi-multidict",
|
||||
"name": "multidict",
|
||||
@@ -1230,7 +1281,7 @@
|
||||
{
|
||||
"SPDXID": "SPDXRef-Package-pypi-onnxruntime",
|
||||
"name": "onnxruntime",
|
||||
"versionInfo": "1.29.0",
|
||||
"versionInfo": "1.23.2",
|
||||
"downloadLocation": "NOASSERTION",
|
||||
"filesAnalyzed": false,
|
||||
"licenseConcluded": "NOASSERTION",
|
||||
@@ -1240,7 +1291,7 @@
|
||||
{
|
||||
"referenceCategory": "PACKAGE-MANAGER",
|
||||
"referenceType": "purl",
|
||||
"referenceLocator": "pkg:pypi/onnxruntime@1.29.0"
|
||||
"referenceLocator": "pkg:pypi/onnxruntime@1.23.2"
|
||||
}
|
||||
]
|
||||
},
|
||||
@@ -1992,6 +2043,23 @@
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"SPDXID": "SPDXRef-Package-pypi-sympy",
|
||||
"name": "sympy",
|
||||
"versionInfo": "1.14.0",
|
||||
"downloadLocation": "NOASSERTION",
|
||||
"filesAnalyzed": false,
|
||||
"licenseConcluded": "NOASSERTION",
|
||||
"licenseDeclared": "NOASSERTION",
|
||||
"copyrightText": "NOASSERTION",
|
||||
"externalRefs": [
|
||||
{
|
||||
"referenceCategory": "PACKAGE-MANAGER",
|
||||
"referenceType": "purl",
|
||||
"referenceLocator": "pkg:pypi/sympy@1.14.0"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"SPDXID": "SPDXRef-Package-pypi-tenacity",
|
||||
"name": "tenacity",
|
||||
@@ -2267,12 +2335,12 @@
|
||||
{
|
||||
"SPDXID": "SPDXRef-Package-hy-memory",
|
||||
"name": "hy-memory",
|
||||
"versionInfo": "2.0.1",
|
||||
"versionInfo": "2.0.2",
|
||||
"downloadLocation": "NOASSERTION",
|
||||
"filesAnalyzed": false,
|
||||
"licenseConcluded": "MIT",
|
||||
"licenseDeclared": "MIT",
|
||||
"copyrightText": "Copyright (c) 2025 HY Memory Contributors"
|
||||
"licenseConcluded": "Apache-2.0",
|
||||
"licenseDeclared": "Apache-2.0",
|
||||
"copyrightText": "Copyright (C) 2026 Tencent. All rights reserved."
|
||||
},
|
||||
{
|
||||
"SPDXID": "SPDXRef-Package-memmy-agent-derived",
|
||||
@@ -2427,6 +2495,11 @@
|
||||
"relationshipType": "DEPENDS_ON",
|
||||
"relatedSpdxElement": "SPDXRef-Package-pypi-cloudpickle"
|
||||
},
|
||||
{
|
||||
"spdxElementId": "SPDXRef-Package-one-memory",
|
||||
"relationshipType": "DEPENDS_ON",
|
||||
"relatedSpdxElement": "SPDXRef-Package-pypi-coloredlogs"
|
||||
},
|
||||
{
|
||||
"spdxElementId": "SPDXRef-Package-one-memory",
|
||||
"relationshipType": "DEPENDS_ON",
|
||||
@@ -2507,6 +2580,11 @@
|
||||
"relationshipType": "DEPENDS_ON",
|
||||
"relatedSpdxElement": "SPDXRef-Package-pypi-huggingface-hub"
|
||||
},
|
||||
{
|
||||
"spdxElementId": "SPDXRef-Package-one-memory",
|
||||
"relationshipType": "DEPENDS_ON",
|
||||
"relatedSpdxElement": "SPDXRef-Package-pypi-humanfriendly"
|
||||
},
|
||||
{
|
||||
"spdxElementId": "SPDXRef-Package-one-memory",
|
||||
"relationshipType": "DEPENDS_ON",
|
||||
@@ -2617,6 +2695,11 @@
|
||||
"relationshipType": "DEPENDS_ON",
|
||||
"relatedSpdxElement": "SPDXRef-Package-pypi-more-itertools"
|
||||
},
|
||||
{
|
||||
"spdxElementId": "SPDXRef-Package-one-memory",
|
||||
"relationshipType": "DEPENDS_ON",
|
||||
"relatedSpdxElement": "SPDXRef-Package-pypi-mpmath"
|
||||
},
|
||||
{
|
||||
"spdxElementId": "SPDXRef-Package-one-memory",
|
||||
"relationshipType": "DEPENDS_ON",
|
||||
@@ -2872,6 +2955,11 @@
|
||||
"relationshipType": "DEPENDS_ON",
|
||||
"relatedSpdxElement": "SPDXRef-Package-pypi-starlette"
|
||||
},
|
||||
{
|
||||
"spdxElementId": "SPDXRef-Package-one-memory",
|
||||
"relationshipType": "DEPENDS_ON",
|
||||
"relatedSpdxElement": "SPDXRef-Package-pypi-sympy"
|
||||
},
|
||||
{
|
||||
"spdxElementId": "SPDXRef-Package-one-memory",
|
||||
"relationshipType": "DEPENDS_ON",
|
||||
|
||||
@@ -9,32 +9,72 @@ Per-package license texts are not reproduced here yet; the license
|
||||
column is filled in as part of release review, and the SBOM records
|
||||
`NOASSERTION` until then rather than guessing.
|
||||
|
||||
## hy-memory 2.0.1
|
||||
## hy-memory 2.0.2
|
||||
|
||||
License: MIT
|
||||
License: Apache-2.0
|
||||
|
||||
```text
|
||||
MIT License
|
||||
Tencent is pleased to support the community by making Hy-Memory available.
|
||||
|
||||
Copyright (c) 2025 HY Memory Contributors
|
||||
Copyright (C) 2026 Tencent. All rights reserved.
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
Hy-Memory is licensed under Apache-2.0. Hy-Memory does not impose any additional restrictions beyond those specified in the license.
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
For the avoidance of doubt, Hy-Memory refers solely to code, parameters, and weights made publicly available by Tencent in accordance with Apache-2.0.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
Terms of the Apache-2.0 License:
|
||||
--------------------------------------------------------------------
|
||||
Apache License
|
||||
Version 2.0, January 2004
|
||||
http://www.apache.org/licenses/
|
||||
|
||||
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
||||
|
||||
1. Definitions.
|
||||
|
||||
"License" shall mean the terms and conditions for use, reproduction, and distribution as defined by Sections 1 through 9 of this document.
|
||||
|
||||
"Licensor" shall mean the copyright owner or entity authorized by the copyright owner that is granting the License.
|
||||
|
||||
"Legal Entity" shall mean the union of the acting entity and all other entities that control, are controlled by, or are under common control with that entity. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity.
|
||||
|
||||
"You" (or "Your") shall mean an individual or Legal Entity exercising permissions granted by this License.
|
||||
|
||||
"Source" form shall mean the preferred form for making modifications, including but not limited to software source code, documentation source, and configuration files.
|
||||
|
||||
"Object" form shall mean any form resulting from mechanical transformation or translation of a Source form, including but not limited to compiled object code, generated documentation, and conversions to other media types.
|
||||
|
||||
"Work" shall mean the work of authorship, whether in Source or Object form, made available under the License, as indicated by a copyright notice that is included in or attached to the work (an example is provided in the Appendix below).
|
||||
|
||||
"Derivative Works" shall mean any work, whether in Source or Object form, that is based on (or derived from) the Work and for which the editorial revisions, annotations, elaborations, or other modifications represent, as a whole, an original work of authorship. For the purposes of this License, Derivative Works shall not include works that remain separable from, or merely link (or bind by name) to the interfaces of, the Work and Derivative Works thereof.
|
||||
|
||||
"Contribution" shall mean any work of authorship, including the original version of the Work and any modifications or additions to that Work or Derivative Works thereof, that is intentionally submitted to Licensor for inclusion in the Work by the copyright owner or by an individual or Legal Entity authorized to submit on behalf of the copyright owner. For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to the Licensor or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, the Licensor for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by the copyright owner as "Not a Contribution."
|
||||
|
||||
"Contributor" shall mean Licensor and any individual or Legal Entity on behalf of whom a Contribution has been received by Licensor and subsequently incorporated within the Work.
|
||||
|
||||
2. Grant of Copyright License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in Source or Object form.
|
||||
|
||||
3. Grant of Patent License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by such Contributor that are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s) with the Work to which such Contribution(s) was submitted. If You institute patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Work or a Contribution incorporated within the Work constitutes direct or contributory patent infringement, then any patent licenses granted to You under this License for that Work shall terminate as of the date such litigation is filed.
|
||||
|
||||
4. Redistribution. You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications, and in Source or Object form, provided that You meet the following conditions:
|
||||
|
||||
You must give any other recipients of the Work or Derivative Works a copy of this License; and
|
||||
You must cause any modified files to carry prominent notices stating that You changed the files; and
|
||||
You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works; and
|
||||
If the Work includes a "NOTICE" text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of the following places: within a NOTICE text file distributed as part of the Derivative Works; within the Source form or documentation, if provided along with the Derivative Works; or, within a display generated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of the NOTICE file are for informational purposes only and do not modify the License. You may add Your own attribution notices within Derivative Works that You distribute, alongside or as an addendum to the NOTICE text from the Work, provided that such additional attribution notices cannot be construed as modifying the License.
|
||||
You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work otherwise complies with the conditions stated in this License.
|
||||
|
||||
5. Submission of Contributions. Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions.
|
||||
|
||||
6. Trademarks. This License does not grant permission to use the trade names, trademarks, service marks, or product names of the Licensor, except as required for reasonable and customary use in describing the origin of the Work and reproducing the content of the NOTICE file.
|
||||
|
||||
7. Disclaimer of Warranty. Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License.
|
||||
|
||||
8. Limitation of Liability. In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall any Contributor be liable to You for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Work (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Contributor has been advised of the possibility of such damages.
|
||||
|
||||
9. Accepting Warranty or Additional Liability. While redistributing the Work or Derivative Works thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other liability obligations and/or rights consistent with this License. However, in accepting such obligations, You may act only on Your own behalf and on Your sole responsibility, not on behalf of any other Contributor, and only if You agree to indemnify, defend, and hold each Contributor harmless for any liability incurred by, or claims asserted against, such Contributor by reason of your accepting any such warranty or additional liability.
|
||||
|
||||
END OF TERMS AND CONDITIONS
|
||||
```
|
||||
|
||||
## MemTensor memmy-agent (adapted)
|
||||
@@ -85,7 +125,7 @@ No Memmy brand, icon, font or mascot assets are distributed.
|
||||
- tar 7.5.22
|
||||
- three 0.185.1
|
||||
|
||||
## Python (PyPI) — 122 packages
|
||||
## Python (PyPI) — 126 packages
|
||||
|
||||
- aiohappyeyeballs 2.7.1
|
||||
- aiohttp 3.14.3
|
||||
@@ -104,6 +144,7 @@ No Memmy brand, icon, font or mascot assets are distributed.
|
||||
- chromadb 1.5.9
|
||||
- click 8.5.0
|
||||
- cloudpickle 3.1.2
|
||||
- coloredlogs 15.0.1
|
||||
- distro 1.9.0
|
||||
- durationpy 0.11
|
||||
- execnet 2.1.2
|
||||
@@ -120,6 +161,7 @@ No Memmy brand, icon, font or mascot assets are distributed.
|
||||
- httptools 0.8.0
|
||||
- httpx 0.28.1
|
||||
- huggingface-hub 1.29.0
|
||||
- humanfriendly 10.0
|
||||
- idna 3.19
|
||||
- importlib-resources 7.1.0
|
||||
- iniconfig 2.3.0
|
||||
@@ -142,13 +184,14 @@ No Memmy brand, icon, font or mascot assets are distributed.
|
||||
- mdurl 0.1.2
|
||||
- mmh3 5.3.0
|
||||
- more-itertools 11.1.0
|
||||
- mpmath 1.3.0
|
||||
- multidict 6.7.1
|
||||
- mypy 2.3.1
|
||||
- mypy-extensions 1.1.0
|
||||
- narwhals 2.25.0
|
||||
- numpy 2.5.2
|
||||
- oauthlib 3.3.1
|
||||
- onnxruntime 1.29.0
|
||||
- onnxruntime 1.23.2
|
||||
- openai 2.54.0
|
||||
- opentelemetry-api 1.44.0
|
||||
- opentelemetry-exporter-otlp-proto-common 1.44.0
|
||||
@@ -193,6 +236,7 @@ No Memmy brand, icon, font or mascot assets are distributed.
|
||||
- six 1.17.0
|
||||
- sniffio 1.3.1
|
||||
- starlette 1.6.0
|
||||
- sympy 1.14.0
|
||||
- tenacity 9.1.4
|
||||
- threadpoolctl 3.6.0
|
||||
- tiktoken 0.14.0
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "@one-memory/cli",
|
||||
"version": "1.0.0",
|
||||
"version": "1.0.2",
|
||||
"description": "MemHub bootstrap CLI for installing and managing the local-first shared memory runtime",
|
||||
"license": "Apache-2.0",
|
||||
"repository": {
|
||||
|
||||
@@ -32,7 +32,6 @@ const REQUIRED_COMMAND_SURFACE = [
|
||||
"calls",
|
||||
"import-file",
|
||||
"notes",
|
||||
"permission",
|
||||
"extraction",
|
||||
"graph",
|
||||
"models",
|
||||
@@ -77,7 +76,7 @@ const REQUIRED_MARKERS = [
|
||||
|
||||
/**
|
||||
* Sub-commands the UI-write parity work added. A package can declare the
|
||||
* parent command (`permission`, `models`) while still missing the new verb, so
|
||||
* parent command (`models`) while still missing the new verb, so
|
||||
* checking command names alone would pass a stale bundle. The marker is the
|
||||
* exact literal the catalog usage string contains.
|
||||
*/
|
||||
@@ -88,9 +87,6 @@ const REQUIRED_SUBCOMMANDS = [
|
||||
{ command: "import-file", marker: "memhub import-file discard" },
|
||||
{ command: "notes", marker: "memhub notes add" },
|
||||
{ command: "notes", marker: "memhub notes process" },
|
||||
{ command: "permission", marker: "memhub permission status" },
|
||||
{ command: "permission", marker: "memhub permission allow" },
|
||||
{ command: "permission", marker: "memhub permission deny" },
|
||||
{ command: "extraction", marker: "memhub extraction set" },
|
||||
{ command: "graph", marker: "memhub graph width" },
|
||||
{ command: "models", marker: "memhub models edit" },
|
||||
@@ -153,8 +149,6 @@ const STOPPED_DESTRUCTIVE_PROBES = [
|
||||
"--confirm",
|
||||
],
|
||||
},
|
||||
{ name: "permission_allow", args: ["permission", "allow"] },
|
||||
{ name: "permission_deny", args: ["permission", "deny"] },
|
||||
{
|
||||
name: "extraction_set_apply",
|
||||
args: ["extraction", "set", "--window", "6m", "--confirm"],
|
||||
|
||||
@@ -190,13 +190,6 @@ import {
|
||||
renderMemoryDetailText,
|
||||
renderMemoryPageText,
|
||||
} from "./memory-report.js";
|
||||
import {
|
||||
type ModelContentPermissionDocument,
|
||||
modelContentPermissionExitCode,
|
||||
readModelContentPermission,
|
||||
renderModelContentPermissionText,
|
||||
setModelContentPermission,
|
||||
} from "./model-content-permission-flow.js";
|
||||
import {
|
||||
executeModelProbe,
|
||||
inspectModelProbePlan,
|
||||
@@ -217,12 +210,10 @@ import {
|
||||
executeModelProfileActivate,
|
||||
executeModelProfileEdit,
|
||||
type ModelWriteDocument,
|
||||
type ModelWritePlan,
|
||||
modelWriteExitCode,
|
||||
planModelProfileActivate,
|
||||
planModelProfileEdit,
|
||||
renderModelWriteText,
|
||||
validateModelWritePlan,
|
||||
} from "./model-write-flows.js";
|
||||
import { claimPlanAttempt } from "./operation-journal.js";
|
||||
import {
|
||||
@@ -2462,45 +2453,6 @@ async function runCommand(
|
||||
);
|
||||
return uiWriteExitCode(document);
|
||||
}
|
||||
if (named === "permission") {
|
||||
const parsed = parsePermissionCommand(arguments_.slice(1));
|
||||
if (!parsed) return usage();
|
||||
const config = resolveRuntimeCliConfig(environment, process.argv[1]);
|
||||
const runtime = await inspectRuntimeStatus({
|
||||
dataDir: config.dataDir,
|
||||
probeIdentity,
|
||||
requireServiceReceipt: true,
|
||||
});
|
||||
const attached =
|
||||
runtime.state === "running"
|
||||
? await attachClients(config.dataDir)
|
||||
: { reader: null, operations: null, failure: runtime.code };
|
||||
if (
|
||||
runtime.state !== "running" ||
|
||||
(parsed === "status" ? !attached.reader : !attached.operations)
|
||||
) {
|
||||
const document: ModelContentPermissionDocument = {
|
||||
schema_version: 1,
|
||||
product: PRODUCT,
|
||||
state: "unknown",
|
||||
code:
|
||||
runtime.state !== "running"
|
||||
? runtime.code
|
||||
: (attached.failure ?? "runtime_attach_failed"),
|
||||
};
|
||||
emitModelContentPermissionDocument(document, asJson);
|
||||
return runtime.state === "stopped" ? 3 : 1;
|
||||
}
|
||||
const document =
|
||||
parsed === "status"
|
||||
? await readModelContentPermission(attached.reader!)
|
||||
: await setModelContentPermission(
|
||||
attached.operations!,
|
||||
parsed === "allow",
|
||||
);
|
||||
emitModelContentPermissionDocument(document, asJson);
|
||||
return modelContentPermissionExitCode(document);
|
||||
}
|
||||
if (named === "notes" || named === "extraction" || named === "graph") {
|
||||
const parsed = parseExtendedWriteCommand(
|
||||
named as "notes" | "extraction" | "graph",
|
||||
@@ -5124,18 +5076,6 @@ function parseHeadlessCommand(
|
||||
: null;
|
||||
}
|
||||
|
||||
function parsePermissionCommand(
|
||||
rest: readonly string[],
|
||||
): "status" | "allow" | "deny" | null {
|
||||
if (rest.filter((item) => item === "--json").length > 1) return null;
|
||||
const tokens = rest.filter((item) => item !== "--json");
|
||||
const action = tokens[0];
|
||||
return tokens.length === 1 &&
|
||||
(action === "status" || action === "allow" || action === "deny")
|
||||
? action
|
||||
: null;
|
||||
}
|
||||
|
||||
function unwrapPlan(value: unknown): unknown {
|
||||
if (
|
||||
value &&
|
||||
@@ -5323,17 +5263,6 @@ function emitModelReportDocument(
|
||||
);
|
||||
}
|
||||
|
||||
function emitModelContentPermissionDocument(
|
||||
document: ModelContentPermissionDocument,
|
||||
asJson: boolean,
|
||||
): void {
|
||||
process.stdout.write(
|
||||
asJson
|
||||
? `${JSON.stringify(document)}\n`
|
||||
: renderModelContentPermissionText(document),
|
||||
);
|
||||
}
|
||||
|
||||
function emitSourceSyncDocument(
|
||||
document: SourceSyncFlowDocument,
|
||||
asJson: boolean,
|
||||
|
||||
@@ -493,14 +493,6 @@ export const COMMANDS: readonly CommandDescriptor[] = [
|
||||
],
|
||||
exit_codes: [EXIT_OK, EXIT_FAILED, EXIT_USAGE, EXIT_RUNTIME_STOPPED],
|
||||
},
|
||||
{
|
||||
name: "permission",
|
||||
summary: "读取或直接设置全局模型内容外发许可。",
|
||||
usage:
|
||||
"memhub permission status [--json]\n memhub permission allow [--json]\n memhub permission deny [--json]",
|
||||
options: [JSON_OPTION],
|
||||
exit_codes: [EXIT_OK, EXIT_FAILED, EXIT_USAGE, EXIT_RUNTIME_STOPPED],
|
||||
},
|
||||
{
|
||||
name: "extraction",
|
||||
summary: "设置提取窗口;默认只出计划,--confirm 后才写入。",
|
||||
|
||||
@@ -81,7 +81,6 @@ const COMMANDS = [
|
||||
"calls",
|
||||
"import-file",
|
||||
"notes",
|
||||
"permission",
|
||||
"extraction",
|
||||
"graph",
|
||||
"host",
|
||||
@@ -259,7 +258,14 @@ describe("CLI-22 shared output contract", () => {
|
||||
|
||||
describe("argument validation", () => {
|
||||
it("fails unknown commands without printing data or running anything", async () => {
|
||||
for (const argv of [["unknown"], ["unknown", "--help"], ["--typo"]]) {
|
||||
for (const argv of [
|
||||
["unknown"],
|
||||
["unknown", "--help"],
|
||||
["--typo"],
|
||||
["permission", "status"],
|
||||
["permission", "allow"],
|
||||
["permission", "deny"],
|
||||
]) {
|
||||
const streams = captureStreams();
|
||||
try {
|
||||
expect(await run(argv, {})).toBe(2);
|
||||
|
||||
@@ -8,7 +8,6 @@ import axios from "axios";
|
||||
*/
|
||||
const READ_ONLY_PATHS = [
|
||||
"/api/v1/call-events",
|
||||
"/api/v1/model-content-permission",
|
||||
"/api/v1/home-summary",
|
||||
"/api/v1/import-jobs",
|
||||
"/api/v1/live-sync",
|
||||
|
||||
@@ -30,7 +30,6 @@ const IMPORT_FILE =
|
||||
const MEMORY_NOTES_PATH = "/api/v1/memory-notes";
|
||||
const MEMORY_NOTE_PROCESS =
|
||||
/^\/api\/v1\/memory-notes\/[A-Za-z0-9][\w.:-]{0,127}\/process$/;
|
||||
const MODEL_CONTENT_PERMISSION_PATH = "/api/v1/model-content-permission";
|
||||
const EXTRACTION_SETTINGS_PATH = "/api/v1/extraction-settings";
|
||||
const GRAPH_WIDTH_PATH = "/api/v1/graph-width";
|
||||
const MANUAL_EXTRACT_PATH = "/api/v1/memory-processing/manual-extract";
|
||||
@@ -134,7 +133,6 @@ export function createCoreOperations(attachment: {
|
||||
!IMPORT_FILE_COMMIT.test(path) &&
|
||||
path !== MEMORY_NOTES_PATH &&
|
||||
!MEMORY_NOTE_PROCESS.test(path) &&
|
||||
path !== MODEL_CONTENT_PERMISSION_PATH &&
|
||||
path !== GRAPH_WIDTH_PATH &&
|
||||
path !== MANUAL_EXTRACT_PATH &&
|
||||
path !== ONBOARDING_COMPLETE_PATH &&
|
||||
|
||||
@@ -259,7 +259,7 @@ describe("A2 dual-mode init", () => {
|
||||
expect(streams.value()).toContain("处理:已开启");
|
||||
});
|
||||
|
||||
it("continues the TTY wizard through model probe and content permission", async () => {
|
||||
it("continues the TTY wizard through model setup and probe", async () => {
|
||||
const profileId = "22222222-2222-4222-8222-222222222222";
|
||||
const profile = {
|
||||
profile_id: profileId,
|
||||
@@ -361,12 +361,6 @@ describe("A2 dual-mode init", () => {
|
||||
};
|
||||
}
|
||||
if (url.endsWith("/activate")) return { status: 200, data: profile };
|
||||
if (url === "/api/v1/model-content-permission") {
|
||||
return {
|
||||
status: 200,
|
||||
data: { allowed: body.allowed },
|
||||
};
|
||||
}
|
||||
if (url === "/api/v1/onboarding/complete") {
|
||||
return {
|
||||
status: 200,
|
||||
@@ -383,7 +377,7 @@ describe("A2 dual-mode init", () => {
|
||||
},
|
||||
);
|
||||
const prompt = extendedPrompter(
|
||||
["", "", "", "", "", "", "", "", "n"],
|
||||
["", "", "", "", "", "", "", "n"],
|
||||
"sk-never-print-this",
|
||||
);
|
||||
const streams = captureStdout();
|
||||
@@ -402,7 +396,6 @@ describe("A2 dual-mode init", () => {
|
||||
streams.restore();
|
||||
}
|
||||
expect(streams.value()).toContain("已接入来源:codex");
|
||||
expect(prompt.written.join("\n")).toContain("模型内容外发:已允许");
|
||||
expect(streams.value()).toContain("首次设置完成。");
|
||||
expect(`${streams.value()}\n${prompt.written.join("\n")}`).not.toContain(
|
||||
"sk-never-print-this",
|
||||
@@ -412,10 +405,10 @@ describe("A2 dual-mode init", () => {
|
||||
expect.objectContaining({ api_key: "sk-never-print-this" }),
|
||||
undefined,
|
||||
);
|
||||
expect(transport.post).toHaveBeenCalledWith(
|
||||
expect(transport.post).not.toHaveBeenCalledWith(
|
||||
"/api/v1/model-content-permission",
|
||||
{ allowed: true },
|
||||
undefined,
|
||||
expect.anything(),
|
||||
expect.anything(),
|
||||
);
|
||||
expect(transport.post).toHaveBeenCalledWith(
|
||||
"/api/v1/onboarding/complete",
|
||||
|
||||
@@ -132,12 +132,6 @@ function clients(options: { failedProbes?: number } = {}) {
|
||||
if (path.endsWith("/activate")) {
|
||||
return { state: "committed" as const, data: profile(true) };
|
||||
}
|
||||
if (path === "/api/v1/model-content-permission") {
|
||||
return {
|
||||
state: "committed" as const,
|
||||
data: { allowed: body.allowed },
|
||||
};
|
||||
}
|
||||
return {
|
||||
state: "rejected" as const,
|
||||
code: "unexpected_path",
|
||||
@@ -160,8 +154,8 @@ function clients(options: { failedProbes?: number } = {}) {
|
||||
}
|
||||
|
||||
describe("interactive model setup", () => {
|
||||
it("uses App defaults, keeps the key out of output, and allows model content", async () => {
|
||||
const prompt = prompter(["", "", "", "", "", ""]);
|
||||
it("uses App defaults and keeps the key out of output", async () => {
|
||||
const prompt = prompter(["", "", ""]);
|
||||
const api = clients();
|
||||
|
||||
const result = await runInteractiveModelSetup(prompt.value, api, {
|
||||
@@ -170,10 +164,7 @@ describe("interactive model setup", () => {
|
||||
installHosts: false,
|
||||
});
|
||||
|
||||
expect(result).toMatchObject({
|
||||
state: "configured",
|
||||
model_content_allowed: true,
|
||||
});
|
||||
expect(result).toMatchObject({ state: "configured" });
|
||||
expect(api.post).toHaveBeenCalledWith(
|
||||
"/api/v1/model-profiles",
|
||||
expect.objectContaining({
|
||||
@@ -184,9 +175,10 @@ describe("interactive model setup", () => {
|
||||
embedding_provider_kind: "local_onnx",
|
||||
}),
|
||||
);
|
||||
expect(api.post).toHaveBeenCalledWith("/api/v1/model-content-permission", {
|
||||
allowed: true,
|
||||
});
|
||||
expect(api.post).not.toHaveBeenCalledWith(
|
||||
"/api/v1/model-content-permission",
|
||||
expect.anything(),
|
||||
);
|
||||
expect(prompt.written.join("\n")).not.toContain("sk-do-not-print");
|
||||
});
|
||||
|
||||
@@ -205,7 +197,7 @@ describe("interactive model setup", () => {
|
||||
});
|
||||
|
||||
it("removes an unverified profile before skipping after a failed probe", async () => {
|
||||
const prompt = prompter(["", "", "", "n", "0"]);
|
||||
const prompt = prompter(["", "", "", "0"]);
|
||||
const api = clients();
|
||||
api.post.mockImplementation(
|
||||
async (path: string, body: Record<string, unknown>) => {
|
||||
@@ -280,7 +272,7 @@ describe("interactive model setup", () => {
|
||||
});
|
||||
|
||||
it("cleans a failed probe and lets the user re-enter the model settings", async () => {
|
||||
const prompt = prompter(["", "", "", "", "", "", "", "", "", "", ""]);
|
||||
const prompt = prompter(["", "", "", "", "", "", ""]);
|
||||
const api = clients({ failedProbes: 1 });
|
||||
|
||||
const result = await runInteractiveModelSetup(prompt.value, api, {
|
||||
@@ -289,10 +281,7 @@ describe("interactive model setup", () => {
|
||||
installHosts: false,
|
||||
});
|
||||
|
||||
expect(result).toMatchObject({
|
||||
state: "configured",
|
||||
model_content_allowed: true,
|
||||
});
|
||||
expect(result).toMatchObject({ state: "configured" });
|
||||
expect(api.remove).toHaveBeenCalledTimes(1);
|
||||
expect(
|
||||
api.post.mock.calls.filter(([path]) => path === "/api/v1/model-profiles"),
|
||||
@@ -305,7 +294,7 @@ describe("interactive model setup", () => {
|
||||
const prompt = prompter(["", "", ""]);
|
||||
prompt.value.ask = async () => {
|
||||
asked += 1;
|
||||
if (asked === 4) throw new Error("interrupted");
|
||||
if (asked === 3) throw new Error("interrupted");
|
||||
return "";
|
||||
};
|
||||
const api = clients();
|
||||
@@ -322,7 +311,7 @@ describe("interactive model setup", () => {
|
||||
});
|
||||
|
||||
it("installs selected detected hosts through preview and fingerprint apply", async () => {
|
||||
const prompt = prompter(["", "", "", "n", "", ""]);
|
||||
const prompt = prompter(["", "", "", "", ""]);
|
||||
const api = clients();
|
||||
const read = vi
|
||||
.fn()
|
||||
@@ -360,7 +349,6 @@ describe("interactive model setup", () => {
|
||||
|
||||
expect(result).toMatchObject({
|
||||
state: "configured",
|
||||
model_content_allowed: false,
|
||||
installed_hosts: ["codex"],
|
||||
});
|
||||
expect(read).toHaveBeenNthCalledWith(
|
||||
@@ -373,7 +361,7 @@ describe("interactive model setup", () => {
|
||||
});
|
||||
|
||||
it("lists and installs only hosts that are actually present", async () => {
|
||||
const prompt = prompter(["", "", "", "n", "", ""]);
|
||||
const prompt = prompter(["", "", "", "", ""]);
|
||||
const api = clients();
|
||||
const read = vi
|
||||
.fn()
|
||||
@@ -420,7 +408,7 @@ describe("interactive model setup", () => {
|
||||
});
|
||||
|
||||
it("completes the wizard and keeps onboarding reachable when one host install fails", async () => {
|
||||
const prompt = prompter(["", "", "", "n", "", ""]);
|
||||
const prompt = prompter(["", "", "", "", ""]);
|
||||
const api = clients();
|
||||
const read = vi
|
||||
.fn()
|
||||
@@ -469,7 +457,6 @@ describe("interactive model setup", () => {
|
||||
// calling the install a failure.
|
||||
expect(result).toMatchObject({
|
||||
state: "configured",
|
||||
model_content_allowed: false,
|
||||
installed_hosts: ["codex"],
|
||||
});
|
||||
const summary = prompt.written.join("\n");
|
||||
@@ -479,7 +466,7 @@ describe("interactive model setup", () => {
|
||||
});
|
||||
|
||||
it("tells the user to install App-only hosts in the App instead of calling them failed", async () => {
|
||||
const prompt = prompter(["", "", "", "n", "", ""]);
|
||||
const prompt = prompter(["", "", "", "", ""]);
|
||||
const api = clients();
|
||||
const read = vi
|
||||
.fn()
|
||||
@@ -506,7 +493,6 @@ describe("interactive model setup", () => {
|
||||
// only the App can install, so nothing counts as "installed".
|
||||
expect(result).toMatchObject({
|
||||
state: "configured",
|
||||
model_content_allowed: false,
|
||||
installed_hosts: [],
|
||||
});
|
||||
const summary = prompt.written.join("\n");
|
||||
|
||||
@@ -11,7 +11,6 @@ import {
|
||||
} from "./headless-flows.js";
|
||||
import { approvalFingerprint, listHosts, readHost } from "./host-flow.js";
|
||||
import { parseSelection, type WizardPrompter } from "./init-wizard.js";
|
||||
import { setModelContentPermission } from "./model-content-permission-flow.js";
|
||||
import { executeModelProbe, planModelProbe } from "./model-probe-flow.js";
|
||||
import {
|
||||
executeModelProfileActivate,
|
||||
@@ -122,7 +121,6 @@ export type InteractiveModelSetupState =
|
||||
| {
|
||||
state: "configured";
|
||||
profile: ModelProfile;
|
||||
model_content_allowed: boolean;
|
||||
installed_hosts: string[];
|
||||
}
|
||||
| { state: "skipped" }
|
||||
@@ -289,15 +287,11 @@ async function installHosts(
|
||||
function writeSummary(
|
||||
prompter: WizardPrompter,
|
||||
profile: ModelProfile,
|
||||
modelContentAllowed: boolean,
|
||||
installedHosts: readonly string[],
|
||||
failedHosts: readonly { hostId: string; label: string }[],
|
||||
): void {
|
||||
prompter.write("配置摘要:");
|
||||
prompter.write(` 模型:${profile.name} / ${profile.model}`);
|
||||
prompter.write(
|
||||
` 模型内容外发:${modelContentAllowed ? "已允许" : "已禁止"}`,
|
||||
);
|
||||
prompter.write(
|
||||
` 应用接入:${installedHosts.length > 0 ? installedHosts.join("、") : "已跳过"}`,
|
||||
);
|
||||
@@ -353,14 +347,6 @@ export async function runInteractiveModelSetup(
|
||||
prompter.write("未填写 API Key。输入 back 返回,或重新选择模型服务。");
|
||||
continue;
|
||||
}
|
||||
const permissionAnswer = (
|
||||
await prompter.ask("允许将对话内容发送给模型以整理记忆?[Y/n]: ")
|
||||
)
|
||||
.trim()
|
||||
.toLowerCase();
|
||||
const modelContentAllowed = !["n", "no", "否", "0"].includes(
|
||||
permissionAnswer,
|
||||
);
|
||||
const hostApi = options.hostApi ?? { list: listHosts, read: readHost };
|
||||
const selectedHosts = options.installHosts
|
||||
? await chooseDetectedHosts(prompter, options.hostHome, hostApi)
|
||||
@@ -452,13 +438,6 @@ export async function runInteractiveModelSetup(
|
||||
return { state: "failed", code: "model_activate_failed" };
|
||||
}
|
||||
|
||||
const permission = await setModelContentPermission(
|
||||
clients.operations,
|
||||
modelContentAllowed,
|
||||
);
|
||||
if (permission.state !== "ok") {
|
||||
return { state: "failed", code: "model_content_permission_set_failed" };
|
||||
}
|
||||
const { installed, failed } = await installHosts(
|
||||
options.dataDir,
|
||||
selectedHosts,
|
||||
@@ -467,7 +446,6 @@ export async function runInteractiveModelSetup(
|
||||
writeSummary(
|
||||
prompter,
|
||||
profile,
|
||||
modelContentAllowed,
|
||||
installed,
|
||||
failed.map((hostId) => ({
|
||||
hostId,
|
||||
@@ -477,7 +455,6 @@ export async function runInteractiveModelSetup(
|
||||
return {
|
||||
state: "configured",
|
||||
profile,
|
||||
model_content_allowed: modelContentAllowed,
|
||||
installed_hosts: installed,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -180,5 +180,8 @@ describe("import-job write allow-list", () => {
|
||||
await expect(
|
||||
operations.post("/api/v1/import-jobs/job-1/delete", {}),
|
||||
).rejects.toThrow("core_operation_path_not_allowed");
|
||||
await expect(
|
||||
operations.post("/api/v1/model-content-permission", { allowed: true }),
|
||||
).rejects.toThrow("core_operation_path_not_allowed");
|
||||
});
|
||||
});
|
||||
@@ -1,154 +0,0 @@
|
||||
import { beforeEach, describe, expect, it, vi } from "vitest";
|
||||
|
||||
import { run } from "./cli.js";
|
||||
|
||||
const transport = vi.hoisted(() => ({
|
||||
get: vi.fn(),
|
||||
post: vi.fn(),
|
||||
put: vi.fn(),
|
||||
patch: vi.fn(),
|
||||
delete: vi.fn(),
|
||||
}));
|
||||
vi.mock("axios", () => ({
|
||||
default: { create: vi.fn(() => transport), get: vi.fn() },
|
||||
}));
|
||||
|
||||
const runtime = vi.hoisted(() => ({
|
||||
inspectRuntimeStatus: vi.fn(),
|
||||
discoverRuntimeAttachment: vi.fn(),
|
||||
}));
|
||||
vi.mock("@one-memory/runtime-control", async (importOriginal) => ({
|
||||
...(await importOriginal<typeof import("@one-memory/runtime-control")>()),
|
||||
...runtime,
|
||||
}));
|
||||
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks();
|
||||
runtime.inspectRuntimeStatus.mockResolvedValue({
|
||||
state: "running",
|
||||
code: "runtime_verified",
|
||||
instanceId: "instance-1",
|
||||
host: "127.0.0.1",
|
||||
port: 8765,
|
||||
version: "1.2.3",
|
||||
});
|
||||
runtime.discoverRuntimeAttachment.mockResolvedValue({
|
||||
instanceId: "instance-1",
|
||||
apiToken: "synthetic-runtime-token",
|
||||
host: "127.0.0.1",
|
||||
port: 8765,
|
||||
servicePid: 4242,
|
||||
health: {
|
||||
status: "ok",
|
||||
service: "one-memory-core",
|
||||
version: "1.2.3",
|
||||
instance_id: "instance-1",
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
describe("model content permission CLI", () => {
|
||||
it("documents only final-state permission and manual extraction commands", async () => {
|
||||
const permission = await capturedRun(["help", "permission"]);
|
||||
expect(permission.code).toBe(0);
|
||||
expect(permission.stdout).toContain("memhub permission allow");
|
||||
expect(permission.stdout).toContain("memhub permission deny");
|
||||
expect(permission.stdout).not.toMatch(
|
||||
/\b(preview|confirm|process|revoke)\b/,
|
||||
);
|
||||
expect(permission.stdout).not.toContain("revision");
|
||||
expect(permission.stdout).not.toContain("receipt");
|
||||
|
||||
const processing = await capturedRun(["help", "processing"]);
|
||||
expect(processing.code).toBe(0);
|
||||
expect(processing.stdout).toContain("memhub processing extract");
|
||||
expect(processing.stdout).not.toMatch(/\b(plan|apply|receipt)\b/);
|
||||
expect(processing.stdout).not.toContain("backlog");
|
||||
});
|
||||
|
||||
it("reads the final state", async () => {
|
||||
transport.get.mockResolvedValue({ status: 200, data: { allowed: true } });
|
||||
|
||||
const result = await capturedRun(["permission", "status", "--json"]);
|
||||
|
||||
expect(result.code).toBe(0);
|
||||
expect(JSON.parse(result.stdout)).toMatchObject({
|
||||
state: "ok",
|
||||
allowed: true,
|
||||
});
|
||||
expect(transport.get).toHaveBeenCalledWith(
|
||||
"/api/v1/model-content-permission",
|
||||
undefined,
|
||||
);
|
||||
});
|
||||
|
||||
it.each([
|
||||
["allow", true],
|
||||
["deny", false],
|
||||
] as const)("sets %s directly with only allowed", async (verb, allowed) => {
|
||||
transport.post.mockResolvedValue({ status: 200, data: { allowed } });
|
||||
|
||||
const result = await capturedRun(["permission", verb, "--json"]);
|
||||
|
||||
expect(result.code).toBe(0);
|
||||
expect(JSON.parse(result.stdout)).toMatchObject({ state: "ok", allowed });
|
||||
expect(transport.post).toHaveBeenCalledWith(
|
||||
"/api/v1/model-content-permission",
|
||||
{ allowed },
|
||||
undefined,
|
||||
);
|
||||
});
|
||||
|
||||
it.each(["preview", "confirm", "process", "revoke"])(
|
||||
"rejects removed %s commands before runtime access",
|
||||
async (verb) => {
|
||||
const result = await capturedRun(["permission", verb, "--json"]);
|
||||
|
||||
expect(result.code).toBe(2);
|
||||
expect(runtime.inspectRuntimeStatus).not.toHaveBeenCalled();
|
||||
expect(transport.get).not.toHaveBeenCalled();
|
||||
expect(transport.post).not.toHaveBeenCalled();
|
||||
},
|
||||
);
|
||||
|
||||
it.each(["plan", "apply", "receipt"])(
|
||||
"rejects removed processing %s commands before runtime access",
|
||||
async (verb) => {
|
||||
const result = await capturedRun(["processing", verb, "--json"]);
|
||||
|
||||
expect(result.code).toBe(2);
|
||||
expect(runtime.inspectRuntimeStatus).not.toHaveBeenCalled();
|
||||
expect(transport.get).not.toHaveBeenCalled();
|
||||
expect(transport.post).not.toHaveBeenCalled();
|
||||
},
|
||||
);
|
||||
});
|
||||
|
||||
async function capturedRun(arguments_: string[]) {
|
||||
let stdout = "";
|
||||
let stderr = "";
|
||||
const stdoutSpy = vi
|
||||
.spyOn(process.stdout, "write")
|
||||
.mockImplementation((chunk) => {
|
||||
stdout += String(chunk);
|
||||
return true;
|
||||
});
|
||||
const stderrSpy = vi
|
||||
.spyOn(process.stderr, "write")
|
||||
.mockImplementation((chunk) => {
|
||||
stderr += String(chunk);
|
||||
return true;
|
||||
});
|
||||
try {
|
||||
return {
|
||||
code: await run(arguments_, {
|
||||
ONE_MEMORY_DATA_DIR: "/synthetic/data",
|
||||
}),
|
||||
stdout,
|
||||
stderr,
|
||||
};
|
||||
} finally {
|
||||
stdoutSpy.mockRestore();
|
||||
stderrSpy.mockRestore();
|
||||
}
|
||||
}
|
||||
@@ -1,66 +0,0 @@
|
||||
import { describe, expect, it, vi } from "vitest";
|
||||
|
||||
import type { CoreReader } from "./core-client.js";
|
||||
import type { CoreOperations } from "./core-operations.js";
|
||||
import {
|
||||
readModelContentPermission,
|
||||
setModelContentPermission,
|
||||
} from "./model-content-permission-flow.js";
|
||||
|
||||
describe("model content permission flow", () => {
|
||||
it("reads the single final-state boolean", async () => {
|
||||
const reader = vi.fn().mockResolvedValue({
|
||||
state: "ok",
|
||||
data: { allowed: true },
|
||||
});
|
||||
|
||||
await expect(
|
||||
readModelContentPermission(reader as CoreReader),
|
||||
).resolves.toMatchObject({ state: "ok", allowed: true });
|
||||
expect(reader).toHaveBeenCalledWith("/api/v1/model-content-permission");
|
||||
});
|
||||
|
||||
it.each([
|
||||
["allow", true],
|
||||
["deny", false],
|
||||
] as const)(
|
||||
"sets %s directly without revision or receipt fields",
|
||||
async (_, allowed) => {
|
||||
const post = vi.fn().mockResolvedValue({
|
||||
state: "committed",
|
||||
data: { allowed },
|
||||
});
|
||||
|
||||
await expect(
|
||||
setModelContentPermission({ post } as CoreOperations, allowed),
|
||||
).resolves.toMatchObject({ state: "ok", allowed });
|
||||
expect(post).toHaveBeenCalledWith("/api/v1/model-content-permission", {
|
||||
allowed,
|
||||
});
|
||||
},
|
||||
);
|
||||
|
||||
it("does not accept a malformed or mismatched success response", async () => {
|
||||
const malformed = vi.fn().mockResolvedValue({
|
||||
state: "committed",
|
||||
data: { allowed: true, revision: 1 },
|
||||
});
|
||||
await expect(
|
||||
setModelContentPermission({ post: malformed } as CoreOperations, true),
|
||||
).resolves.toMatchObject({
|
||||
state: "unknown",
|
||||
code: "model_content_permission_response_invalid",
|
||||
});
|
||||
|
||||
const mismatched = vi.fn().mockResolvedValue({
|
||||
state: "committed",
|
||||
data: { allowed: false },
|
||||
});
|
||||
await expect(
|
||||
setModelContentPermission({ post: mismatched } as CoreOperations, true),
|
||||
).resolves.toMatchObject({
|
||||
state: "unknown",
|
||||
code: "model_content_permission_state_mismatch",
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -1,96 +0,0 @@
|
||||
import type { CoreReader } from "./core-client.js";
|
||||
import type { CoreOperations } from "./core-operations.js";
|
||||
import { PRODUCT } from "./version.js";
|
||||
|
||||
export type ModelContentPermissionDocument =
|
||||
| {
|
||||
schema_version: 1;
|
||||
product: string;
|
||||
state: "ok";
|
||||
allowed: boolean;
|
||||
}
|
||||
| {
|
||||
schema_version: 1;
|
||||
product: string;
|
||||
state: "rejected" | "unknown";
|
||||
code: string;
|
||||
};
|
||||
|
||||
export async function readModelContentPermission(
|
||||
reader: CoreReader,
|
||||
): Promise<ModelContentPermissionDocument> {
|
||||
const outcome = await reader<unknown>("/api/v1/model-content-permission");
|
||||
if (outcome.state !== "ok") {
|
||||
return failure("unknown", outcome.code);
|
||||
}
|
||||
return permissionState(
|
||||
outcome.data,
|
||||
"model_content_permission_response_invalid",
|
||||
);
|
||||
}
|
||||
|
||||
export async function setModelContentPermission(
|
||||
operations: CoreOperations,
|
||||
allowed: boolean,
|
||||
): Promise<ModelContentPermissionDocument> {
|
||||
const outcome = await operations.post<unknown>(
|
||||
"/api/v1/model-content-permission",
|
||||
{ allowed },
|
||||
);
|
||||
if (outcome.state === "rejected") {
|
||||
return failure("rejected", outcome.code);
|
||||
}
|
||||
if (outcome.state === "unknown") {
|
||||
return failure("unknown", outcome.code);
|
||||
}
|
||||
const document = permissionState(
|
||||
outcome.data,
|
||||
"model_content_permission_response_invalid",
|
||||
);
|
||||
return document.state === "ok" && document.allowed !== allowed
|
||||
? failure("unknown", "model_content_permission_state_mismatch")
|
||||
: document;
|
||||
}
|
||||
|
||||
export function modelContentPermissionExitCode(
|
||||
document: ModelContentPermissionDocument,
|
||||
): number {
|
||||
return document.state === "ok" ? 0 : 1;
|
||||
}
|
||||
|
||||
export function renderModelContentPermissionText(
|
||||
document: ModelContentPermissionDocument,
|
||||
): string {
|
||||
if (document.state === "ok") {
|
||||
return `模型内容外发:${document.allowed ? "允许" : "禁止"}\n`;
|
||||
}
|
||||
return `模型内容权限未更新:${document.state}(${document.code})\n`;
|
||||
}
|
||||
|
||||
function permissionState(
|
||||
value: unknown,
|
||||
invalidCode: string,
|
||||
): ModelContentPermissionDocument {
|
||||
if (
|
||||
value === null ||
|
||||
typeof value !== "object" ||
|
||||
Array.isArray(value) ||
|
||||
Object.keys(value).length !== 1 ||
|
||||
typeof (value as Record<string, unknown>).allowed !== "boolean"
|
||||
) {
|
||||
return failure("unknown", invalidCode);
|
||||
}
|
||||
return {
|
||||
schema_version: 1,
|
||||
product: PRODUCT,
|
||||
state: "ok",
|
||||
allowed: (value as { allowed: boolean }).allowed,
|
||||
};
|
||||
}
|
||||
|
||||
function failure(
|
||||
state: "rejected" | "unknown",
|
||||
code: string,
|
||||
): ModelContentPermissionDocument {
|
||||
return { schema_version: 1, product: PRODUCT, state, code };
|
||||
}
|
||||
@@ -137,8 +137,8 @@ describe("package matrix harness", () => {
|
||||
};
|
||||
expect(document.state).toBe("verified");
|
||||
const results = document.commands.stopped_destructive.results;
|
||||
// 13 destructive commands, each unknown/runtime_not_running with exit 3.
|
||||
expect(Object.keys(results)).toHaveLength(13);
|
||||
// 11 destructive commands, each unknown/runtime_not_running with exit 3.
|
||||
expect(Object.keys(results)).toHaveLength(11);
|
||||
for (const [name, result] of Object.entries(results)) {
|
||||
expect(result, `${name} did not fail closed`).toEqual({
|
||||
exit: 3,
|
||||
@@ -162,7 +162,7 @@ describe("package matrix harness", () => {
|
||||
const shim = [
|
||||
"#!/usr/bin/env node",
|
||||
"const a=process.argv.slice(2);",
|
||||
"if(['calls','import-file','notes','permission','extraction','graph','models'].includes(a[0])||(a[0]==='processing'&&a[1]==='extract')){",
|
||||
"if(['calls','import-file','notes','extraction','graph','models'].includes(a[0])||(a[0]==='processing'&&a[1]==='extract')){",
|
||||
' process.stdout.write(\'{"schema_version":1,"product":"memhub","state":"committed","operation_id":"x","receipt":{}}\\n\');',
|
||||
" process.exit(0);",
|
||||
"}",
|
||||
@@ -188,7 +188,7 @@ describe("package matrix harness", () => {
|
||||
expect(failure?.state).toBe("failed");
|
||||
expect(
|
||||
failure?.failures.filter((item) => item.includes("did not fail closed")),
|
||||
).toHaveLength(13);
|
||||
).toHaveLength(11);
|
||||
});
|
||||
|
||||
it("can require an expected bundle SHA and reports bundle metadata", {
|
||||
@@ -280,9 +280,8 @@ describe("package matrix harness", () => {
|
||||
});
|
||||
|
||||
it("rejects a catalog that declares the parent command but not the new verb", async () => {
|
||||
// A bundle can still declare `permission`/`models` while missing
|
||||
// `permission allow` or `models activate`. Names alone would pass it; the
|
||||
// usage marker must not.
|
||||
// A bundle can still declare `models` while missing `models activate`.
|
||||
// Names alone would pass it; the usage marker must not.
|
||||
const declared = [
|
||||
"install",
|
||||
"init",
|
||||
@@ -292,7 +291,6 @@ describe("package matrix harness", () => {
|
||||
"calls",
|
||||
"import-file",
|
||||
"notes",
|
||||
"permission",
|
||||
"extraction",
|
||||
"graph",
|
||||
"models",
|
||||
@@ -342,9 +340,6 @@ describe("package matrix harness", () => {
|
||||
},
|
||||
};
|
||||
const failures = harness.validateCatalog(catalog);
|
||||
expect(failures).toContain(
|
||||
'command "permission" is missing usage "memhub permission allow"',
|
||||
);
|
||||
expect(failures).toContain(
|
||||
'command "models" is missing usage "memhub models activate"',
|
||||
);
|
||||
|
||||
@@ -151,9 +151,6 @@ describe("memhub short-command contract", () => {
|
||||
"memhub import-file discard",
|
||||
"memhub notes add",
|
||||
"memhub notes process",
|
||||
"memhub permission status",
|
||||
"memhub permission allow",
|
||||
"memhub permission deny",
|
||||
"memhub models setup",
|
||||
"memhub models activate",
|
||||
"memhub models edit",
|
||||
@@ -163,6 +160,9 @@ describe("memhub short-command contract", () => {
|
||||
]) {
|
||||
expect(flattened).toContain(shortForm);
|
||||
}
|
||||
expect(COMMANDS.some((command) => command.name === "permission")).toBe(
|
||||
false,
|
||||
);
|
||||
});
|
||||
|
||||
it("ships a memhub bin entry pointing at a real bundle path", () => {
|
||||
|
||||
@@ -840,6 +840,7 @@ describe("CLI-21 memory reads", () => {
|
||||
expect(isReadOnlyPath("/api/v1/memories/graph")).toBe(false);
|
||||
expect(isReadOnlyPath("/api/v1/memories/ask")).toBe(false);
|
||||
expect(isReadOnlyPath("/api/v1/memories/m1")).toBe(true);
|
||||
expect(isReadOnlyPath("/api/v1/model-content-permission")).toBe(false);
|
||||
expect(
|
||||
isReadOnlyPath(
|
||||
"/api/v1/memory-mutations/123e4567-e89b-12d3-a456-426614174000",
|
||||
|
||||
@@ -11,7 +11,7 @@ folder_layout = layout == 'folder'
|
||||
datas = [('src/one_memory/embedding_assets', 'one_memory/embedding_assets')]
|
||||
binaries = []
|
||||
# Loaded lazily by the API; static dependency analysis cannot discover it.
|
||||
hiddenimports = ['one_memory.hy_permission_context']
|
||||
hiddenimports = []
|
||||
tmp_ret = collect_all('hy_memory')
|
||||
datas += tmp_ret[0]; binaries += tmp_ret[1]; hiddenimports += tmp_ret[2]
|
||||
tmp_ret = collect_all('chromadb')
|
||||
|
||||
@@ -4,7 +4,7 @@ build-backend = "uv_build"
|
||||
|
||||
[project]
|
||||
name = "one-memory"
|
||||
version = "1.0.0"
|
||||
version = "1.0.2"
|
||||
description = "MemHub local-first memory core"
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.12"
|
||||
@@ -12,7 +12,7 @@ dependencies = [
|
||||
"fastapi>=0.141.1",
|
||||
"filelock>=3.32.4",
|
||||
# Public SDK version validated against MemHub's permission and recall contracts.
|
||||
"hy-memory==2.0.1",
|
||||
"hy-memory==2.0.2",
|
||||
# Retrieval's derived vector index directly consumes these locked versions.
|
||||
"chromadb==1.5.9",
|
||||
"numpy==2.5.2",
|
||||
@@ -24,7 +24,7 @@ dependencies = [
|
||||
# transitive version we do not ask for is one an unrelated upgrade could
|
||||
# drop. Deliberately NOT sentence-transformers or torch: those would take
|
||||
# the installer past 2GB, and onnxruntime alone runs the exported model.
|
||||
"onnxruntime>=1.22.0",
|
||||
"onnxruntime==1.23.2",
|
||||
"tokenizers>=0.21.0",
|
||||
"pydantic-settings>=2.15.0",
|
||||
"uvicorn[standard]>=0.52.4",
|
||||
|
||||
@@ -20,7 +20,6 @@ from copy import copy
|
||||
from datetime import UTC, datetime, timedelta
|
||||
from enum import StrEnum
|
||||
from functools import partial
|
||||
from importlib import import_module
|
||||
from pathlib import Path
|
||||
from typing import Any, Literal, cast
|
||||
from uuid import UUID
|
||||
@@ -164,7 +163,6 @@ from one_memory.memory_library import (
|
||||
)
|
||||
from one_memory.memory_notes import MemoryNoteStore, memory_note_router
|
||||
from one_memory.memory_processing import (
|
||||
MemoryBridgeProtocol,
|
||||
MemoryProcessingSupervisor,
|
||||
_conversation_chunks,
|
||||
memory_task_timeout_seconds,
|
||||
@@ -178,10 +176,8 @@ from one_memory.memory_queue import (
|
||||
)
|
||||
from one_memory.memory_server.config_state import ConfigStore
|
||||
from one_memory.memory_tasks import MemoryTask, MemoryTaskStatus, MemoryTaskStore
|
||||
from one_memory.model_content_routes import model_content_router
|
||||
from one_memory.model_presets import seed_preset_profile
|
||||
from one_memory.model_probe import ModelProber, ModelProbeResult
|
||||
from one_memory.model_processing_permission_guard import ModelProcessingPermissionLease
|
||||
from one_memory.model_profiles import (
|
||||
ActiveModelProfileError,
|
||||
ModelCreationOperationConflictError,
|
||||
@@ -818,6 +814,13 @@ def create_app(
|
||||
evidence=app.state.evidence_store,
|
||||
)
|
||||
app.state.memory_write_queue = memory_write_queue
|
||||
# Builds through 1.0.2 parked pre-invocation jobs behind the
|
||||
# retired model-content consent bit. The embedded SDK is trusted
|
||||
# now, so make those durable jobs claimable on the first startup.
|
||||
await asyncio.to_thread(
|
||||
memory_write_queue.readmit_parked,
|
||||
states=("waiting_for_permission",),
|
||||
)
|
||||
repaired_identity_failures = await asyncio.to_thread(
|
||||
memory_write_queue.repair_pre_send_identity_failures
|
||||
)
|
||||
@@ -960,7 +963,7 @@ def create_app(
|
||||
# Not fatal: without the tables every read checks exactly, as
|
||||
# it does today.
|
||||
app.state.memory_visibility_error = str(error)
|
||||
app.state.model_request_budget = (
|
||||
model_request_budget = (
|
||||
ModelRequestBudget(
|
||||
database_path,
|
||||
request_limit=runtime_settings.model_request_limit,
|
||||
@@ -970,6 +973,7 @@ def create_app(
|
||||
if runtime_settings.model_request_limit is not None
|
||||
else None
|
||||
)
|
||||
app.state.model_request_budget = model_request_budget
|
||||
app.state.memory_answerer = MemoryAnswerer(
|
||||
governance=app.state.memory_governance_store,
|
||||
filter_memories=app.state.source_read_access.memories,
|
||||
@@ -1008,6 +1012,16 @@ def create_app(
|
||||
governance=app.state.memory_governance_store,
|
||||
data_dir=runtime_settings.data_dir,
|
||||
extraction_settings=app.state.extraction_settings_store,
|
||||
bridge_factory=lambda profile, api_key, data_dir: HyMemoryBridge(
|
||||
profile=profile,
|
||||
api_key=api_key,
|
||||
data_dir=data_dir,
|
||||
http_transport_factory=(
|
||||
(lambda: BudgetedModelTransport(model_request_budget))
|
||||
if model_request_budget is not None
|
||||
else None
|
||||
),
|
||||
),
|
||||
# `None` means "whatever the active profile allows". Pinned to
|
||||
# 1 in `a59f4b5` as temporary load-shedding for an acceptance
|
||||
# package, and left pinned after it: a hardcoded 1 is rule 9's
|
||||
@@ -1016,10 +1030,6 @@ def create_app(
|
||||
# profile concurrency 4, recall latency is unchanged while
|
||||
# precipitation goes 4x.
|
||||
max_concurrent_tasks=runtime_settings.precipitation_max_concurrent_tasks,
|
||||
model_content_config=app.state.config_store,
|
||||
permission_bridge_factory=lambda lease, secret, directory: _permission_context(
|
||||
app, lease, secret, directory
|
||||
),
|
||||
source_writes_permitted=lambda source_id: _source_writes_on(app, source_id),
|
||||
service_levels=app.state.service_levels,
|
||||
)
|
||||
@@ -1039,10 +1049,15 @@ def create_app(
|
||||
job.job_id,
|
||||
)
|
||||
await asyncio.to_thread(
|
||||
memory_write_queue.retry_timed_out_unknown_after_reconcile,
|
||||
older_than=datetime.now(UTC) - timedelta(minutes=5),
|
||||
memory_write_queue.retry_first_unknown_after_reconcile,
|
||||
older_than=datetime.now(UTC) - timedelta(seconds=10),
|
||||
limit=32,
|
||||
)
|
||||
await asyncio.to_thread(
|
||||
app.state.memory_task_store.retry_first_unknown_after_reconcile,
|
||||
older_than=datetime.now(UTC) - timedelta(seconds=10),
|
||||
limit=128,
|
||||
)
|
||||
|
||||
async def admit_hy_invocation(
|
||||
source_id: str,
|
||||
@@ -1134,7 +1149,6 @@ def create_app(
|
||||
app.state.matter_discovery.processor = app.state.memory_processing_supervisor
|
||||
app.state.matter_discovery.write_intake = app.state.memory_write_intake
|
||||
app.state.matter_discovery.write_queue = memory_write_queue
|
||||
app.state.processing_coordinator = None
|
||||
app.state.memory_note_store = MemoryNoteStore(
|
||||
app.state.evidence_store,
|
||||
app.state.memory_processing_supervisor,
|
||||
@@ -1446,7 +1460,6 @@ def create_app(
|
||||
app.include_router(matter_router(protected))
|
||||
app.include_router(memory_note_router(protected))
|
||||
app.include_router(file_import_router(protected))
|
||||
app.include_router(model_content_router(protected))
|
||||
app.include_router(context_router(protected))
|
||||
|
||||
def require_workbuddy_boundary(request: Request) -> None:
|
||||
@@ -2328,7 +2341,7 @@ def create_app(
|
||||
await asyncio.to_thread(
|
||||
app.state.memory_task_store.park_source_for_permission,
|
||||
source_id=source_id,
|
||||
error_code="model_content_permission_required",
|
||||
error_code="hy_memory_source_revoked",
|
||||
)
|
||||
await _live_sync(app).stop_source(source_id)
|
||||
_supervisor(app).unregister_adapter(source_id)
|
||||
@@ -3894,12 +3907,7 @@ async def _authorize_answer_generation(
|
||||
# The explicit question is this operation's input, not a standing permission for history.
|
||||
if not question.strip() or not grounding:
|
||||
raise MemoryAnswerError("memory_answer_generation_not_authorized")
|
||||
if not await asyncio.to_thread(_config_store(app).model_content_allowed):
|
||||
raise MemoryAnswerError("memory_answer_generation_not_authorized")
|
||||
|
||||
def check() -> None:
|
||||
if not _config_store(app).model_content_allowed():
|
||||
raise MemoryAnswerError("memory_answer_generation_not_authorized")
|
||||
if _model_profile_store(app).active_identity() != (
|
||||
profile.profile_id,
|
||||
profile.revision,
|
||||
@@ -3922,30 +3930,6 @@ async def _authorize_answer_generation(
|
||||
await recheck()
|
||||
return recheck
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
async def _permission_context(
|
||||
app: FastAPI,
|
||||
lease: ModelProcessingPermissionLease,
|
||||
secret: str | None,
|
||||
directory: Path,
|
||||
) -> AsyncIterator[MemoryBridgeProtocol]:
|
||||
module = await asyncio.to_thread(import_module, "one_memory.hy_permission_context")
|
||||
budget = cast(ModelRequestBudget | None, app.state.model_request_budget)
|
||||
source_id = lease.records[0].source_id if lease.records else "manual"
|
||||
factory = module.permission_bridge_factory(
|
||||
lambda profile, api_key, data_dir: _memory_processing_supervisor(app)._bridge(
|
||||
profile,
|
||||
api_key,
|
||||
data_dir,
|
||||
scope_key=source_id,
|
||||
),
|
||||
**({"http_transport": BudgetedModelTransport(budget)} if budget else {}),
|
||||
)
|
||||
async with factory(lease, secret, directory) as bridge:
|
||||
yield cast(MemoryBridgeProtocol, bridge)
|
||||
|
||||
|
||||
class _LazyASGIApp:
|
||||
"""Compatibility ASGI app that is built only when a server imports it.
|
||||
|
||||
@@ -4723,8 +4707,6 @@ async def _run_backfill(app: FastAPI) -> None:
|
||||
app.state.memory_backfill_error = None
|
||||
if not app.state.settings.automatic_memory_processing:
|
||||
return
|
||||
if not await asyncio.to_thread(_config_store(app).model_content_allowed):
|
||||
return
|
||||
try:
|
||||
queue = app.state.memory_write_queue
|
||||
intake = app.state.memory_write_intake
|
||||
|
||||
@@ -21,7 +21,6 @@ from pydantic import BaseModel, ConfigDict, Field
|
||||
|
||||
from one_memory.diagnostic_log import diagnostic_event, diagnostic_exception, elapsed_ms
|
||||
from one_memory.hy_lifecycle import HyLifecycle, HyLifecycleUnreadable, read_hy_lifecycle
|
||||
from one_memory.hy_permission_store import native_store_read_slot
|
||||
from one_memory.hy_presence import HyPresence, confirm_deleted
|
||||
from one_memory.hy_runtime_config import (
|
||||
HY_MEMORY_MODE,
|
||||
@@ -32,6 +31,7 @@ from one_memory.hy_runtime_config import (
|
||||
diagnostic_log_level,
|
||||
)
|
||||
from one_memory.hy_source_context import HySourceContext, HyWriteBindings
|
||||
from one_memory.hy_store_concurrency import native_store_read_slot
|
||||
from one_memory.model_profiles import ModelProfileSnapshot, ProviderKind, attach_disable_thinking
|
||||
from one_memory.semantic_deadline import SemanticMemoryHit
|
||||
|
||||
@@ -376,11 +376,13 @@ class HyMemoryBridge:
|
||||
api_key: str | None,
|
||||
data_dir: Path,
|
||||
client: HyMemoryClientProtocol | None = None,
|
||||
http_transport_factory: Callable[[], Any] | None = None,
|
||||
) -> None:
|
||||
self.profile = profile
|
||||
self.api_key = api_key
|
||||
self.data_dir = data_dir
|
||||
self._client = client
|
||||
self._http_transport_factory = http_transport_factory
|
||||
self._client_lock = asyncio.Lock()
|
||||
self._write_lock = asyncio.Lock()
|
||||
self._last_bindings: HyWriteBindings | None = None
|
||||
@@ -427,6 +429,13 @@ class HyMemoryBridge:
|
||||
)
|
||||
client = await self._get_client()
|
||||
guards = getattr(client, "_one_memory_write_guards", ())
|
||||
from one_memory.hy_response_guard import HyReconcileGuard
|
||||
|
||||
reconcile_guards = tuple(
|
||||
guard for guard in guards if isinstance(guard, HyReconcileGuard)
|
||||
)
|
||||
for guard in reconcile_guards:
|
||||
guard.begin_write()
|
||||
before = [len(guard.failures) for guard in guards]
|
||||
bindings = HyWriteBindings(request.source_context) if request.source_context else None
|
||||
if bindings:
|
||||
@@ -500,6 +509,12 @@ class HyMemoryBridge:
|
||||
if len(guard.failures) > count:
|
||||
raise HyMemoryBridgeError(guard.failures[count])
|
||||
result = await self._result(client, request, raw, expected_contents=None, bindings=bindings)
|
||||
result = await _with_native_mutation_delta(
|
||||
client,
|
||||
result,
|
||||
raw,
|
||||
reconcile_guards,
|
||||
)
|
||||
if bindings:
|
||||
outputs = []
|
||||
for memory in result.memories:
|
||||
@@ -553,6 +568,8 @@ class HyMemoryBridge:
|
||||
from one_memory.hy_response_guard import HyReconcileGuard
|
||||
|
||||
report_guards = tuple(guard for guard in guards if isinstance(guard, HyReconcileGuard))
|
||||
for guard in report_guards:
|
||||
guard.begin_write()
|
||||
expected_contents = frozenset(content.strip() for content in contents if content.strip())
|
||||
for guard in report_guards:
|
||||
guard.imported_contents = expected_contents
|
||||
@@ -599,6 +616,12 @@ class HyMemoryBridge:
|
||||
raw,
|
||||
expected_contents=expected_contents,
|
||||
)
|
||||
result = await _with_native_mutation_delta(
|
||||
client,
|
||||
result,
|
||||
raw,
|
||||
report_guards,
|
||||
)
|
||||
diagnostic_event(
|
||||
level="info",
|
||||
component="hy.bridge",
|
||||
@@ -1087,6 +1110,7 @@ class HyMemoryBridge:
|
||||
self.profile,
|
||||
api_key=self.api_key,
|
||||
data_dir=self.data_dir,
|
||||
http_transport_factory=self._http_transport_factory,
|
||||
)
|
||||
|
||||
await _joined_thread(initialize)
|
||||
@@ -1136,6 +1160,7 @@ def create_hy_memory_client(
|
||||
*,
|
||||
api_key: str | None,
|
||||
data_dir: Path,
|
||||
http_transport_factory: Callable[[], Any] | None = None,
|
||||
) -> HyMemoryClientProtocol:
|
||||
configure_hy_memory_environment(data_dir)
|
||||
config = build_hy_memory_config(
|
||||
@@ -1143,6 +1168,7 @@ def create_hy_memory_client(
|
||||
api_key=api_key,
|
||||
data_dir=data_dir,
|
||||
)
|
||||
config.http_transport_factory = http_transport_factory
|
||||
reattach_legacy_vector_collection(profile, config)
|
||||
mismatch = inspect_graph_width_mismatch(config)
|
||||
if mismatch is not None:
|
||||
@@ -1287,6 +1313,74 @@ async def _coding_result(
|
||||
return HyMemoryIngestResult(success=True, memories=tuple(outputs))
|
||||
|
||||
|
||||
async def _with_native_mutation_delta(
|
||||
client: HyMemoryClientProtocol,
|
||||
result: HyMemoryIngestResult,
|
||||
raw: Mapping[str, object],
|
||||
reconcile_guards: Sequence[Any],
|
||||
) -> HyMemoryIngestResult:
|
||||
"""Map native SDK mutations back into MemHub governance.
|
||||
|
||||
Store permission wrappers used to discover this by intercepting every
|
||||
native CRUD call. The embedded SDK now owns unrestricted local CRUD, so
|
||||
chat targets come from the reconcile observer and coding targets come from
|
||||
the SDK's public result document.
|
||||
"""
|
||||
|
||||
retired = set(result.retired_memory_ids)
|
||||
superseded = {
|
||||
key: tuple(values) for key, values in result.superseded_memory_ids.items()
|
||||
}
|
||||
chat_targets = {
|
||||
key
|
||||
for guard in reconcile_guards
|
||||
for key in getattr(guard, "mutation_target_ids", ())
|
||||
if isinstance(key, str) and key
|
||||
}
|
||||
|
||||
for item in raw.get("ops", ()) if isinstance(raw.get("ops"), list) else ():
|
||||
if not isinstance(item, Mapping):
|
||||
continue
|
||||
if item.get("action") == "DELETE":
|
||||
key = item.get("target_experience_id")
|
||||
if isinstance(key, str) and key:
|
||||
retired.add(key)
|
||||
|
||||
for item in raw.get("fact_ops", ()) if isinstance(raw.get("fact_ops"), list) else ():
|
||||
if not isinstance(item, Mapping) or item.get("action") != "SUPERSEDE":
|
||||
continue
|
||||
old, new = item.get("fact_id"), item.get("superseded_by")
|
||||
if isinstance(old, str) and old:
|
||||
retired.add(old)
|
||||
if isinstance(new, str) and new:
|
||||
superseded[old] = (new,)
|
||||
|
||||
for key in chat_targets:
|
||||
record = await _joined_thread(_read_hy_record, client, key)
|
||||
if record is None:
|
||||
retired.add(key)
|
||||
continue
|
||||
status = str(record.get("status", "")).lower()
|
||||
if status in {"deleted", "shadow", "superseded"}:
|
||||
retired.add(key)
|
||||
replacement = record.get("superseded_by")
|
||||
if isinstance(replacement, str) and replacement:
|
||||
superseded[key] = (replacement,)
|
||||
elif isinstance(replacement, list):
|
||||
values = tuple(
|
||||
value for value in replacement if isinstance(value, str) and value
|
||||
)
|
||||
if values:
|
||||
superseded[key] = values
|
||||
|
||||
return result.model_copy(
|
||||
update={
|
||||
"retired_memory_ids": tuple(sorted(retired)),
|
||||
"superseded_memory_ids": superseded,
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
def _read_hy_record(
|
||||
client: HyMemoryClientProtocol,
|
||||
memory_id: str,
|
||||
|
||||
@@ -1,607 +0,0 @@
|
||||
"""Operation-bound read/write capability for the pinned native HY coding store.
|
||||
|
||||
HY's coding route writes two layers, experiences and working facts. Rows this
|
||||
lease cannot attest stay in the same backend but are invisible here, so they
|
||||
never reach a coding prompt and cannot be updated, superseded or linked.
|
||||
|
||||
HY's reconcilers catch store exceptions and carry on, so a denial latches and
|
||||
recheck() raises it again at finish; a swallowed denial still fails the task.
|
||||
Any native method not implemented here is denied rather than passed through.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
from collections.abc import Iterator, Mapping, Sequence
|
||||
from copy import deepcopy
|
||||
from hashlib import sha256
|
||||
from types import MappingProxyType
|
||||
from typing import Any, NoReturn
|
||||
|
||||
from hy_memory.coding.types import CodingMemory, WorkingFact # type: ignore[import-untyped]
|
||||
|
||||
from one_memory.hy_permission_store import HyNodePermission
|
||||
from one_memory.model_processing_permission_guard import (
|
||||
ModelProcessingPermissionLease,
|
||||
permission_error,
|
||||
)
|
||||
|
||||
_NATIVE_WRITES = frozenset(
|
||||
{
|
||||
"insert",
|
||||
"update",
|
||||
"delete",
|
||||
"insert_fact",
|
||||
"supersede_fact",
|
||||
"set_fact_derived_experiences",
|
||||
"set_experience_source_facts",
|
||||
# Trace rows are raw conversation: writes must be gated like any other.
|
||||
"append_trace",
|
||||
}
|
||||
)
|
||||
|
||||
_TIMESTAMPS = frozenset({"created_at", "updated_at"})
|
||||
|
||||
# SDK 2.0.1 reads `config` off the coding store while routing a coding
|
||||
# turn. The wrapper must answer it, but the native `MemoryConfig` carries
|
||||
# credentials (llm/embedder api keys, graph and cache passwords) and is a live
|
||||
# handle into the backend, so it is never handed out. Only this redacted
|
||||
# projection is exposed: credential-bearing keys collapse to None.
|
||||
_CONFIG_REDACTED_KEYS = frozenset(
|
||||
{
|
||||
"api_key",
|
||||
"password",
|
||||
"username",
|
||||
"mysql_password",
|
||||
"mysql_user",
|
||||
"extra_headers",
|
||||
"extra_body",
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
class _RedactedConfig(Mapping[str, Any]):
|
||||
"""Read-only config view: readable by key and attribute, never the native one.
|
||||
|
||||
The SDK reaches config values both ways, so this keeps attribute access
|
||||
working. It is deliberately not the native MemoryConfig: it exposes only
|
||||
plain copied values, so no backend handle, connection or callable escapes.
|
||||
"""
|
||||
|
||||
__slots__ = ("_values",)
|
||||
|
||||
def __init__(self, values: Mapping[str, Any]) -> None:
|
||||
object.__setattr__(self, "_values", MappingProxyType(dict(values)))
|
||||
|
||||
def __getitem__(self, key: str) -> Any:
|
||||
return self._values[key]
|
||||
|
||||
def __iter__(self) -> Iterator[str]:
|
||||
return iter(self._values)
|
||||
|
||||
def __len__(self) -> int:
|
||||
return len(self._values)
|
||||
|
||||
def __getattr__(self, name: str) -> Any:
|
||||
values = object.__getattribute__(self, "_values")
|
||||
try:
|
||||
return values[name]
|
||||
except KeyError:
|
||||
raise AttributeError(name) from None
|
||||
|
||||
def __setattr__(self, name: str, value: Any) -> NoReturn:
|
||||
raise AttributeError("redacted config is read-only")
|
||||
|
||||
def __repr__(self) -> str:
|
||||
return f"_RedactedConfig({sorted(self._values)})"
|
||||
|
||||
|
||||
def _redact_config(value: Any, depth: int = 0) -> Any:
|
||||
"""Copy config data into read-only views, collapsing credentials to None.
|
||||
|
||||
The SDK reads this both by attribute and by key, so each config section
|
||||
becomes a mapping-like view rather than a plain dict: attribute access stays
|
||||
available while the native object itself is never handed out.
|
||||
"""
|
||||
if depth > 4:
|
||||
return None
|
||||
if value is None or isinstance(value, (str, int, float, bool)):
|
||||
return value
|
||||
if isinstance(value, Mapping):
|
||||
return _RedactedConfig(
|
||||
{
|
||||
key: None if key in _CONFIG_REDACTED_KEYS else _redact_config(item, depth + 1)
|
||||
for key, item in value.items()
|
||||
}
|
||||
)
|
||||
if isinstance(value, (list, tuple)):
|
||||
return [_redact_config(item, depth + 1) for item in value]
|
||||
if hasattr(value, "__dict__") and type(value).__module__.startswith("hy_memory"):
|
||||
return _redact_config(vars(value), depth + 1)
|
||||
return None
|
||||
|
||||
|
||||
def _config_projection(native: Any) -> Any:
|
||||
try:
|
||||
return _redact_config(native.config)
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
|
||||
def hy_coding_fingerprint(row: CodingMemory | WorkingFact) -> str:
|
||||
"""Hash the complete native row as read back, including links and timestamps."""
|
||||
if type(row) is CodingMemory:
|
||||
kind = "experience"
|
||||
elif type(row) is WorkingFact:
|
||||
kind = "fact"
|
||||
else:
|
||||
raise permission_error("hy_coding_store_row_shape")
|
||||
payload = {"kind": kind, **row.to_dict()}
|
||||
return sha256(
|
||||
json.dumps(payload, sort_keys=True, separators=(",", ":"), allow_nan=False).encode()
|
||||
).hexdigest()
|
||||
|
||||
|
||||
def _row_id(row: CodingMemory | WorkingFact) -> str:
|
||||
return str(row.experience_id if type(row) is CodingMemory else row.fact_id)
|
||||
|
||||
|
||||
class HyPermissionCodingStore:
|
||||
"""Closed, task-exclusive adapter; initialize/close never own the native store.
|
||||
|
||||
node_permissions is the same attested ledger the chat vector store uses;
|
||||
coding rows are keyed by experience or fact id. written_nodes carries the
|
||||
whole lease footprint for each row written here, as the chat store does.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
native: Any,
|
||||
lease: ModelProcessingPermissionLease,
|
||||
*,
|
||||
agent_id: str,
|
||||
session_id: str,
|
||||
node_permissions: Mapping[str, HyNodePermission] | None = None,
|
||||
) -> None:
|
||||
self._native = native
|
||||
self._lease = lease
|
||||
self._agent_id = agent_id
|
||||
self._session_id = session_id
|
||||
self._user_id = lease.user_id
|
||||
self._failure: str | None = None
|
||||
self._closed = False
|
||||
self._lock = asyncio.Lock()
|
||||
self._deleted: set[str] = set()
|
||||
self._written: dict[str, HyNodePermission] = {}
|
||||
self._superseded: dict[str, str] = {}
|
||||
self._evidence_ids = tuple(record.evidence_id for record in lease.records)
|
||||
self._permissions = {
|
||||
key: proof
|
||||
for key, proof in (node_permissions or {}).items()
|
||||
if set(proof.evidence_ids) <= set(self._evidence_ids)
|
||||
}
|
||||
self.recheck()
|
||||
if not agent_id or agent_id == "*" or not session_id or not self._evidence_ids:
|
||||
self._deny("scope")
|
||||
|
||||
def __getattr__(self, name: str) -> NoReturn:
|
||||
if name.startswith("__"):
|
||||
raise AttributeError(name)
|
||||
self._deny("unsupported_operation")
|
||||
|
||||
@property
|
||||
def written_nodes(self) -> Mapping[str, HyNodePermission]:
|
||||
self.recheck()
|
||||
return MappingProxyType(dict(self._written))
|
||||
|
||||
@property
|
||||
def config(self) -> Any:
|
||||
"""Read-only, redacted view of the native coding config.
|
||||
|
||||
SDK 2.0.1 reads `config` when it routes a coding turn. Answering
|
||||
it is safe only as a projection: the native MemoryConfig is a live
|
||||
backend handle and carries credentials, so callers get plain values
|
||||
with every credential-bearing key collapsed to None. This carries no
|
||||
identity, permission or tenant state and is not the native object.
|
||||
"""
|
||||
self._assert_open()
|
||||
return _config_projection(self._native)
|
||||
|
||||
@property
|
||||
def retired_ids(self) -> frozenset[str]:
|
||||
"""Experiences HY deleted in this operation."""
|
||||
self.recheck()
|
||||
return frozenset(self._deleted)
|
||||
|
||||
@property
|
||||
def superseded_ids(self) -> Mapping[str, tuple[str, ...]]:
|
||||
"""Facts HY superseded in this operation, with the fact that replaced each.
|
||||
|
||||
A successor linked before it existed must have been written here by
|
||||
now; otherwise the old fact points at a row nobody attested.
|
||||
"""
|
||||
self.recheck()
|
||||
if any(new not in self._written for new in self._superseded.values()):
|
||||
self._deny("link_denied")
|
||||
return MappingProxyType({old: (new,) for old, new in self._superseded.items()})
|
||||
|
||||
def recheck(self) -> None:
|
||||
self._assert_open()
|
||||
try:
|
||||
self._lease.recheck()
|
||||
except Exception as error:
|
||||
code = str(error)
|
||||
self._failure = (
|
||||
code
|
||||
if code.startswith("model_content_") and len(code) <= 100
|
||||
else "model_content_hy_coding_store_lease_failed"
|
||||
)
|
||||
raise permission_error(self._failure) from None
|
||||
|
||||
def _assert_open(self) -> None:
|
||||
if self._closed:
|
||||
self._deny("closed")
|
||||
if self._failure:
|
||||
raise permission_error(self._failure)
|
||||
|
||||
async def close(self) -> None:
|
||||
async with self._lock:
|
||||
self._closed = True
|
||||
self._permissions.clear()
|
||||
self._written.clear()
|
||||
|
||||
def _deny(self, suffix: str) -> NoReturn:
|
||||
self._failure = self._failure or f"model_content_hy_coding_store_{suffix}"
|
||||
raise permission_error(self._failure)
|
||||
|
||||
def _scope(self, user_id: str | None) -> None:
|
||||
if user_id not in (None, self._user_id):
|
||||
self._deny("scope")
|
||||
|
||||
async def _invoke(self, method: str, *args: Any, **kwargs: Any) -> Any:
|
||||
writing = method in _NATIVE_WRITES
|
||||
self.recheck() if writing else self._assert_open()
|
||||
try:
|
||||
result = await getattr(self._native, method)(*args, **kwargs)
|
||||
except Exception:
|
||||
self._deny("backend_failed")
|
||||
self.recheck() if writing else self._assert_open()
|
||||
return result
|
||||
|
||||
def _visible(self, row: Any, expected: Any) -> bool:
|
||||
if row is None:
|
||||
return False
|
||||
if type(row) is not expected:
|
||||
self._deny("row_shape")
|
||||
key = _row_id(row)
|
||||
proof = self._permissions.get(key)
|
||||
if proof is None or key in self._deleted or row.user_id != self._user_id:
|
||||
return False
|
||||
if hy_coding_fingerprint(row) != proof.fingerprint:
|
||||
self._deny("row_changed")
|
||||
return True
|
||||
|
||||
async def _experience(self, key: str) -> CodingMemory | None:
|
||||
if key not in self._permissions or key in self._deleted:
|
||||
return None
|
||||
row = await self._invoke("get_by_id", key, user_id=self._user_id)
|
||||
return deepcopy(row) if self._visible(row, CodingMemory) else None
|
||||
|
||||
async def _fact(self, key: str) -> WorkingFact | None:
|
||||
if key not in self._permissions or key in self._deleted:
|
||||
return None
|
||||
rows = await self._invoke("get_facts", [key], user_id=self._user_id)
|
||||
row = rows[0] if rows else None
|
||||
return deepcopy(row) if self._visible(row, WorkingFact) else None
|
||||
|
||||
def _new_row_in_scope(self, row: CodingMemory | WorkingFact) -> bool:
|
||||
return bool(
|
||||
row.user_id == self._user_id
|
||||
and row.agent_id == self._agent_id
|
||||
and row.session_id == self._session_id
|
||||
)
|
||||
|
||||
async def _capture(
|
||||
self, key: str, kind: Any, expected: CodingMemory | WorkingFact | None = None
|
||||
) -> None:
|
||||
row: Any
|
||||
if kind is CodingMemory:
|
||||
row = await self._invoke("get_by_id", key, user_id=self._user_id)
|
||||
else:
|
||||
rows = await self._invoke("get_facts", [key], user_id=self._user_id)
|
||||
row = rows[0] if rows else None
|
||||
if type(row) is not kind or row.user_id != self._user_id:
|
||||
self._deny("write_mismatch")
|
||||
if expected is not None:
|
||||
stored, planned = row.to_dict(), expected.to_dict()
|
||||
for field in _TIMESTAMPS:
|
||||
stored.pop(field, None)
|
||||
planned.pop(field, None)
|
||||
if stored != planned:
|
||||
self._deny("write_mismatch")
|
||||
proof = HyNodePermission(hy_coding_fingerprint(row), self._evidence_ids)
|
||||
self._permissions[key] = proof
|
||||
self._written[key] = proof
|
||||
|
||||
async def _plan_new(self, row: CodingMemory | WorkingFact, kind: Any) -> None:
|
||||
if type(row) is not kind or not self._new_row_in_scope(row):
|
||||
self._deny("scope")
|
||||
key = _row_id(row)
|
||||
if not key or key in self._permissions or key in self._deleted:
|
||||
self._deny("scope")
|
||||
if kind is CodingMemory:
|
||||
existing = await self._invoke("get_by_id", key)
|
||||
else:
|
||||
existing = next(iter(await self._invoke("get_facts", [key])), None)
|
||||
if existing is not None:
|
||||
self._deny("scope")
|
||||
|
||||
# Reads
|
||||
|
||||
async def get_by_id(
|
||||
self, experience_id: str, user_id: str | None = None
|
||||
) -> CodingMemory | None:
|
||||
async with self._lock:
|
||||
self._scope(user_id)
|
||||
return await self._experience(experience_id)
|
||||
|
||||
async def get_many(
|
||||
self, experience_ids: list[str], user_id: str | None = None
|
||||
) -> list[CodingMemory]:
|
||||
async with self._lock:
|
||||
self._scope(user_id)
|
||||
wanted = [
|
||||
key
|
||||
for key in dict.fromkeys(experience_ids)
|
||||
if key in self._permissions and key not in self._deleted
|
||||
]
|
||||
if not wanted:
|
||||
return []
|
||||
rows = await self._invoke("get_many", wanted, user_id=self._user_id)
|
||||
return [deepcopy(row) for row in rows if self._visible(row, CodingMemory)]
|
||||
|
||||
async def search_by_query_embedding(
|
||||
self,
|
||||
query_embedding: list[float],
|
||||
user_id: str,
|
||||
*,
|
||||
workspace_id: str | None = None,
|
||||
branch: str | None = None,
|
||||
top: int = 40,
|
||||
) -> list[dict[str, Any]]:
|
||||
async with self._lock:
|
||||
self._scope(user_id)
|
||||
hits = await self._invoke(
|
||||
"search_by_query_embedding",
|
||||
deepcopy(query_embedding),
|
||||
self._user_id,
|
||||
workspace_id=workspace_id,
|
||||
branch=branch,
|
||||
top=top * 4,
|
||||
)
|
||||
result = []
|
||||
for hit in hits:
|
||||
key = hit.get("experience_id")
|
||||
if not isinstance(key, str) or await self._experience(key) is None:
|
||||
continue
|
||||
result.append(
|
||||
{
|
||||
"experience_id": key,
|
||||
"score": float(hit.get("score", 0.0)),
|
||||
"matched_key_kind": hit.get("matched_key_kind"),
|
||||
"matched_key_text": hit.get("matched_key_text"),
|
||||
}
|
||||
)
|
||||
return result[:top]
|
||||
|
||||
async def get_fact(self, fact_id: str, user_id: str | None = None) -> WorkingFact | None:
|
||||
async with self._lock:
|
||||
self._scope(user_id)
|
||||
return await self._fact(fact_id)
|
||||
|
||||
async def get_facts(self, fact_ids: list[str], user_id: str | None = None) -> list[WorkingFact]:
|
||||
async with self._lock:
|
||||
self._scope(user_id)
|
||||
found = [await self._fact(key) for key in dict.fromkeys(fact_ids)]
|
||||
return [row for row in found if row is not None]
|
||||
|
||||
async def search_facts(
|
||||
self,
|
||||
query_embedding: list[float],
|
||||
user_id: str,
|
||||
*,
|
||||
workspace_id: str | None = None,
|
||||
branch: str | None = None,
|
||||
top: int = 5,
|
||||
query_text: str | None = None,
|
||||
min_score: float = 0.0,
|
||||
) -> list[tuple[WorkingFact, float]]:
|
||||
async with self._lock:
|
||||
self._scope(user_id)
|
||||
hits = await self._invoke(
|
||||
"search_facts",
|
||||
deepcopy(query_embedding),
|
||||
self._user_id,
|
||||
workspace_id=workspace_id,
|
||||
branch=branch,
|
||||
top=top * 4,
|
||||
query_text=query_text,
|
||||
min_score=min_score,
|
||||
)
|
||||
result = []
|
||||
for row, score in hits:
|
||||
if self._visible(row, WorkingFact):
|
||||
result.append((deepcopy(row), float(score)))
|
||||
return result[:top]
|
||||
|
||||
async def get_fact_ancestors(
|
||||
self, fact_id: str, *, user_id: str | None = None, max_depth: int = 8
|
||||
) -> list[WorkingFact]:
|
||||
async with self._lock:
|
||||
self._scope(user_id)
|
||||
if await self._fact(fact_id) is None:
|
||||
return []
|
||||
rows = await self._invoke(
|
||||
"get_fact_ancestors", fact_id, user_id=self._user_id, max_depth=max_depth
|
||||
)
|
||||
# Oldest first; the chain is only as long as its nearest unapproved link.
|
||||
kept: list[WorkingFact] = []
|
||||
for row in reversed(rows):
|
||||
if not self._visible(row, WorkingFact):
|
||||
break
|
||||
kept.append(deepcopy(row))
|
||||
return list(reversed(kept))
|
||||
|
||||
# Writes
|
||||
|
||||
async def insert(self, memory: CodingMemory) -> None:
|
||||
async with self._lock:
|
||||
self.recheck()
|
||||
await self._plan_new(memory, CodingMemory)
|
||||
await self._invoke("insert", memory)
|
||||
await self._capture(memory.experience_id, CodingMemory, memory)
|
||||
|
||||
async def update(self, memory: CodingMemory) -> None:
|
||||
async with self._lock:
|
||||
self.recheck()
|
||||
if type(memory) is not CodingMemory or not self._new_row_in_scope(memory):
|
||||
self._deny("scope")
|
||||
if await self._experience(memory.experience_id) is None:
|
||||
self._deny("target_denied")
|
||||
await self._invoke("update", memory)
|
||||
await self._capture(memory.experience_id, CodingMemory, memory)
|
||||
|
||||
async def delete(self, experience_id: str, user_id: str) -> bool:
|
||||
async with self._lock:
|
||||
self._scope(user_id)
|
||||
if await self._experience(experience_id) is None:
|
||||
self._deny("target_denied")
|
||||
if await self._invoke("delete", experience_id, user_id=self._user_id) is not True:
|
||||
self._deny("write_failed")
|
||||
if await self._invoke("get_by_id", experience_id) is not None:
|
||||
self._deny("write_mismatch")
|
||||
self._deleted.add(experience_id)
|
||||
self._permissions.pop(experience_id, None)
|
||||
self._written.pop(experience_id, None)
|
||||
return True
|
||||
|
||||
async def insert_fact(self, fact: WorkingFact) -> None:
|
||||
async with self._lock:
|
||||
self.recheck()
|
||||
await self._plan_new(fact, WorkingFact)
|
||||
await self._invoke("insert_fact", fact)
|
||||
await self._capture(fact.fact_id, WorkingFact, fact)
|
||||
|
||||
async def supersede_fact(self, old_fact_id: str, new_fact_id: str) -> bool:
|
||||
async with self._lock:
|
||||
self.recheck()
|
||||
if await self._fact(old_fact_id) is None or old_fact_id == new_fact_id:
|
||||
self._deny("target_denied")
|
||||
# HY links the old fact before it inserts the new one. A successor
|
||||
# that does not exist yet may only become a row this lease writes;
|
||||
# an existing row it cannot attest stays out of reach.
|
||||
if await self._fact(new_fact_id) is None:
|
||||
if not new_fact_id or new_fact_id in self._deleted:
|
||||
self._deny("target_denied")
|
||||
if next(iter(await self._invoke("get_facts", [new_fact_id])), None) is not None:
|
||||
self._deny("target_denied")
|
||||
changed = await self._invoke("supersede_fact", old_fact_id, new_fact_id)
|
||||
await self._capture(old_fact_id, WorkingFact)
|
||||
if changed:
|
||||
self._superseded[old_fact_id] = new_fact_id
|
||||
return bool(changed)
|
||||
|
||||
async def set_fact_derived_experiences(self, fact_id: str, experience_ids: list[str]) -> None:
|
||||
async with self._lock:
|
||||
self.recheck()
|
||||
if await self._fact(fact_id) is None:
|
||||
self._deny("target_denied")
|
||||
for key in experience_ids:
|
||||
if await self._experience(key) is None:
|
||||
self._deny("link_denied")
|
||||
await self._invoke("set_fact_derived_experiences", fact_id, list(experience_ids))
|
||||
await self._capture(fact_id, WorkingFact)
|
||||
|
||||
async def set_experience_source_facts(self, experience_id: str, fact_ids: list[str]) -> None:
|
||||
async with self._lock:
|
||||
self.recheck()
|
||||
if await self._experience(experience_id) is None:
|
||||
self._deny("target_denied")
|
||||
for key in fact_ids:
|
||||
if await self._fact(key) is None:
|
||||
self._deny("link_denied")
|
||||
await self._invoke("set_experience_source_facts", experience_id, list(fact_ids))
|
||||
await self._capture(experience_id, CodingMemory)
|
||||
|
||||
async def get_recent_trace_messages(
|
||||
self,
|
||||
*,
|
||||
user_id: str,
|
||||
agent_id: str = "default_agent",
|
||||
max_user_turns: int = 5,
|
||||
) -> list[Any]:
|
||||
"""Read recent raw trace rows, still bounded by this lease.
|
||||
|
||||
Public SDK 2.0.1 asks for recent history before a coding write. The native call is
|
||||
scoped to the lease's own user and agent, and rows this lease cannot
|
||||
attest are filtered out, so unapproved conversations never reach a
|
||||
coding prompt through this path.
|
||||
"""
|
||||
self._scope(user_id)
|
||||
if agent_id not in (self._agent_id, "*"):
|
||||
self._deny("scope")
|
||||
rows = await self._invoke(
|
||||
"get_recent_trace_messages",
|
||||
user_id=self._user_id,
|
||||
agent_id=self._agent_id,
|
||||
max_user_turns=max_user_turns,
|
||||
)
|
||||
return [row for row in (rows or []) if self._trace_visible(row)]
|
||||
|
||||
async def append_trace(
|
||||
self,
|
||||
messages: Sequence[Any],
|
||||
*,
|
||||
request_id: str,
|
||||
user_id: str,
|
||||
agent_id: str = "default_agent",
|
||||
session_id: str | None = None,
|
||||
workspace_id: str | None = None,
|
||||
branch: str | None = None,
|
||||
) -> None:
|
||||
"""Append raw trace rows for this lease only.
|
||||
|
||||
Trace rows are raw conversation, so this is a write path: the lease is
|
||||
rechecked around the call and every message must belong to the leased
|
||||
user and session. Nothing outside the lease may be recorded.
|
||||
"""
|
||||
self._scope(user_id)
|
||||
if agent_id not in (self._agent_id, "*"):
|
||||
self._deny("scope")
|
||||
if session_id not in (None, self._session_id):
|
||||
self._deny("scope")
|
||||
for row in messages or []:
|
||||
if not self._trace_visible(row):
|
||||
self._deny("scope")
|
||||
await self._invoke(
|
||||
"append_trace",
|
||||
list(messages or []),
|
||||
request_id=request_id,
|
||||
user_id=self._user_id,
|
||||
agent_id=self._agent_id,
|
||||
session_id=self._session_id,
|
||||
workspace_id=workspace_id,
|
||||
branch=branch,
|
||||
)
|
||||
|
||||
def _trace_visible(self, row: Any) -> bool:
|
||||
"""A trace row may be read or written only inside this lease's scope."""
|
||||
if row is None:
|
||||
return False
|
||||
owner = getattr(row, "user_id", None)
|
||||
session = getattr(row, "session_id", None)
|
||||
return not (
|
||||
(owner is not None and owner != self._user_id)
|
||||
or (session is not None and session != self._session_id)
|
||||
)
|
||||
@@ -1,582 +0,0 @@
|
||||
"""Task-exclusive HY wiring; provenance metadata stays beside the existing Core DB."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
from collections.abc import AsyncIterator, Callable
|
||||
from contextlib import asynccontextmanager, nullcontext, suppress
|
||||
from pathlib import Path
|
||||
from typing import Any, cast
|
||||
|
||||
import httpx
|
||||
|
||||
from one_memory.database import take_the_write_lock_first
|
||||
from one_memory.database_access import main_database
|
||||
from one_memory.hy_memory_bridge import (
|
||||
HyMemoryBridge,
|
||||
HyMemoryBridgeError,
|
||||
HyMemoryIngestRequest,
|
||||
HyMemoryIngestResult,
|
||||
HyMemoryOutput,
|
||||
LocalHyEmbedService,
|
||||
_joined_thread,
|
||||
)
|
||||
from one_memory.hy_permission_coding_store import HyPermissionCodingStore
|
||||
from one_memory.hy_permission_store import (
|
||||
HyNodePermission,
|
||||
HyPermissionGraphStore,
|
||||
HyPermissionVectorStore,
|
||||
)
|
||||
from one_memory.hy_permission_transport import HyPermissionTransport, HyPermissionTransportError
|
||||
from one_memory.hy_response_guard import HyResponseGuard
|
||||
from one_memory.hy_step_timing import HyStepTimer
|
||||
from one_memory.memory_server.migrations import hy_node_permissions_ddl
|
||||
from one_memory.memory_tasks import evidence_ref
|
||||
from one_memory.model_processing_permission_guard import ModelProcessingPermissionLease
|
||||
|
||||
|
||||
class _NativeProofs:
|
||||
def __init__(self, database: Path) -> None:
|
||||
self.database = database
|
||||
self._loaded_states: dict[str, tuple[int, str]] | None = None
|
||||
with main_database(database).write() as db, db:
|
||||
# Schema owned by `memory_server.migrations` (migration 26).
|
||||
db.execute(hy_node_permissions_ddl("sqlite"))
|
||||
|
||||
def load(self, lease: ModelProcessingPermissionLease) -> dict[str, HyNodePermission]:
|
||||
allowed = {record.evidence_id: evidence_ref(record) for record in lease.records}
|
||||
with main_database(self.database).read() as db:
|
||||
rows = db.execute(
|
||||
"SELECT p.node_id,p.fingerprint,p.evidence_json,g.revision,g.status "
|
||||
"FROM hy_node_permissions p LEFT JOIN governed_memories g ON g.memory_id=p.node_id "
|
||||
# Superseded nodes stay attested: every later head links back to them.
|
||||
"WHERE g.status IS NULL OR g.status IN ('confirmed','superseded')"
|
||||
).fetchall()
|
||||
proofs = {}
|
||||
states = {}
|
||||
for identifier, fingerprint, evidence_json, revision, status in rows:
|
||||
refs = json.loads(evidence_json)
|
||||
if (
|
||||
not isinstance(refs, list)
|
||||
or not refs
|
||||
or not all(isinstance(ref, dict) for ref in refs)
|
||||
):
|
||||
continue
|
||||
if all(allowed.get(ref.get("evidence_id")) == ref for ref in refs):
|
||||
proofs[identifier] = HyNodePermission(
|
||||
fingerprint, tuple(ref["evidence_id"] for ref in refs)
|
||||
)
|
||||
if revision is not None:
|
||||
states[identifier] = (revision, status)
|
||||
self._loaded_states = states
|
||||
return proofs
|
||||
|
||||
def guard(self, values: dict[str, HyNodePermission]) -> Callable[[], None]:
|
||||
identifiers = tuple(values)
|
||||
|
||||
def state() -> dict[str, tuple[int, str]]:
|
||||
found: dict[str, tuple[int, str]] = {}
|
||||
with main_database(self.database).read() as db:
|
||||
for start in range(0, len(identifiers), 900):
|
||||
batch = identifiers[start : start + 900]
|
||||
placeholders = ",".join("?" for _ in batch)
|
||||
for identifier, revision, status in db.execute(
|
||||
"SELECT memory_id,revision,status FROM governed_memories "
|
||||
f"WHERE memory_id IN ({placeholders})",
|
||||
batch,
|
||||
):
|
||||
found[identifier] = (revision, status)
|
||||
return found
|
||||
|
||||
if self._loaded_states is None:
|
||||
raise HyMemoryBridgeError("model_content_context_unverified")
|
||||
before = dict(self._loaded_states)
|
||||
|
||||
def check() -> None:
|
||||
if state() != before:
|
||||
raise HyMemoryBridgeError("model_content_context_changed")
|
||||
|
||||
check()
|
||||
return check
|
||||
|
||||
def save(
|
||||
self, lease: ModelProcessingPermissionLease, values: dict[str, HyNodePermission]
|
||||
) -> None:
|
||||
refs = {record.evidence_id: evidence_ref(record) for record in lease.records}
|
||||
with main_database(self.database).write(autocommit=True) as db:
|
||||
take_the_write_lock_first(db)
|
||||
with db:
|
||||
lease.recheck()
|
||||
db.executemany(
|
||||
"INSERT INTO hy_node_permissions VALUES (?,?,?) ON CONFLICT(node_id) "
|
||||
"DO UPDATE SET fingerprint=excluded.fingerprint,"
|
||||
"evidence_json=excluded.evidence_json",
|
||||
[
|
||||
(
|
||||
identifier,
|
||||
proof.fingerprint,
|
||||
json.dumps([refs[key] for key in proof.evidence_ids]),
|
||||
)
|
||||
for identifier, proof in values.items()
|
||||
],
|
||||
)
|
||||
|
||||
def refresh(
|
||||
self, lease: ModelProcessingPermissionLease, node_id: str
|
||||
) -> HyNodePermission | None:
|
||||
del lease
|
||||
with main_database(self.database).read() as db:
|
||||
row = db.execute(
|
||||
"SELECT fingerprint,evidence_json FROM hy_node_permissions WHERE node_id=?",
|
||||
(node_id,),
|
||||
).fetchone()
|
||||
if row is None:
|
||||
return None
|
||||
fingerprint = str(row["fingerprint"])
|
||||
refs = json.loads(row["evidence_json"])
|
||||
if (
|
||||
len(fingerprint) != 64
|
||||
or any(value not in "0123456789abcdef" for value in fingerprint)
|
||||
or not isinstance(refs, list)
|
||||
or not refs
|
||||
or not all(isinstance(ref, dict) for ref in refs)
|
||||
):
|
||||
return None
|
||||
evidence_ids = tuple(ref.get("evidence_id") for ref in refs)
|
||||
if any(not isinstance(value, str) or not value for value in evidence_ids):
|
||||
return None
|
||||
return HyNodePermission(fingerprint, evidence_ids)
|
||||
|
||||
|
||||
class _BoundLease:
|
||||
def __init__(
|
||||
self,
|
||||
lease: ModelProcessingPermissionLease,
|
||||
proof_check: Callable[[], None],
|
||||
step_timer: HyStepTimer | None = None,
|
||||
) -> None:
|
||||
self.lease = lease
|
||||
self.step_timer = step_timer
|
||||
self.user_id = lease.user_id
|
||||
self.profile = lease.profile
|
||||
self.task_id = lease.task_id
|
||||
self.records = lease.records
|
||||
self.source_context = lease.source_context
|
||||
self.proof_check = proof_check
|
||||
self.vector: HyPermissionVectorStore | None = None
|
||||
self.coding: HyPermissionCodingStore | None = None
|
||||
|
||||
def recheck(self) -> None:
|
||||
timer = self.step_timer
|
||||
with timer.step("permission_check") if timer is not None else nullcontext():
|
||||
self.lease.recheck()
|
||||
self.proof_check()
|
||||
if self.vector is not None:
|
||||
self.vector.recheck()
|
||||
if self.coding is not None:
|
||||
self.coding.recheck()
|
||||
|
||||
|
||||
class _Wiring:
|
||||
def __init__(
|
||||
self,
|
||||
client: Any,
|
||||
lease: ModelProcessingPermissionLease,
|
||||
proofs_store: _NativeProofs,
|
||||
proofs: dict[str, HyNodePermission],
|
||||
agent_id: str,
|
||||
session_id: str,
|
||||
transport: httpx.AsyncBaseTransport | None,
|
||||
proof_check: Callable[[], None],
|
||||
step_timer: HyStepTimer,
|
||||
) -> None:
|
||||
self.client, self.lease, self.proofs_store, self.proofs = (
|
||||
client,
|
||||
lease,
|
||||
proofs_store,
|
||||
proofs,
|
||||
)
|
||||
self.step_timer = step_timer
|
||||
self.agent_id, self.session_id, self.http_transport = agent_id, session_id, transport
|
||||
self.cross_evidence_scope = False
|
||||
self.proof_check = proof_check
|
||||
self.saved: list[tuple[Any, str, Any]] = []
|
||||
self.transport: HyPermissionTransport
|
||||
self.vector: HyPermissionVectorStore
|
||||
self.graph: HyPermissionGraphStore | None = None
|
||||
self.coding: HyPermissionCodingStore
|
||||
self.active = False
|
||||
|
||||
def replace(self, target: Any, name: str, value: Any) -> None:
|
||||
self.saved.append((target, name, getattr(target, name)))
|
||||
setattr(target, name, value)
|
||||
|
||||
async def enter(self) -> None:
|
||||
client, registry = self.client, self.client._registry
|
||||
config = client._config
|
||||
if config.llm.enable_summary or config.llm.enable_query_rewrite:
|
||||
raise HyMemoryBridgeError("model_content_context_unverified")
|
||||
bound = _BoundLease(self.lease, self.proof_check, self.step_timer)
|
||||
self.transport = await HyPermissionTransport(
|
||||
bound, http_transport=self.http_transport, step_timer=self.step_timer
|
||||
).__aenter__()
|
||||
self.active = True
|
||||
try:
|
||||
for guard in getattr(client, "_one_memory_write_guards", ()):
|
||||
if hasattr(type(guard), "step_timer"):
|
||||
self.replace(guard, "step_timer", self.step_timer)
|
||||
system2 = await registry.get_system2_writer()
|
||||
writers = (await registry.get_writer(), system2._writer)
|
||||
native_vector = client._vector_store
|
||||
store_lease = _BoundLease(self.lease, self.proof_check, self.step_timer)
|
||||
self.vector = HyPermissionVectorStore(
|
||||
native_vector,
|
||||
cast(ModelProcessingPermissionLease, store_lease),
|
||||
agent_id=self.agent_id,
|
||||
session_id=self.session_id,
|
||||
cross_evidence_scope=self.cross_evidence_scope,
|
||||
node_permissions=self.proofs,
|
||||
refresh_permission=lambda node_id: asyncio.to_thread(
|
||||
self.proofs_store.refresh, self.lease, node_id
|
||||
),
|
||||
step_timer=self.step_timer,
|
||||
)
|
||||
bound.vector = self.vector
|
||||
embed = client._embed_service
|
||||
if isinstance(embed, LocalHyEmbedService):
|
||||
self.transport.bind_local_embed_service(embed)
|
||||
else:
|
||||
self.transport.bind_embed_service(embed)
|
||||
for writer in writers:
|
||||
provider = writer.mem_agent.llm_provider
|
||||
self.transport.bind_provider(
|
||||
provider.provider if isinstance(provider, HyResponseGuard) else provider
|
||||
)
|
||||
reconciler = writer.reconciler
|
||||
provider = reconciler._get_llm()
|
||||
self.transport.bind_provider(
|
||||
provider.provider if isinstance(provider, HyResponseGuard) else provider
|
||||
)
|
||||
self.replace(writer, "_external_vector_store", self.vector)
|
||||
self.replace(writer, "_vector_store", self.vector)
|
||||
self.replace(writer, "_vector_store_initialized", True)
|
||||
self.replace(writer, "_cache", None)
|
||||
self.replace(client, "_vector_store", self.vector)
|
||||
native_graph = client._graph_store
|
||||
if self.cross_evidence_scope and native_graph is not None:
|
||||
self.graph = HyPermissionGraphStore(native_graph, self.proof_check)
|
||||
self.replace(client, "_graph_store", self.graph)
|
||||
else:
|
||||
self.replace(client, "_graph_store", None)
|
||||
self.replace(registry, "_shared_vector_store", self.vector)
|
||||
self.replace(
|
||||
registry,
|
||||
"_shared_graph_store",
|
||||
self.graph if self.graph is not None else None,
|
||||
)
|
||||
self.replace(registry, "_shared_cache", None)
|
||||
self.replace(system2, "_vector_store", self.vector)
|
||||
self.replace(
|
||||
system2,
|
||||
"_graph_store",
|
||||
self.graph if self.graph is not None else None,
|
||||
)
|
||||
self.replace(system2, "_cache", None)
|
||||
# Result fallback may call search; rebuild only its non-generative reader.
|
||||
self.replace(registry, "_reader", None)
|
||||
self.replace(registry, "_readers", {})
|
||||
await self._enter_coding(bound, embed)
|
||||
self.transport.assert_allowed()
|
||||
except BaseException:
|
||||
await self.exit()
|
||||
raise
|
||||
|
||||
async def _enter_coding(self, bound: _BoundLease, embed: Any) -> None:
|
||||
"""HY judges and writes the coding route itself; every seam it uses is bound here."""
|
||||
client = self.client
|
||||
# Created now, so the judge and every coding stage share one bound provider.
|
||||
writer = await client._get_coding_writer()
|
||||
stages = (writer, writer.extractor, writer.reconciler, writer.fact_reconciler)
|
||||
native_store = client._coding_store
|
||||
if any(
|
||||
stage.store is not native_store
|
||||
for stage in (writer, writer.reconciler, writer.fact_reconciler)
|
||||
) or any(
|
||||
item.embed_service is not embed
|
||||
for item in (native_store, writer, writer.reconciler, writer.fact_reconciler)
|
||||
):
|
||||
raise HyMemoryBridgeError("model_content_context_unverified")
|
||||
providers = {
|
||||
id(p): p for p in (client._coding_llm_provider, *(s.llm_provider for s in stages))
|
||||
}
|
||||
for provider in providers.values():
|
||||
self.transport.bind_provider(provider)
|
||||
self.coding = HyPermissionCodingStore(
|
||||
native_store,
|
||||
cast(
|
||||
ModelProcessingPermissionLease,
|
||||
_BoundLease(self.lease, self.proof_check, self.step_timer),
|
||||
),
|
||||
agent_id=self.agent_id,
|
||||
session_id=self.session_id,
|
||||
node_permissions=self.proofs,
|
||||
)
|
||||
bound.coding = self.coding
|
||||
for stage in (writer, writer.reconciler, writer.fact_reconciler):
|
||||
self.replace(stage, "store", self.coding)
|
||||
self.replace(client, "_coding_store", self.coding)
|
||||
for stage in (writer, writer.extractor, writer.reconciler):
|
||||
self.replace(stage, "cache", None)
|
||||
# The judge logs its prompt through the client's own cache.
|
||||
self.replace(client, "_cache", None)
|
||||
|
||||
async def finish(self) -> dict[str, HyNodePermission]:
|
||||
self.transport.assert_allowed()
|
||||
self.vector.recheck()
|
||||
self.coding.recheck()
|
||||
return {**self.vector.written_nodes, **self.coding.written_nodes}
|
||||
|
||||
async def retired(self) -> tuple[tuple[str, ...], dict[str, tuple[str, ...]]]:
|
||||
self.transport.assert_allowed()
|
||||
return (
|
||||
tuple(sorted(self.vector.retired_node_ids | self.coding.retired_ids)),
|
||||
{**self.vector.superseded_nodes, **self.coding.superseded_ids},
|
||||
)
|
||||
|
||||
async def exit(self) -> None:
|
||||
if not self.active:
|
||||
return
|
||||
self.active = False
|
||||
try:
|
||||
await self.transport.__aexit__(None, None, None)
|
||||
finally:
|
||||
for target, name, value in reversed(self.saved):
|
||||
setattr(target, name, value)
|
||||
self.saved.clear()
|
||||
if hasattr(self, "vector"):
|
||||
await self.vector.close()
|
||||
if self.graph is not None:
|
||||
await self.graph.close()
|
||||
if hasattr(self, "coding"):
|
||||
await self.coding.close()
|
||||
|
||||
|
||||
class _LeasedBridge:
|
||||
def __init__(
|
||||
self,
|
||||
bridge: HyMemoryBridge,
|
||||
lease: ModelProcessingPermissionLease,
|
||||
proofs: _NativeProofs,
|
||||
transport: httpx.AsyncBaseTransport | None,
|
||||
) -> None:
|
||||
self.bridge, self.lease, self.proofs, self.transport = bridge, lease, proofs, transport
|
||||
self.wiring: _Wiring | None = None
|
||||
self.finished = False
|
||||
self.step_timer = HyStepTimer()
|
||||
self.request_id: str | None = None
|
||||
|
||||
def _step(self, name: str) -> Any:
|
||||
return self.step_timer.step(name)
|
||||
|
||||
async def _recheck(self) -> None:
|
||||
def recheck() -> None:
|
||||
with self._step("permission_check"):
|
||||
self.lease.recheck()
|
||||
|
||||
await _joined_thread(recheck)
|
||||
|
||||
async def _start(self, request: HyMemoryIngestRequest) -> None:
|
||||
if self.wiring is not None or self.finished:
|
||||
raise HyMemoryBridgeError("model_content_context_reused")
|
||||
if request.user_id != self.lease.user_id or set(request.evidence_ids) - {
|
||||
item.evidence_id for item in self.lease.records
|
||||
}:
|
||||
raise HyMemoryBridgeError("model_content_context_unverified")
|
||||
await self._start_operation(
|
||||
user_id=request.user_id,
|
||||
agent_id=request.agent_id,
|
||||
request_id=request.request_id,
|
||||
session_id=request.session_id,
|
||||
)
|
||||
|
||||
async def _start_operation(
|
||||
self,
|
||||
*,
|
||||
user_id: str,
|
||||
agent_id: str,
|
||||
request_id: str,
|
||||
session_id: str,
|
||||
cross_evidence_scope: bool = False,
|
||||
) -> None:
|
||||
if self.wiring is not None or self.finished:
|
||||
raise HyMemoryBridgeError("model_content_context_reused")
|
||||
if user_id != self.lease.user_id:
|
||||
raise HyMemoryBridgeError("model_content_context_unverified")
|
||||
self.request_id = request_id
|
||||
await self._recheck()
|
||||
client = cast(Any, await self.bridge._get_client())
|
||||
with self._step("permission_store_read"):
|
||||
known = await asyncio.to_thread(self.proofs.load, self.lease)
|
||||
proof_check = await asyncio.to_thread(self.proofs.guard, known)
|
||||
self.wiring = _Wiring(
|
||||
client,
|
||||
self.lease,
|
||||
self.proofs,
|
||||
known,
|
||||
agent_id,
|
||||
session_id,
|
||||
self.transport,
|
||||
proof_check,
|
||||
self.step_timer,
|
||||
)
|
||||
self.wiring.cross_evidence_scope = cross_evidence_scope
|
||||
await _joined_thread(client._loop_thread.run, self.wiring.enter())
|
||||
|
||||
async def list_agent_ids(self, *, user_id: str, limit: int = 1_000) -> tuple[str, ...]:
|
||||
if user_id != self.lease.user_id or self.wiring is not None or self.finished:
|
||||
raise HyMemoryBridgeError("model_content_context_unverified")
|
||||
await self._recheck()
|
||||
result = await self.bridge.list_agent_ids(user_id=user_id, limit=limit)
|
||||
await self._recheck()
|
||||
return result
|
||||
|
||||
async def digest(self, *, user_id: str, agent_id: str) -> dict[str, object]:
|
||||
await self._start_operation(
|
||||
user_id=user_id,
|
||||
agent_id=agent_id,
|
||||
request_id=f"digest:{self.lease.task_id}:{agent_id}",
|
||||
session_id=f"digest:{self.lease.task_id}",
|
||||
cross_evidence_scope=True,
|
||||
)
|
||||
try:
|
||||
result = dict(await self.bridge.digest(user_id=user_id, agent_id=agent_id))
|
||||
except HyMemoryBridgeError:
|
||||
await self._check()
|
||||
raise
|
||||
proofs = await self._check()
|
||||
with self._step("permission_store_write"):
|
||||
await asyncio.to_thread(self.proofs.save, self.lease, proofs)
|
||||
await self.close()
|
||||
await self._recheck()
|
||||
return result
|
||||
|
||||
async def graph_snapshot(self, *, user_id: str, agent_id: str, limit: int = 300) -> Any:
|
||||
await self._start_operation(
|
||||
user_id=user_id,
|
||||
agent_id=agent_id,
|
||||
request_id=f"digest-graph:{self.lease.task_id}:{agent_id}",
|
||||
session_id=f"digest:{self.lease.task_id}",
|
||||
cross_evidence_scope=True,
|
||||
)
|
||||
try:
|
||||
result = await self.bridge.graph_snapshot(
|
||||
user_id=user_id,
|
||||
agent_id=agent_id,
|
||||
limit=limit,
|
||||
)
|
||||
except HyMemoryBridgeError:
|
||||
await self._check()
|
||||
raise
|
||||
await self._check()
|
||||
await self.close()
|
||||
await self._recheck()
|
||||
return result
|
||||
|
||||
async def ingest(self, request: HyMemoryIngestRequest) -> HyMemoryIngestResult:
|
||||
await self._start(request)
|
||||
try:
|
||||
result = await self.bridge.ingest(request)
|
||||
except HyMemoryBridgeError:
|
||||
await self._check()
|
||||
raise
|
||||
return await self._with_retired(result)
|
||||
|
||||
async def ingest_extracted(
|
||||
self, request: HyMemoryIngestRequest, contents: tuple[str, ...]
|
||||
) -> HyMemoryIngestResult:
|
||||
await self._start(request)
|
||||
try:
|
||||
result = await self.bridge.ingest_extracted(request, contents)
|
||||
except HyMemoryBridgeError:
|
||||
await self._check()
|
||||
raise
|
||||
return await self._with_retired(result)
|
||||
|
||||
async def _with_retired(self, result: HyMemoryIngestResult) -> HyMemoryIngestResult:
|
||||
await self._check()
|
||||
if self.wiring is None:
|
||||
raise HyMemoryBridgeError("model_content_context_unverified")
|
||||
try:
|
||||
retired, superseded = await _joined_thread(
|
||||
self.wiring.client._loop_thread.run, self.wiring.retired()
|
||||
)
|
||||
except HyPermissionTransportError as error:
|
||||
await self._recheck()
|
||||
raise HyMemoryBridgeError(error.code) from None
|
||||
return result.model_copy(
|
||||
update={"retired_memory_ids": retired, "superseded_memory_ids": superseded}
|
||||
)
|
||||
|
||||
async def _check(self) -> dict[str, HyNodePermission]:
|
||||
if self.wiring is None or self.finished:
|
||||
raise HyMemoryBridgeError("model_content_context_unverified")
|
||||
await self._recheck()
|
||||
try:
|
||||
return await _joined_thread(self.wiring.client._loop_thread.run, self.wiring.finish())
|
||||
except HyPermissionTransportError as error:
|
||||
await self._recheck()
|
||||
raise HyMemoryBridgeError(error.code) from None
|
||||
|
||||
async def profile_memories(self, *, user_id: str, agent_id: str) -> tuple[HyMemoryOutput, ...]:
|
||||
if self.wiring is None or user_id != self.lease.user_id or agent_id != self.wiring.agent_id:
|
||||
raise HyMemoryBridgeError("model_content_context_unverified")
|
||||
result = await self.bridge.profile_memories(user_id=user_id, agent_id=agent_id)
|
||||
proofs = await self._check()
|
||||
with self._step("permission_store_write"):
|
||||
await asyncio.to_thread(self.proofs.save, self.lease, proofs)
|
||||
# SDK context closes before governance can update a previously known memory.
|
||||
await self.close()
|
||||
return result
|
||||
|
||||
async def close(self) -> None:
|
||||
if self.wiring is not None and not self.finished:
|
||||
self.finished = True
|
||||
try:
|
||||
await _joined_thread(self.wiring.client._loop_thread.run, self.wiring.exit())
|
||||
finally:
|
||||
self.step_timer.emit(request_id=self.request_id, task_id=self.lease.task_id)
|
||||
|
||||
|
||||
def permission_bridge_factory(
|
||||
resolve_bridge: Callable[..., Any],
|
||||
*,
|
||||
http_transport: httpx.AsyncBaseTransport | None = None,
|
||||
) -> Callable[..., Any]:
|
||||
@asynccontextmanager
|
||||
async def factory(
|
||||
lease: ModelProcessingPermissionLease, secret: str | None, data_dir: Path
|
||||
) -> AsyncIterator[Any]:
|
||||
bridge = resolve_bridge(lease.profile, secret, data_dir)
|
||||
if not isinstance(bridge, HyMemoryBridge):
|
||||
raise HyMemoryBridgeError("model_content_context_unverified")
|
||||
proofs = _NativeProofs(data_dir / "one-memory.sqlite3")
|
||||
async with bridge.exclusive_access():
|
||||
await bridge.prepare()
|
||||
bound = _LeasedBridge(bridge, lease, proofs, http_transport)
|
||||
try:
|
||||
yield bound
|
||||
except BaseException as error:
|
||||
with suppress(Exception):
|
||||
await bound.close()
|
||||
if isinstance(error, HyPermissionTransportError):
|
||||
await _joined_thread(lease.recheck)
|
||||
raise HyMemoryBridgeError(error.code) from None
|
||||
raise
|
||||
else:
|
||||
try:
|
||||
await bound.close()
|
||||
except HyPermissionTransportError as error:
|
||||
raise HyMemoryBridgeError(error.code) from None
|
||||
|
||||
return factory
|
||||
@@ -1,880 +0,0 @@
|
||||
"""Operation-bound read/write capability for the pinned native HY chat vector store.
|
||||
|
||||
No semantic storage or extraction lives here. Pre-existing nodes require a full
|
||||
native-node attestation with complete approved Evidence provenance; known content
|
||||
alone cannot authorize tags, custom fields, vectors or links. Unknown nodes stay
|
||||
in the same HY backend but are not exposed to this operation.
|
||||
|
||||
The caller must exclusively own the native backend's writes during execution:
|
||||
its API has no compare-and-swap, and a swallowed get_by_id error can look missing.
|
||||
This component cannot make that check/upsert interval atomic. Factory integration
|
||||
must replace client/registry/both writers/cached-reader references, bind physical
|
||||
LLM/embedding sends to the lease AND this denial latch, and guard publication.
|
||||
Coding rows go through hy_permission_coding_store instead; graph, entities,
|
||||
digest, sweeper and summary remain unsupported.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import math
|
||||
import struct
|
||||
import weakref
|
||||
from collections.abc import AsyncIterator, Awaitable, Callable, Mapping
|
||||
from contextlib import asynccontextmanager, nullcontext
|
||||
from copy import deepcopy
|
||||
from dataclasses import dataclass
|
||||
from hashlib import sha256
|
||||
from types import MappingProxyType
|
||||
from typing import Any, NoReturn
|
||||
|
||||
from hy_memory.models.memory import ( # type: ignore[import-untyped]
|
||||
MemoryLayer,
|
||||
MemoryNode,
|
||||
MemoryStatus,
|
||||
)
|
||||
|
||||
from one_memory.hy_step_timing import HyStepTimer
|
||||
from one_memory.model_processing_permission_guard import (
|
||||
ModelProcessingPermissionLease,
|
||||
permission_error,
|
||||
)
|
||||
|
||||
_WRITE_LOCKS: weakref.WeakKeyDictionary[asyncio.AbstractEventLoop, asyncio.Lock] = (
|
||||
weakref.WeakKeyDictionary()
|
||||
)
|
||||
|
||||
|
||||
def _native_write_lock() -> asyncio.Lock:
|
||||
loop = asyncio.get_running_loop()
|
||||
lock = _WRITE_LOCKS.get(loop)
|
||||
if lock is None:
|
||||
lock = asyncio.Lock()
|
||||
_WRITE_LOCKS[loop] = lock
|
||||
return lock
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
async def native_store_read_slot() -> AsyncIterator[bool]:
|
||||
"""Admit one native HY read only while no write is active.
|
||||
|
||||
Chroma's native reader may retain the GIL after an asyncio timeout. Running
|
||||
it concurrently with HY's multi-step store writes has deadlocked the Core
|
||||
under sustained imports. Reads therefore share the physical store lock,
|
||||
but never wait behind a writer: host recall can immediately use the local
|
||||
governance ledger instead.
|
||||
"""
|
||||
lock = _native_write_lock()
|
||||
if lock.locked():
|
||||
yield False
|
||||
return
|
||||
await lock.acquire()
|
||||
try:
|
||||
yield True
|
||||
finally:
|
||||
lock.release()
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class HyNodePermission:
|
||||
"""Trusted caller's full-node attestation, not permission inferred from an ID."""
|
||||
|
||||
fingerprint: str
|
||||
evidence_ids: tuple[str, ...]
|
||||
|
||||
|
||||
def hy_node_fingerprint(node: MemoryNode) -> str:
|
||||
"""Hash the complete native serialized node, including vectors and metadata.
|
||||
|
||||
Timestamp precision follows HY's to_dict contract (integer Unix seconds).
|
||||
Attest the node read from the backend, not an unpersisted input object.
|
||||
"""
|
||||
if type(node) is not MemoryNode:
|
||||
raise permission_error("hy_store_node_shape")
|
||||
return sha256(
|
||||
json.dumps(node.to_dict(), sort_keys=True, separators=(",", ":"), allow_nan=False).encode()
|
||||
).hexdigest()
|
||||
|
||||
|
||||
_CHAT_LAYERS = frozenset(
|
||||
{
|
||||
MemoryLayer.L0_BASIC_INFO,
|
||||
MemoryLayer.L1_RAW,
|
||||
MemoryLayer.L2_FACT,
|
||||
MemoryLayer.L4_IDENTITY,
|
||||
MemoryLayer.L7_INTENTION,
|
||||
}
|
||||
)
|
||||
_UPDATE_FIELDS = frozenset(
|
||||
{
|
||||
"content",
|
||||
"embedding",
|
||||
"tags",
|
||||
"tag_ners",
|
||||
"layer",
|
||||
"memory_at",
|
||||
"gmt_modified",
|
||||
"is_latest",
|
||||
"status",
|
||||
"supersedes",
|
||||
"superseded_by",
|
||||
"source_raw_memory_id",
|
||||
# SDK 2.0.1 sends both of these on an in-place reconcile UPDATE.
|
||||
# They are metadata only and must never carry identity, permission or
|
||||
# tenant state: `owner` is a provenance label whose only values are
|
||||
# "user" / "agent" (who said this line), and `custom` is the SDK's own
|
||||
# free-form notes field. Neither may hold a user_id, scope, grant or
|
||||
# tenancy marker, so allowing the field names does not widen the
|
||||
# cross-user boundary that the store still enforces per node.
|
||||
"owner",
|
||||
"custom",
|
||||
}
|
||||
)
|
||||
_DEDUP_AUDIT_FIELDS = frozenset({"shadow_reason", "dedup_kept_id"})
|
||||
_DIGEST_COUNTER_FIELD = "s2_evidence_count"
|
||||
|
||||
|
||||
def _write_view(node: MemoryNode) -> dict[str, Any]:
|
||||
"""Only native Chroma's empty-string and float32 storage normalization."""
|
||||
payload: dict[str, Any] = node.to_dict()
|
||||
payload["source_session_id"] = payload["source_session_id"] or ""
|
||||
# The pinned public SDK serializes tag_ners as JSON, but its Chroma read path omits this
|
||||
# field from the JSON decode list. Decode only the exact list-of-strings
|
||||
# shape that the write path produced; malformed values still fail the
|
||||
# complete-node comparison below.
|
||||
tag_ners = payload.get("tag_ners")
|
||||
if (
|
||||
isinstance(tag_ners, list)
|
||||
and tag_ners
|
||||
and all(isinstance(value, str) and len(value) == 1 for value in tag_ners)
|
||||
and tag_ners[0] == "["
|
||||
and tag_ners[-1] == "]"
|
||||
):
|
||||
tag_ners = "".join(tag_ners)
|
||||
if isinstance(tag_ners, str):
|
||||
try:
|
||||
decoded = json.loads(tag_ners)
|
||||
except (json.JSONDecodeError, TypeError):
|
||||
decoded = None
|
||||
if isinstance(decoded, list) and all(isinstance(value, str) for value in decoded):
|
||||
payload["tag_ners"] = decoded
|
||||
if payload.get("tag_ners") in ([], ["[", "]"]):
|
||||
payload["tag_ners"] = []
|
||||
# HY Chroma encodes None as ""; these JSON fields keep "" when json.loads
|
||||
# fails (vector_store_chroma.py:741-788). Only an absent link is equivalent.
|
||||
for field in ("supersedes", "superseded_by"):
|
||||
if payload[field] == "":
|
||||
payload[field] = None
|
||||
if payload["embedding"] is not None:
|
||||
payload["embedding"] = [
|
||||
struct.unpack("f", struct.pack("f", n))[0] for n in payload["embedding"]
|
||||
]
|
||||
if not all(math.isfinite(n) for n in payload["embedding"]):
|
||||
raise ValueError("nonfinite_embedding")
|
||||
return payload
|
||||
|
||||
|
||||
def _same_written_vector(actual: list[float] | None, expected: list[float] | None) -> bool:
|
||||
if actual == expected:
|
||||
return True
|
||||
if actual is None or expected is None or len(actual) != len(expected):
|
||||
return False
|
||||
# Chroma's cosine index can renormalize float32 vectors on readback. Bound
|
||||
# only this write roundtrip; the saved full-node proof uses exact readback.
|
||||
return all(
|
||||
math.isclose(left, right, rel_tol=4 * 2**-23, abs_tol=0.0)
|
||||
for left, right in zip(actual, expected, strict=True)
|
||||
)
|
||||
|
||||
|
||||
_NATIVE_WRITES = frozenset(
|
||||
{"upsert", "upsert_batch", "update_payload", "update_embedding", "delete"}
|
||||
)
|
||||
|
||||
|
||||
class HyPermissionGraphStore:
|
||||
"""Lease-gated graph store for HY System 2's digest transaction.
|
||||
|
||||
The native graph is deliberately absent from ordinary ingest leases. Digest
|
||||
needs it to write L6 schemas, but every operation must remain inside the
|
||||
same frozen permission lease rather than exposing the raw store.
|
||||
"""
|
||||
|
||||
def __init__(self, native: Any, recheck: Any) -> None:
|
||||
self._native = native
|
||||
self._recheck = recheck
|
||||
self._closed = False
|
||||
|
||||
def recheck(self) -> None:
|
||||
if self._closed:
|
||||
raise permission_error("hy_graph_store_closed")
|
||||
self._recheck()
|
||||
|
||||
async def close(self) -> None:
|
||||
self._closed = True
|
||||
|
||||
def __getattr__(self, name: str) -> Any:
|
||||
if name.startswith("_") and name != "_run":
|
||||
raise AttributeError(name)
|
||||
attribute = getattr(self._native, name)
|
||||
if not callable(attribute):
|
||||
return attribute
|
||||
|
||||
async def guarded(*args: Any, **kwargs: Any) -> Any:
|
||||
self.recheck()
|
||||
result = await attribute(*args, **kwargs)
|
||||
self.recheck()
|
||||
return result
|
||||
|
||||
return guarded
|
||||
|
||||
|
||||
class HyPermissionVectorStore:
|
||||
"""Closed, task-exclusive adapter; initialize/close never own the native store.
|
||||
|
||||
node_permissions may attest additional native raw/profile nodes not represented
|
||||
in known_memories. Every attestation must cover all its contributors, and all
|
||||
Evidence IDs must already be in this lease. Known entries also constrain exact
|
||||
content/owner. Without a full attestation even a known entry is denied.
|
||||
|
||||
written_nodes records successful writes with the entire lease footprint, a
|
||||
conservative provenance bound, not a claim of semantic citation accuracy.
|
||||
These transient fingerprints must be persisted by the integrating factory if
|
||||
future operations are to reuse raw/profile nodes. No content is stored here.
|
||||
"""
|
||||
|
||||
__slots__ = (
|
||||
"__native",
|
||||
"__lease",
|
||||
"__agent_id",
|
||||
"__session_id",
|
||||
"__cross_evidence_scope",
|
||||
"__user_id",
|
||||
"__known",
|
||||
"__permissions",
|
||||
"__written",
|
||||
"__evidence_ids",
|
||||
"__deleted",
|
||||
"__shadowed",
|
||||
"__superseded",
|
||||
"__heads",
|
||||
"__lock",
|
||||
"__closed",
|
||||
"__failure",
|
||||
"__step_timer",
|
||||
"__refresh_permission",
|
||||
)
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
native: Any,
|
||||
lease: ModelProcessingPermissionLease,
|
||||
*,
|
||||
agent_id: str,
|
||||
session_id: str,
|
||||
cross_evidence_scope: bool = False,
|
||||
node_permissions: Mapping[str, HyNodePermission] | None = None,
|
||||
refresh_permission: Callable[[str], Awaitable[HyNodePermission | None]] | None = None,
|
||||
step_timer: HyStepTimer | None = None,
|
||||
) -> None:
|
||||
self.__step_timer = step_timer
|
||||
self.__native = native
|
||||
self.__lease = lease
|
||||
self.__agent_id = agent_id
|
||||
self.__session_id = session_id
|
||||
self.__cross_evidence_scope = cross_evidence_scope
|
||||
self.__user_id = lease.user_id
|
||||
self.__failure: str | None = None
|
||||
self.__closed = False
|
||||
self.__lock = asyncio.Lock()
|
||||
self.__deleted: set[str] = set()
|
||||
self.__written: dict[str, HyNodePermission] = {}
|
||||
self.__shadowed: set[str] = set()
|
||||
self.__superseded: dict[str, tuple[str, ...]] = {}
|
||||
self.__heads: dict[str, tuple[str, ...]] = {}
|
||||
self.__refresh_permission = refresh_permission
|
||||
self.recheck()
|
||||
self.__evidence_ids = tuple(record.evidence_id for record in lease.records)
|
||||
current = lease.source_context.current if lease.source_context else lease.records
|
||||
conversation_mismatch = any(r.conversation_id != session_id for r in current)
|
||||
if (
|
||||
not lease.task_id
|
||||
or not agent_id
|
||||
or agent_id == "*"
|
||||
or not session_id
|
||||
or not current
|
||||
or (conversation_mismatch and not cross_evidence_scope)
|
||||
or not self.__evidence_ids
|
||||
or len(set(self.__evidence_ids)) != len(lease.records)
|
||||
or any(r.user_id != self.__user_id for r in lease.records)
|
||||
):
|
||||
self.__deny("scope")
|
||||
known = lease.source_context.known_memories if lease.source_context else ()
|
||||
self.__known = {item.hy_memory_id: item for item in known}
|
||||
if len(self.__known) != len(known):
|
||||
self.__deny("provenance")
|
||||
self.__permissions = dict(node_permissions or {})
|
||||
for key, proof in self.__permissions.items():
|
||||
if (
|
||||
not key
|
||||
or not isinstance(proof, HyNodePermission)
|
||||
or len(proof.fingerprint) != 64
|
||||
or any(c not in "0123456789abcdef" for c in proof.fingerprint)
|
||||
or not isinstance(proof.evidence_ids, tuple)
|
||||
or not proof.evidence_ids
|
||||
or len(set(proof.evidence_ids)) != len(proof.evidence_ids)
|
||||
or not set(proof.evidence_ids) <= set(self.__evidence_ids)
|
||||
):
|
||||
self.__deny("provenance")
|
||||
for key, item in self.__known.items():
|
||||
if not item.evidence_ids or not set(item.evidence_ids) <= set(self.__evidence_ids):
|
||||
self.__deny("provenance")
|
||||
known_proof = self.__permissions.get(key)
|
||||
if known_proof and not set(item.evidence_ids) <= set(known_proof.evidence_ids):
|
||||
self.__deny("provenance")
|
||||
|
||||
@property
|
||||
def written_nodes(self) -> Mapping[str, HyNodePermission]:
|
||||
"""Latest verified write proofs; each access returns a new immutable snapshot.
|
||||
|
||||
Export after SDK writes have drained and before close/governance projection.
|
||||
Merge these rows into the lineage ledger, then remove deleted_node_ids;
|
||||
this write delta does not include untouched pre-existing permissions.
|
||||
"""
|
||||
self.recheck()
|
||||
return MappingProxyType(dict(self.__written))
|
||||
|
||||
@property
|
||||
def deleted_node_ids(self) -> frozenset[str]:
|
||||
"""Verified native deletions in this operation, for lineage-ledger cleanup."""
|
||||
self.recheck()
|
||||
return frozenset(self.__deleted)
|
||||
|
||||
@property
|
||||
def retired_node_ids(self) -> frozenset[str]:
|
||||
"""Nodes HY deleted or shadowed (its logical delete) in this operation."""
|
||||
self.recheck()
|
||||
return frozenset(self.__deleted | self.__shadowed)
|
||||
|
||||
@property
|
||||
def superseded_nodes(self) -> Mapping[str, tuple[str, ...]]:
|
||||
"""Nodes HY marked superseded in this operation, with what replaced them.
|
||||
|
||||
HY writes the two directions separately and may fail either one, so a
|
||||
replacement counts whether it was recorded on the old node or only on
|
||||
the new head.
|
||||
"""
|
||||
self.recheck()
|
||||
result = {}
|
||||
for key, by in self.__superseded.items():
|
||||
heads = set(by) | {head for head, olds in self.__heads.items() if key in olds}
|
||||
heads -= self.__deleted | {key}
|
||||
if key not in self.__deleted and heads:
|
||||
result[key] = tuple(sorted(heads))
|
||||
return MappingProxyType(result)
|
||||
|
||||
@property
|
||||
def supports_fulltext(self) -> bool:
|
||||
return False
|
||||
|
||||
def recheck(self) -> None:
|
||||
"""Must also gate transport/publication after HY swallows a store failure."""
|
||||
self.__assert_open()
|
||||
try:
|
||||
self.__lease.recheck()
|
||||
except Exception as error:
|
||||
code = str(error)
|
||||
self.__failure = (
|
||||
code
|
||||
if code.startswith("model_content_") and len(code) <= 100
|
||||
else "model_content_hy_store_lease_failed"
|
||||
)
|
||||
raise permission_error(self.__failure) from None
|
||||
|
||||
def __assert_open(self) -> None:
|
||||
if self.__closed:
|
||||
self.__deny("closed")
|
||||
if self.__failure:
|
||||
raise permission_error(self.__failure)
|
||||
|
||||
async def initialize(self) -> None:
|
||||
self.recheck()
|
||||
|
||||
async def close(self) -> None:
|
||||
"""Revoke this wrapper locally without rechecking context or closing HY.
|
||||
|
||||
Cleanup must remain possible after governance projection changes the
|
||||
lease's known-memory revisions, or after permission has been revoked.
|
||||
"""
|
||||
async with self.__lock:
|
||||
self.__closed = True
|
||||
self.__permissions.clear()
|
||||
self.__written.clear()
|
||||
|
||||
async def has_tag_embedding(self, user_id: str, tag: str) -> bool:
|
||||
self.recheck()
|
||||
# Native chat calls this best-effort helper. NotImplemented prevents it
|
||||
# from embedding tags or falling through to an unguarded entity store.
|
||||
raise NotImplementedError("hy_permission_store_tag_index_unsupported")
|
||||
|
||||
def __deny(self, suffix: str) -> NoReturn:
|
||||
self.__failure = self.__failure or f"model_content_hy_store_{suffix}"
|
||||
raise permission_error(self.__failure)
|
||||
|
||||
async def __invoke(self, method: str, *args: Any, **kwargs: Any) -> Any:
|
||||
writing = method in _NATIVE_WRITES
|
||||
self.recheck() if writing else self.__assert_open()
|
||||
step = "vector_store_write" if writing else "vector_store_read"
|
||||
first = args[0] if args else None
|
||||
try:
|
||||
with (
|
||||
self.__step_timer.step(step, items=len(first) if isinstance(first, list) else 1)
|
||||
if self.__step_timer is not None
|
||||
else nullcontext()
|
||||
):
|
||||
result = await getattr(self.__native, method)(*args, **kwargs)
|
||||
except Exception:
|
||||
self.__deny("backend_failed")
|
||||
self.recheck() if writing else self.__assert_open()
|
||||
return result
|
||||
|
||||
async def __read_native(self, key: str) -> MemoryNode | None:
|
||||
value = await self.__invoke("get_by_id", key)
|
||||
if value is not None and (type(value) is not MemoryNode or value.node_id != key):
|
||||
self.__deny("node_shape")
|
||||
return deepcopy(value)
|
||||
|
||||
def __fingerprint(self, node: MemoryNode) -> str:
|
||||
try:
|
||||
return hy_node_fingerprint(node)
|
||||
except (TypeError, ValueError, AttributeError, OverflowError):
|
||||
self.__deny("node_shape")
|
||||
|
||||
def __write_view(self, node: MemoryNode) -> dict[str, Any]:
|
||||
self.__fingerprint(node)
|
||||
try:
|
||||
return _write_view(node)
|
||||
except (TypeError, ValueError, OverflowError, struct.error):
|
||||
self.__deny("node_shape")
|
||||
|
||||
def __in_scope(self, node: MemoryNode) -> bool:
|
||||
if node.user_id != self.__user_id or node.layer not in _CHAT_LAYERS:
|
||||
return False
|
||||
if node.layer is MemoryLayer.L0_BASIC_INFO:
|
||||
# Public SDK 2.0.1 migrates verified legacy agent-level profiles into one
|
||||
# user-level chain. Existing nodes still require a full proof in
|
||||
# __load; accepting their historical agent id does not authorize
|
||||
# unproven cross-agent memories.
|
||||
return True
|
||||
if self.__cross_evidence_scope:
|
||||
return bool(node.agent_id == self.__agent_id)
|
||||
return bool(node.agent_id == self.__agent_id and node.session_id == self.__session_id)
|
||||
|
||||
def __new_node_in_scope(self, node: MemoryNode) -> bool:
|
||||
if not self.__in_scope(node):
|
||||
return False
|
||||
if node.layer is MemoryLayer.L0_BASIC_INFO:
|
||||
return node.agent_id in ("", self.__agent_id) and node.session_id in (
|
||||
"",
|
||||
self.__session_id,
|
||||
)
|
||||
return True
|
||||
|
||||
def __references(self, node: MemoryNode) -> list[str]:
|
||||
refs = []
|
||||
for field in ("supersedes", "superseded_by", "evidence_chain"):
|
||||
values = getattr(node, field)
|
||||
if field != "evidence_chain" and values == "":
|
||||
continue
|
||||
if values is not None:
|
||||
if not isinstance(values, list) or any(
|
||||
not isinstance(v, str) or not v for v in values
|
||||
):
|
||||
self.__deny("node_shape")
|
||||
refs.extend(values)
|
||||
if node.source_raw_memory_id is not None:
|
||||
if not isinstance(node.source_raw_memory_id, str) or not node.source_raw_memory_id:
|
||||
self.__deny("node_shape")
|
||||
refs.append(node.source_raw_memory_id)
|
||||
return refs
|
||||
|
||||
async def __load(
|
||||
self,
|
||||
key: str,
|
||||
first: MemoryNode | None = None,
|
||||
*,
|
||||
allow_deleted_links: bool = False,
|
||||
) -> MemoryNode | None:
|
||||
pending = [key]
|
||||
checked: dict[str, MemoryNode] = {}
|
||||
while pending:
|
||||
current = pending.pop()
|
||||
if current in checked:
|
||||
continue
|
||||
# HY dedup deletes head then ancestors. Only this operation's
|
||||
# verified tombstones may terminate a closure during deletion.
|
||||
if allow_deleted_links and current != key and current in self.__deleted:
|
||||
continue
|
||||
proof = self.__permissions.get(current)
|
||||
if proof is None or current in self.__deleted:
|
||||
return None
|
||||
value = (
|
||||
first if current == key and first is not None else await self.__read_native(current)
|
||||
)
|
||||
if value is None or not self.__in_scope(value):
|
||||
return None
|
||||
if self.__fingerprint(value) != proof.fingerprint:
|
||||
refreshed = (
|
||||
await self.__refresh_permission(current)
|
||||
if self.__refresh_permission is not None
|
||||
else None
|
||||
)
|
||||
if refreshed is None or self.__fingerprint(value) != refreshed.fingerprint:
|
||||
self.__deny("node_changed")
|
||||
self.__permissions[current] = refreshed
|
||||
proof = refreshed
|
||||
known = self.__known.get(current) if current not in self.__written else None
|
||||
if known and (value.content != known.content or value.owner != known.owner):
|
||||
self.__deny("node_changed")
|
||||
checked[current] = value
|
||||
pending.extend(self.__references(value))
|
||||
return deepcopy(checked[key])
|
||||
|
||||
async def get_by_id(self, node_id: str) -> MemoryNode | None:
|
||||
async with self.__lock:
|
||||
self.recheck()
|
||||
result = await self.__load(node_id)
|
||||
self.recheck()
|
||||
return result
|
||||
|
||||
async def get_by_ids(self, node_ids: list[str]) -> list[MemoryNode]:
|
||||
async with self.__lock:
|
||||
self.recheck()
|
||||
result = [
|
||||
n for key in dict.fromkeys(node_ids) if (n := await self.__load(key)) is not None
|
||||
]
|
||||
self.recheck()
|
||||
return result
|
||||
|
||||
async def get_embeddings(self, node_ids: list[str]) -> dict[str, list[float]]:
|
||||
return {
|
||||
n.node_id: list(n.embedding)
|
||||
for n in await self.get_by_ids(node_ids)
|
||||
if n.embedding is not None
|
||||
}
|
||||
|
||||
def __scope(self, user_id: str | None, agent_id: str | None) -> None:
|
||||
if user_id not in (None, self.__user_id) or agent_id not in (None, self.__agent_id):
|
||||
self.__deny("scope")
|
||||
|
||||
async def list_by_user(
|
||||
self,
|
||||
user_id: str | None = None,
|
||||
agent_id: str | None = None,
|
||||
limit: int = 10000,
|
||||
status_filter: list[Any] | None = None,
|
||||
layers: list[Any] | None = None,
|
||||
user_ids: list[str] | None = None,
|
||||
agent_ids: list[str] | None = None,
|
||||
only_latest: bool = False,
|
||||
with_vectors: bool = True,
|
||||
allow_raw_layer: bool = False,
|
||||
) -> list[MemoryNode]:
|
||||
async with self.__lock:
|
||||
self.recheck()
|
||||
effective_users = user_ids or ([user_id] if user_id else [])
|
||||
effective_agents = agent_ids or ([agent_id] if agent_id else [])
|
||||
if effective_users != [self.__user_id] or any(
|
||||
value != self.__agent_id for value in effective_agents
|
||||
):
|
||||
self.__deny("scope")
|
||||
rows = await self.__invoke(
|
||||
"list_by_user",
|
||||
user_id=self.__user_id,
|
||||
agent_id=agent_id,
|
||||
limit=limit,
|
||||
status_filter=status_filter,
|
||||
layers=layers,
|
||||
user_ids=[self.__user_id],
|
||||
agent_ids=effective_agents or None,
|
||||
only_latest=only_latest,
|
||||
with_vectors=with_vectors,
|
||||
allow_raw_layer=allow_raw_layer,
|
||||
)
|
||||
result = []
|
||||
for row in rows:
|
||||
if type(row) is not MemoryNode:
|
||||
self.__deny("node_shape")
|
||||
value = await self.__load(row.node_id)
|
||||
if (
|
||||
value is not None
|
||||
and (not layers or value.layer in layers or value.layer.value in layers)
|
||||
and (
|
||||
not status_filter
|
||||
or value.status in status_filter
|
||||
or value.status.value in status_filter
|
||||
)
|
||||
):
|
||||
result.append(value)
|
||||
self.recheck()
|
||||
return result[:limit]
|
||||
|
||||
async def search(
|
||||
self,
|
||||
query_embedding: list[float],
|
||||
isolation_key: str = "",
|
||||
isolation_keys: list[str] | None = None,
|
||||
user_id: str | None = None,
|
||||
user_ids: list[str] | None = None,
|
||||
agent_ids: list[str] | None = None,
|
||||
layers: list[Any] | None = None,
|
||||
limit: int = 10,
|
||||
score_threshold: float = 0.0,
|
||||
status_filter: list[Any] | None = None,
|
||||
only_latest: bool = True,
|
||||
tags_match_any: list[str] | None = None,
|
||||
created_after: float | None = None,
|
||||
) -> list[dict[str, Any]]:
|
||||
async with self.__lock:
|
||||
self.recheck()
|
||||
self.__scope(user_id, None)
|
||||
key = MemoryNode.build_isolation_key(self.__user_id, self.__agent_id, self.__session_id)
|
||||
if (
|
||||
any(uid != self.__user_id for uid in user_ids or [])
|
||||
or any(aid != self.__agent_id for aid in agent_ids or [])
|
||||
or isolation_key not in ("", key)
|
||||
or any(ik != key for ik in isolation_keys or [])
|
||||
):
|
||||
self.__deny("scope")
|
||||
hits = await self.__invoke(
|
||||
"search",
|
||||
query_embedding=deepcopy(query_embedding),
|
||||
isolation_key=isolation_key,
|
||||
isolation_keys=isolation_keys,
|
||||
user_id=self.__user_id,
|
||||
user_ids=[self.__user_id],
|
||||
agent_ids=[self.__agent_id],
|
||||
layers=layers,
|
||||
limit=limit,
|
||||
score_threshold=score_threshold,
|
||||
status_filter=status_filter,
|
||||
only_latest=only_latest,
|
||||
tags_match_any=tags_match_any,
|
||||
created_after=created_after,
|
||||
)
|
||||
result = []
|
||||
for hit in hits:
|
||||
value = await self.__load(hit["node_id"])
|
||||
if value is None:
|
||||
continue
|
||||
score = float(hit["score"])
|
||||
if not math.isfinite(score):
|
||||
self.__deny("node_shape")
|
||||
# Do not forward arbitrary backend hit metadata. Rehydrate the
|
||||
# full fingerprinted node; native search often omits embedding.
|
||||
result.append({"node_id": value.node_id, "node": value, "score": score})
|
||||
self.recheck()
|
||||
return result[:limit]
|
||||
|
||||
async def __target(
|
||||
self,
|
||||
key: str,
|
||||
first: MemoryNode | None = None,
|
||||
*,
|
||||
allow_deleted_links: bool = False,
|
||||
) -> MemoryNode:
|
||||
value = await self.__load(key, first, allow_deleted_links=allow_deleted_links)
|
||||
if value is None:
|
||||
self.__deny("target_denied")
|
||||
return value
|
||||
|
||||
async def __check_links(self, value: MemoryNode) -> None:
|
||||
for ref in self.__references(value):
|
||||
if await self.__load(ref) is None:
|
||||
self.__deny("link_denied")
|
||||
|
||||
async def __plan_upsert(self, value: MemoryNode) -> MemoryNode:
|
||||
value = deepcopy(value)
|
||||
if type(value) is not MemoryNode or not value.node_id or not self.__in_scope(value):
|
||||
self.__deny("scope")
|
||||
existing = await self.__read_native(value.node_id)
|
||||
if existing is not None:
|
||||
await self.__target(value.node_id, existing)
|
||||
elif (
|
||||
not self.__new_node_in_scope(value)
|
||||
or value.node_id in self.__permissions
|
||||
or value.node_id in self.__known
|
||||
or value.node_id in self.__deleted
|
||||
):
|
||||
self.__deny("scope")
|
||||
await self.__check_links(value)
|
||||
self.__write_view(value)
|
||||
return value
|
||||
|
||||
async def __capture(
|
||||
self,
|
||||
expected: MemoryNode,
|
||||
*,
|
||||
allow_native_modified_time: bool = False,
|
||||
) -> None:
|
||||
actual = await self.__read_native(expected.node_id)
|
||||
if actual is None:
|
||||
self.__deny("write_mismatch")
|
||||
actual_view, expected_view = self.__write_view(actual), self.__write_view(expected)
|
||||
if allow_native_modified_time:
|
||||
if actual.gmt_modified < expected.gmt_modified:
|
||||
self.__deny("write_mismatch")
|
||||
actual_view["gmt_modified"] = expected_view["gmt_modified"]
|
||||
actual_vector = actual_view.pop("embedding")
|
||||
expected_vector = expected_view.pop("embedding")
|
||||
if actual_view != expected_view or not _same_written_vector(actual_vector, expected_vector):
|
||||
self.__deny("write_mismatch")
|
||||
proof = HyNodePermission(self.__fingerprint(actual), self.__evidence_ids)
|
||||
self.__permissions[actual.node_id] = proof
|
||||
self.__written[actual.node_id] = proof
|
||||
status = getattr(actual.status, "value", actual.status)
|
||||
if status == MemoryStatus.SHADOW.value:
|
||||
self.__shadowed.add(actual.node_id)
|
||||
else:
|
||||
self.__shadowed.discard(actual.node_id)
|
||||
if status == MemoryStatus.SUPERSEDED.value:
|
||||
by = actual.superseded_by
|
||||
self.__superseded[actual.node_id] = tuple(by) if isinstance(by, list) else ()
|
||||
else:
|
||||
self.__superseded.pop(actual.node_id, None)
|
||||
olds = actual.supersedes
|
||||
if isinstance(olds, list) and olds:
|
||||
self.__heads[actual.node_id] = tuple(olds)
|
||||
else:
|
||||
self.__heads.pop(actual.node_id, None)
|
||||
|
||||
async def upsert(self, node: MemoryNode) -> str:
|
||||
async with _native_write_lock(), self.__lock:
|
||||
self.recheck()
|
||||
expected = await self.__plan_upsert(node)
|
||||
result = await self.__invoke("upsert", deepcopy(expected))
|
||||
if result != expected.node_id:
|
||||
self.__deny("write_mismatch")
|
||||
await self.__capture(expected)
|
||||
return str(result)
|
||||
|
||||
async def upsert_batch(self, nodes: list[MemoryNode]) -> list[str]:
|
||||
async with _native_write_lock(), self.__lock:
|
||||
self.recheck()
|
||||
if len({n.node_id for n in nodes}) != len(nodes):
|
||||
self.__deny("node_shape")
|
||||
# No forward links to uncommitted batch members. Preflight every
|
||||
# existing target before the backend can make a partial write.
|
||||
planned = [await self.__plan_upsert(n) for n in nodes]
|
||||
ids = [n.node_id for n in planned]
|
||||
result = await self.__invoke("upsert_batch", deepcopy(planned))
|
||||
if result != ids:
|
||||
self.__deny("write_mismatch")
|
||||
for expected in planned:
|
||||
await self.__capture(expected)
|
||||
return ids
|
||||
|
||||
async def update_payload(self, node_id: str, updates: dict[str, Any]) -> bool:
|
||||
async with _native_write_lock(), self.__lock:
|
||||
self.recheck()
|
||||
current = await self.__target(node_id)
|
||||
updates = deepcopy(updates)
|
||||
dedup_audit = False
|
||||
if set(updates) & _DEDUP_AUDIT_FIELDS:
|
||||
if (
|
||||
not set(updates) <= (_UPDATE_FIELDS | _DEDUP_AUDIT_FIELDS)
|
||||
or updates.get("status") != MemoryStatus.SHADOW.value
|
||||
or updates.get("is_latest") is not False
|
||||
or not isinstance(updates.get("shadow_reason"), str)
|
||||
or not updates["shadow_reason"].startswith("dedup:")
|
||||
or not isinstance(updates.get("dedup_kept_id"), str)
|
||||
or not updates["dedup_kept_id"]
|
||||
):
|
||||
self.__deny("update_fields")
|
||||
await self.__target(updates["dedup_kept_id"])
|
||||
custom = deepcopy(current.custom)
|
||||
custom["hy_dedup"] = {
|
||||
"reason": updates.pop("shadow_reason"),
|
||||
"kept_id": updates.pop("dedup_kept_id"),
|
||||
}
|
||||
updates["custom"] = custom
|
||||
dedup_audit = True
|
||||
allowed = _UPDATE_FIELDS | ({"custom"} if dedup_audit else set())
|
||||
if self.__cross_evidence_scope and set(updates) == {"custom"}:
|
||||
custom = updates.get("custom")
|
||||
if not isinstance(custom, dict):
|
||||
self.__deny("update_fields")
|
||||
current_custom = current.custom if isinstance(current.custom, dict) else {}
|
||||
previous = current_custom.get(_DIGEST_COUNTER_FIELD, 0)
|
||||
proposed = custom.get(_DIGEST_COUNTER_FIELD)
|
||||
if (
|
||||
not isinstance(previous, int)
|
||||
or isinstance(previous, bool)
|
||||
or not isinstance(proposed, int)
|
||||
or isinstance(proposed, bool)
|
||||
or proposed != previous + 1
|
||||
or set(custom) != set(current_custom) | {_DIGEST_COUNTER_FIELD}
|
||||
or any(custom[key] != value for key, value in current_custom.items())
|
||||
):
|
||||
self.__deny("update_fields")
|
||||
allowed |= {"custom"}
|
||||
# `custom` is an allowed field name (SDK 2.0.1 sends it on a
|
||||
# reconcile UPDATE), but the System 2 evidence counter it can hold
|
||||
# stays digest-only: only the digest scope may advance it, and only
|
||||
# by one per write. Letting the name through must not let an
|
||||
# ordinary write move that counter.
|
||||
if not self.__cross_evidence_scope and _DIGEST_COUNTER_FIELD in (
|
||||
updates.get("custom") or {}
|
||||
):
|
||||
self.__deny("update_fields")
|
||||
if not set(updates) <= allowed:
|
||||
self.__deny("update_fields")
|
||||
try:
|
||||
if "layer" in updates:
|
||||
MemoryLayer(updates["layer"])
|
||||
if "status" in updates:
|
||||
MemoryStatus(updates["status"])
|
||||
if "tag_ners" in updates and (
|
||||
not isinstance(updates["tag_ners"], list)
|
||||
or any(not isinstance(value, str) for value in updates["tag_ners"])
|
||||
):
|
||||
self.__deny("node_shape")
|
||||
expected = MemoryNode.from_dict({**current.to_dict(), **updates})
|
||||
self.__fingerprint(expected)
|
||||
except (TypeError, ValueError):
|
||||
self.__deny("node_shape")
|
||||
if not self.__in_scope(expected):
|
||||
self.__deny("scope")
|
||||
await self.__check_links(expected)
|
||||
self.__write_view(expected)
|
||||
if await self.__invoke("update_payload", node_id, updates) is not True:
|
||||
self.__deny("write_failed")
|
||||
await self.__capture(expected, allow_native_modified_time=True)
|
||||
return True
|
||||
|
||||
async def update_embedding(self, node_id: str, embedding: list[float]) -> bool:
|
||||
async with _native_write_lock(), self.__lock:
|
||||
self.recheck()
|
||||
expected = await self.__target(node_id)
|
||||
expected.embedding = deepcopy(embedding)
|
||||
self.__write_view(expected)
|
||||
if await self.__invoke("update_embedding", node_id, deepcopy(embedding)) is not True:
|
||||
self.__deny("write_failed")
|
||||
await self.__capture(expected, allow_native_modified_time=True)
|
||||
return True
|
||||
|
||||
async def delete(self, node_id: str) -> bool:
|
||||
async with _native_write_lock(), self.__lock:
|
||||
self.recheck()
|
||||
await self.__target(node_id, allow_deleted_links=True)
|
||||
if await self.__invoke("delete", node_id) is not True:
|
||||
self.__deny("write_failed")
|
||||
if await self.__read_native(node_id) is not None:
|
||||
self.__deny("write_mismatch")
|
||||
self.__deleted.add(node_id)
|
||||
self.__permissions.pop(node_id, None)
|
||||
self.__written.pop(node_id, None)
|
||||
self.__superseded.pop(node_id, None)
|
||||
self.__heads.pop(node_id, None)
|
||||
return True
|
||||
File diff suppressed because it is too large.
Load diff
@@ -144,6 +144,10 @@ class HyReconcileGuard:
|
||||
self.failures: list[str] = []
|
||||
self.bindings: HyWriteBindings | None = None
|
||||
self.imported_contents: frozenset[str] | None = None
|
||||
self.mutation_target_ids: set[str] = set()
|
||||
|
||||
def begin_write(self) -> None:
|
||||
self.mutation_target_ids.clear()
|
||||
|
||||
def __getattr__(self, name: str) -> Any:
|
||||
return getattr(self.reconciler, name)
|
||||
@@ -210,6 +214,15 @@ class HyReconcileGuard:
|
||||
for operation in result.ops:
|
||||
raw_name = getattr(operation, "op", "")
|
||||
name = str(getattr(raw_name, "value", raw_name)).upper()
|
||||
if name in {"UPDATE", "SUPERSEDE", "DELETE", "SHADOW"}:
|
||||
target = getattr(operation, "memory_id", None)
|
||||
if isinstance(target, str) and target:
|
||||
self.mutation_target_ids.add(target)
|
||||
self.mutation_target_ids.update(
|
||||
key
|
||||
for key in getattr(operation, "memory_ids", ())
|
||||
if isinstance(key, str) and key
|
||||
)
|
||||
if name == "ADD":
|
||||
operation_counts["ADD"] += 1
|
||||
elif name in {"UPDATE", "MERGE", "NOOP"}:
|
||||
@@ -232,20 +245,6 @@ class HyReconcileGuard:
|
||||
"result_count": len(result.ops),
|
||||
},
|
||||
)
|
||||
timer = self.step_timer
|
||||
try:
|
||||
with (
|
||||
timer.step("write_verification", items=len(result.ops))
|
||||
if timer is not None
|
||||
else nullcontext()
|
||||
):
|
||||
await self._check_targets(result, kwargs)
|
||||
except Exception as error:
|
||||
code = str(error)
|
||||
if not code.startswith("hy_memory_write_verification_") or len(code) > 100:
|
||||
code = "hy_memory_write_verification_failed"
|
||||
self.failures.append(code)
|
||||
raise ValueError(code) from error
|
||||
return result
|
||||
|
||||
async def _check_targets(self, result: Any, kwargs: dict[str, Any]) -> None:
|
||||
|
||||
@@ -14,7 +14,7 @@ from pathlib import Path
|
||||
from typing import Literal, cast
|
||||
|
||||
HY_MEMORY_DISTRIBUTION = "hy-memory"
|
||||
HY_MEMORY_VERSION = "2.0.1"
|
||||
HY_MEMORY_VERSION = "2.0.2"
|
||||
HY_MEMORY_NAMESPACE = "governed-v2"
|
||||
DiagnosticLogLevel = Literal["error", "warning", "info", "debug"]
|
||||
|
||||
@@ -35,7 +35,7 @@ _COLLECTION_LOGGERS = (
|
||||
# One Memory deliberately stays on ultra because its existing, user-visible
|
||||
# profile/graph flow calls digest(), which the pinned public SDK exposes only
|
||||
# for ultra clients. Reader and non-reranker behavior remain pinned for
|
||||
# compatibility with public SDK 2.0.1.
|
||||
# compatibility with public SDK 2.0.2.
|
||||
HY_MEMORY_MODE = "ultra"
|
||||
# The SDK only recognises `legacy` and `hybrid` (see
|
||||
# hy_memory/pipelines/_retrieval/config.py: `ALL_READERS = (READER_LEGACY,
|
||||
|
||||
@@ -0,0 +1,41 @@
|
||||
"""Process-local coordination for HY's native stores."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import weakref
|
||||
from collections.abc import AsyncIterator
|
||||
from contextlib import asynccontextmanager
|
||||
|
||||
_WRITE_LOCKS: weakref.WeakKeyDictionary[asyncio.AbstractEventLoop, asyncio.Lock] = (
|
||||
weakref.WeakKeyDictionary()
|
||||
)
|
||||
|
||||
|
||||
def native_store_write_lock() -> asyncio.Lock:
|
||||
"""Return the write lock for the current event loop."""
|
||||
loop = asyncio.get_running_loop()
|
||||
lock = _WRITE_LOCKS.get(loop)
|
||||
if lock is None:
|
||||
lock = asyncio.Lock()
|
||||
_WRITE_LOCKS[loop] = lock
|
||||
return lock
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
async def native_store_read_slot() -> AsyncIterator[bool]:
|
||||
"""Admit a native HY read only while no write is active.
|
||||
|
||||
Chroma's native reader may retain the GIL after an asyncio timeout. Reads
|
||||
therefore share the physical-store lock, but never wait behind a writer:
|
||||
host recall can immediately fall back to the governance ledger.
|
||||
"""
|
||||
lock = native_store_write_lock()
|
||||
if lock.locked():
|
||||
yield False
|
||||
return
|
||||
await lock.acquire()
|
||||
try:
|
||||
yield True
|
||||
finally:
|
||||
lock.release()
|
||||
@@ -316,6 +316,10 @@ class ImportSupervisor:
|
||||
"source_id": current.source_id,
|
||||
"mode": current.operation,
|
||||
"count": current.stored,
|
||||
"stored_count": current.stored,
|
||||
"revised_count": current.revised,
|
||||
"skipped_count": current.skipped,
|
||||
"error_count": current.errors,
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
@@ -10,7 +10,6 @@ from fastapi.params import Depends
|
||||
from pydantic import BaseModel, ConfigDict, Field, field_validator
|
||||
|
||||
from one_memory.database_access import main_database
|
||||
from one_memory.hy_memory_bridge import HyMemoryBridgeError
|
||||
from one_memory.imports.models import (
|
||||
EvidenceInput,
|
||||
EvidenceOrigin,
|
||||
@@ -186,6 +185,10 @@ class MemoryNoteStore:
|
||||
]
|
||||
if task is None or task.status == MemoryTaskStatus.WAITING_FOR_MODEL:
|
||||
state = "waiting_for_model"
|
||||
elif task.status == MemoryTaskStatus.WAITING_FOR_PERMISSION:
|
||||
# Upgrade compatibility: old rows are claimable and will leave
|
||||
# this state as soon as the direct SDK runner claims them.
|
||||
state = "waiting_for_permission"
|
||||
elif task.status == MemoryTaskStatus.COMPLETED:
|
||||
state = "ready" if memory_ids else "no_memories"
|
||||
elif task.status == MemoryTaskStatus.RUNNING:
|
||||
@@ -195,22 +198,6 @@ class MemoryNoteStore:
|
||||
else:
|
||||
state = "failed"
|
||||
error = task.last_error_code if task else None
|
||||
if self.processor.model_content_config is not None and state in {
|
||||
"waiting_for_model",
|
||||
"failed",
|
||||
}:
|
||||
try:
|
||||
active = self.processor.profiles.active(self.processor.secrets)
|
||||
if active is not None:
|
||||
self.processor._guard_records((record,), active)
|
||||
except HyMemoryBridgeError as failure:
|
||||
if str(failure).startswith("model_content_"):
|
||||
state = "waiting_for_permission"
|
||||
error = str(failure)
|
||||
else:
|
||||
raise
|
||||
except SecretStoreUnavailableError:
|
||||
error = "secure_storage_unavailable"
|
||||
return MemoryNoteReceipt(
|
||||
evidence_id=record.evidence_id,
|
||||
text=sanitize_model_input(record.verbatim),
|
||||
@@ -274,11 +261,8 @@ def memory_note_router(dependencies: Sequence[Depends]) -> APIRouter:
|
||||
# unchanged; replaying the same request returns the original
|
||||
# `entry:<evidence_id>` job instead of a second HY call.
|
||||
if readmit:
|
||||
# `/{evidence_id}/process` is the explicit "the condition
|
||||
# that parked it may have changed" signal: the person just
|
||||
# granted consent or fixed the model. `submit_evidence`
|
||||
# alone would return an existing parked row unchanged, so
|
||||
# without this readmit the note stays parked forever.
|
||||
# `/{evidence_id}/process` is the explicit retry signal.
|
||||
# It also revives pre-upgrade permission waiters.
|
||||
await asyncio.to_thread(notes.intake.queue.readmit_parked)
|
||||
submitted = await asyncio.to_thread(
|
||||
notes.intake.submit_evidence,
|
||||
@@ -296,10 +280,6 @@ def memory_note_router(dependencies: Sequence[Depends]) -> APIRouter:
|
||||
last_round=1,
|
||||
priority="live",
|
||||
)
|
||||
except HyMemoryBridgeError as error:
|
||||
if not str(error).startswith("model_content_"):
|
||||
raise
|
||||
return await asyncio.to_thread(notes.receipt, record.evidence_id)
|
||||
except SecretStoreUnavailableError:
|
||||
saved = await asyncio.to_thread(notes.receipt, record.evidence_id)
|
||||
return saved.model_copy(
|
||||
|
||||
File diff suppressed because it is too large.
Load diff
@@ -29,7 +29,6 @@ class ConfigDocument(BaseModel):
|
||||
model_config = ConfigDict(frozen=True, extra="forbid", strict=True)
|
||||
|
||||
schema_version: int = Field(default=1, ge=1, le=1)
|
||||
model_content_allowed: StrictBool = False
|
||||
sources: dict[str, SourceConfig] = Field(default_factory=dict)
|
||||
|
||||
@field_validator("sources")
|
||||
@@ -64,10 +63,6 @@ class ConfigStore:
|
||||
source = self._document.sources.get(source_id)
|
||||
return bool(source and source.sync_enabled)
|
||||
|
||||
def model_content_allowed(self) -> bool:
|
||||
with self._lock:
|
||||
return self._document.model_content_allowed
|
||||
|
||||
def set_sync(self, source_id: str, enabled: bool) -> ConfigDocument:
|
||||
return self.set_sync_many({source_id: enabled})
|
||||
|
||||
@@ -86,20 +81,17 @@ class ConfigStore:
|
||||
self._document = updated
|
||||
return updated
|
||||
|
||||
def set_model_content_allowed(self, allowed: bool) -> ConfigDocument:
|
||||
if type(allowed) is not bool:
|
||||
raise ValueError("invalid_model_content_config")
|
||||
with self._lock:
|
||||
updated = self._document.model_copy(update={"model_content_allowed": allowed})
|
||||
self._write(updated)
|
||||
self._document = updated
|
||||
return updated
|
||||
|
||||
def _read(self) -> ConfigDocument:
|
||||
if not self.path.exists():
|
||||
return ConfigDocument()
|
||||
try:
|
||||
return ConfigDocument.model_validate_json(self.path.read_text(encoding="utf-8"))
|
||||
value = json.loads(self.path.read_text(encoding="utf-8"))
|
||||
if isinstance(value, dict):
|
||||
# Builds through 1.0.2 persisted this global consent bit.
|
||||
# The embedded SDK is now trusted to use all local memory, so
|
||||
# accept existing files while discarding the retired setting.
|
||||
value.pop("model_content_allowed", None)
|
||||
return ConfigDocument.model_validate(value)
|
||||
except (OSError, ValueError) as error:
|
||||
raise ConfigFileError("config_invalid") from error
|
||||
|
||||
|
||||
@@ -306,7 +306,7 @@ LEGACY_MEMORY_WRITE_JOB_COLUMNS = (
|
||||
("last_error", "TEXT"),
|
||||
)
|
||||
|
||||
_FIFO_BLOCKING_STATES = (PENDING, RUNNING, WAITING_FOR_PERMISSION, WAITING_FOR_MODEL, UNKNOWN)
|
||||
_FIFO_BLOCKING_STATES = (PENDING, RUNNING, WAITING_FOR_PERMISSION, WAITING_FOR_MODEL)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
@@ -1492,6 +1492,59 @@ class DurableWriteQueue:
|
||||
)
|
||||
return len(job_ids)
|
||||
|
||||
def retry_first_unknown_after_reconcile(
|
||||
self,
|
||||
*,
|
||||
older_than: datetime,
|
||||
limit: int = 32,
|
||||
) -> int:
|
||||
"""Retry one unresolved unknown outcome once, after reconciliation.
|
||||
|
||||
The caller first asks HY and checks linked governed output. Rows still
|
||||
unknown may have lost the response after the SDK call, so one bounded
|
||||
retry is useful. `attempts=1` is the guard: the retry claim increments
|
||||
it to two, and a second unknown is left for explicit user recovery.
|
||||
"""
|
||||
if type(limit) is not int or limit < 1:
|
||||
raise ValueError("limit must be positive")
|
||||
now = datetime.now(UTC).isoformat()
|
||||
with (
|
||||
main_database(self.database_path).write(row_factory=sqlite3.Row) as connection,
|
||||
connection,
|
||||
):
|
||||
rows = connection.execute(
|
||||
"SELECT job_id FROM memory_write_jobs "
|
||||
"WHERE state=? AND attempts=1 AND attempts < max_attempts "
|
||||
"AND updated_at <= ? ORDER BY updated_at, job_id LIMIT ?",
|
||||
(UNKNOWN, older_than.isoformat(), limit),
|
||||
).fetchall()
|
||||
job_ids = tuple(str(row["job_id"]) for row in rows)
|
||||
if not job_ids:
|
||||
return 0
|
||||
placeholders = ",".join("?" for _ in job_ids)
|
||||
connection.execute(
|
||||
f"UPDATE memory_write_jobs SET state=?, lease_owner=NULL, "
|
||||
f"lease_expires_at=NULL, hy_invoked=0, "
|
||||
f"last_error='model_content_unknown_auto_retry', updated_at=? "
|
||||
f"WHERE state=? AND attempts=1 AND job_id IN ({placeholders})",
|
||||
(PENDING, now, UNKNOWN, *job_ids),
|
||||
)
|
||||
if connection.execute(
|
||||
"SELECT 1 FROM sqlite_master WHERE type='table' AND name='memory_tasks'"
|
||||
).fetchone():
|
||||
connection.execute(
|
||||
f"UPDATE memory_tasks SET status=?, runner_token=NULL, "
|
||||
f"last_error_code='model_content_unknown_auto_retry', updated_at=? "
|
||||
f"WHERE status=? AND task_id IN ("
|
||||
f"SELECT inputs.task_id FROM memory_task_inputs AS inputs "
|
||||
f"JOIN memory_write_job_evidence AS linked ON EXISTS ("
|
||||
f"SELECT 1 FROM json_each(inputs.refs_json) AS ref "
|
||||
f"WHERE json_extract(ref.value, '$.evidence_id')=linked.evidence_id) "
|
||||
f"WHERE linked.job_id IN ({placeholders}))",
|
||||
(MemoryTaskStatus.PENDING, now, MemoryTaskStatus.UNKNOWN, *job_ids),
|
||||
)
|
||||
return len(job_ids)
|
||||
|
||||
def resolve_unknown_from_linked_output(self, job_id: str) -> bool:
|
||||
"""Close an unknown durable row when local governance output proves success."""
|
||||
with main_database(self.database_path).read(row_factory=sqlite3.Row) as connection:
|
||||
@@ -1896,6 +1949,13 @@ class DurableWriteDrain:
|
||||
#: Reaping is a write; it runs at most once per this interval on the
|
||||
#: idle path so an empty queue does not issue one write per poll.
|
||||
self._reap_interval = reap_interval
|
||||
# Unknown recovery is user-visible and lightweight enough to check
|
||||
# sooner than general database housekeeping. Tests that set the common
|
||||
# interval to zero keep immediate maintenance.
|
||||
self._unknown_reconcile_interval = min(
|
||||
reap_interval,
|
||||
timedelta(seconds=10),
|
||||
)
|
||||
#: The maintenance writes below are recoverability/retention work, not
|
||||
#: idle housekeeping, so each has its own interval and runs at the start
|
||||
#: of every drain tick. Sharing one timestamp would let a busy tick
|
||||
@@ -2308,7 +2368,8 @@ class DurableWriteDrain:
|
||||
)
|
||||
if self.reconcile_unknown is not None and (
|
||||
self._last_unknown_reconcile is None
|
||||
or tick_started - self._last_unknown_reconcile >= self._reap_interval
|
||||
or tick_started - self._last_unknown_reconcile
|
||||
>= self._unknown_reconcile_interval
|
||||
):
|
||||
self._last_unknown_reconcile = tick_started
|
||||
try:
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
The result decides whether the `unknown` job state can ever be retired, so it
|
||||
is probed against the installed SDK rather than written down from memory.
|
||||
|
||||
What was verified against `hy_memory-internal==2.0.1`:
|
||||
What was verified against `hy-memory==2.0.2`:
|
||||
|
||||
- `HyMemoryClient.add(..., request_id=...)` accepts a caller-supplied id and
|
||||
echoes it back. It is described as a request/receipt handle, and the SDK
|
||||
|
||||
@@ -1404,7 +1404,14 @@ async def _024_matter_model_cache(cursor: Any) -> None:
|
||||
|
||||
|
||||
async def _026_hy_node_permissions(cursor: Any) -> None:
|
||||
await cursor.execute(hy_node_permissions_ddl(_dialect(cursor)))
|
||||
# Historical migration retained as a ledger slot. Fresh databases no
|
||||
# longer create this retired proof table; migration 51 removes it from
|
||||
# databases that ran the original v26 body.
|
||||
return None
|
||||
|
||||
|
||||
async def _051_drop_hy_node_permissions(cursor: Any) -> None:
|
||||
await cursor.execute("DROP TABLE IF EXISTS hy_node_permissions")
|
||||
|
||||
|
||||
def model_profiles_ddl(dialect: str = "mysql") -> str:
|
||||
@@ -2756,7 +2763,7 @@ MIGRATIONS: tuple[Migration, ...] = (
|
||||
),
|
||||
Migration(23, "source_candidates", _023_source_candidates, ("source_candidates",)),
|
||||
Migration(24, "matter_model_cache", _024_matter_model_cache, ("matter_model_cache",)),
|
||||
Migration(26, "hy_node_permissions", _026_hy_node_permissions, ("hy_node_permissions",)),
|
||||
Migration(26, "hy_node_permissions", _026_hy_node_permissions),
|
||||
Migration(
|
||||
27,
|
||||
"model_profiles",
|
||||
@@ -2841,4 +2848,5 @@ MIGRATIONS: tuple[Migration, ...] = (
|
||||
"model_probe_operations_reserved_at",
|
||||
_050_model_probe_operations_reserved_at,
|
||||
),
|
||||
Migration(51, "drop_hy_node_permissions", _051_drop_hy_node_permissions),
|
||||
)
|
||||
@@ -267,15 +267,13 @@ class MemoryTaskOutput(BaseModel):
|
||||
MEMORY_TASK_SCHEMA = memory_tasks_ddl("sqlite")
|
||||
|
||||
|
||||
#: Transport refusals raised before the physical send, so provably nothing
|
||||
#: reached HY: nothing can have been written and retrying cannot duplicate.
|
||||
#: Legacy transport refusals raised before the physical send, so provably
|
||||
#: nothing reached HY: nothing can have been written and retrying cannot
|
||||
#: duplicate. Kept to recover rows written by builds through 1.0.2.
|
||||
#:
|
||||
#: Every name here is denied by `HyPermissionTransport._check_request` (the
|
||||
#: httpx request hook) or by `_PhysicalTransport.handle_async_request` before
|
||||
#: it counts the request — both run inside httpx, before the socket. Explicit
|
||||
#: on purpose (D88): the transport also denies `budget_exceeded`, `redirect`,
|
||||
#: `lease_*` and `request_failed`, all of which can arrive after HY persisted
|
||||
#: a node, so a prefix rule would requeue work that may already exist.
|
||||
#: The retired request hook denied every name below before opening a socket.
|
||||
#: This stays explicit: other old transport codes may have happened after HY
|
||||
#: persisted a node and must not be blindly requeued.
|
||||
_PRE_SEND_REFUSALS = (
|
||||
"model_content_transport_headers_mismatch",
|
||||
"model_content_transport_body_mismatch",
|
||||
@@ -2163,6 +2161,43 @@ class MemoryTaskStore:
|
||||
)
|
||||
return changed.rowcount
|
||||
|
||||
def retry_first_unknown_after_reconcile(
|
||||
self,
|
||||
*,
|
||||
older_than: datetime,
|
||||
limit: int = 128,
|
||||
) -> int:
|
||||
"""Automatically retry a task-only unknown exactly once.
|
||||
|
||||
Reconciliation runs before this transition. Attempts are preserved, so
|
||||
the retry claim moves `attempts=1` to two and another unknown is left
|
||||
for explicit user recovery.
|
||||
"""
|
||||
if type(limit) is not int or limit < 1:
|
||||
raise ValueError("limit must be positive")
|
||||
now = datetime.now(UTC).isoformat()
|
||||
with self._connect(writing=True) as connection:
|
||||
changed = connection.execute(
|
||||
"""
|
||||
UPDATE memory_tasks
|
||||
SET status=?, runner_token=NULL,
|
||||
last_error_code='model_content_unknown_auto_retry', updated_at=?
|
||||
WHERE task_id IN (
|
||||
SELECT task_id FROM memory_tasks
|
||||
WHERE status=? AND attempts=1 AND updated_at <= ?
|
||||
ORDER BY updated_at, task_id LIMIT ?
|
||||
)
|
||||
""",
|
||||
(
|
||||
MemoryTaskStatus.PENDING,
|
||||
now,
|
||||
MemoryTaskStatus.UNKNOWN,
|
||||
older_than.isoformat(),
|
||||
limit,
|
||||
),
|
||||
)
|
||||
return changed.rowcount
|
||||
|
||||
def recover_running(self) -> int:
|
||||
"""Return tasks orphaned by a previous process to the pending queue."""
|
||||
return self.recover_orphan_running()
|
||||
|
||||
@@ -1,49 +0,0 @@
|
||||
"""One global final-state permission for remote model content egress."""
|
||||
|
||||
import asyncio
|
||||
from collections.abc import Sequence
|
||||
from typing import Annotated, cast
|
||||
|
||||
from fastapi import APIRouter, Request
|
||||
from fastapi.params import Depends
|
||||
from pydantic import BaseModel, ConfigDict, StrictBool
|
||||
|
||||
from one_memory.memory_server.config_state import ConfigStore
|
||||
|
||||
|
||||
class ModelContentPermissionState(BaseModel):
|
||||
model_config = ConfigDict(frozen=True, extra="forbid")
|
||||
|
||||
allowed: bool
|
||||
|
||||
|
||||
class SetModelContentPermission(BaseModel):
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
|
||||
allowed: StrictBool
|
||||
|
||||
|
||||
def model_content_router(dependencies: Sequence[Depends]) -> APIRouter:
|
||||
router = APIRouter(prefix="/api/v1/model-content-permission", dependencies=dependencies)
|
||||
|
||||
def config(request: Request) -> ConfigStore:
|
||||
return cast(ConfigStore, request.app.state.config_store)
|
||||
|
||||
Config = Annotated[ConfigStore, Depends(config)]
|
||||
|
||||
@router.get("", response_model=ModelContentPermissionState)
|
||||
async def current(store: Config) -> ModelContentPermissionState:
|
||||
return ModelContentPermissionState(allowed=store.model_content_allowed())
|
||||
|
||||
@router.post("", response_model=ModelContentPermissionState)
|
||||
async def set_permission(
|
||||
request: Request,
|
||||
value: SetModelContentPermission,
|
||||
store: Config,
|
||||
) -> ModelContentPermissionState:
|
||||
await asyncio.to_thread(store.set_model_content_allowed, value.allowed)
|
||||
if value.allowed:
|
||||
request.app.state.start_memory_backfill()
|
||||
return ModelContentPermissionState(allowed=value.allowed)
|
||||
|
||||
return router
|
||||
@@ -1,180 +0,0 @@
|
||||
"""Live global permission checks for remote model-content egress.
|
||||
|
||||
``config.json`` is the only durable permission state. Tasks do not carry a
|
||||
permission receipt, revision, source scope, or model pin. Every remote send
|
||||
and every publication boundary comes back through the lease and reads the
|
||||
current final-state boolean.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Iterator
|
||||
from contextlib import contextmanager
|
||||
from dataclasses import dataclass, field
|
||||
from datetime import UTC, datetime
|
||||
|
||||
from one_memory.hy_source_context import HySourceContext
|
||||
from one_memory.imports.models import EvidenceRecord
|
||||
from one_memory.memory_governance import MemoryGovernanceStore, MemoryStatus
|
||||
from one_memory.memory_server.config_state import ConfigStore
|
||||
from one_memory.memory_tasks import MemoryTask, MemoryTaskStatus
|
||||
from one_memory.model_profiles import ModelProfileSnapshot, ModelProfileStore
|
||||
|
||||
|
||||
def permission_error(code: str) -> RuntimeError:
|
||||
from one_memory.hy_memory_bridge import HyMemoryBridgeError
|
||||
|
||||
return HyMemoryBridgeError(
|
||||
code if code.startswith("model_content_") else f"model_content_{code}"
|
||||
)
|
||||
|
||||
|
||||
def error_is_policy_stop(code: str) -> bool:
|
||||
"""A disabled final-state permission is an intentional stop."""
|
||||
|
||||
return code in {
|
||||
"model_content_source_disabled",
|
||||
"hy_memory_evidence_changed",
|
||||
}
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ModelProcessingPermissionLease:
|
||||
task_id: str
|
||||
user_id: str
|
||||
profile: ModelProfileSnapshot
|
||||
records: tuple[EvidenceRecord, ...]
|
||||
source_context: HySourceContext | None
|
||||
_guard: ModelProcessingPermissionGuard = field(repr=False, compare=False)
|
||||
|
||||
def recheck(self) -> None:
|
||||
"""Read current config/profile immediately before an egress boundary."""
|
||||
|
||||
self.recheck_permission()
|
||||
self._guard.check_context(self.records, self.source_context)
|
||||
|
||||
def recheck_permission(self) -> None:
|
||||
self._guard.check(self.profile, self.records)
|
||||
|
||||
|
||||
class ModelProcessingPermissionGuard:
|
||||
def __init__(
|
||||
self,
|
||||
config: ConfigStore,
|
||||
governance: MemoryGovernanceStore,
|
||||
profiles: ModelProfileStore,
|
||||
) -> None:
|
||||
self.config = config
|
||||
self.governance = governance
|
||||
self.profiles = profiles
|
||||
|
||||
def require_allowed(self) -> None:
|
||||
if not self.config.model_content_allowed():
|
||||
raise permission_error("permission_required")
|
||||
|
||||
def check(
|
||||
self,
|
||||
profile: ModelProfileSnapshot,
|
||||
records: tuple[EvidenceRecord, ...] = (),
|
||||
) -> None:
|
||||
"""Validate only live global consent and the currently active model."""
|
||||
|
||||
self.require_allowed()
|
||||
if self.profiles.active_identity() != (profile.profile_id, profile.revision):
|
||||
raise permission_error("profile_changed")
|
||||
if records and any(record.user_id != records[0].user_id for record in records):
|
||||
raise permission_error("context_scope_invalid")
|
||||
|
||||
def lease(
|
||||
self,
|
||||
task: MemoryTask,
|
||||
records: tuple[EvidenceRecord, ...],
|
||||
context: HySourceContext | None,
|
||||
) -> ModelProcessingPermissionLease:
|
||||
combined = self.context_records(records, context)
|
||||
lease = ModelProcessingPermissionLease(
|
||||
task_id=task.task_id,
|
||||
user_id=task.user_id,
|
||||
profile=task.profile_snapshot,
|
||||
records=combined,
|
||||
source_context=context,
|
||||
_guard=self,
|
||||
)
|
||||
lease.recheck()
|
||||
return lease
|
||||
|
||||
@staticmethod
|
||||
def context_records(
|
||||
records: tuple[EvidenceRecord, ...],
|
||||
context: HySourceContext | None,
|
||||
) -> tuple[EvidenceRecord, ...]:
|
||||
by_id = {record.evidence_id: record for record in records}
|
||||
if len(by_id) != len(records):
|
||||
raise permission_error("context_scope_invalid")
|
||||
if context is not None:
|
||||
for record in (*context.current, *context.history, *context.supporting):
|
||||
previous = by_id.get(record.evidence_id)
|
||||
if previous is not None and previous != record:
|
||||
raise permission_error("evidence_changed")
|
||||
by_id[record.evidence_id] = record
|
||||
return tuple(by_id.values())
|
||||
|
||||
def check_context(
|
||||
self,
|
||||
records: tuple[EvidenceRecord, ...],
|
||||
context: HySourceContext | None,
|
||||
) -> None:
|
||||
if context is None:
|
||||
return
|
||||
ids = {record.evidence_id for record in records}
|
||||
if not context.current:
|
||||
raise permission_error("context_scope_invalid")
|
||||
context_records = (*context.current, *context.history, *context.supporting)
|
||||
if len({record.evidence_id for record in context_records}) != len(context_records):
|
||||
raise permission_error("context_scope_invalid")
|
||||
for known in context.known_memories:
|
||||
if not known.evidence_ids or not set(known.evidence_ids) <= ids:
|
||||
raise permission_error("context_support_missing")
|
||||
memory = self.governance.get(
|
||||
known.governed_memory_id,
|
||||
user_id=context.current[0].user_id,
|
||||
)
|
||||
if (
|
||||
memory is None
|
||||
or memory.revision != known.revision
|
||||
or memory.content != known.content
|
||||
or set(memory.evidence_ids) != set(known.evidence_ids)
|
||||
or memory.status is not MemoryStatus.CONFIRMED
|
||||
):
|
||||
raise permission_error("context_changed")
|
||||
|
||||
def reject_unstarted(self, task_id: str, code: str) -> None:
|
||||
stopped = error_is_policy_stop(code)
|
||||
with self._task_write() as tasks:
|
||||
tasks.execute(
|
||||
"UPDATE memory_tasks SET status=?, last_error_code=?, updated_at=? "
|
||||
"WHERE task_id=? AND status IN (?,?,?,?)",
|
||||
(
|
||||
MemoryTaskStatus.CANCELLED if stopped else MemoryTaskStatus.FAILED,
|
||||
str(permission_error(code)),
|
||||
datetime.now(UTC).isoformat(),
|
||||
task_id,
|
||||
MemoryTaskStatus.PENDING,
|
||||
MemoryTaskStatus.WAITING_FOR_PERMISSION,
|
||||
MemoryTaskStatus.WAITING_FOR_MODEL,
|
||||
MemoryTaskStatus.DEFERRED_OUTSIDE_WINDOW,
|
||||
),
|
||||
)
|
||||
|
||||
@contextmanager
|
||||
def _task_write(self) -> Iterator[object]:
|
||||
from one_memory.database import take_the_write_lock_first
|
||||
from one_memory.database_access import main_database
|
||||
|
||||
with main_database(self.governance.database_path).write(
|
||||
autocommit=True,
|
||||
foreign_keys=True,
|
||||
) as connection:
|
||||
take_the_write_lock_first(connection)
|
||||
with connection:
|
||||
yield connection
|
||||
@@ -1,81 +0,0 @@
|
||||
"""Continuous recovery monitor for model processing.
|
||||
|
||||
The monitor is intentionally retained. It observes the global config final
|
||||
state plus ``memory_tasks`` and asks the supervisor to recover work; it owns no
|
||||
second request/receipt state machine.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import time
|
||||
from collections.abc import Hashable
|
||||
|
||||
from one_memory.memory_processing import MemoryProcessingSupervisor
|
||||
from one_memory.memory_server.config_state import ConfigStore
|
||||
|
||||
|
||||
class ProcessingCoordinator:
|
||||
RESCAN_SECONDS = 60.0
|
||||
POLL_SECONDS = 2.0
|
||||
|
||||
def __init__(self, config: ConfigStore, supervisor: MemoryProcessingSupervisor):
|
||||
self.config = config
|
||||
self.supervisor = supervisor
|
||||
self._stopping = False
|
||||
self._monitor: asyncio.Task[None] | None = None
|
||||
self._error: str | None = None
|
||||
self._observed: tuple[Hashable, ...] | None = None
|
||||
self._last_scan = 0.0
|
||||
self._clock = time.monotonic
|
||||
|
||||
async def recover(self) -> None:
|
||||
"""Start the one process-owned monitor, whether permission is on or off."""
|
||||
|
||||
if self._stopping:
|
||||
return
|
||||
if self._monitor is None or self._monitor.done():
|
||||
self._monitor = asyncio.create_task(
|
||||
self._monitor_loop(),
|
||||
name="one-memory-processing-monitor",
|
||||
)
|
||||
|
||||
def _identity(self) -> tuple[Hashable, ...]:
|
||||
counts = self.supervisor.tasks.counts()
|
||||
return (
|
||||
self.config.model_content_allowed(),
|
||||
tuple(sorted((str(status), count) for status, count in counts.items())),
|
||||
)
|
||||
|
||||
async def _tick(self) -> None:
|
||||
identity = await asyncio.to_thread(self._identity)
|
||||
now = self._clock()
|
||||
if identity[0] and (
|
||||
identity != self._observed or now - self._last_scan >= self.RESCAN_SECONDS
|
||||
):
|
||||
await self.supervisor.resume_allowed()
|
||||
self._last_scan = now
|
||||
identity = await asyncio.to_thread(self._identity)
|
||||
self._observed = identity
|
||||
self._error = None
|
||||
|
||||
async def _monitor_loop(self) -> None:
|
||||
while not self._stopping:
|
||||
try:
|
||||
await self._tick()
|
||||
except asyncio.CancelledError:
|
||||
raise
|
||||
except Exception:
|
||||
self._error = "model_content_recovery_blocked"
|
||||
await asyncio.sleep(self.POLL_SECONDS)
|
||||
|
||||
def status(self) -> dict[str, object]:
|
||||
return {
|
||||
"monitoring": self._monitor is not None and not self._monitor.done(),
|
||||
"allowed": self._observed[0] if self._observed else None,
|
||||
"error": self._error,
|
||||
}
|
||||
|
||||
async def shutdown(self) -> None:
|
||||
self._stopping = True
|
||||
if self._monitor is not None:
|
||||
self._monitor.cancel()
|
||||
await asyncio.gather(self._monitor, return_exceptions=True)
|
||||
@@ -13,10 +13,7 @@
|
||||
| `test_archive_extract.py` | `contract` | Unpacking an archive a stranger sent, onto this user's disk. |
|
||||
| `test_ask_workspace_path.py` | `contract` | Ask must honour the same workspace_path expansion recall uses. |
|
||||
| `test_async_bridge.py` | `contract` | 覆盖 reader rejects parallel workers that could close during next、cancelling a read stops the worker before the batch finishes、starting a read does not stop the app answering 等 10 个场景。 |
|
||||
| `test_async_dispatch_checks.py` | `contract` | 覆盖 cancelling quiet timer does not start any permission reader、dispatch check does not queue work after shutdown、dispatch permission check denies before queue after classification 等 6 个场景。 |
|
||||
| `test_auth.py` | `integration` | 覆盖 local api requires bearer token、browser origin is checked in addition to token 场景。 |
|
||||
| `test_backfill_acceptance_lifecycle.py` | `contract` | A processing acknowledgement belongs to one consent revision and live worker. |
|
||||
| `test_backfill_conversation_isolation.py` | `contract` | One stale conversation cannot invalidate an entire backfill scan. |
|
||||
| `test_background_work.py` | `contract` | 覆盖 a request does not wait behind a catch up、bulk work is kept to a few threads at a time、the pool outlives one hand off 等 4 个场景。 |
|
||||
| `test_backup_destroy_restore_reconciliation.py` | `contract` | Backup -> destroy -> restore, reconciled row for row. |
|
||||
| `test_backup_endpoint.py` | `integration` | The 'POST /api/v1/backup' endpoint the CLI's 'backup create' calls. |
|
||||
@@ -64,7 +61,6 @@
|
||||
| `test_cli_lifecycle.py` | `integration` | 覆盖 cli starts and stops without residual runtime 场景。 |
|
||||
| `test_cli_restore.py` | `contract` | 覆盖 restore requires a source、restore refuses while the runtime is running、restore previews without confirmation 等 4 个场景。 |
|
||||
| `test_codex_header_provenance.py` | `contract` | 覆盖 header workspace versions and native ids reach file origin、new adapter rereads header after a real checkpoint、machine and unknown session sources do not establish human or agent sender 等 14 个场景。 |
|
||||
| `test_compact_model_content.py` | `contract` | Server-held previews preserve exact grants without round-tripping all refs. |
|
||||
| `test_connection_lifetime.py` | `integration` | Every SQLite connection a request opens must be closed when the request ends. |
|
||||
| `test_context_access.py` | `contract` | 覆盖 memory only never reads raw authority、raw permission does not grant memory and removes revoked sources、raw requires all apps and source 等 13 个场景。 |
|
||||
| `test_context_api.py` | `integration` | 覆盖 no model local query returns real raw and governed items、existing single lane authorization never runs denied reader、authentication and origin remain required 等 14 个场景。 |
|
||||
@@ -76,12 +72,6 @@
|
||||
| `test_context_search.py` | `contract` | 覆盖 authorization and requested paths never invoke other lane、empty partial zero unavailable and safe failures、parallel common deadline and finalizers wait for both reads 等 24 个场景。 |
|
||||
| `test_context_search_both_paths.py` | `integration` | The objective names 'recall' and requires BOTH paths: raw and memory. |
|
||||
| `test_context_v2_contract_parity.py` | `unit` | Mechanical parity checks for the shared v2 coverage receipt contract. |
|
||||
| `test_continuous_completed_input_review.py` | `contract` | Independent review of completed-input reuse; real stores and controlled HY only. |
|
||||
| `test_continuous_coordinator.py` | `contract` | 覆盖 activation hook and startup recovery start single monitor then stop、failed preparation does not advance observed watermark、new journal event and source regrant wake replay without legacy request 等 6 个场景。 |
|
||||
| `test_continuous_outcome_unknown.py` | `contract` | 覆盖 a timed out continuous task becomes unknown not pending、a timed out non continuous task also becomes unknown、an unsettled continuous send becomes unknown not failed 等 8 个场景。 |
|
||||
| `test_continuous_policy.py` | `contract` | 覆盖 backlog snapshot over one hundred thousand is compact、caller rollback includes policy and backlog、enable idempotence rejects changed payload and superseded operation 等 19 个场景。 |
|
||||
| `test_continuous_processing_bridge.py` | `contract` | 覆盖 bound source regrant refuses old task、continuous inactive never falls back to legacy、continuous prepare lease ingest has no legacy receipt or old context 等 17 个场景。 |
|
||||
| `test_continuous_processing_integration.py` | `contract` | Real temporary Core/stores with controlled probe and HY boundaries, never real data. |
|
||||
| `test_contract_parity.py` | `contract` | Python 的枚举和 TypeScript 里那份手写的联合类型必须一致。 |
|
||||
| `test_conversation_search.py` | `integration` | Searching imported conversations before any generation model is configured. |
|
||||
| `test_conversation_seed_ids.py` | `contract` | 覆盖 seed ids read metadata only without hydrating、seed eligibility keeps person and memory content only、latest activity includes nonseed turns and import time fallback 等 13 个场景。 |
|
||||
@@ -97,7 +87,6 @@
|
||||
| `test_database_contention.py` | `contract` | Writers meet in this one file, and the loser used to lose data. |
|
||||
| `test_database_file_permissions.py` | `contract` | 覆盖 a new main database is readable by its owner alone、the wal sidecars are not left world readable、a database from an older build is tightened on open 场景。 |
|
||||
| `test_db_session_utc.py` | `contract` | The MySQL session must be pinned to UTC. |
|
||||
| `test_delete_no_conversation_freeze.py` | `contract` | 删掉一条记忆,不得让同一对话之后的新 turn 停止沉淀(RC10)。 |
|
||||
| `test_delivery_board.py` | `contract` | 覆盖 parses only the marked structured record、verified requires evidence with a positive control、rejects unknown dependency 等 5 个场景。 |
|
||||
| `test_derived_echo.py` | `contract` | A host that saves our injection back must not mint a new user decision. |
|
||||
| `test_detailed_logging.py` | `contract` | 覆盖 detailed pipeline logging defaults off and survives restart、live log level applies to full hy and provider call chain、info collects hy info without enabling debug pipeline content 等 7 个场景。 |
|
||||
@@ -111,7 +100,6 @@
|
||||
| `test_dual_engine_helper.py` | `contract` | Tests for the dual-engine harness itself. |
|
||||
| `test_event_loop_blocking.py` | `contract` | Proof, as a test, that blocking work on the loop freezes it — and the fix works. |
|
||||
| `test_evidence_reconcile_primitive.py` | `integration` | A bounded, read-only discovery primitive for the H23 Evidence reconcile. |
|
||||
| `test_evidence_revision_read_review.py` | `contract` | Read invalidation review: real guard/provenance, controlled HY, no model/network. |
|
||||
| `test_evidence_store.py` | `contract` | 覆盖 same source message merges origins without duplication、identical text from different sources is not deduplicated、rescanning an unchanged file does not write the message again 等 26 个场景。 |
|
||||
| `test_export_probe.py` | `contract` | What a probe may say about a stranger's chat history, and what it may not. |
|
||||
| `test_external_endpoint_scopes.py` | `integration` | Core boundary tests using the broker's envelope, never a host's claimed identity. |
|
||||
@@ -125,10 +113,9 @@
|
||||
| `test_file_import_operations.py` | `integration` | Bounded synthetic uploads; operation identity is not a content dedup key. |
|
||||
| `test_file_import_operations_review.py` | `contract` | Upload refusal/legacy compatibility review with bounded synthetic files only. |
|
||||
| `test_file_publish_job_boundary.py` | `contract` | Staged uploads cannot widen an earlier directory import's uncaptured input. |
|
||||
| `test_four_app_live_ingest.py` | `integration` | Actual source readers + OS watch + Core routes; no model or native host calls. |
|
||||
| `test_four_app_live_ingest.py` | `contract` | Actual source readers + OS watch + Core routes; no model or native host calls. |
|
||||
| `test_frozen_core_layout.py` | `contract` | 覆盖 single file layout remains available without private preset、folder layout keeps same core and all native dependencies、unknown layout does not silently build another package 等 5 个场景。 |
|
||||
| `test_golden_corpus.py` | `contract` | A7-01: the golden corpus comparison. |
|
||||
| `test_governance_authority_review.py` | `contract` | D04 authority and workspace scope through the real continuous permission guard. |
|
||||
| `test_governance_permission_commit.py` | `contract` | 覆盖 permission guard is rechecked inside write transaction and rolls back、guard context does not leak into subsequent manual actions、guard check and commit hold database write authority 场景。 |
|
||||
| `test_governance_read_path_does_not_take_the_writer.py` | `contract` | The recall read path must not acquire the single global writer. |
|
||||
| `test_governance_store_root.py` | `contract` | 生产代码里治理 store 只在 Core 启动的组合根建一次。 |
|
||||
@@ -144,32 +131,19 @@
|
||||
| `test_hourly_c1_peak_replay_contract.py` | `contract` | Guard the package peak-replay probe's error accounting. |
|
||||
| `test_hourly_c1_recall_stage_probe_contract.py` | `contract` | Guard the packaged recall stage probe's command and error semantics. |
|
||||
| `test_hy_baseline_source.py` | `contract` | 覆盖 first import replay copies raw prefix without selecting message ids、frozen transcript keeps middle user message and excludes later records 场景。 |
|
||||
| `test_hy_coding_lease.py` | `contract` | HY's own coding/working route under the permission lease, on the pinned SDK. |
|
||||
| `test_hy_coding_reply.py` | `contract` | R-07: HY's coding-shaped replies read as "no memories at all". |
|
||||
| `test_hy_lease_responsiveness.py` | `contract` | Blocked lease checks, local failing doubles only; no network or credentials. |
|
||||
| `test_hy_lifecycle_sync.py` | `contract` | HY decides deletions; the ledger mirrors them. Pinned SDK, scripted HTTP. |
|
||||
| `test_hy_log_privacy.py` | `contract` | HY SDK records are not rewritten by One Memory's logging setup. |
|
||||
| `test_hy_memory_bridge.py` | `contract` | 覆盖 hy client initialization is serial across bridges、native hy clients share one runtime per data directory、ollama profile ignores ambient remote storage and models 等 41 个场景。 |
|
||||
| `test_hy_metrics_lifecycle.py` | `contract` | What the pinned public SDK's MetricsCollector does, and what that costs us. |
|
||||
| `test_hy_native_write_chain.py` | `contract` | HY writes with its own prompts and judgment; the product adds no audit of its own. |
|
||||
| `test_hy_network_egress.py` | `contract` | Where a real HY client is allowed to send bytes, and where it is not. |
|
||||
| `test_hy_permission_context.py` | `contract` | Whole native HY client with synthetic content and HTTP MockTransport, never cloud. |
|
||||
| `test_hy_permission_context_races.py` | `contract` | Bounded factory races with a temporary native HY client and no model calls. |
|
||||
| `test_hy_permission_raw_context.py` | `contract` | Real pinned HY SDK/Chroma and guards; scripted HTTP is not model acceptance. |
|
||||
| `test_hy_permission_store.py` | `contract` | Native HY node/algorithm tests with synthetic data and an in-memory backend only. |
|
||||
| `test_hy_permission_transport.py` | `contract` | Native HY entry points, synthetic inputs and MockTransport only. |
|
||||
| `test_hy_response_guard.py` | `contract` | 覆盖 invalid response retries same hy provider without changing prompt、repeated empty response fails instead of becoming no op、failure receipt survives a later successful model call 等 23 个场景。 |
|
||||
| `test_hy_response_shapes.py` | `contract` | HY response shape selection only; no provider, write routing or model calls. |
|
||||
| `test_hy_runtime_config.py` | `contract` | MEMORY.md is the audited record of what we send HY; keep it answerable. |
|
||||
| `test_hy_sdk_idempotency_probe.py` | `contract` | A0-03: probe whether the HY SDK can make a write idempotent. |
|
||||
| `test_hy_source_context.py` | `contract` | 覆盖 bound context rejects a same named conversation in another workspace、bound context rejects a duplicated evidence id、bound context rejects a same conversation from another source 等 9 个场景。 |
|
||||
| `test_hy_stage_trace.py` | `contract` | 覆盖 observer returns native results without editing or requesting models、observer redacts content without recording prompts or exception details 场景。 |
|
||||
| `test_hy_state_source.py` | `contract` | RC4:记忆生命周期(有效 / 被取代 / 去重隐藏 / 过期)直接读 HY,产品账本不再逐类镜像。 |
|
||||
| `test_hy_step_timing.py` | `contract` | Where an HY write spends the time that is not a model call. |
|
||||
| `test_hy_step_trace_switch.py` | `contract` | Rule 15: the per-step HY trace is off unless the debug tier asks for it. |
|
||||
| `test_hy_supersede_sync.py` | `contract` | HY decides supersessions; the ledger mirrors them. Pinned SDK, scripted HTTP. |
|
||||
| `test_hy_trial_budget.py` | `contract` | Offline proofs for the real-model acceptance budget (no network, no Keychain). |
|
||||
| `test_hy_turn_input.py` | `contract` | HY gets one turn at a time, exactly as it happened, and picks its own route. |
|
||||
| `test_hy_writeback.py` | `integration` | HY 的结果只有一扇门进治理表,而且每次进门都带着调用开始时的证据快照。 |
|
||||
| `test_import_api.py` | `integration` | 覆盖 resume during cleanup returns conflict without internal error、authorize detect import and revoke api flow、protected api rejects missing token and foreign origin 等 5 个场景。 |
|
||||
| `test_import_control_races_review.py` | `contract` | Read-only review repro: real runner/store, synthetic delayed adapter cleanup. |
|
||||
@@ -181,7 +155,6 @@
|
||||
| `test_import_three_scenarios.py` | `contract` | The three import scenarios a user actually meets. |
|
||||
| `test_import_three_scenarios_real.py` | `contract` | The three import scenarios, on a real Core at a real volume. |
|
||||
| `test_imported_index_background.py` | `contract` | The ordering index is built off the startup path. |
|
||||
| `test_incremental_memory_backfill.py` | `contract` | 覆盖 accept backfill returns after first batch while later history is still read、backfill budget yields between rounds and resumes same cursor、backfill holds an unanswered raw turn until its reply arrives 等 18 个场景。 |
|
||||
| `test_ingest_journal.py` | `contract` | 覆盖 baseline and empty initialize are explicit transactional and idempotent、old date new import and revise noop and rollbacks、import metadata changes invalidate without authorizing content 等 12 个场景。 |
|
||||
| `test_input_sanitization.py` | `unit` | 覆盖 redacts cli and assignment secrets without removing context、redacts bearer known tokens urls and private keys、redacts quoted json assignments and basic authorization 等 8 个场景。 |
|
||||
| `test_installer.py` | `contract` | 覆盖 installer create verify and rollback is transactional、installer updates owned file and restores backup、installer refuses unowned or changed target 场景。 |
|
||||
@@ -209,7 +182,6 @@
|
||||
| `test_main_database_fairness.py` | `contract` | Waiting for the one database has to be a queue, not a scramble. |
|
||||
| `test_main_database_isolation.py` | `contract` | What one checkout of the main database can do, and what it leaves for the next. |
|
||||
| `test_managed_restore.py` | `contract` | 覆盖 restore rejects a source outside the managed backup directory、restore rejects a symlinked source、restore rejects a non one memory database 等 9 个场景。 |
|
||||
| `test_manual_memory_via_hy.py` | `contract` | T04: what the person explicitly asks to remember is written through HY's import path. |
|
||||
| `test_manual_transcript_import.py` | `integration` | 覆盖 manual json keeps authors unknown time and exact text、marked text keeps code fences and never invents timestamps、ambiguous partial or invalid manual data never leaves partial import 等 4 个场景。 |
|
||||
| `test_matter_collection.py` | `integration` | 覆盖 hy windows cover every turn including leading and boundary assistants、oversize turn is split without silently dropping content、partial timestamps preserve source sequence 等 8 个场景。 |
|
||||
| `test_matter_discovery.py` | `contract` | 覆盖 synthesis materializes only known verbatim citations、duplicate claims keep all governed refs and distinct states、synthesis cannot echo the original sixty claim list 等 27 个场景。 |
|
||||
@@ -267,30 +239,18 @@
|
||||
| `test_memory_write_registry.py` | `contract` | A3-01: acceptance and outcome recording for the unified write path. |
|
||||
| `test_memory_write_routes.py` | `integration` | A3-01: the HTTP write entry, tested against a real FastAPI app. |
|
||||
| `test_migrate_off_sqlite.py` | `contract` | A1-06: the migration is only done when reconciliation agrees. |
|
||||
| `test_model_content_bulk.py` | `contract` | Bulk consent reads preserve exact identities and SQLite snapshot boundaries. |
|
||||
| `test_model_content_parse_cost.py` | `contract` | Memoize parsing only; every permission decision still reads current SQLite state. |
|
||||
| `test_model_content_permissions.py` | `contract` | Content permission tests use only a temporary database and synthetic sources. |
|
||||
| `test_model_content_preview_races.py` | `contract` | Confirmation must tolerate append-only import without broadening consent. |
|
||||
| `test_model_content_routes.py` | `integration` | 覆盖 permission route reads and writes only config final state、core composition starts with config permission and durable drain 场景。 |
|
||||
| `test_model_content_scope_seals.py` | `contract` | Indexed scope invalidation retains complete-snapshot semantics without rereading it. |
|
||||
| `test_model_content_snapshot_cost.py` | `contract` | Reuse derived fingerprints only after reading every raw row in this transaction. |
|
||||
| `test_model_creation_api.py` | `integration` | Normal protected model creation routes with temporary data and in-memory secrets. |
|
||||
| `test_model_creation_operations.py` | `contract` | 覆盖 lookup without table is read only and legacy create does not initialize、operation commits with profile and version without autoactivation、any replay conflicts before secret put 等 21 个场景。 |
|
||||
| `test_model_creation_validation_boundary.py` | `integration` | Real ASGI validation must precede creation reservation and secret writes. |
|
||||
| `test_model_policy_activation.py` | `contract` | 覆盖 activation helper is transactional and restores row factory、backlog scan does not hold the single writer、cloud credential is checked without keychain write 等 21 个场景。 |
|
||||
| `test_model_policy_routes.py` | `contract` | 覆盖 all routes require auth、backlog is required not defaulted、enable and transition safe shape and callback once after commit 等 10 个场景。 |
|
||||
| `test_model_probe.py` | `contract` | 覆盖 probe still measures embedding when structured output fails、probe records the real embedding width on the profile、openai compatible probe checks model and structured output 等 23 个场景。 |
|
||||
| `test_model_probe_input_revision.py` | `contract` | 覆盖 input revision precedes measured width update、concurrent update still rejects width or capability cas、early returns record the actual starting revision 等 6 个场景。 |
|
||||
| `test_model_probe_operations.py` | `integration` | Persisted probe identity, real SQLite/API and synthetic transport only. |
|
||||
| `test_model_processing_permissions.py` | `contract` | 覆盖 lease reads current allowed before each boundary、lease refuses a task profile that is no longer active 场景。 |
|
||||
| `test_model_profile_api.py` | `integration` | 覆盖 model profile api never returns key and enforces revision、model profile probe route returns only machine results、model profile validation never echoes api key 等 4 个场景。 |
|
||||
| `test_model_profile_context_reset.py` | `integration` | 覆盖 create context default and inclusive bounds、explicit null clears override and preserves history and secret、omitted or same value is noop with unchanged revision and metadata 等 11 个场景。 |
|
||||
| `test_model_profile_identity.py` | `contract` | Stable model editing and configured-vs-available credentials; no Keychain/network. |
|
||||
| `test_model_profile_queue_handover.py` | `contract` | Queued work when the model changes or its configuration is deleted (D75). |
|
||||
| `test_model_profiles.py` | `contract` | 覆盖 profile uses one model and keeps key out of database、api key updates reuse one stable secret reference、startup cleanup removes only secrets not referenced by current profiles 等 15 个场景。 |
|
||||
| `test_model_request_budget.py` | `contract` | 覆盖 persists failed request reservations without content、parallel leases cannot exceed request limit、unknown or excessive output and body are denied 等 6 个场景。 |
|
||||
| `test_model_request_budget_integration.py` | `contract` | Real HY leaf retries plus durable budget accounting; no network or provider. |
|
||||
| `test_model_scope_lifecycle.py` | `contract` | Bound derived/temporary data without changing consent or original history. |
|
||||
| `test_mysql_lifecycle.py` | `contract` | A1-07: the decisions Runtime makes around the bundled MySQL. |
|
||||
| `test_mysql_platform.py` | `contract` | A1-07: the machine-facing half of managing the bundled MySQL. |
|
||||
| `test_obs03_health_probe.py` | `contract` | OBS-03: '/health' answered with the web app instead of a health payload. |
|
||||
@@ -299,14 +259,11 @@
|
||||
| `test_oversized_file_guard.py` | `contract` | One pass over a source file is bounded by a byte budget, not skipped. |
|
||||
| `test_parent_watchdog.py` | `integration` | A Core that outlives its launcher makes the next launch impossible. |
|
||||
| `test_policy_api_wiring.py` | `integration` | 覆盖 global model permission off does not backfill、backfill puts history on write jobs without scheduling memory tasks、source pause skips history until the source is reenabled 场景。 |
|
||||
| `test_policy_operation_lookup.py` | `contract` | 覆盖 corrupt current row cannot make history or missing operation look safe、current operation uses safe receipt without callback or writes、lookup connection rejects writes and preserves database 等 12 个场景。 |
|
||||
| `test_policy_production_entry.py` | `contract` | 覆盖 production serve passes only supported app options、maintenance serve does not own the runtime 场景。 |
|
||||
| `test_precipitation_concurrency_setting.py` | `contract` | Rule 2/9: precipitation concurrency is configurable, not hardcoded to 1. |
|
||||
| `test_primary_source_adapters.py` | `contract` | 覆盖 pipeline json is host generated even when truncated、workbuddy record parsing never blocks the event loop、cursor store user info envelope is not attributed to the person 等 34 个场景。 |
|
||||
| `test_probe_code_contract_parity.py` | `contract` | 探测码在三处必须同时存在:python 枚举、TS 契约、界面文案表。 |
|
||||
| `test_probe_payment_required.py` | `contract` | 覆盖 http 402 without a useful body is still understood as unpaid、a wrong address is still a missing model not an unpaid one 场景。 |
|
||||
| `test_processing_coordinator.py` | `contract` | 覆盖 monitor is retained and observes config plus tasks 场景。 |
|
||||
| `test_processing_merged_context.py` | `contract` | 覆盖 new chunk omits known memory with changed support but keeps original history 场景。 |
|
||||
| `test_processing_requests.py` | `contract` | 覆盖 concurrent claim only one snapshot wins、concurrent create same grant has one id and profile、concurrent different profile never reuses another scope 等 14 个场景。 |
|
||||
| `test_quality_acceptance.py` | `contract` | 覆盖 acceptance pack is deterministic and scores personal boundary、acceptance score fails missing personal memory 场景。 |
|
||||
| `test_raw_search_without_model.py` | `integration` | Search imported conversations with no generative model configured at all. |
|
||||
@@ -375,8 +332,6 @@
|
||||
| `test_session_inbox.py` | `contract` | A session inbox is not a shared mailbox and a pull is not adoption. |
|
||||
| `test_session_lease.py` | `contract` | A session file on disk is not proof that the session is online. |
|
||||
| `test_session_registry.py` | `contract` | Trusted session identity is not a workspace path. |
|
||||
| `test_settings_policy_lifecycle.py` | `contract` | Settings data-plane lifecycle: real temporary Core, fake probe/HY, no UI or package. |
|
||||
| `test_simplified_model_permission_migrations.py` | `contract` | 覆盖 fresh schema omits removed permission state machines 场景。 |
|
||||
| `test_snapshot_adapter.py` | `contract` | Selection-only binding reopens checksum-verified frozen copies. |
|
||||
| `test_snapshot_pool.py` | `contract` | A whole-directory snapshot must not hold a thread everything else is waiting for. |
|
||||
| `test_soak_probe_effectiveness.py` | `integration` | The soak probe must observe the load it reports on. |
|
||||
@@ -392,7 +347,7 @@
|
||||
| `test_source_parsing.py` | `contract` | 覆盖 reminders are removed only at message edges、original supported outer context and query envelopes、attributed reminder is opaque before finding the outer query 等 11 个场景。 |
|
||||
| `test_source_permission_races.py` | `integration` | Permission execution regressions using temporary stores and Core ASGI instances. |
|
||||
| `test_source_query_preview.py` | `contract` | 覆盖 preserves exact user text and span、supported outer wrapper is stripped without paraphrasing、cursor host prefixes are not shown as the user query 等 8 个场景。 |
|
||||
| `test_source_read_revocation.py` | `integration` | 覆盖 source revocation stops every outward read but keeps other sources、revocation during model wait prevents late answer 场景。 |
|
||||
| `test_source_read_revocation.py` | `contract` | 覆盖 revocation during model wait prevents late answer、source revocation stops every outward read but keeps other sources 场景。 |
|
||||
| `test_source_readers.py` | `contract` | 覆盖 jsonl is resumable and does not advance over an incomplete tail、sqlite reader uses monotonic row checkpoint、zstd jsonl reader resumes by completed record 等 7 个场景。 |
|
||||
| `test_source_registry_and_grants.py` | `contract` | 覆盖 catalog resolves defaults and marks environment overrides、recall hosts that can append join the live sync poller、declarations with no reader are named rather than mistaken for behaviour 等 15 个场景。 |
|
||||
| `test_source_revocation_parking.py` | `integration` | Revoking a source parks its unfinished extraction instead of failing it (D73). |
|
||||
@@ -402,7 +357,7 @@
|
||||
| `test_source_sync_idempotent.py` | `integration` | Final-state Harness sync is idempotent and resumes missed source work. |
|
||||
| `test_source_sync_permissions.py` | `integration` | S1.3 source permission tests use temporary files only, never real host history. |
|
||||
| `test_source_totals.py` | `contract` | 覆盖 the first ask is answered from the rows、same named sessions in two workspaces count as two conversations、insert during count is not marked as already counted 等 7 个场景。 |
|
||||
| `test_source_upload_intents.py` | `integration` | Upload identity binds retries without replacing file receipts or directory consent. |
|
||||
| `test_source_upload_intents.py` | `contract` | Upload identity binds retries without replacing file receipts or directory consent. |
|
||||
| `test_source_write_switch.py` | `contract` | A frozen Harness produces no extraction tasks. |
|
||||
| `test_sql_compat.py` | `contract` | The shared compatibility layer, especially 'rowcount'. |
|
||||
| `test_sqlite_semantics_gate.py` | `contract` | D56: prevent SQLite implementation details spreading outside the data layer. |
|
||||
@@ -411,13 +366,10 @@
|
||||
| `test_store_binding.py` | `contract` | A2: which store is in force, and honest reporting of it. |
|
||||
| `test_store_binding_ratchet.py` | `contract` | D56: ratchet the retired dual-engine 'store_for' seam out of the Core. |
|
||||
| `test_store_port.py` | `contract` | A1-04/A2: the store port, and that real implementations satisfy it. |
|
||||
| `test_subagent_sessions.py` | `contract` | D98: a sub agent's transcript is its own conversation, not part of the main one. |
|
||||
| `test_subagent_turns.py` | `contract` | D98: a sub agent's turn is ordinary user-assistant data. |
|
||||
| `test_subscription_visibility.py` | `unit` | A shared directory is not a read grant. |
|
||||
| `test_sync_facade.py` | `contract` | A2: the sync facade must keep the sync shape call sites already use. |
|
||||
| `test_tail_metadata_lane.py` | `contract` | Post-response metadata retries must be bounded and never block the route. |
|
||||
| `test_terminal_payload_reaping.py` | `contract` | Rule 14: dead-end payloads are actually reaped, not just reapable. |
|
||||
| `test_tool_evidence_hy_turns.py` | `contract` | Every adapter's tool records reach HY inside the turn they happened in. |
|
||||
| `test_two_level_discovery.py` | `contract` | 3925a kernel item 1: two-level discovery. |
|
||||
| `test_ui_session.py` | `contract` | 覆盖 exchange is single use and creates a session、expired exchange cannot create a session、exchange is bound to the core instance 等 5 个场景。 |
|
||||
| `test_ui_session_api.py` | `integration` | 覆盖 ui session exchange sets httponly strict cookie、ui session requires bearer for exchange、ui session reports missing web assets 场景。 |
|
||||
@@ -429,8 +381,8 @@
|
||||
| `test_windows_process_control.py` | `contract` | R-29:Windows 上 'os.kill' 不是发信号,是无条件杀进程。 |
|
||||
| `test_workbuddy_boundary.py` | `contract` | WorkBuddy only injects on SessionStart and UserPromptSubmit. |
|
||||
| `test_workbuddy_parsing.py` | `contract` | 覆盖 automation preserves query without claiming fresh human authorship、automation context attributes do not depend on order or quote style、quoted full automation envelope is not scheduled provenance 等 17 个场景。 |
|
||||
| `test_workbuddy_prompt_backfill.py` | `integration` | When the prompt hook's capture fails, the transcript file is what brings the words in. |
|
||||
| `test_workbuddy_prompt_backfill.py` | `contract` | When the prompt hook's capture fails, the transcript file is what brings the words in. |
|
||||
| `test_workspace_recall.py` | `contract` | Workspace memories were unreachable from every host; these pin the way in. |
|
||||
| `test_write_queue_lease_renewal.py` | `contract` | Rule 8: a lease must outlive the work it covers, and be renewed while it runs. |
|
||||
|
||||
共 428 个测试文件。
|
||||
共 380 个测试文件。
|
||||
@@ -14,8 +14,8 @@ from pathlib import Path
|
||||
import pytest
|
||||
|
||||
PRIMARY_MARKERS = ("unit", "contract", "integration", "performance", "live")
|
||||
EXPECTED_TEST_FILE_COUNT = 428
|
||||
EXPECTED_TEST_FILES_SHA256 = "bd3bf4fc9cda2a8e4427d106283c40d22dad026fd452db954046b6f28d21b896"
|
||||
EXPECTED_TEST_FILE_COUNT = 380
|
||||
EXPECTED_TEST_FILES_SHA256 = "9f075d18403328882c25325d29ebe77d0e3365d675ba19d56134a6a0fb9b64b4"
|
||||
|
||||
# Pure transformations, validation models, parity checks and state machines.
|
||||
# This list is intentionally small: uncertain files default to contract.
|
||||
@@ -68,7 +68,6 @@ INTEGRATION_FILES = frozenset(
|
||||
"test_external_installation_permissions.py",
|
||||
"test_fault_matrix.py",
|
||||
"test_file_import_operations.py",
|
||||
"test_four_app_live_ingest.py",
|
||||
"test_graph_width_endpoint.py",
|
||||
"test_home_summary.py",
|
||||
"test_hy_writeback.py",
|
||||
@@ -100,7 +99,6 @@ INTEGRATION_FILES = frozenset(
|
||||
"test_memory_supersede.py",
|
||||
"test_memory_visibility.py",
|
||||
"test_memory_write_routes.py",
|
||||
"test_model_content_routes.py",
|
||||
"test_model_creation_api.py",
|
||||
"test_model_creation_validation_boundary.py",
|
||||
"test_model_probe_operations.py",
|
||||
@@ -124,17 +122,14 @@ INTEGRATION_FILES = frozenset(
|
||||
"test_source_connection_intents.py",
|
||||
"test_source_import_receipts.py",
|
||||
"test_source_permission_races.py",
|
||||
"test_source_read_revocation.py",
|
||||
"test_source_revocation_parking.py",
|
||||
"test_source_session_count_api.py",
|
||||
"test_source_status_latency.py",
|
||||
"test_source_sync_idempotent.py",
|
||||
"test_source_sync_permissions.py",
|
||||
"test_source_upload_intents.py",
|
||||
"test_ui_session_api.py",
|
||||
"test_unified_write_entry_wiring.py",
|
||||
"test_visibility_background.py",
|
||||
"test_workbuddy_prompt_backfill.py",
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
@@ -26,67 +26,6 @@ async def thaw_source(client, source_id, receipt):
|
||||
return response.json()
|
||||
|
||||
|
||||
async def confirm_model_content(
|
||||
client,
|
||||
*,
|
||||
source_ids,
|
||||
evidence_ids=(),
|
||||
all_imported=None,
|
||||
future_additions=True,
|
||||
process=True,
|
||||
):
|
||||
"""Grant model processing the same way the product does: preview, then confirm.
|
||||
|
||||
Live suites need this because importing Evidence is not consent to send it
|
||||
to a model; without it the processing guard refuses, which is the correct
|
||||
product behaviour and not something a test may bypass by writing the store.
|
||||
"""
|
||||
profiles = await client.get("/api/v1/model-profiles")
|
||||
profiles.raise_for_status()
|
||||
active = next(item for item in profiles.json() if item["is_active"])
|
||||
preview = await client.post(
|
||||
"/api/v1/model-content-permission/preview",
|
||||
json={
|
||||
"profile_id": active["profile_id"],
|
||||
"profile_revision": active["revision"],
|
||||
"source_ids": list(source_ids),
|
||||
"evidence_ids": list(evidence_ids),
|
||||
# The manual source only accepts named originals, so "everything
|
||||
# from these sources" is not a default the caller can ignore.
|
||||
"all_imported_from_selected_sources": (
|
||||
not evidence_ids if all_imported is None else all_imported
|
||||
),
|
||||
"include_future_additions": future_additions,
|
||||
},
|
||||
)
|
||||
# Report the refusal code, not just the status: every content refusal is a
|
||||
# named product decision and a live run that hides it cannot be diagnosed.
|
||||
assert preview.status_code == 200, preview.text
|
||||
current = await client.get("/api/v1/model-content-permission")
|
||||
current.raise_for_status()
|
||||
existing = current.json()
|
||||
confirmed = await client.post(
|
||||
"/api/v1/model-content-permission",
|
||||
json={
|
||||
"snapshot": preview.json(),
|
||||
"expected_revision": existing["revision"] if existing else 0,
|
||||
"user_confirmed": True,
|
||||
},
|
||||
)
|
||||
confirmed.raise_for_status()
|
||||
receipt = confirmed.json()
|
||||
if process:
|
||||
# What the product does after the user confirms: take the already
|
||||
# imported Evidence through the pipeline instead of waiting for the
|
||||
# next import to trigger a debounce.
|
||||
accepted = await client.post(
|
||||
"/api/v1/model-content-permission/process",
|
||||
json={"expected_revision": receipt["revision"], "user_confirmed": True},
|
||||
)
|
||||
accepted.raise_for_status()
|
||||
return receipt
|
||||
|
||||
|
||||
async def connection_request(client, source_ids, **options):
|
||||
roots = {}
|
||||
for source_id in source_ids:
|
||||
|
||||
@@ -65,12 +65,7 @@ HY_BRIDGE = f"{PACKAGE_LABEL}/hy_memory_bridge.py"
|
||||
SDK_CALLER = HY_BRIDGE
|
||||
|
||||
#: Type-only importers of HY datatypes; they hold no client and write nothing.
|
||||
HY_TYPE_ONLY = frozenset(
|
||||
{
|
||||
f"{PACKAGE_LABEL}/hy_permission_store.py",
|
||||
f"{PACKAGE_LABEL}/hy_permission_coding_store.py",
|
||||
}
|
||||
)
|
||||
HY_TYPE_ONLY: frozenset[str] = frozenset()
|
||||
|
||||
|
||||
def _relative(path: Path) -> str:
|
||||
|
||||
@@ -1,248 +0,0 @@
|
||||
import asyncio
|
||||
import threading
|
||||
|
||||
import pytest
|
||||
|
||||
from one_memory.model_profiles import ModelProfilePatch
|
||||
from tests.test_model_processing_permissions import Harness
|
||||
|
||||
|
||||
def test_quiet_arrival_guard_is_off_loop_and_5000_events_coalesce(tmp_path, monkeypatch):
|
||||
h = Harness(tmp_path)
|
||||
supervisor = h.supervisor()
|
||||
entered, release = threading.Event(), threading.Event()
|
||||
checked = []
|
||||
original = supervisor._guard_records
|
||||
|
||||
async def scenario():
|
||||
loop_thread = threading.get_ident()
|
||||
|
||||
def guard(records, *args, **kwargs):
|
||||
checked.append(threading.get_ident())
|
||||
assert threading.get_ident() != loop_thread, "arrival parsed permission on the loop"
|
||||
entered.set()
|
||||
assert release.wait(5)
|
||||
return original(records, *args, **kwargs)
|
||||
|
||||
def credentials(*_args, **_kwargs):
|
||||
pytest.fail("denied preparation reached credential/profile resolution")
|
||||
|
||||
monkeypatch.setattr(supervisor, "_guard_records", guard)
|
||||
monkeypatch.setattr(h.profiles, "active", credentials)
|
||||
monkeypatch.setattr(h.profiles, "resolve_snapshot_secret", credentials)
|
||||
try:
|
||||
for _ in range(5000):
|
||||
supervisor.enqueue_evidence(h.record)
|
||||
assert checked == []
|
||||
assert len(supervisor._debounces) == 1
|
||||
assert await asyncio.to_thread(entered.wait, 3)
|
||||
# This callback must run while the actual permission reader is blocked.
|
||||
responsive = asyncio.Event()
|
||||
asyncio.get_running_loop().call_soon(responsive.set)
|
||||
await asyncio.wait_for(responsive.wait(), 0.5)
|
||||
assert checked == [checked[0]]
|
||||
release.set()
|
||||
await asyncio.gather(*tuple(supervisor._debounces.values()))
|
||||
assert supervisor._preparations == {}
|
||||
assert h.tasks.list() == []
|
||||
assert len(supervisor._queue) == 0 and supervisor._workers == {}
|
||||
assert h.factory_calls == [] and h.bridge.calls == []
|
||||
finally:
|
||||
release.set()
|
||||
await supervisor.shutdown()
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
@pytest.mark.parametrize("stop_mode", ["cancel", "shutdown"])
|
||||
def test_live_preparation_cancel_joins_thread_without_late_dispatch(
|
||||
tmp_path, monkeypatch, stop_mode
|
||||
):
|
||||
h = Harness(tmp_path)
|
||||
h.grant()
|
||||
supervisor = h.supervisor()
|
||||
entered, release = threading.Event(), threading.Event()
|
||||
original = supervisor._prepare_conversation_work
|
||||
completed = []
|
||||
|
||||
def blocked(seed, **kwargs):
|
||||
entered.set()
|
||||
assert release.wait(5)
|
||||
result = original(seed, **kwargs)
|
||||
completed.append(True)
|
||||
return result
|
||||
|
||||
monkeypatch.setattr(supervisor, "_prepare_conversation_work", blocked)
|
||||
|
||||
async def scenario():
|
||||
stopping = None
|
||||
preparation = None
|
||||
try:
|
||||
supervisor.enqueue_evidence(h.record)
|
||||
assert await asyncio.to_thread(entered.wait, 3)
|
||||
preparation = next(iter(supervisor._preparations.values()))
|
||||
if stop_mode == "cancel":
|
||||
preparation.cancel()
|
||||
await asyncio.sleep(0.02)
|
||||
preparation.cancel()
|
||||
else:
|
||||
stopping = asyncio.create_task(supervisor.shutdown())
|
||||
await asyncio.sleep(0.02)
|
||||
assert not preparation.done(), "cancel detached the actual preparation thread"
|
||||
if stopping is not None:
|
||||
assert not stopping.done()
|
||||
finally:
|
||||
release.set()
|
||||
if preparation is not None:
|
||||
await asyncio.gather(preparation, return_exceptions=True)
|
||||
if stopping is not None:
|
||||
await stopping
|
||||
await supervisor.shutdown()
|
||||
assert completed == [True]
|
||||
assert supervisor._preparations == {} and supervisor._debounces == {}
|
||||
assert len(supervisor._queue) == 0 and supervisor._workers == {}
|
||||
assert h.factory_calls == [] and h.bridge.calls == []
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"change", ["model-revoked", "source-revoked", "profile-changed", "regranted-other-input"]
|
||||
)
|
||||
def test_quiet_preparation_rechecks_permission_before_credentials_or_durable_work(
|
||||
tmp_path, monkeypatch, change
|
||||
):
|
||||
h = Harness(tmp_path)
|
||||
receipt = h.grant()
|
||||
supervisor = h.supervisor()
|
||||
entered, release = threading.Event(), threading.Event()
|
||||
original = supervisor._prepare_conversation_work
|
||||
guarded = []
|
||||
original_guard = supervisor._guard_records
|
||||
|
||||
def blocked(seed, **kwargs):
|
||||
entered.set()
|
||||
assert release.wait(5)
|
||||
return original(seed, **kwargs)
|
||||
|
||||
def guard(*args, **kwargs):
|
||||
guarded.append(True)
|
||||
return original_guard(*args, **kwargs)
|
||||
|
||||
def credentials(*_args, **_kwargs):
|
||||
pytest.fail("denied preparation reached credential/profile resolution")
|
||||
|
||||
monkeypatch.setattr(supervisor, "_prepare_conversation_work", blocked)
|
||||
monkeypatch.setattr(supervisor, "_guard_records", guard)
|
||||
|
||||
async def scenario():
|
||||
try:
|
||||
supervisor.enqueue_evidence(h.record)
|
||||
assert await asyncio.to_thread(entered.wait, 3)
|
||||
if change in {"model-revoked", "regranted-other-input"}:
|
||||
h.permissions.revoke(
|
||||
user_id="local", expected_revision=receipt.revision, user_confirmed=True
|
||||
)
|
||||
if change == "regranted-other-input":
|
||||
h.grant((h.put("other", conversation="other"),))
|
||||
elif change == "source-revoked":
|
||||
h.sources.revoke(user_id="local", source_id="cursor")
|
||||
else:
|
||||
h.profiles.update(
|
||||
h.profile.profile_id,
|
||||
ModelProfilePatch(expected_revision=h.profile.revision, model="changed"),
|
||||
h.secrets,
|
||||
)
|
||||
monkeypatch.setattr(h.profiles, "active", credentials)
|
||||
monkeypatch.setattr(h.profiles, "resolve_snapshot_secret", credentials)
|
||||
release.set()
|
||||
await asyncio.gather(*tuple(supervisor._debounces.values()))
|
||||
assert guarded == [True]
|
||||
assert h.tasks.list() == []
|
||||
assert h.factory_calls == [] and h.bridge.calls == []
|
||||
assert len(supervisor._queue) == 0 and supervisor._workers == {}
|
||||
finally:
|
||||
release.set()
|
||||
await supervisor.shutdown()
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_cancelling_quiet_timer_does_not_start_any_permission_reader(tmp_path, monkeypatch):
|
||||
h = Harness(tmp_path)
|
||||
h.grant()
|
||||
supervisor = h.supervisor()
|
||||
checks = []
|
||||
original = supervisor._guard_records
|
||||
|
||||
def guard(*args, **kwargs):
|
||||
checks.append(True)
|
||||
return original(*args, **kwargs)
|
||||
|
||||
monkeypatch.setattr(supervisor, "_guard_records", guard)
|
||||
|
||||
async def scenario():
|
||||
try:
|
||||
supervisor.enqueue_evidence(h.record)
|
||||
timer = next(iter(supervisor._debounces.values()))
|
||||
await asyncio.sleep(0)
|
||||
timer.cancel()
|
||||
await asyncio.gather(timer, return_exceptions=True)
|
||||
assert checks == []
|
||||
assert supervisor._debounces == {} and supervisor._preparations == {}
|
||||
assert h.tasks.list() == [] and h.factory_calls == [] and h.bridge.calls == []
|
||||
finally:
|
||||
await supervisor.shutdown()
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_dispatch_permission_check_denies_before_queue_after_classification(tmp_path, monkeypatch):
|
||||
h = Harness(tmp_path)
|
||||
h.grant()
|
||||
supervisor = h.supervisor()
|
||||
work = supervisor._classify(supervisor.prepare_conversation(h.record))
|
||||
checked = []
|
||||
|
||||
def reject(task):
|
||||
checked.append(task.task_id)
|
||||
return False
|
||||
|
||||
async def scenario():
|
||||
monkeypatch.setattr(supervisor, "_guard_task", reject)
|
||||
try:
|
||||
supervisor._schedule_tasks(work)
|
||||
assert len(checked) == 1
|
||||
assert len(supervisor._queue) == 0 and supervisor._workers == {}
|
||||
assert h.factory_calls == []
|
||||
finally:
|
||||
await supervisor.shutdown()
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_dispatch_check_does_not_queue_work_after_shutdown(tmp_path, monkeypatch):
|
||||
h = Harness(tmp_path)
|
||||
h.grant()
|
||||
supervisor = h.supervisor()
|
||||
work = supervisor._classify(supervisor.prepare_conversation(h.record))
|
||||
checked = []
|
||||
|
||||
def allowed(task):
|
||||
checked.append(task.task_id)
|
||||
return True
|
||||
|
||||
monkeypatch.setattr(supervisor, "_guard_task", allowed)
|
||||
|
||||
async def scenario():
|
||||
stopping = asyncio.create_task(supervisor.shutdown())
|
||||
try:
|
||||
await stopping
|
||||
supervisor._schedule_tasks(work)
|
||||
finally:
|
||||
await supervisor.shutdown()
|
||||
assert len(supervisor._queue) == 0 and supervisor._workers == {}
|
||||
assert checked == []
|
||||
assert h.factory_calls == []
|
||||
|
||||
asyncio.run(scenario())
|
||||
@@ -1,244 +0,0 @@
|
||||
"""A processing acknowledgement belongs to one consent revision and live worker."""
|
||||
|
||||
import asyncio
|
||||
|
||||
import pytest
|
||||
|
||||
from one_memory.hy_memory_bridge import HyMemoryBridgeError
|
||||
from tests.test_model_processing_permissions import Harness
|
||||
|
||||
|
||||
def test_changed_consent_does_not_acknowledge_an_older_running_request(tmp_path, monkeypatch):
|
||||
async def scenario():
|
||||
h = Harness(tmp_path)
|
||||
first = h.grant()
|
||||
supervisor = h.supervisor()
|
||||
started = asyncio.Event()
|
||||
release = asyncio.Event()
|
||||
calls = []
|
||||
|
||||
async def backfill(*, expected_permission_revision=None, on_first_scheduled=None):
|
||||
calls.append(expected_permission_revision)
|
||||
if on_first_scheduled is not None:
|
||||
on_first_scheduled()
|
||||
started.set()
|
||||
await release.wait()
|
||||
|
||||
monkeypatch.setattr(supervisor, "backfill_off_thread", backfill)
|
||||
try:
|
||||
await asyncio.wait_for(
|
||||
supervisor.accept_backfill_off_thread(expected_permission_revision=first.revision),
|
||||
timeout=5,
|
||||
)
|
||||
await asyncio.wait_for(started.wait(), timeout=5)
|
||||
second = h.grant()
|
||||
await asyncio.wait_for(
|
||||
supervisor.accept_backfill_off_thread(expected_permission_revision=second.revision),
|
||||
timeout=5,
|
||||
)
|
||||
started.clear()
|
||||
await asyncio.wait_for(started.wait(), timeout=5)
|
||||
assert calls == [first.revision, second.revision]
|
||||
finally:
|
||||
release.set()
|
||||
await supervisor.shutdown()
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_late_failure_is_observed_in_processing_status(tmp_path, monkeypatch):
|
||||
async def scenario():
|
||||
h = Harness(tmp_path)
|
||||
receipt = h.grant()
|
||||
supervisor = h.supervisor()
|
||||
release = asyncio.Event()
|
||||
|
||||
async def backfill(*, expected_permission_revision=None, on_first_scheduled=None):
|
||||
if on_first_scheduled is not None:
|
||||
on_first_scheduled()
|
||||
await release.wait()
|
||||
raise HyMemoryBridgeError("model_content_source_permission_changed")
|
||||
|
||||
monkeypatch.setattr(supervisor, "backfill_off_thread", backfill)
|
||||
try:
|
||||
await supervisor.accept_backfill_off_thread(
|
||||
expected_permission_revision=receipt.revision
|
||||
)
|
||||
assert supervisor.accepted_backfill_status() == ("running", None)
|
||||
release.set()
|
||||
await supervisor._live_backfill
|
||||
assert supervisor.accepted_backfill_status() == (
|
||||
"failed",
|
||||
"model_content_source_permission_changed",
|
||||
)
|
||||
finally:
|
||||
release.set()
|
||||
await supervisor.shutdown()
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_same_consent_concurrent_requests_share_one_worker(tmp_path, monkeypatch):
|
||||
async def scenario():
|
||||
h = Harness(tmp_path)
|
||||
receipt = h.grant()
|
||||
supervisor = h.supervisor()
|
||||
entered, acknowledge, finish = asyncio.Event(), asyncio.Event(), asyncio.Event()
|
||||
calls = []
|
||||
|
||||
async def backfill(*, expected_permission_revision=None, on_first_scheduled=None):
|
||||
calls.append(expected_permission_revision)
|
||||
entered.set()
|
||||
await acknowledge.wait()
|
||||
if on_first_scheduled is not None:
|
||||
on_first_scheduled()
|
||||
await finish.wait()
|
||||
|
||||
monkeypatch.setattr(supervisor, "backfill_off_thread", backfill)
|
||||
first = asyncio.create_task(
|
||||
supervisor.accept_backfill_off_thread(expected_permission_revision=receipt.revision)
|
||||
)
|
||||
try:
|
||||
await entered.wait()
|
||||
second = asyncio.create_task(
|
||||
supervisor.accept_backfill_off_thread(expected_permission_revision=receipt.revision)
|
||||
)
|
||||
await asyncio.sleep(0)
|
||||
acknowledge.set()
|
||||
await asyncio.wait_for(asyncio.gather(first, second), 2)
|
||||
assert calls == [receipt.revision]
|
||||
finally:
|
||||
acknowledge.set()
|
||||
finish.set()
|
||||
await supervisor.shutdown()
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_background_failure_is_reported_after_acceptance_and_can_retry(tmp_path, monkeypatch):
|
||||
async def scenario():
|
||||
h = Harness(tmp_path)
|
||||
receipt = h.grant()
|
||||
supervisor = h.supervisor()
|
||||
|
||||
async def broken(**_kwargs):
|
||||
raise RuntimeError("secret or provider body must not be returned")
|
||||
|
||||
monkeypatch.setattr(supervisor, "backfill_off_thread", broken)
|
||||
try:
|
||||
await supervisor.accept_backfill_off_thread(
|
||||
expected_permission_revision=receipt.revision
|
||||
)
|
||||
await supervisor._live_backfill
|
||||
assert supervisor.accepted_backfill_status() == (
|
||||
"failed",
|
||||
"model_content_processing_unavailable",
|
||||
)
|
||||
|
||||
async def complete(**kwargs):
|
||||
callback = kwargs.get("on_first_scheduled")
|
||||
if callback is not None:
|
||||
callback()
|
||||
|
||||
monkeypatch.setattr(supervisor, "backfill_off_thread", complete)
|
||||
await supervisor.accept_backfill_off_thread(
|
||||
expected_permission_revision=receipt.revision
|
||||
)
|
||||
await supervisor._live_backfill
|
||||
assert supervisor.accepted_backfill_status() == ("done", None)
|
||||
finally:
|
||||
await supervisor.shutdown()
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_shutdown_rejects_new_processing_acceptance(tmp_path):
|
||||
async def scenario():
|
||||
h = Harness(tmp_path)
|
||||
receipt = h.grant()
|
||||
supervisor = h.supervisor()
|
||||
await supervisor.shutdown()
|
||||
with pytest.raises(HyMemoryBridgeError, match="model_content_processing_unavailable"):
|
||||
await supervisor.accept_backfill_off_thread(
|
||||
expected_permission_revision=receipt.revision
|
||||
)
|
||||
assert h.tasks.list() == []
|
||||
assert h.factory_calls == []
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_health_reports_late_explicit_processing_failure(tmp_path, monkeypatch):
|
||||
from httpx import ASGITransport, AsyncClient
|
||||
|
||||
from one_memory.api import create_app
|
||||
from one_memory.config import Settings
|
||||
|
||||
monkeypatch.setattr("one_memory.model_presets._preset_data", lambda: None)
|
||||
|
||||
async def scenario():
|
||||
app = create_app(Settings(_env_file=None, data_dir=tmp_path, imports_enabled=False))
|
||||
release = asyncio.Event()
|
||||
async with app.router.lifespan_context(app):
|
||||
await app.state.memory_backfill
|
||||
supervisor = app.state.memory_processing_supervisor
|
||||
supervisor._permission_guard = None
|
||||
|
||||
async def failed_later(**kwargs):
|
||||
callback = kwargs.get("on_first_scheduled")
|
||||
if callback is not None:
|
||||
callback()
|
||||
await release.wait()
|
||||
raise HyMemoryBridgeError("model_content_source_permission_changed")
|
||||
|
||||
monkeypatch.setattr(supervisor, "backfill_off_thread", failed_later)
|
||||
try:
|
||||
await supervisor.accept_backfill_off_thread()
|
||||
async with AsyncClient(
|
||||
transport=ASGITransport(app=app), base_url="http://test"
|
||||
) as client:
|
||||
running = (await client.get("/api/v1/health")).json()
|
||||
assert running["memory_backfill"] == "running"
|
||||
release.set()
|
||||
await supervisor._live_backfill
|
||||
failed = (await client.get("/api/v1/health")).json()
|
||||
assert failed["memory_backfill"] == "failed"
|
||||
assert (
|
||||
failed["memory_backfill_error"] == "model_content_source_permission_changed"
|
||||
)
|
||||
finally:
|
||||
release.set()
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_completed_caller_does_not_cancel_accepted_background_work(tmp_path, monkeypatch):
|
||||
async def scenario():
|
||||
h = Harness(tmp_path)
|
||||
receipt = h.grant()
|
||||
supervisor = h.supervisor()
|
||||
entered, release = asyncio.Event(), asyncio.Event()
|
||||
|
||||
async def backfill(**kwargs):
|
||||
entered.set()
|
||||
await release.wait()
|
||||
callback = kwargs.get("on_first_scheduled")
|
||||
if callback is not None:
|
||||
callback()
|
||||
|
||||
monkeypatch.setattr(supervisor, "backfill_off_thread", backfill)
|
||||
caller = asyncio.create_task(
|
||||
supervisor.accept_backfill_off_thread(expected_permission_revision=receipt.revision)
|
||||
)
|
||||
try:
|
||||
await asyncio.wait_for(caller, timeout=2)
|
||||
assert not supervisor._live_backfill.done()
|
||||
await asyncio.wait_for(entered.wait(), timeout=5)
|
||||
release.set()
|
||||
await supervisor._live_backfill
|
||||
assert supervisor.accepted_backfill_status() == ("done", None)
|
||||
finally:
|
||||
release.set()
|
||||
await supervisor.shutdown()
|
||||
|
||||
asyncio.run(scenario())
|
||||
@@ -1,54 +0,0 @@
|
||||
"""One stale conversation cannot invalidate an entire backfill scan."""
|
||||
|
||||
import asyncio
|
||||
from contextlib import suppress
|
||||
|
||||
from one_memory.hy_memory_bridge import HyMemoryBridgeError
|
||||
from one_memory.imports.models import EvidenceInput
|
||||
from tests.test_continuous_processing_integration import IntegrationCore
|
||||
|
||||
|
||||
def test_one_changed_conversation_does_not_block_other_conversations(tmp_path, monkeypatch):
|
||||
core = IntegrationCore(tmp_path, monkeypatch)
|
||||
|
||||
async def scenario():
|
||||
async with core.running(automatic_memory_processing=False):
|
||||
monkeypatch.setenv("ONE_MEMORY_DIAGNOSTICS", "1")
|
||||
core.grant_source()
|
||||
core.put_complete("conversation-good", conversation="good")
|
||||
changed = core.put_complete("conversation-changed", conversation="changed")
|
||||
await core.save_and_probe()
|
||||
core.kernel_enable("include")
|
||||
|
||||
evidence_store = core.app.state.evidence_store
|
||||
changed_input = EvidenceInput.model_validate(
|
||||
{
|
||||
**changed.model_dump(include=set(EvidenceInput.model_fields)),
|
||||
"verbatim": changed.verbatim + " edited",
|
||||
}
|
||||
)
|
||||
evidence_store.put(changed_input)
|
||||
assert evidence_store.get(changed.evidence_id).revision > changed.revision
|
||||
|
||||
# Production reaches completed-input reuse with the live chunk when
|
||||
# an earlier task already covered it; that reuse rechecks the
|
||||
# stored input against the row now in Evidence. Model the observed
|
||||
# stale-row refusal at the exact boundary that raises in production.
|
||||
guard = core.app.state.memory_processing_supervisor._permission_guard
|
||||
original_completed_lookup = guard.completed_input_task
|
||||
|
||||
def refuse_changed_conversation(receipt, records, **kwargs):
|
||||
if any(record.conversation_id == "changed" for record in records):
|
||||
raise HyMemoryBridgeError("model_content_evidence_changed")
|
||||
return original_completed_lookup(receipt, records, **kwargs)
|
||||
|
||||
monkeypatch.setattr(guard, "completed_input_task", refuse_changed_conversation)
|
||||
|
||||
supervisor = core.app.state.memory_processing_supervisor
|
||||
with suppress(Exception):
|
||||
await supervisor.backfill_off_thread()
|
||||
|
||||
await core.wait_completed(timeout=10, minimum_tasks=1)
|
||||
assert len(core.bridge.requests) == 1
|
||||
|
||||
asyncio.run(scenario())
|
||||
@@ -1,9 +1,8 @@
|
||||
"""Diagnostic: why does an all-imported preview see zero Evidence?
|
||||
|
||||
The C0 integration red asserts a preview snapshot has two Evidence rows after a
|
||||
real WorkBuddy import. Reproduces the exact query path from
|
||||
`ModelContentPermissionStore.snapshot_from_ids` and reports what the import
|
||||
actually wrote, so the owner can be named from evidence instead of inference.
|
||||
The C0 integration red asserted a preview snapshot had two Evidence rows after
|
||||
a real WorkBuddy import. This keeps the useful import-path diagnostic after
|
||||
the retired model-content preview layer was removed.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
|
||||
@@ -1,105 +0,0 @@
|
||||
"""Server-held previews preserve exact grants without round-tripping all refs."""
|
||||
|
||||
import json
|
||||
import sqlite3
|
||||
|
||||
import pytest
|
||||
|
||||
from one_memory.model_content_permissions import (
|
||||
ModelContentPermissionError,
|
||||
ModelContentPermissionStore,
|
||||
)
|
||||
from tests.test_model_content_bulk import many_records
|
||||
from tests.test_model_content_permissions import USER, evidence_value
|
||||
from tests.test_model_content_permissions import env as env
|
||||
|
||||
|
||||
def test_compact_preview_grants_exact_scope_and_survives_restart(env):
|
||||
records = many_records(env, 1000)
|
||||
snapshot = env.snapshot(records=tuple(records), future=True)
|
||||
result = env.permissions.retain_preview(snapshot)
|
||||
assert result["evidence"] == [] and result["evidence_count"] == 1000
|
||||
assert len(json.dumps(result)) < 5000
|
||||
assert env.permissions.receipt(user_id=USER) is None
|
||||
restarted = ModelContentPermissionStore(env.database)
|
||||
receipt = restarted.grant_preview(
|
||||
result["preview_id"], user_id=USER, expected_revision=0, user_confirmed=True
|
||||
)
|
||||
assert receipt.snapshot == snapshot
|
||||
assert env.check(receipt) == receipt
|
||||
with pytest.raises(ModelContentPermissionError, match="permission_changed"):
|
||||
restarted.grant_preview(
|
||||
result["preview_id"], user_id=USER, expected_revision=0, user_confirmed=True
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"change", ["edit", "revoke-source", "grant-race", "expiry", "wrong-owner", "confirmation"]
|
||||
)
|
||||
def test_compact_preview_does_not_bypass_live_checks(env, change):
|
||||
result = env.permissions.retain_preview(env.snapshot(future=True))
|
||||
if change == "edit":
|
||||
with sqlite3.connect(env.database) as db:
|
||||
db.execute("UPDATE evidence SET verbatim='changed raw original'")
|
||||
elif change == "revoke-source":
|
||||
env.sources.revoke(user_id=USER, source_id=env.record.source_id)
|
||||
elif change == "grant-race":
|
||||
env.grant()
|
||||
elif change == "expiry":
|
||||
with sqlite3.connect(env.database) as db:
|
||||
db.execute("UPDATE model_content_previews SET expires_at='2000-01-01T00:00:00+00:00'")
|
||||
with pytest.raises(ModelContentPermissionError):
|
||||
env.permissions.grant_preview(
|
||||
result["preview_id"],
|
||||
user_id="other" if change == "wrong-owner" else USER,
|
||||
expected_revision=0,
|
||||
user_confirmed=change != "confirmation",
|
||||
)
|
||||
|
||||
|
||||
def test_new_originals_keep_explicit_future_scope_but_not_exact_scope(env):
|
||||
result = env.permissions.retain_preview(env.snapshot(future=True))
|
||||
later = env.evidence.put(evidence_value("new-arrival")).record
|
||||
receipt = env.permissions.grant_preview(
|
||||
result["preview_id"], user_id=USER, expected_revision=0, user_confirmed=True
|
||||
)
|
||||
assert len(receipt.snapshot.evidence) == 1
|
||||
assert env.check(receipt, records=(later,)) == receipt
|
||||
|
||||
|
||||
def test_compact_receipt_is_metadata_only_and_version_bound(env):
|
||||
result = env.permissions.retain_preview(env.snapshot())
|
||||
receipt = env.permissions.grant_preview(
|
||||
result["preview_id"], user_id=USER, expected_revision=0, user_confirmed=True
|
||||
)
|
||||
compact = env.permissions.compact_receipt(receipt)
|
||||
assert compact["revision"] == receipt.revision
|
||||
assert compact["snapshot"]["snapshot_hash"] == result["snapshot_hash"]
|
||||
assert compact["snapshot"]["source_counts"] == {env.record.source_id: 1}
|
||||
assert env.record.verbatim not in json.dumps(compact)
|
||||
assert compact["snapshot"]["evidence"] == []
|
||||
|
||||
|
||||
@pytest.mark.parametrize("future", [False, True])
|
||||
def test_preview_cannot_rebase_over_a_grant_or_revoke_between_reads(env, future):
|
||||
first = env.grant(future=future)
|
||||
preview = env.snapshot(future=future)
|
||||
env.permissions.revoke(user_id=USER, expected_revision=first.revision, user_confirmed=True)
|
||||
with pytest.raises(ModelContentPermissionError, match="permission_changed"):
|
||||
env.permissions.retain_preview(preview)
|
||||
assert env.permissions.receipt(user_id=USER).revoked_at is not None
|
||||
|
||||
|
||||
def test_shared_migration_declares_the_retained_preview_table() -> None:
|
||||
"""Retained previews are product state; migration 28 must recreate them.
|
||||
|
||||
This is a red-state guard for the A1-04 schema extraction. The table exists
|
||||
in the historical f6d624a schema and is still read/written by this module,
|
||||
so migration 28's generator must declare it for both SQLite and MySQL.
|
||||
"""
|
||||
from one_memory.memory_server import migrations
|
||||
|
||||
sqlite_schema = migrations.model_content_permissions_ddl("sqlite")
|
||||
mysql_schema = migrations.model_content_permissions_ddl("mysql")
|
||||
assert "model_content_previews" in sqlite_schema
|
||||
assert "model_content_previews" in mysql_schema
|
||||
@@ -12,15 +12,12 @@ import asyncio
|
||||
import gc
|
||||
import sqlite3
|
||||
from contextlib import contextmanager
|
||||
from types import SimpleNamespace
|
||||
|
||||
from httpx import ASGITransport, AsyncClient
|
||||
from pydantic import SecretStr
|
||||
|
||||
from one_memory.api import create_app
|
||||
from one_memory.config import Settings
|
||||
from one_memory.database_access import close_main_databases, main_database_stats
|
||||
from one_memory.hy_permission_context import _NativeProofs
|
||||
from one_memory.imports.sources.registry import build_source_catalog
|
||||
|
||||
|
||||
@@ -81,24 +78,3 @@ def test_polled_endpoints_close_every_connection_they_open(tmp_path, monkeypatch
|
||||
assert left_open == 0, f"{left_open} connections left for the collector"
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_hy_permission_proofs_close_their_connections(tmp_path):
|
||||
database = tmp_path / "one-memory.sqlite3"
|
||||
with sqlite3.connect(database) as db:
|
||||
db.execute("CREATE TABLE governed_memories (memory_id TEXT, revision INT, status TEXT)")
|
||||
db.close()
|
||||
proofs = _NativeProofs(database)
|
||||
lease = SimpleNamespace(records=())
|
||||
# The first read opens the pooled reader; it is meant to stay open.
|
||||
proofs.guard(proofs.load(lease))()
|
||||
|
||||
with _without_cyclic_collection():
|
||||
before = _open_connections()
|
||||
for _ in range(5):
|
||||
check = proofs.guard(proofs.load(lease))
|
||||
check()
|
||||
left_open = _open_connections() - before
|
||||
assert left_open == 0, f"{left_open} connections left for the collector"
|
||||
close_main_databases()
|
||||
assert main_database_stats()["open_connections"] == 0
|
||||
@@ -1,341 +0,0 @@
|
||||
"""Independent review of completed-input reuse; real stores and controlled HY only."""
|
||||
|
||||
import asyncio
|
||||
import sqlite3
|
||||
from contextlib import closing
|
||||
from uuid import uuid4
|
||||
|
||||
import pytest
|
||||
|
||||
from one_memory import continuous_policy
|
||||
from one_memory.continuous_permission_bridge import ContinuousPermissionReceipt
|
||||
from one_memory.hy_memory_bridge import HyMemoryBridgeError
|
||||
from one_memory.imports.models import EvidenceInput
|
||||
from one_memory.imports.sources.registry import RootCandidateKind
|
||||
from one_memory.memory_processing import MEMORY_INGEST_POLICY_VERSION
|
||||
from one_memory.memory_tasks import MemoryTaskStatus
|
||||
from tests.test_continuous_processing_integration import IntegrationCore
|
||||
|
||||
|
||||
def transition(core, action):
|
||||
with closing(sqlite3.connect(core.database)) as db, db:
|
||||
db.execute("BEGIN IMMEDIATE")
|
||||
policy = continuous_policy.current(db)
|
||||
return continuous_policy.transition(
|
||||
db,
|
||||
action=action,
|
||||
expected_revision=policy.revision,
|
||||
operation_id=str(uuid4()),
|
||||
user_confirmed=True,
|
||||
)
|
||||
|
||||
|
||||
def bindings(core):
|
||||
with closing(sqlite3.connect(core.database)) as db:
|
||||
return db.execute(
|
||||
"SELECT task_id,policy_revision,policy_epoch,receipt_json "
|
||||
"FROM memory_task_continuous_permissions ORDER BY task_id"
|
||||
).fetchall()
|
||||
|
||||
|
||||
async def completed(core):
|
||||
core.grant_source()
|
||||
await core.save_and_probe()
|
||||
core.kernel_enable("none")
|
||||
record = core.put_complete("review-completed", conversation="review")
|
||||
tasks = await core.run_prepared(record)
|
||||
assert len(tasks) == 1 and tasks[0].status == MemoryTaskStatus.COMPLETED
|
||||
assert len(core.bridge.requests) == 1
|
||||
return record, tasks[0]
|
||||
|
||||
|
||||
def frozen(core, records):
|
||||
guard = core.app.state.memory_processing_supervisor._permission_guard
|
||||
receipt = guard.freeze(guard.current("local"), records)
|
||||
assert isinstance(receipt, ContinuousPermissionReceipt)
|
||||
return guard, receipt
|
||||
|
||||
|
||||
def lookup(guard, receipt, records, seed):
|
||||
return guard.completed_input_task(
|
||||
receipt,
|
||||
records,
|
||||
input_ref=seed.evidence_id,
|
||||
request_prefix=f"conversation:{MEMORY_INGEST_POLICY_VERSION}:{seed.source_id}:",
|
||||
)
|
||||
|
||||
|
||||
def test_resume_reuses_completed_without_rebinding_historical_receipt(tmp_path, monkeypatch):
|
||||
core = IntegrationCore(tmp_path, monkeypatch)
|
||||
|
||||
async def scenario():
|
||||
async with core.running():
|
||||
record, task = await completed(core)
|
||||
before = bindings(core)
|
||||
transition(core, "pause")
|
||||
resumed = transition(core, "resume")
|
||||
supervisor = core.app.state.memory_processing_supervisor
|
||||
profile = core.app.state.model_profile_store.active(core.secrets)
|
||||
|
||||
def cannot_rebind(*args, **kwargs):
|
||||
pytest.fail("completed history was rebound to a new execution receipt")
|
||||
|
||||
with monkeypatch.context() as patch:
|
||||
patch.setattr(supervisor._permission_guard, "bind", cannot_rebind)
|
||||
reused = supervisor._enqueue_chunk(core.turn_records(record), profile)
|
||||
assert reused.task_id == task.task_id
|
||||
assert reused.status == MemoryTaskStatus.COMPLETED
|
||||
assert resumed.revision == 3 and before[0][1] == 1
|
||||
assert bindings(core) == before
|
||||
assert supervisor._classify([reused]) == []
|
||||
assert len(core.bridge.requests) == 1
|
||||
assert len(core.app.state.memory_task_store.list()) == 1
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
@pytest.mark.parametrize("change", ["chunk", "event-revision", "profile"])
|
||||
def test_changed_input_identity_is_not_reused_as_completed(tmp_path, monkeypatch, change):
|
||||
core = IntegrationCore(tmp_path, monkeypatch)
|
||||
|
||||
async def scenario():
|
||||
async with core.running():
|
||||
record, task = await completed(core)
|
||||
before = bindings(core)
|
||||
records = core.turn_records(record)
|
||||
if change == "chunk":
|
||||
other = core.put("review-added", conversation="review")
|
||||
# Same input_ref deliberately: input_ref alone cannot prove chunk identity.
|
||||
records = (other, *records)
|
||||
elif change == "event-revision":
|
||||
changed = EvidenceInput.model_validate(
|
||||
{
|
||||
**record.model_dump(include=set(EvidenceInput.model_fields)),
|
||||
"verbatim": record.verbatim + " changed",
|
||||
}
|
||||
)
|
||||
record = core.app.state.evidence_store.put(changed).record
|
||||
assert record.revision > 1
|
||||
records = (record, *records[1:])
|
||||
elif change == "profile":
|
||||
transition(core, "revoke")
|
||||
# Persist a new profile through the real API; probe remains controlled.
|
||||
saved = await core.client.post(
|
||||
"/api/v1/model-profiles",
|
||||
json={
|
||||
"name": "review-second-profile",
|
||||
"provider_kind": "openai_compatible",
|
||||
"base_url": "https://integration.invalid/v1",
|
||||
"model": "integration-model",
|
||||
"api_key": "synthetic-key-not-a-credential",
|
||||
"embedding_model": "integration-vector",
|
||||
"embedding_provider_kind": "openai_compatible",
|
||||
"embedding_base_url": "https://integration.invalid/v1",
|
||||
},
|
||||
)
|
||||
assert saved.status_code == 200
|
||||
result = await core.client.post(
|
||||
f"/api/v1/model-profiles/{saved.json()['profile_id']}/probe"
|
||||
)
|
||||
assert result.status_code == 200
|
||||
core.profile = next(
|
||||
p
|
||||
for p in (await core.client.get("/api/v1/model-profiles")).json()
|
||||
if p["profile_id"] == saved.json()["profile_id"]
|
||||
)
|
||||
current = core.policy_current()
|
||||
core.app.state.model_policy_activation.enable(
|
||||
operation_id=str(uuid4()),
|
||||
profile_id=core.profile["profile_id"],
|
||||
profile_revision=core.profile["revision"],
|
||||
expected_policy_revision=current.revision,
|
||||
expected_legacy_revision=current.legacy_revision,
|
||||
backlog_mode="include",
|
||||
user_confirmed=True,
|
||||
)
|
||||
guard, receipt = frozen(core, records)
|
||||
assert lookup(guard, receipt, records, record) is None
|
||||
assert bindings(core) == before
|
||||
assert (
|
||||
core.app.state.memory_task_store.require(task.task_id).status
|
||||
== MemoryTaskStatus.COMPLETED
|
||||
)
|
||||
assert len(core.bridge.requests) == 1
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_changed_source_root_is_not_reused_as_completed(tmp_path, monkeypatch):
|
||||
core = IntegrationCore(tmp_path, monkeypatch)
|
||||
|
||||
async def scenario():
|
||||
async with core.running():
|
||||
record, task = await completed(core)
|
||||
before = bindings(core)
|
||||
grants = core.app.state.source_grant_store
|
||||
grants.revoke(user_id="local", source_id="cursor")
|
||||
replacement = core.home / "cursor-replacement"
|
||||
replacement.mkdir()
|
||||
grants.grant(
|
||||
user_id="local",
|
||||
source_id="cursor",
|
||||
root=replacement,
|
||||
candidate_kind=RootCandidateKind.DEFAULT,
|
||||
)
|
||||
records = core.turn_records(record)
|
||||
guard, receipt = frozen(core, records)
|
||||
|
||||
assert lookup(guard, receipt, records, record) is None
|
||||
assert bindings(core) == before
|
||||
assert (
|
||||
core.app.state.memory_task_store.require(task.task_id).status
|
||||
== MemoryTaskStatus.COMPLETED
|
||||
)
|
||||
assert len(core.bridge.requests) == 1
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_reenabled_policy_reuses_exact_completed_input_without_duplicate_hy_call(
|
||||
tmp_path, monkeypatch
|
||||
):
|
||||
core = IntegrationCore(tmp_path, monkeypatch)
|
||||
|
||||
async def scenario():
|
||||
async with core.running():
|
||||
record, task = await completed(core)
|
||||
before = bindings(core)
|
||||
transition(core, "revoke")
|
||||
prior = core.policy_current()
|
||||
core.app.state.model_policy_activation.enable(
|
||||
operation_id=str(uuid4()),
|
||||
profile_id=core.profile["profile_id"],
|
||||
profile_revision=core.profile["revision"],
|
||||
expected_policy_revision=prior.revision,
|
||||
expected_legacy_revision=prior.legacy_revision,
|
||||
backlog_mode="include",
|
||||
user_confirmed=True,
|
||||
)
|
||||
assert core.policy_current().epoch != before[0][2]
|
||||
|
||||
supervisor = core.app.state.memory_processing_supervisor
|
||||
profile = core.app.state.model_profile_store.active(core.secrets)
|
||||
reused = supervisor._enqueue_chunk(core.turn_records(record), profile)
|
||||
|
||||
assert reused.task_id == task.task_id
|
||||
assert reused.status == MemoryTaskStatus.COMPLETED
|
||||
assert bindings(core) == before
|
||||
assert len(core.bridge.requests) == 1
|
||||
assert len(core.app.state.memory_task_store.list()) == 1
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
@pytest.mark.parametrize("when", ["before", "after-first-check"])
|
||||
@pytest.mark.parametrize("authority", ["source", "policy"])
|
||||
def test_current_source_permission_is_rechecked_around_completed_lookup(
|
||||
tmp_path, monkeypatch, when, authority
|
||||
):
|
||||
core = IntegrationCore(tmp_path, monkeypatch)
|
||||
|
||||
async def scenario():
|
||||
async with core.running():
|
||||
record, _ = await completed(core)
|
||||
before = bindings(core)
|
||||
records = core.turn_records(record)
|
||||
guard, receipt = frozen(core, records)
|
||||
|
||||
def revoke():
|
||||
if authority == "source":
|
||||
core.app.state.source_grant_store.revoke(user_id="local", source_id="cursor")
|
||||
else:
|
||||
transition(core, "pause")
|
||||
|
||||
calls = []
|
||||
original = guard.check
|
||||
|
||||
def checked(*args, **kwargs):
|
||||
value = original(*args, **kwargs)
|
||||
calls.append(True)
|
||||
if len(calls) == 1:
|
||||
revoke()
|
||||
return value
|
||||
|
||||
if when == "before":
|
||||
revoke()
|
||||
else:
|
||||
monkeypatch.setattr(guard, "check", checked)
|
||||
with pytest.raises(HyMemoryBridgeError):
|
||||
lookup(guard, receipt, records, record)
|
||||
assert bindings(core) == before
|
||||
assert len(core.bridge.requests) == 1
|
||||
if when == "after-first-check":
|
||||
assert calls == [True]
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_corrupt_historical_binding_is_not_accepted_as_completion(tmp_path, monkeypatch):
|
||||
core = IntegrationCore(tmp_path, monkeypatch)
|
||||
|
||||
async def scenario():
|
||||
async with core.running():
|
||||
record, task = await completed(core)
|
||||
with sqlite3.connect(core.database) as db:
|
||||
db.execute(
|
||||
"UPDATE memory_task_continuous_permissions "
|
||||
"SET policy_revision=99 WHERE task_id=?",
|
||||
(task.task_id,),
|
||||
)
|
||||
records = core.turn_records(record)
|
||||
guard, receipt = frozen(core, records)
|
||||
with pytest.raises(HyMemoryBridgeError, match="continuous_binding_invalid"):
|
||||
lookup(guard, receipt, records, record)
|
||||
assert len(core.bridge.requests) == 1
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_sdk_admitted_unknown_is_not_reused_or_resent_on_resume(tmp_path, monkeypatch):
|
||||
core = IntegrationCore(tmp_path, monkeypatch)
|
||||
|
||||
async def scenario():
|
||||
async with core.running():
|
||||
core.grant_source()
|
||||
await core.save_and_probe()
|
||||
core.kernel_enable()
|
||||
record = core.put_complete("review-unknown")
|
||||
supervisor = core.app.state.memory_processing_supervisor
|
||||
tasks = supervisor.prepare_conversation(record)
|
||||
entered = asyncio.Event()
|
||||
|
||||
async def interrupted(request):
|
||||
core.bridge.requests.append(request)
|
||||
entered.set()
|
||||
await asyncio.Event().wait()
|
||||
|
||||
monkeypatch.setattr(core.bridge, "ingest", interrupted)
|
||||
running = asyncio.create_task(supervisor._create_runner().run(tasks[0].task_id))
|
||||
try:
|
||||
await asyncio.wait_for(entered.wait(), 3)
|
||||
finally:
|
||||
running.cancel()
|
||||
with pytest.raises(asyncio.CancelledError):
|
||||
await running
|
||||
# Actual restart recovery marks a previously admitted RUNNING task unknown.
|
||||
core.app.state.memory_task_store.recover_running()
|
||||
before = bindings(core)
|
||||
transition(core, "pause")
|
||||
transition(core, "resume")
|
||||
with pytest.raises(HyMemoryBridgeError, match="outcome_unknown"):
|
||||
supervisor._enqueue_chunk(
|
||||
core.turn_records(record),
|
||||
core.app.state.model_profile_store.active(core.secrets),
|
||||
)
|
||||
assert bindings(core) == before
|
||||
assert len(core.bridge.requests) == 1
|
||||
task = core.app.state.memory_task_store.require(tasks[0].task_id)
|
||||
assert task.last_error_code == "model_content_outcome_unknown" and task.attempts == 1
|
||||
assert len(core.app.state.memory_task_store.list()) == 1
|
||||
|
||||
asyncio.run(scenario())
|
||||
@@ -1,190 +0,0 @@
|
||||
import asyncio
|
||||
|
||||
import pytest
|
||||
|
||||
from one_memory.hy_memory_bridge import HyMemoryBridgeError
|
||||
from one_memory.processing_coordinator import ProcessingCoordinator
|
||||
from one_memory.processing_requests import ProcessingRequestStore
|
||||
from tests.test_continuous_processing_bridge import enable, put_after_policy, transition
|
||||
from tests.test_model_processing_permissions import h as h # noqa: F401
|
||||
|
||||
|
||||
def coordinator(h, monkeypatch):
|
||||
supervisor = h.supervisor()
|
||||
scheduled = []
|
||||
|
||||
async def schedule(work):
|
||||
scheduled.extend(work)
|
||||
|
||||
monkeypatch.setattr(supervisor, "_schedule_tasks_off_thread", schedule)
|
||||
return ProcessingCoordinator(ProcessingRequestStore(h.database), supervisor), scheduled
|
||||
|
||||
|
||||
def test_restart_replays_lost_callback_into_same_durable_task_queue(h, monkeypatch):
|
||||
policy = enable(h)
|
||||
fresh = put_after_policy(h, "committed-before-crash")
|
||||
owner, scheduled = coordinator(h, monkeypatch)
|
||||
|
||||
async def run():
|
||||
try:
|
||||
await owner._continuous_tick()
|
||||
assert scheduled
|
||||
task_ids = {task.task_id for task in h.tasks.list()}
|
||||
assert len(task_ids) == 1
|
||||
assert {
|
||||
ref["evidence_id"]
|
||||
for task in h.tasks.list()
|
||||
for ref in h.tasks.inputs(task.task_id)
|
||||
} == {fresh.evidence_id}
|
||||
assert owner.continuous_status()["observed_seq"] > policy.after_seq
|
||||
await owner._continuous_tick()
|
||||
assert {task.task_id for task in h.tasks.list()} == task_ids
|
||||
restarted, _ = coordinator(h, monkeypatch)
|
||||
try:
|
||||
await restarted._continuous_tick()
|
||||
assert {task.task_id for task in h.tasks.list()} == task_ids
|
||||
finally:
|
||||
await restarted.shutdown()
|
||||
await restarted.supervisor.shutdown()
|
||||
finally:
|
||||
await owner.shutdown()
|
||||
await owner.supervisor.shutdown()
|
||||
|
||||
asyncio.run(run())
|
||||
|
||||
|
||||
def test_new_journal_event_and_source_regrant_wake_replay_without_legacy_request(h, monkeypatch):
|
||||
from one_memory.imports.sources.registry import RootCandidateKind
|
||||
|
||||
enable(h)
|
||||
owner, scheduled = coordinator(h, monkeypatch)
|
||||
|
||||
async def run():
|
||||
try:
|
||||
await owner._continuous_tick()
|
||||
assert not scheduled
|
||||
fresh = put_after_policy(h, "new")
|
||||
h.sources.revoke(user_id="local", source_id="cursor")
|
||||
await owner._continuous_tick()
|
||||
assert not scheduled
|
||||
h.sources.grant(
|
||||
user_id="local",
|
||||
source_id="cursor",
|
||||
root=h.root / "synthetic-source",
|
||||
candidate_kind=RootCandidateKind.DEFAULT,
|
||||
)
|
||||
await owner._continuous_tick()
|
||||
assert scheduled
|
||||
assert h.tasks.inputs(scheduled[-1].task_id)[0]["evidence_id"] == fresh.evidence_id
|
||||
with h.evidence._connect() as db:
|
||||
assert db.execute("SELECT count(*) FROM processing_requests").fetchone()[0] == 0
|
||||
finally:
|
||||
await owner.shutdown()
|
||||
await owner.supervisor.shutdown()
|
||||
|
||||
asyncio.run(run())
|
||||
|
||||
|
||||
def test_activation_hook_and_startup_recovery_start_single_monitor_then_stop(h, monkeypatch):
|
||||
policy = enable(h)
|
||||
put_after_policy(h, "new")
|
||||
owner, scheduled = coordinator(h, monkeypatch)
|
||||
|
||||
async def run():
|
||||
try:
|
||||
await owner.recover()
|
||||
worker = owner._continuous_worker
|
||||
assert worker is not None
|
||||
await owner.resume_continuous(expected_policy_revision=policy.revision)
|
||||
assert owner._continuous_worker is worker
|
||||
for _ in range(100):
|
||||
if scheduled:
|
||||
break
|
||||
await asyncio.sleep(0.01)
|
||||
assert scheduled
|
||||
finally:
|
||||
await owner.shutdown()
|
||||
await owner.supervisor.shutdown()
|
||||
assert worker.done()
|
||||
with pytest.raises(HyMemoryBridgeError):
|
||||
await owner.resume_continuous(expected_policy_revision=policy.revision)
|
||||
|
||||
asyncio.run(run())
|
||||
|
||||
|
||||
def test_pause_refuses_recovery_without_fallback_and_resume_replays(h, monkeypatch):
|
||||
h.grant()
|
||||
enable(h)
|
||||
fresh = put_after_policy(h, "new")
|
||||
owner, scheduled = coordinator(h, monkeypatch)
|
||||
transition(h, "pause")
|
||||
|
||||
async def run():
|
||||
try:
|
||||
await owner.recover()
|
||||
assert not scheduled and owner._continuous_worker is None
|
||||
with pytest.raises(HyMemoryBridgeError):
|
||||
await owner.resume_continuous()
|
||||
transition(h, "resume")
|
||||
await owner._continuous_tick()
|
||||
assert scheduled
|
||||
assert h.tasks.inputs(scheduled[-1].task_id)[0]["evidence_id"] == fresh.evidence_id
|
||||
finally:
|
||||
await owner.shutdown()
|
||||
await owner.supervisor.shutdown()
|
||||
|
||||
asyncio.run(run())
|
||||
|
||||
|
||||
def test_failed_preparation_does_not_advance_observed_watermark(h, monkeypatch):
|
||||
enable(h)
|
||||
put_after_policy(h, "new")
|
||||
owner, _ = coordinator(h, monkeypatch)
|
||||
original = owner.supervisor.resume_continuous
|
||||
|
||||
async def fail(**kwargs):
|
||||
raise RuntimeError("synthetic failed preparation")
|
||||
|
||||
monkeypatch.setattr(owner.supervisor, "resume_continuous", fail)
|
||||
|
||||
async def run():
|
||||
try:
|
||||
with pytest.raises(RuntimeError):
|
||||
await owner._continuous_tick()
|
||||
assert owner.continuous_status()["observed_seq"] is None
|
||||
monkeypatch.setattr(owner.supervisor, "resume_continuous", original)
|
||||
await owner._continuous_tick()
|
||||
assert owner.continuous_status()["observed_seq"] is not None
|
||||
finally:
|
||||
await owner.shutdown()
|
||||
await owner.supervisor.shutdown()
|
||||
|
||||
asyncio.run(run())
|
||||
|
||||
|
||||
def test_unchanged_continuous_identity_is_rescanned_every_sixty_seconds(h, monkeypatch):
|
||||
enable(h)
|
||||
owner, _scheduled = coordinator(h, monkeypatch)
|
||||
calls = []
|
||||
now = 100.0
|
||||
|
||||
async def resume(*, expected_policy_revision=None):
|
||||
calls.append(expected_policy_revision)
|
||||
|
||||
monkeypatch.setattr(owner.supervisor, "resume_continuous", resume)
|
||||
owner._clock = lambda: now
|
||||
|
||||
async def run():
|
||||
nonlocal now
|
||||
try:
|
||||
await owner._continuous_tick()
|
||||
now = 159.9
|
||||
await owner._continuous_tick()
|
||||
now = 160.0
|
||||
await owner._continuous_tick()
|
||||
assert calls == [1, 1]
|
||||
finally:
|
||||
await owner.shutdown()
|
||||
await owner.supervisor.shutdown()
|
||||
|
||||
asyncio.run(run())
|
||||
@@ -1,176 +0,0 @@
|
||||
import asyncio
|
||||
|
||||
import pytest
|
||||
|
||||
from one_memory.hy_memory_bridge import HyMemoryBridgeError
|
||||
from one_memory.memory_tasks import MemoryTaskOwnershipError, MemoryTaskStatus
|
||||
from tests.test_continuous_processing_bridge import enable, put_after_policy
|
||||
from tests.test_model_processing_permissions import h as h # noqa: F401
|
||||
|
||||
|
||||
def test_crashed_continuous_send_is_not_replayed_but_new_event_progresses(h):
|
||||
enable(h)
|
||||
fresh = put_after_policy(h, "sent-before-crash")
|
||||
supervisor = h.supervisor()
|
||||
task = supervisor.prepare_conversation(fresh)[0]
|
||||
claimed, _ = h.tasks.claim(task.task_id)
|
||||
runner = supervisor._create_runner()
|
||||
lease, request, _ = runner._prepare_permission_lease(claimed)
|
||||
h.bridge.fail = False
|
||||
asyncio.run(h.bridge.ingest(request)) # Controlled send returned, process lost its receipt.
|
||||
assert len(h.bridge.calls) == 1
|
||||
assert h.tasks.recover_running() == 1
|
||||
recovered = h.tasks.require(task.task_id)
|
||||
assert recovered.status == MemoryTaskStatus.UNKNOWN
|
||||
assert recovered.last_error_code == "model_content_outcome_unknown"
|
||||
with pytest.raises(MemoryTaskOwnershipError):
|
||||
h.tasks.claim(task.task_id)
|
||||
assert supervisor.prepare_resume() == []
|
||||
assert supervisor.prepare_backfill() == []
|
||||
# The continuous-policy filter now excludes an already possibly-sent
|
||||
# Evidence before task preparation. Either explicit unknown or outside
|
||||
# policy proves it cannot be blindly replayed.
|
||||
with pytest.raises(HyMemoryBridgeError, match="outcome_unknown|evidence_outside_policy"):
|
||||
asyncio.run(supervisor.process_records((fresh,)))
|
||||
# A new message in the same conversation must not resend the unknown original as context.
|
||||
newest = put_after_policy(h, "new-after-restart")
|
||||
tasks = supervisor.prepare_conversation(newest)
|
||||
assert len(tasks) == 1 and tasks[0].task_id != task.task_id
|
||||
|
||||
async def run():
|
||||
try:
|
||||
result = await supervisor._create_runner().run(tasks[0].task_id)
|
||||
assert result.status == MemoryTaskStatus.COMPLETED
|
||||
sent = [
|
||||
request
|
||||
for kind, request in h.bridge.calls
|
||||
if kind in {"ingest", "ingest_extracted"}
|
||||
]
|
||||
assert len(sent) == 2
|
||||
assert sent[-1].evidence_ids == (newest.evidence_id,)
|
||||
assert all(r.evidence_id != fresh.evidence_id for r in h.leases[-1].records)
|
||||
assert supervisor.prepare_backfill() == []
|
||||
finally:
|
||||
await supervisor.shutdown()
|
||||
|
||||
asyncio.run(run())
|
||||
assert h.tasks.require(task.task_id).last_error_code == "model_content_outcome_unknown"
|
||||
|
||||
|
||||
def test_legacy_running_recovery_remains_pending_and_no_continuous_table(h):
|
||||
legacy = h.legacy_task()
|
||||
h.tasks.claim(legacy.task_id)
|
||||
assert h.tasks.recover_running() == 1
|
||||
restored = h.tasks.require(legacy.task_id)
|
||||
assert restored.status == MemoryTaskStatus.PENDING
|
||||
assert restored.last_error_code == "worker_restarted"
|
||||
|
||||
|
||||
def test_a_timed_out_non_continuous_task_also_becomes_unknown(h):
|
||||
"""The timeout reason alone is enough to make a task possibly-sent.
|
||||
|
||||
The continuous-permission record is not the only way HY can have been
|
||||
reached: `MemoryTaskRunner.run` wraps the HY call in `asyncio.wait_for`, so
|
||||
a wall-clock timeout can fire after the request was in flight. A
|
||||
non-continuous task must therefore also become sticky `unknown`, or the
|
||||
scheduler re-selects it and sends a second copy (HY has no idempotency key).
|
||||
"""
|
||||
# A legacy (non-continuous) task: no continuous permission record exists.
|
||||
task = h.legacy_task()
|
||||
_claimed, runner_token = h.tasks.claim(task.task_id)
|
||||
h.tasks.record_hy_invocation(task.task_id, runner_token=runner_token)
|
||||
assert h.tasks.recover_one_running(task.task_id, reason="hy_memory_timeout") is True
|
||||
recovered = h.tasks.require(task.task_id)
|
||||
assert recovered.status == MemoryTaskStatus.UNKNOWN
|
||||
assert recovered.last_error_code == "model_content_outcome_unknown"
|
||||
with pytest.raises(MemoryTaskOwnershipError):
|
||||
h.tasks.claim(task.task_id)
|
||||
|
||||
|
||||
def test_claim_without_proven_send_is_still_conservatively_unknown(h):
|
||||
enable(h)
|
||||
fresh = put_after_policy(h, "claim-only")
|
||||
task = h.supervisor().prepare_conversation(fresh)[0]
|
||||
h.tasks.claim(task.task_id)
|
||||
h.tasks.recover_running()
|
||||
recovered = h.tasks.require(task.task_id)
|
||||
assert recovered.status == MemoryTaskStatus.UNKNOWN
|
||||
assert recovered.last_error_code == "model_content_outcome_unknown"
|
||||
assert h.bridge.calls == []
|
||||
|
||||
|
||||
def test_a_timed_out_continuous_task_becomes_unknown_not_pending(h):
|
||||
"""The one-shot recovery must be continuous-aware, like process-start recovery.
|
||||
|
||||
`recover_one_running` is the path a timeout takes. A continuous task may
|
||||
already have reached HY, so returning it to `pending` would let it be
|
||||
re-sent and duplicate the memory; it must become sticky `unknown` exactly
|
||||
as `recover_running` marks it at process start.
|
||||
"""
|
||||
enable(h)
|
||||
fresh = put_after_policy(h, "timed-out-continuous-send")
|
||||
task = h.supervisor().prepare_conversation(fresh)[0]
|
||||
_claimed, runner_token = h.tasks.claim(task.task_id)
|
||||
h.tasks.record_hy_invocation(task.task_id, runner_token=runner_token)
|
||||
|
||||
assert h.tasks.recover_one_running(task.task_id, reason="hy_memory_timeout") is True
|
||||
|
||||
recovered = h.tasks.require(task.task_id)
|
||||
assert recovered.status == MemoryTaskStatus.UNKNOWN
|
||||
assert recovered.last_error_code == "model_content_outcome_unknown"
|
||||
with pytest.raises(MemoryTaskOwnershipError):
|
||||
h.tasks.claim(task.task_id)
|
||||
|
||||
|
||||
def test_restart_preserves_the_original_unknown_ordering_time(h):
|
||||
"""Recovery must not collapse old in-flight work into the restart instant.
|
||||
|
||||
`unknown_jobs()` pages in `(updated_at, job_id)` order. If a process
|
||||
restart rewrote every recovered task's `updated_at`, a crash with many
|
||||
in-flight tasks would erase their original request order and make the
|
||||
inquiry drain start from an arbitrary same-timestamp batch. The recovery
|
||||
transition changes status/error only; the timestamp remains the durable
|
||||
moment the row last changed before the process died.
|
||||
"""
|
||||
enable(h)
|
||||
fresh = put_after_policy(h, "restart-order")
|
||||
task = h.supervisor().prepare_conversation(fresh)[0]
|
||||
h.tasks.claim(task.task_id)
|
||||
original = h.tasks.require(task.task_id).updated_at
|
||||
|
||||
assert h.tasks.recover_running() == 1
|
||||
|
||||
recovered = h.tasks.require(task.task_id)
|
||||
assert recovered.status is MemoryTaskStatus.UNKNOWN
|
||||
assert recovered.updated_at == original
|
||||
|
||||
|
||||
def test_an_unsettled_continuous_send_becomes_unknown_not_failed(h):
|
||||
"""`failed` is retryable, so an abandoned continuous send must be `unknown`.
|
||||
|
||||
`fail_unsettled` runs when a runner raised without settling. That exception
|
||||
may have arrived after HY was already called; `failed` would be re-claimed
|
||||
and re-sent, duplicating the memory. Only a task that never reached
|
||||
`running` (still pending) is provably unsent and may be `failed`.
|
||||
"""
|
||||
enable(h)
|
||||
fresh = put_after_policy(h, "unsettled-continuous-send")
|
||||
task = h.supervisor().prepare_conversation(fresh)[0]
|
||||
h.tasks.claim(task.task_id)
|
||||
|
||||
assert h.tasks.fail_unsettled(task.task_id, error_code="memory_task_dispatch_failed") is True
|
||||
|
||||
settled = h.tasks.require(task.task_id)
|
||||
assert settled.status == MemoryTaskStatus.UNKNOWN
|
||||
with pytest.raises(MemoryTaskOwnershipError):
|
||||
h.tasks.claim(task.task_id)
|
||||
|
||||
|
||||
def test_an_unsettled_non_continuous_task_is_still_failed(h):
|
||||
"""The continuous rule must not change the ordinary path."""
|
||||
legacy = h.legacy_task()
|
||||
h.tasks.claim(legacy.task_id)
|
||||
|
||||
assert h.tasks.fail_unsettled(legacy.task_id, error_code="memory_task_dispatch_failed") is True
|
||||
|
||||
assert h.tasks.require(legacy.task_id).status == MemoryTaskStatus.FAILED
|
||||
@@ -1,336 +0,0 @@
|
||||
import sqlite3
|
||||
from contextlib import contextmanager
|
||||
from uuid import uuid4
|
||||
|
||||
import pytest
|
||||
|
||||
from one_memory import continuous_policy as policy
|
||||
from one_memory import ingest_journal as journal
|
||||
from one_memory.imports.sources.registry import RootCandidateKind
|
||||
from one_memory.model_profiles import ModelProfilePatch
|
||||
from tests.test_model_content_permissions import SOURCE, evidence_value
|
||||
from tests.test_model_content_permissions import env as env # noqa: F401
|
||||
|
||||
|
||||
@contextmanager
|
||||
def transaction(environment):
|
||||
db = sqlite3.connect(environment.database)
|
||||
db.row_factory = sqlite3.Row
|
||||
db.execute("BEGIN IMMEDIATE")
|
||||
try:
|
||||
journal.initialize(db)
|
||||
policy.initialize(db)
|
||||
yield db
|
||||
db.commit()
|
||||
except BaseException:
|
||||
db.rollback()
|
||||
raise
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
|
||||
def enable(db, environment, **overrides):
|
||||
values = dict(
|
||||
profile=environment.profile,
|
||||
expected_revision=0,
|
||||
expected_legacy_revision=0,
|
||||
operation_id=str(uuid4()),
|
||||
backlog_mode="none",
|
||||
user_confirmed=True,
|
||||
)
|
||||
values.update(overrides)
|
||||
return policy.enable(db, **values)
|
||||
|
||||
|
||||
def test_explicit_policy_never_upgrades_legacy_or_sends_or_creates_tasks(env):
|
||||
with transaction(env) as db:
|
||||
assert policy.current(db) is None
|
||||
receipt = enable(db, env)
|
||||
assert policy.current(db) == receipt
|
||||
assert db.execute("SELECT COUNT(*) FROM model_content_permissions").fetchone()[0] == 0
|
||||
assert receipt.backlog_events == ()
|
||||
with pytest.raises(policy.ContinuousPolicyError, match="evidence_outside_policy"):
|
||||
policy.check(db, receipt, evidence_ids=(env.record.evidence_id,))
|
||||
|
||||
|
||||
def test_backlog_snapshot_over_one_hundred_thousand_is_compact(monkeypatch, env) -> None:
|
||||
count = 100_001
|
||||
|
||||
class Database:
|
||||
in_transaction = True
|
||||
|
||||
def execute(self, _sql, _values):
|
||||
return iter(((SOURCE, count, 1),))
|
||||
|
||||
monkeypatch.setattr(policy.ingest_journal, "watermark", lambda _db: count)
|
||||
monkeypatch.setattr(
|
||||
policy.ModelContentPermissionStore,
|
||||
"_source",
|
||||
lambda _db, _user, _source: object(),
|
||||
)
|
||||
|
||||
snapshot = policy.snapshot_backlog(Database()) # type: ignore[arg-type]
|
||||
|
||||
assert snapshot.after_seq == count
|
||||
assert snapshot.source_ids == (SOURCE,)
|
||||
assert snapshot.evidence_count == count
|
||||
assert snapshot.conversation_count == 1
|
||||
with transaction(env) as db:
|
||||
receipt = enable(db, env, backlog_mode="include", backlog_snapshot=snapshot)
|
||||
assert receipt.backlog_through_seq == count
|
||||
assert receipt.backlog_source_ids == (SOURCE,)
|
||||
assert receipt.backlog_count == count
|
||||
assert receipt.backlog_conversation_count == 1
|
||||
assert receipt.backlog_events == ()
|
||||
assert len(receipt.model_dump_json()) < 5_000
|
||||
|
||||
|
||||
def test_legacy_backlog_receipt_remains_readable(env) -> None:
|
||||
with transaction(env) as db:
|
||||
receipt = enable(db, env)
|
||||
legacy = receipt.model_dump(
|
||||
exclude={"backlog_through_seq", "backlog_source_ids", "backlog_count"}
|
||||
)
|
||||
legacy["backlog_mode"] = "include"
|
||||
legacy["backlog_events"] = (1,)
|
||||
restored = policy.ContinuousPolicy.model_validate(legacy)
|
||||
db.execute(
|
||||
"UPDATE continuous_policy_history SET receipt_json=? WHERE user_id=?",
|
||||
(restored.model_dump_json(), restored.user_id),
|
||||
)
|
||||
assert policy.check(db, restored, evidence_ids=(env.record.evidence_id,)) == restored
|
||||
|
||||
|
||||
@pytest.mark.parametrize("confirmation", [False, 1, "true", None])
|
||||
def test_explicit_confirmation_is_required(env, confirmation):
|
||||
with transaction(env) as db:
|
||||
with pytest.raises(policy.ContinuousPolicyError):
|
||||
enable(db, env, user_confirmed=confirmation)
|
||||
assert policy.current(db) is None
|
||||
|
||||
|
||||
def test_future_old_dated_import_and_new_allowed_source_are_eligible(env, tmp_path):
|
||||
with transaction(env) as db:
|
||||
receipt = enable(db, env)
|
||||
other_root = tmp_path / "other-source"
|
||||
other_root.mkdir()
|
||||
env.sources.grant(
|
||||
user_id="local",
|
||||
source_id="cursor",
|
||||
root=other_root,
|
||||
candidate_kind=RootCandidateKind.DEFAULT,
|
||||
)
|
||||
with transaction(env) as db:
|
||||
value = evidence_value("after", source_id="cursor")
|
||||
result = env.evidence.put(value, connection=db)
|
||||
event = journal.record(db, result.record.evidence_id)
|
||||
assert event.seq > receipt.after_seq
|
||||
assert policy.check(db, receipt, evidence_ids=(result.record.evidence_id,)) == receipt
|
||||
|
||||
|
||||
def test_include_uses_current_source_authority_not_the_enable_time_snapshot(env, tmp_path):
|
||||
old_ungranted = env.evidence.put(evidence_value("old-cursor", source_id="cursor")).record
|
||||
with transaction(env) as db:
|
||||
receipt = enable(db, env, backlog_mode="include")
|
||||
assert receipt.backlog_source_ids == (SOURCE,)
|
||||
|
||||
other_root = tmp_path / "later-authorized-source"
|
||||
other_root.mkdir()
|
||||
env.sources.grant(
|
||||
user_id="local",
|
||||
source_id="cursor",
|
||||
root=other_root,
|
||||
candidate_kind=RootCandidateKind.DEFAULT,
|
||||
)
|
||||
with transaction(env) as db:
|
||||
assert policy.check(db, receipt, evidence_ids=(old_ungranted.evidence_id,)) == receipt
|
||||
|
||||
|
||||
def test_include_keeps_legacy_backlog_fields_readable_without_using_them_as_a_boundary(env):
|
||||
with transaction(env) as db:
|
||||
receipt = enable(db, env, backlog_mode="include")
|
||||
assert receipt.backlog_events == ()
|
||||
assert receipt.backlog_through_seq == 1
|
||||
assert receipt.backlog_source_ids == (SOURCE,)
|
||||
policy.check(db, receipt, evidence_ids=(env.record.evidence_id,))
|
||||
db.execute(
|
||||
"UPDATE evidence SET workspace='new identity' WHERE evidence_id=?",
|
||||
(env.record.evidence_id,),
|
||||
)
|
||||
journal.record(db, env.record.evidence_id, cause="maintenance")
|
||||
assert policy.check(db, receipt, evidence_ids=(env.record.evidence_id,)) == receipt
|
||||
|
||||
|
||||
def test_only_new_mode_keeps_the_enable_time_boundary(env):
|
||||
with transaction(env) as db:
|
||||
receipt = enable(db, env, backlog_mode="none")
|
||||
with pytest.raises(policy.ContinuousPolicyError, match="evidence_outside_policy"):
|
||||
policy.check(db, receipt, evidence_ids=(env.record.evidence_id,))
|
||||
|
||||
|
||||
def test_pause_resume_and_revoke_are_cas_and_old_receipts_never_reactivate(env):
|
||||
with transaction(env) as db:
|
||||
receipt = enable(db, env)
|
||||
paused = policy.transition(
|
||||
db, action="pause", operation_id=str(uuid4()), user_confirmed=True, expected_revision=1
|
||||
)
|
||||
with pytest.raises(policy.ContinuousPolicyError):
|
||||
policy.check(db, receipt)
|
||||
with pytest.raises(policy.ContinuousPolicyError):
|
||||
policy.check(db, paused)
|
||||
resumed = policy.transition(
|
||||
db, action="resume", operation_id=str(uuid4()), user_confirmed=True, expected_revision=2
|
||||
)
|
||||
assert resumed.epoch == receipt.epoch and resumed.after_seq == receipt.after_seq
|
||||
revoked = policy.transition(
|
||||
db, action="revoke", operation_id=str(uuid4()), user_confirmed=True, expected_revision=3
|
||||
)
|
||||
with pytest.raises(policy.ContinuousPolicyError):
|
||||
policy.check(db, resumed)
|
||||
with pytest.raises(policy.ContinuousPolicyError, match="requires_new_enable"):
|
||||
policy.transition(
|
||||
db,
|
||||
action="resume",
|
||||
operation_id=str(uuid4()),
|
||||
user_confirmed=True,
|
||||
expected_revision=4,
|
||||
)
|
||||
new = enable(db, env, expected_revision=revoked.revision)
|
||||
assert new.epoch != receipt.epoch
|
||||
|
||||
|
||||
def test_enable_idempotence_rejects_changed_payload_and_superseded_operation(env):
|
||||
with transaction(env) as db:
|
||||
request_id = str(uuid4())
|
||||
receipt = enable(db, env, operation_id=request_id)
|
||||
assert enable(db, env, operation_id=request_id) == receipt
|
||||
assert db.execute("SELECT COUNT(*) FROM continuous_policy_history").fetchone()[0] == 1
|
||||
with pytest.raises(policy.ContinuousPolicyError, match="payload_changed"):
|
||||
enable(db, env, operation_id=request_id, backlog_mode="include")
|
||||
policy.transition(
|
||||
db, action="revoke", operation_id=str(uuid4()), user_confirmed=True, expected_revision=1
|
||||
)
|
||||
with pytest.raises(policy.ContinuousPolicyError, match="superseded"):
|
||||
enable(db, env, operation_id=request_id)
|
||||
|
||||
|
||||
def test_source_revoke_profile_change_and_legacy_changes_block_existing_policy(env):
|
||||
with transaction(env) as db:
|
||||
receipt = enable(db, env, backlog_mode="include")
|
||||
env.sources.revoke(user_id="local", source_id=SOURCE)
|
||||
with transaction(env) as db, pytest.raises(RuntimeError):
|
||||
policy.check(db, receipt, evidence_ids=(env.record.evidence_id,))
|
||||
env.profiles.update(
|
||||
env.profile.profile_id,
|
||||
ModelProfilePatch(expected_revision=1, model="different"),
|
||||
env.secrets,
|
||||
)
|
||||
with transaction(env) as db, pytest.raises(RuntimeError):
|
||||
policy.check(db, receipt)
|
||||
|
||||
|
||||
def test_caller_rollback_includes_policy_and_backlog(env):
|
||||
with pytest.raises(RuntimeError, match="rollback"), transaction(env) as db:
|
||||
enable(db, env)
|
||||
raise RuntimeError("rollback")
|
||||
with transaction(env) as db:
|
||||
assert policy.current(db) is None
|
||||
|
||||
|
||||
@pytest.mark.parametrize("value", [True, -1, "0", 0.5])
|
||||
def test_revision_values_are_not_coerced(env, value):
|
||||
with (
|
||||
transaction(env) as db,
|
||||
pytest.raises(policy.ContinuousPolicyError, match="invalid_revision"),
|
||||
):
|
||||
enable(db, env, expected_revision=value)
|
||||
|
||||
|
||||
def test_legacy_permission_changes_do_not_invalidate_continuous_policy(env):
|
||||
legacy = env.grant()
|
||||
with transaction(env) as db:
|
||||
receipt = enable(db, env, expected_legacy_revision=legacy.revision)
|
||||
env.permissions.revoke(user_id="local", expected_revision=legacy.revision, user_confirmed=True)
|
||||
with transaction(env) as db:
|
||||
assert policy.check(db, receipt) == receipt
|
||||
paused = policy.transition(
|
||||
db,
|
||||
expected_revision=receipt.revision,
|
||||
action="pause",
|
||||
operation_id=str(uuid4()),
|
||||
user_confirmed=True,
|
||||
)
|
||||
resumed = policy.transition(
|
||||
db,
|
||||
expected_revision=paused.revision,
|
||||
action="resume",
|
||||
operation_id=str(uuid4()),
|
||||
user_confirmed=True,
|
||||
)
|
||||
assert resumed.state == "enabled"
|
||||
|
||||
|
||||
def test_paused_policy_cannot_resume_after_model_identity_changes(env):
|
||||
with transaction(env) as db:
|
||||
enable(db, env)
|
||||
policy.transition(
|
||||
db, expected_revision=1, action="pause", operation_id=str(uuid4()), user_confirmed=True
|
||||
)
|
||||
env.profiles.update(
|
||||
env.profile.profile_id,
|
||||
ModelProfilePatch(expected_revision=1, model="changed-model"),
|
||||
env.secrets,
|
||||
)
|
||||
with transaction(env) as db:
|
||||
with pytest.raises(RuntimeError):
|
||||
policy.transition(
|
||||
db,
|
||||
expected_revision=2,
|
||||
action="resume",
|
||||
operation_id=str(uuid4()),
|
||||
user_confirmed=True,
|
||||
)
|
||||
assert policy.current(db).state == "paused"
|
||||
|
||||
|
||||
def test_read_check_restores_callers_row_factory(env):
|
||||
with transaction(env) as db:
|
||||
receipt = enable(db, env)
|
||||
db.row_factory = None
|
||||
assert policy.check(db, receipt) == receipt
|
||||
assert db.row_factory is None
|
||||
|
||||
|
||||
def test_prepared_input_cannot_be_replaced_by_a_newer_allowed_revision(env):
|
||||
with transaction(env) as db:
|
||||
receipt = enable(db, env)
|
||||
first = env.evidence.put(evidence_value("new-input"), connection=db).record
|
||||
lease = policy.bind_inputs(db, receipt, (first.evidence_id,))
|
||||
policy.recheck_inputs(db, lease)
|
||||
env.evidence.put(evidence_value("new-input", text="changed material"), connection=db)
|
||||
policy.check(db, receipt, evidence_ids=(first.evidence_id,))
|
||||
with pytest.raises(policy.ContinuousPolicyError, match="bound_input_changed"):
|
||||
policy.recheck_inputs(db, lease)
|
||||
|
||||
|
||||
def test_prepared_source_cannot_survive_regrant_without_revalidation(env):
|
||||
with transaction(env) as db:
|
||||
receipt = enable(db, env, backlog_mode="include")
|
||||
lease = policy.bind_inputs(db, receipt, (env.record.evidence_id,))
|
||||
env.sources.revoke(user_id="local", source_id=SOURCE)
|
||||
env.sources.grant(
|
||||
user_id="local", source_id=SOURCE, root=env.root, candidate_kind=RootCandidateKind.DEFAULT
|
||||
)
|
||||
with (
|
||||
transaction(env) as db,
|
||||
pytest.raises(policy.ContinuousPolicyError, match="bound_source_authority_changed"),
|
||||
):
|
||||
policy.recheck_inputs(db, lease)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("inputs", [(), ("duplicate", "duplicate")])
|
||||
def test_lease_requires_explicit_unique_inputs(env, inputs):
|
||||
with transaction(env) as db:
|
||||
receipt = enable(db, env)
|
||||
with pytest.raises(policy.ContinuousPolicyError, match="unique_inputs"):
|
||||
policy.bind_inputs(db, receipt, inputs)
|
||||
@@ -1,472 +0,0 @@
|
||||
import asyncio
|
||||
import json
|
||||
from uuid import uuid4
|
||||
|
||||
import pytest
|
||||
|
||||
from one_memory import continuous_policy as cp
|
||||
from one_memory import ingest_journal
|
||||
from one_memory.hy_memory_bridge import HyMemoryBridgeError
|
||||
from one_memory.imports.sources.registry import RootCandidateKind
|
||||
from one_memory.memory_tasks import MemoryTaskStatus
|
||||
from one_memory.model_processing_permission_guard import ModelProcessingPermissionGuard
|
||||
from tests.test_model_processing_permissions import h as h # noqa: F401
|
||||
|
||||
|
||||
def enable(h, backlog="none"):
|
||||
with h.evidence._connect(writing=True) as db:
|
||||
ingest_journal.initialize(db)
|
||||
cp.initialize(db)
|
||||
legacy = h.permissions.receipt(user_id="local")
|
||||
return cp.enable(
|
||||
db,
|
||||
profile=h.profiles.snapshot(h.profile.profile_id, h.profile.revision),
|
||||
expected_revision=0,
|
||||
expected_legacy_revision=legacy.revision if legacy else 0,
|
||||
operation_id=str(uuid4()),
|
||||
backlog_mode=backlog,
|
||||
user_confirmed=True,
|
||||
)
|
||||
|
||||
|
||||
def transition(h, action):
|
||||
with h.evidence._connect(writing=True) as db:
|
||||
policy = cp.current(db)
|
||||
return cp.transition(
|
||||
db,
|
||||
expected_revision=policy.revision,
|
||||
action=action,
|
||||
operation_id=str(uuid4()),
|
||||
user_confirmed=True,
|
||||
)
|
||||
|
||||
|
||||
def put_after_policy(h, key, *, source="cursor"):
|
||||
"""Create one complete continuous input and record its ingest identity.
|
||||
|
||||
These policy tests used a raw user-only record, which the complete-turn
|
||||
contract correctly holds back. MEMORY_CONTENT is a complete input by
|
||||
itself; recording it after policy enable proves the `after_seq` boundary
|
||||
without inventing an assistant reply that the assertions do not inspect.
|
||||
"""
|
||||
record = h.put(key, source=source, memory_content=True)
|
||||
with h.evidence._connect(writing=True) as db:
|
||||
ingest_journal.record(db, record.evidence_id, cause="import")
|
||||
return h.evidence.get(record.evidence_id)
|
||||
|
||||
|
||||
def test_continuous_prepare_lease_ingest_has_no_legacy_receipt_or_old_context(h):
|
||||
policy = enable(h)
|
||||
fresh = put_after_policy(h, "new")
|
||||
supervisor = h.supervisor()
|
||||
|
||||
async def scenario():
|
||||
try:
|
||||
tasks = supervisor.prepare_conversation(fresh)
|
||||
assert len(tasks) == 1
|
||||
result = await supervisor._create_runner().run(tasks[0].task_id)
|
||||
assert result.status == MemoryTaskStatus.WAITING_FOR_MODEL # Controlled HY failure.
|
||||
assert result.last_error_code == "hy_memory_unavailable"
|
||||
lease = h.leases[0]
|
||||
assert lease.receipt.schema_version == 2
|
||||
assert lease.receipt.authority_kind == "continuous"
|
||||
assert lease.receipt.policy == policy
|
||||
assert {r.evidence_id for r in lease.records} == {fresh.evidence_id}
|
||||
assert h.record.evidence_id not in {r.evidence_id for r in lease.records}
|
||||
assert h.permissions.receipt(user_id="local") is None
|
||||
finally:
|
||||
await supervisor.shutdown()
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
@pytest.mark.parametrize("action", ["pause", "revoke"])
|
||||
def test_continuous_inactive_never_falls_back_to_legacy(h, action):
|
||||
h.grant()
|
||||
enable(h, "include")
|
||||
transition(h, action)
|
||||
supervisor = h.supervisor()
|
||||
with pytest.raises(HyMemoryBridgeError, match="model_content_"):
|
||||
supervisor.prepare_conversation(h.record)
|
||||
assert not h.bridge.calls
|
||||
|
||||
|
||||
@pytest.mark.parametrize("phase", ["before_run", "ingest", "profile"])
|
||||
def test_revocation_rechecked_at_runner_and_publication_boundaries(h, phase):
|
||||
enable(h)
|
||||
fresh = put_after_policy(h, "new")
|
||||
supervisor = h.supervisor()
|
||||
task = supervisor.prepare_conversation(fresh)[0]
|
||||
h.bridge.fail = False
|
||||
if phase == "before_run":
|
||||
transition(h, "revoke")
|
||||
elif phase == "ingest":
|
||||
h.bridge.on_ingest = lambda: transition(h, "revoke")
|
||||
else:
|
||||
h.bridge.on_profile = lambda: transition(h, "revoke")
|
||||
|
||||
async def scenario():
|
||||
try:
|
||||
result = await supervisor._create_runner().run(task.task_id)
|
||||
assert result.status == MemoryTaskStatus.CANCELLED
|
||||
assert result.last_error_code.startswith("model_content_")
|
||||
assert h.governance.list(user_id="local") == []
|
||||
finally:
|
||||
await supervisor.shutdown()
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_durable_binding_restarts_and_cannot_adopt_new_policy_or_new_revision(h):
|
||||
enable(h)
|
||||
fresh = put_after_policy(h, "new")
|
||||
supervisor = h.supervisor()
|
||||
task = supervisor.prepare_conversation(fresh)[0]
|
||||
guard = ModelProcessingPermissionGuard(h.permissions, h.governance)
|
||||
saved = guard.bound(task)
|
||||
assert saved.authority_kind == "continuous" and saved.inputs
|
||||
guard.lease(task, (fresh,), None).recheck()
|
||||
with h.evidence._connect(writing=True) as db:
|
||||
db.execute(
|
||||
"UPDATE evidence SET revision=revision+1,verbatim='changed' WHERE evidence_id=?",
|
||||
(fresh.evidence_id,),
|
||||
)
|
||||
ingest_journal.record(db, fresh.evidence_id, cause="import")
|
||||
with pytest.raises(HyMemoryBridgeError):
|
||||
guard.lease(task, (fresh,), None).recheck()
|
||||
|
||||
|
||||
def test_legacy_task_not_promoted_by_continuous_policy(h):
|
||||
legacy = h.grant()
|
||||
task = h.legacy_task()
|
||||
guard = ModelProcessingPermissionGuard(h.permissions, h.governance)
|
||||
guard.bind(task, legacy)
|
||||
enable(h, "include")
|
||||
with pytest.raises(HyMemoryBridgeError):
|
||||
guard.bound(task)
|
||||
|
||||
|
||||
def test_future_new_source_old_date_uses_continuous_ingest_boundary(h, tmp_path):
|
||||
enable(h)
|
||||
folder = tmp_path / "second"
|
||||
folder.mkdir()
|
||||
h.sources.grant(
|
||||
user_id="local",
|
||||
source_id="new-source",
|
||||
root=folder,
|
||||
candidate_kind=RootCandidateKind.DEFAULT,
|
||||
)
|
||||
fresh = put_after_policy(h, "new", source="new-source")
|
||||
with h.evidence._connect(writing=True) as db:
|
||||
db.execute(
|
||||
"UPDATE evidence SET created_at='1990-01-01T00:00:00+00:00' WHERE evidence_id=?",
|
||||
(fresh.evidence_id,),
|
||||
)
|
||||
# Actual import material revision, not authorizing an attribution repair.
|
||||
db.execute(
|
||||
"UPDATE evidence SET verbatim='Newly imported old-dated content' WHERE evidence_id=?",
|
||||
(fresh.evidence_id,),
|
||||
)
|
||||
ingest_journal.record(db, fresh.evidence_id, cause="import")
|
||||
fresh = h.evidence.get(fresh.evidence_id)
|
||||
supervisor = h.supervisor()
|
||||
tasks = supervisor.prepare_conversation(fresh)
|
||||
assert len(tasks) == 1
|
||||
assert [work.task_id for work in supervisor._classify(tasks)] == [tasks[0].task_id]
|
||||
assert supervisor.tasks.require(tasks[0].task_id).status is MemoryTaskStatus.PENDING
|
||||
|
||||
|
||||
def test_schema2_roundtrip_rejects_unknown_and_preserves_bound_events(h):
|
||||
from one_memory.continuous_permission_bridge import decode_authority
|
||||
|
||||
enable(h)
|
||||
fresh = put_after_policy(h, "new")
|
||||
supervisor = h.supervisor()
|
||||
task = supervisor.prepare_conversation(fresh)[0]
|
||||
receipt = supervisor._permission_guard.bound(task)
|
||||
assert decode_authority(receipt.model_dump_json()) == receipt
|
||||
payload = receipt.model_dump(mode="json")
|
||||
payload["schema_version"] = 99
|
||||
with pytest.raises(ValueError):
|
||||
decode_authority(json.dumps(payload))
|
||||
|
||||
|
||||
def test_transport_rechecks_continuous_lease_without_forged_snapshot(h):
|
||||
from one_memory.hy_permission_transport import HyPermissionTransport, HyPermissionTransportError
|
||||
|
||||
enable(h)
|
||||
fresh = put_after_policy(h, "new")
|
||||
supervisor = h.supervisor()
|
||||
task = supervisor.prepare_conversation(fresh)[0]
|
||||
lease = supervisor._create_runner()._prepare_permission_lease(task)[0]
|
||||
transport = HyPermissionTransport(lease)
|
||||
transport._active = True
|
||||
transport.assert_allowed()
|
||||
transition(h, "pause")
|
||||
with pytest.raises(HyPermissionTransportError, match="lease_denied"):
|
||||
transport.assert_allowed()
|
||||
|
||||
|
||||
def test_transport_denial_names_why_the_lease_failed(h):
|
||||
from one_memory.hy_permission_transport import HyPermissionTransport, HyPermissionTransportError
|
||||
|
||||
enable(h)
|
||||
task = h.supervisor().prepare_conversation(put_after_policy(h, "new"))[0]
|
||||
lease = h.supervisor()._create_runner()._prepare_permission_lease(task)[0]
|
||||
transport = HyPermissionTransport(lease)
|
||||
transport._active = True
|
||||
transition(h, "pause")
|
||||
with pytest.raises(Exception) as direct:
|
||||
lease.recheck()
|
||||
reason = str(direct.value).removeprefix("model_content_")
|
||||
assert reason and reason != "lease_denied"
|
||||
|
||||
with pytest.raises(HyPermissionTransportError) as denied:
|
||||
transport.assert_allowed()
|
||||
|
||||
assert denied.value.code == f"model_content_transport_lease_denied_{reason}"
|
||||
|
||||
|
||||
def test_successful_continuous_runner_completes_without_old_backlog(h):
|
||||
enable(h)
|
||||
fresh = put_after_policy(h, "new")
|
||||
supervisor = h.supervisor()
|
||||
h.bridge.fail = False
|
||||
|
||||
async def run():
|
||||
try:
|
||||
task = supervisor.prepare_conversation(fresh)[0]
|
||||
result = await supervisor._create_runner().run(task.task_id)
|
||||
assert result.status == MemoryTaskStatus.COMPLETED
|
||||
assert [kind for kind, _ in h.bridge.calls] == [
|
||||
"ingest_extracted",
|
||||
"profile",
|
||||
]
|
||||
request = h.bridge.calls[0][1]
|
||||
assert request.evidence_ids == (fresh.evidence_id,)
|
||||
assert request.source_context is None
|
||||
finally:
|
||||
await supervisor.shutdown()
|
||||
|
||||
asyncio.run(run())
|
||||
|
||||
|
||||
def test_prepare_backfill_filters_old_inputs_and_paused_policy_refuses(h):
|
||||
policy = enable(h)
|
||||
fresh = put_after_policy(h, "new")
|
||||
supervisor = h.supervisor()
|
||||
tasks = supervisor.prepare_backfill(expected_permission_revision=policy.revision)
|
||||
assert tasks
|
||||
for work in tasks:
|
||||
task = h.tasks.require(work.task_id)
|
||||
assert {ref["evidence_id"] for ref in h.tasks.inputs(task.task_id)} == {fresh.evidence_id}
|
||||
transition(h, "pause")
|
||||
with pytest.raises(HyMemoryBridgeError):
|
||||
supervisor.prepare_backfill()
|
||||
|
||||
|
||||
def test_bound_source_regrant_refuses_old_task(h):
|
||||
enable(h)
|
||||
fresh = put_after_policy(h, "new")
|
||||
supervisor = h.supervisor()
|
||||
task = supervisor.prepare_conversation(fresh)[0]
|
||||
h.sources.revoke(user_id="local", source_id="cursor")
|
||||
h.sources.grant(
|
||||
user_id="local",
|
||||
source_id="cursor",
|
||||
root=h.root / "synthetic-source",
|
||||
candidate_kind=RootCandidateKind.DEFAULT,
|
||||
)
|
||||
with pytest.raises(HyMemoryBridgeError):
|
||||
supervisor._create_runner()._prepare_permission_lease(task)
|
||||
|
||||
|
||||
def test_schema2_native_sdk_child_send_denies_after_policy_revoke(h):
|
||||
import httpx
|
||||
|
||||
from one_memory.hy_permission_transport import HyPermissionTransport, HyPermissionTransportError
|
||||
from tests.test_hy_permission_transport import native, response
|
||||
|
||||
enable(h)
|
||||
fresh = put_after_policy(h, "new")
|
||||
supervisor = h.supervisor()
|
||||
task = supervisor.prepare_conversation(fresh)[0]
|
||||
lease = supervisor._create_runner()._prepare_permission_lease(task)[0]
|
||||
provider, _ = native(lease)
|
||||
requests = []
|
||||
|
||||
def transport(request):
|
||||
requests.append(request.url.path)
|
||||
return response(request)
|
||||
|
||||
async def run():
|
||||
with pytest.raises(HyPermissionTransportError):
|
||||
async with HyPermissionTransport(
|
||||
lease, http_transport=httpx.MockTransport(transport)
|
||||
) as boundary:
|
||||
boundary.bind_provider(provider)
|
||||
await provider.complete("Synthetic permitted input")
|
||||
assert len(requests) == 1
|
||||
transition(h, "revoke")
|
||||
await provider.complete("This request must not leave")
|
||||
assert len(requests) == 1
|
||||
|
||||
asyncio.run(run())
|
||||
|
||||
|
||||
def test_recreated_event_identity_produces_new_task_even_same_revision(h):
|
||||
enable(h)
|
||||
fresh = put_after_policy(h, "new")
|
||||
supervisor = h.supervisor()
|
||||
first = supervisor.prepare_conversation(fresh)[0]
|
||||
with h.evidence._connect(writing=True) as db:
|
||||
db.execute(
|
||||
"UPDATE evidence SET verbatim='changed without revision' WHERE evidence_id=?",
|
||||
(fresh.evidence_id,),
|
||||
)
|
||||
ingest_journal.record(db, fresh.evidence_id, cause="import")
|
||||
new = h.evidence.get(fresh.evidence_id)
|
||||
second = supervisor.prepare_conversation(new)[0]
|
||||
assert second.task_id != first.task_id
|
||||
|
||||
|
||||
def test_default_guard_does_not_create_continuous_schema(h):
|
||||
ModelProcessingPermissionGuard(h.permissions, h.governance)
|
||||
with h.evidence._connect() as db:
|
||||
assert not db.execute(
|
||||
"SELECT 1 FROM sqlite_master WHERE name='memory_task_continuous_permissions'"
|
||||
).fetchone()
|
||||
assert not db.execute(
|
||||
"SELECT 1 FROM sqlite_master WHERE name='continuous_policy_history'"
|
||||
).fetchone()
|
||||
|
||||
|
||||
def test_publish_guard_rechecks_continuous_after_last_bridge_read(h, monkeypatch):
|
||||
enable(h)
|
||||
fresh = put_after_policy(h, "new")
|
||||
supervisor = h.supervisor()
|
||||
h.bridge.fail = False
|
||||
task = supervisor.prepare_conversation(fresh)[0]
|
||||
runner = supervisor._create_runner()
|
||||
original = runner._record_outputs
|
||||
|
||||
def revoke_at_commit(**kwargs):
|
||||
transition(h, "revoke")
|
||||
return original(**kwargs)
|
||||
|
||||
monkeypatch.setattr(runner, "_record_outputs", revoke_at_commit)
|
||||
result = asyncio.run(runner.run(task.task_id))
|
||||
assert result.status == MemoryTaskStatus.CANCELLED
|
||||
assert result.last_error_code.startswith("model_content_")
|
||||
assert h.tasks.extraction(task.task_id) is None
|
||||
assert h.governance.list(user_id="local") == []
|
||||
|
||||
|
||||
def test_native_hy_continuous_source_to_governance_with_mock_network(tmp_path):
|
||||
import httpx
|
||||
|
||||
from one_memory.hy_permission_context import permission_bridge_factory
|
||||
from one_memory.imports.models import (
|
||||
EvidenceContentKind,
|
||||
EvidenceInput,
|
||||
EvidenceOrigin,
|
||||
OriginMetadata,
|
||||
WrittenBy,
|
||||
)
|
||||
from tests.test_hy_permission_context import whole_client
|
||||
|
||||
async def run():
|
||||
with whole_client(tmp_path) as (supervisor, old, permissions, legacy):
|
||||
with supervisor.evidence._connect(writing=True) as db:
|
||||
ingest_journal.initialize(db)
|
||||
cp.initialize(db)
|
||||
cp.enable(
|
||||
db,
|
||||
profile=legacy.snapshot.profile,
|
||||
expected_revision=0,
|
||||
expected_legacy_revision=legacy.revision,
|
||||
operation_id=str(uuid4()),
|
||||
backlog_mode="none",
|
||||
user_confirmed=True,
|
||||
)
|
||||
fresh = supervisor.evidence.put(
|
||||
EvidenceInput(
|
||||
source_id=old.source_id,
|
||||
conversation_id=old.conversation_id,
|
||||
message_id="new",
|
||||
role="user",
|
||||
written_by=WrittenBy.PERSON,
|
||||
content_kind=EvidenceContentKind.MEMORY_CONTENT,
|
||||
verbatim="Synthetic fresh continuous memory, not old backlog.",
|
||||
origin_meta={EvidenceOrigin.FILE: OriginMetadata()},
|
||||
)
|
||||
).record
|
||||
sends = []
|
||||
|
||||
def handler(request):
|
||||
body = json.loads(request.content)
|
||||
sends.append(body)
|
||||
assert old.verbatim not in request.content.decode()
|
||||
if request.url.path.endswith("embeddings"):
|
||||
return httpx.Response(
|
||||
200,
|
||||
json={
|
||||
"object": "list",
|
||||
"model": "test-embedding",
|
||||
"data": [
|
||||
{
|
||||
"index": i,
|
||||
"object": "embedding",
|
||||
"embedding": [1.0] + [0.0] * 383,
|
||||
}
|
||||
for i, _ in enumerate(body["input"])
|
||||
],
|
||||
"usage": {"prompt_tokens": 1, "total_tokens": 1},
|
||||
},
|
||||
)
|
||||
return httpx.Response(
|
||||
200,
|
||||
json={
|
||||
"id": "fixture",
|
||||
"object": "chat.completion",
|
||||
"created": 1,
|
||||
"model": "test-generation",
|
||||
"choices": [
|
||||
{
|
||||
"index": 0,
|
||||
"finish_reason": "stop",
|
||||
"message": {
|
||||
"role": "assistant",
|
||||
"content": json.dumps(
|
||||
[
|
||||
{
|
||||
"op": "ADD",
|
||||
"content": fresh.verbatim,
|
||||
"layer": "L2_FACT",
|
||||
"owner": "user",
|
||||
}
|
||||
]
|
||||
),
|
||||
},
|
||||
}
|
||||
],
|
||||
"usage": {"prompt_tokens": 1, "completion_tokens": 1, "total_tokens": 2},
|
||||
},
|
||||
)
|
||||
|
||||
supervisor.permission_bridge_factory = permission_bridge_factory(
|
||||
supervisor._bridge, http_transport=httpx.MockTransport(handler)
|
||||
)
|
||||
try:
|
||||
task = supervisor.prepare_conversation(fresh)[0]
|
||||
result = await supervisor._create_runner().run(task.task_id)
|
||||
assert result.status == "completed", result.last_error_code
|
||||
memories = supervisor.governance.list(user_id="local")
|
||||
assert sends and len(memories) == 1
|
||||
assert memories[0].content == fresh.verbatim
|
||||
assert memories[0].evidence_ids == (fresh.evidence_id,)
|
||||
assert permissions.receipt(user_id="local") == legacy
|
||||
finally:
|
||||
await supervisor.shutdown()
|
||||
|
||||
asyncio.run(run())
|
||||
File diff suppressed because it is too large.
Load diff
@@ -1,223 +0,0 @@
|
||||
"""删掉一条记忆,不得让同一对话之后的新 turn 停止沉淀(RC10)。
|
||||
|
||||
沙箱真 HY 实验里实测过:人删掉一条记忆后,同一对话之后的新消息也不再成为记忆。
|
||||
处理一段对话时,每个 chunk 都要过一遍上下文预检;已处理过的旧 turn 的证据
|
||||
正托着被删的那条记忆,于是预检在旧 chunk 上抛 evidence_changed,整段对话中止,
|
||||
新 turn 连同它一起被丢下。live 路径还把这个错误吞成一行日志,实时同步就此静默停住。
|
||||
|
||||
删除删的是那条记忆实体,不是"永远不许再说这件事":
|
||||
新 turn 是新的陈述,应当照常沉淀;同一段旧原话(按证据 id)不得把已删记忆抽回。
|
||||
|
||||
三条入口各守一条:即时 prepare、live 到达、backfill 回放。
|
||||
全部走真实的临时 Core、真实的连续处理授权与守卫,只把 HY 换成可控的替身。
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
|
||||
import pytest
|
||||
|
||||
from one_memory.imports.models import WrittenBy
|
||||
from one_memory.imports.runner import live_sync_arrival
|
||||
from one_memory.memory_governance import MemoryStatus
|
||||
from one_memory.memory_tasks import MemoryTaskStatus
|
||||
from tests.test_continuous_processing_integration import IntegrationCore
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def core(tmp_path, monkeypatch):
|
||||
return IntegrationCore(tmp_path, monkeypatch)
|
||||
|
||||
|
||||
def _memory_for(core, record):
|
||||
return core.app.state.memory_governance_store.get(
|
||||
"integration-hy-" + record.evidence_id, user_id="local"
|
||||
)
|
||||
|
||||
|
||||
async def _first_turn_then_delete(core):
|
||||
"""第一个 turn 沉淀成记忆,然后人把它删掉。返回那一轮的原话与连续授权。"""
|
||||
core.grant_source()
|
||||
await core.save_and_probe()
|
||||
receipt = core.kernel_enable()
|
||||
first = core.put_complete("first-turn")
|
||||
results = await core.run_prepared(first)
|
||||
assert [t.status for t in results] == [MemoryTaskStatus.COMPLETED]
|
||||
memory = _memory_for(core, first)
|
||||
# 正向前提:第一轮确实沉淀出了一条可删的记忆。
|
||||
assert memory is not None and memory.status is MemoryStatus.CONFIRMED
|
||||
forgotten = []
|
||||
|
||||
async def forget_in_hy(memory_id):
|
||||
# HY 那头的遗忘另有守护;这里只要治理侧的删除照常落地。
|
||||
forgotten.append(memory_id)
|
||||
return {"deleted": True}
|
||||
|
||||
core.app.state.memory_processing_supervisor.delete_hy_memory = forget_in_hy
|
||||
deleted = await core.client.post(
|
||||
f"/api/v1/memories/{memory.memory_id}/actions/delete",
|
||||
json={"request_id": "delete-first", "expected_revision": memory.revision},
|
||||
)
|
||||
assert deleted.status_code == 200, deleted.text
|
||||
assert _memory_for(core, first).status is MemoryStatus.DELETED
|
||||
assert forgotten == [memory.memory_id]
|
||||
return first, receipt
|
||||
|
||||
|
||||
def test_prepare_after_a_deletion_still_settles_the_new_turn(core):
|
||||
async def scenario():
|
||||
async with core.running():
|
||||
first, _receipt = await _first_turn_then_delete(core)
|
||||
second = core.put_complete("second-turn")
|
||||
supervisor = core.app.state.memory_processing_supervisor
|
||||
|
||||
prepared = await asyncio.to_thread(supervisor.prepare_conversation, second)
|
||||
# 已完成的旧 turn 在合并前剔除、不再预检重跑;只有新 turn 是待做的。
|
||||
by_input = {t.input_ref: t for t in prepared}
|
||||
assert set(by_input) == {second.evidence_id}
|
||||
pending = [t for t in prepared if t.status is not MemoryTaskStatus.COMPLETED]
|
||||
assert [t.input_ref for t in pending] == [second.evidence_id]
|
||||
for task in pending:
|
||||
await supervisor._create_runner().run(task.task_id)
|
||||
|
||||
memory = _memory_for(core, second)
|
||||
assert memory is not None and memory.status is MemoryStatus.CONFIRMED
|
||||
assert _memory_for(core, first).status is MemoryStatus.DELETED
|
||||
# 新 turn 照常处理,但被删的记忆和它托着的原话都不进这一轮的上下文。
|
||||
sent = next(r for r in core.bridge.requests if second.evidence_id in r.evidence_ids)
|
||||
context = sent.source_context
|
||||
assert context is not None
|
||||
assert first.evidence_id not in {
|
||||
r.evidence_id for r in (*context.history, *context.supporting)
|
||||
}
|
||||
deleted_id = "integration-hy-" + first.evidence_id
|
||||
assert deleted_id not in {m.governed_memory_id for m in context.known_memories}
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
async def _live_settled(supervisor, tasks):
|
||||
"""等防抖、准备与派发都走完,而不是等固定秒数:慢机器上也不会早判。"""
|
||||
loop = asyncio.get_running_loop()
|
||||
end = loop.time() + 30
|
||||
while loop.time() < end:
|
||||
idle = not supervisor._debounces and not supervisor._preparations
|
||||
busy = any(
|
||||
t.status in {MemoryTaskStatus.PENDING, MemoryTaskStatus.RUNNING} for t in tasks.list()
|
||||
)
|
||||
if idle and not busy:
|
||||
return
|
||||
await asyncio.sleep(0.05)
|
||||
pytest.fail("live 到达的准备与派发没有在界限内走完")
|
||||
|
||||
|
||||
def test_live_arrival_after_a_deletion_still_settles_the_new_turn(core):
|
||||
async def scenario():
|
||||
async with core.running():
|
||||
first, _receipt = await _first_turn_then_delete(core)
|
||||
second = core.put("second-turn")
|
||||
reply = core.put(
|
||||
"second-turn-reply",
|
||||
conversation="same-conversation",
|
||||
role="assistant",
|
||||
written_by="agent",
|
||||
)
|
||||
supervisor = core.app.state.memory_processing_supervisor
|
||||
assert _memory_for(core, second) is None
|
||||
|
||||
token = live_sync_arrival.set(True)
|
||||
try:
|
||||
supervisor.enqueue_evidence(second)
|
||||
supervisor.enqueue_evidence(reply)
|
||||
finally:
|
||||
live_sync_arrival.reset(token)
|
||||
await _live_settled(supervisor, core.app.state.memory_task_store)
|
||||
|
||||
memory = _memory_for(core, second)
|
||||
assert memory is not None and memory.status is MemoryStatus.CONFIRMED
|
||||
assert _memory_for(core, first).status is MemoryStatus.DELETED
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
@pytest.mark.parametrize("deleted", [True, False], ids=["deleted", "control-kept"])
|
||||
def test_reprocessing_the_same_old_evidence_does_not_bring_a_deleted_memory_back(core, deleted):
|
||||
"""按证据 id 挡住:同一段旧原话再交给 HY 一次,也不得重新长出被删的那条记忆。
|
||||
|
||||
对照组不删:同一条重跑路径确实会把同一段原话再抽成一条新记忆——
|
||||
证明挡住它的是删除,而不是这条路径本来就到不了写入。
|
||||
"""
|
||||
|
||||
async def scenario():
|
||||
async with core.running():
|
||||
if deleted:
|
||||
first, _receipt = await _first_turn_then_delete(core)
|
||||
else:
|
||||
core.grant_source()
|
||||
await core.save_and_probe()
|
||||
core.kernel_enable()
|
||||
first = core.put_complete("first-turn")
|
||||
await core.run_prepared(first)
|
||||
original = core.bridge.ingest
|
||||
|
||||
async def reextract(request):
|
||||
# 真 HY 重抽同一段原话时给的是新 id,内容相同。
|
||||
result = await original(request)
|
||||
return result.model_copy(
|
||||
update={
|
||||
"memories": tuple(
|
||||
m.model_copy(update={"memory_id": "reextracted-" + m.memory_id})
|
||||
for m in result.memories
|
||||
)
|
||||
}
|
||||
)
|
||||
|
||||
core.bridge.ingest = reextract
|
||||
tasks = core.app.state.memory_task_store
|
||||
done = next(t for t in tasks.list() if t.input_ref == first.evidence_id)
|
||||
tasks.requeue_completed(done.task_id, reason="reprocess_same_evidence")
|
||||
|
||||
finished = await core.app.state.memory_processing_supervisor._create_runner().run(
|
||||
done.task_id
|
||||
)
|
||||
|
||||
back = core.app.state.memory_governance_store.get(
|
||||
"reextracted-integration-hy-" + first.evidence_id, user_id="local"
|
||||
)
|
||||
if deleted:
|
||||
assert back is None or back.status is not MemoryStatus.CONFIRMED
|
||||
assert finished.status is not MemoryTaskStatus.COMPLETED
|
||||
else:
|
||||
assert finished.status is MemoryTaskStatus.COMPLETED, finished.last_error_code
|
||||
assert back is not None and back.status is MemoryStatus.CONFIRMED
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_backfill_after_a_deletion_still_settles_the_new_turn(core):
|
||||
"""backfill 修复前就不冻结:已完成的旧 chunk 在入队时跳过,出错的 chunk 也只丢它自己。
|
||||
|
||||
只拿掉其中一道它仍是绿的;两道都拿掉时才红(实测)。
|
||||
它守的是 backfill 这一路的结果,不单独守"已完成跳过"——那一道由 prepare 的守卫看住。
|
||||
"""
|
||||
|
||||
async def scenario():
|
||||
async with core.running():
|
||||
first, receipt = await _first_turn_then_delete(core)
|
||||
second = core.put_complete("second-turn")
|
||||
core.put(
|
||||
"second-turn-answer",
|
||||
conversation=second.conversation_id,
|
||||
role="assistant",
|
||||
written_by=WrittenBy.AGENT,
|
||||
)
|
||||
supervisor = core.app.state.memory_processing_supervisor
|
||||
assert _memory_for(core, second) is None
|
||||
|
||||
await supervisor.backfill_off_thread(expected_permission_revision=receipt.revision)
|
||||
await core.wait_completed(minimum_tasks=2)
|
||||
|
||||
memory = _memory_for(core, second)
|
||||
assert memory is not None and memory.status is MemoryStatus.CONFIRMED
|
||||
assert _memory_for(core, first).status is MemoryStatus.DELETED
|
||||
|
||||
asyncio.run(scenario())
|
||||
@@ -1,10 +1,8 @@
|
||||
import json
|
||||
import logging
|
||||
from importlib import import_module
|
||||
from inspect import getsource
|
||||
from pathlib import Path
|
||||
|
||||
from one_memory.hy_permission_transport import HyPermissionTransport
|
||||
from one_memory.hy_runtime_config import (
|
||||
configure_hy_runtime_environment,
|
||||
detailed_logging_enabled,
|
||||
@@ -66,13 +64,6 @@ def test_info_collects_hy_info_without_enabling_debug_pipeline_content(
|
||||
assert __import__("os").environ["MEMORY_LOG_LLM_CONTENT"] == "false"
|
||||
|
||||
|
||||
def test_transport_does_not_reject_debug_wire_logging() -> None:
|
||||
source = getsource(HyPermissionTransport)
|
||||
|
||||
assert "_check_wire_logging" not in source
|
||||
assert "unsafe_wire_logging" not in source
|
||||
|
||||
|
||||
def test_live_debug_switch_makes_sdk_pipeline_writer_keep_original_content(
|
||||
tmp_path: Path,
|
||||
) -> None:
|
||||
|
||||
@@ -165,6 +165,13 @@ def test_import_request_trace_reaches_the_background_job_without_original_text(t
|
||||
assert "import.queued" in names
|
||||
assert "import.started" in names
|
||||
assert "import.finished" in names
|
||||
finished = next(event for event in events if event["event"] == "import.finished")
|
||||
assert {
|
||||
"stored_count",
|
||||
"revised_count",
|
||||
"skipped_count",
|
||||
"error_count",
|
||||
} <= finished["details"].keys()
|
||||
assert secret not in written
|
||||
|
||||
|
||||
|
||||
@@ -1,142 +0,0 @@
|
||||
"""Read invalidation review: real guard/provenance, controlled HY, no model/network."""
|
||||
|
||||
import asyncio
|
||||
import sqlite3
|
||||
from contextlib import closing
|
||||
|
||||
import pytest
|
||||
|
||||
from one_memory import context_routes
|
||||
from one_memory.hy_memory_bridge import HyMemoryOutput
|
||||
from one_memory.imports.models import WrittenBy
|
||||
from one_memory.memory_governance import MemoryStatus
|
||||
from one_memory.memory_tasks import evidence_ref
|
||||
from tests.test_continuous_processing_integration import IntegrationCore
|
||||
from tests.test_governance_authority_review import WORKSPACE, prepare, process, put, search
|
||||
|
||||
OLD = "revisionneedle the project deploys only on Windows"
|
||||
NEW = "revisionneedle the project deploys only on macOS"
|
||||
|
||||
|
||||
async def derived(core):
|
||||
await prepare(core)
|
||||
original = put(core, "edited-turn", OLD, WrittenBy.PERSON)
|
||||
put(core, "edited-turn-reply", "Acknowledged.", WrittenBy.AGENT)
|
||||
outputs = await process(
|
||||
core,
|
||||
original,
|
||||
[
|
||||
HyMemoryOutput(
|
||||
memory_id="review-old-claim",
|
||||
content=OLD,
|
||||
layer="L2_FACT",
|
||||
owner="user",
|
||||
evidence_ids=(original.evidence_id,),
|
||||
)
|
||||
],
|
||||
)
|
||||
output = outputs[0]
|
||||
assert output.support_refs == (evidence_ref(original),)
|
||||
assert output.memory_revision == 1
|
||||
before = await search(core, "revisionneedle", [WORKSPACE])
|
||||
assert before.status_code == 200 and len(before.json()["items"]) == 1
|
||||
assert before.json()["items"][0]["content"] == OLD
|
||||
return original, output
|
||||
|
||||
|
||||
@pytest.mark.parametrize("during_read", [False, True], ids=["before-query", "before-publish"])
|
||||
def test_revised_evidence_does_not_supply_old_derived_claim(tmp_path, monkeypatch, during_read):
|
||||
async def scenario():
|
||||
core = IntegrationCore(tmp_path, monkeypatch)
|
||||
async with core.running():
|
||||
original, output = await derived(core)
|
||||
revised = None
|
||||
|
||||
def revise():
|
||||
nonlocal revised
|
||||
revised = put(core, "edited-turn", NEW, WrittenBy.PERSON)
|
||||
assert revised.evidence_id == original.evidence_id
|
||||
assert revised.revision == original.revision + 1
|
||||
assert revised.content_hash != original.content_hash
|
||||
|
||||
if during_read:
|
||||
read = context_routes.read_memory_context
|
||||
|
||||
def revision_after_selection(*args, **kwargs):
|
||||
result = read(*args, **kwargs)
|
||||
assert len(result.items) == 1
|
||||
revise()
|
||||
return result
|
||||
|
||||
monkeypatch.setattr(context_routes, "read_memory_context", revision_after_selection)
|
||||
else:
|
||||
revise()
|
||||
response = await search(core, "revisionneedle", [WORKSPACE])
|
||||
assert response.status_code == 200
|
||||
assert revised is not None
|
||||
stored = core.app.state.memory_task_store.outputs(output.task_id)[0]
|
||||
assert stored.support_refs == output.support_refs
|
||||
memory = core.app.state.memory_governance_store.get(output.memory_id, user_id="local")
|
||||
assert memory.status is MemoryStatus.CONFIRMED and memory.content == OLD
|
||||
assert len(core.bridge.requests) == 1
|
||||
with closing(sqlite3.connect(core.database)) as db:
|
||||
versions = db.execute(
|
||||
"SELECT revision FROM evidence_versions WHERE evidence_id=? ORDER BY revision",
|
||||
(original.evidence_id,),
|
||||
).fetchall()
|
||||
events = db.execute(
|
||||
"SELECT evidence_revision FROM evidence_ingest_events "
|
||||
"WHERE evidence_id=? ORDER BY seq",
|
||||
(original.evidence_id,),
|
||||
).fetchall()
|
||||
assert versions == [(1,), (2,)] and events == [(1,), (2,)]
|
||||
print("revision review:", during_read, output.support_refs, response.json())
|
||||
assert response.json()["items"] == []
|
||||
library = await core.client.get("/api/v1/memory-library")
|
||||
# D22: the total skips the per-row freshness scan (product accepted a
|
||||
# small drift), but the stale claim itself must never be listed.
|
||||
assert library.status_code == 200 and library.json()["items"] == []
|
||||
home = await core.client.get("/api/v1/home-summary")
|
||||
# Same D22 drift for the homepage count; the latest card must not be it.
|
||||
assert home.status_code == 200 and home.json()["latest"] is None
|
||||
assert (await core.client.get("/api/v1/memories")).json() == []
|
||||
detail = await core.client.get("/api/v1/memories/" + output.memory_id)
|
||||
assert detail.status_code == 403
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_explicit_correction_is_distinguishable_from_unchanged_sdk_projection(
|
||||
tmp_path, monkeypatch
|
||||
):
|
||||
async def scenario():
|
||||
core = IntegrationCore(tmp_path, monkeypatch)
|
||||
async with core.running():
|
||||
original, output = await derived(core)
|
||||
corrected = core.app.state.person_writes.correct(
|
||||
output.memory_id,
|
||||
user_id="local",
|
||||
request_id="explicit-user-correction",
|
||||
expected_revision=output.memory_revision,
|
||||
content=NEW,
|
||||
)
|
||||
assert corrected.revision > output.memory_revision
|
||||
assert corrected.origin.value == "extracted"
|
||||
assert corrected.evidence_ids == (original.evidence_id,)
|
||||
with closing(sqlite3.connect(core.database)) as db:
|
||||
operation = db.execute(
|
||||
"SELECT operation FROM governed_memory_mutations WHERE request_id=?",
|
||||
("explicit-user-correction",),
|
||||
).fetchone()
|
||||
version = db.execute(
|
||||
"SELECT action,content FROM governed_memory_versions "
|
||||
"WHERE memory_id=? AND revision=?",
|
||||
(corrected.memory_id, corrected.revision),
|
||||
).fetchone()
|
||||
assert operation == ("correct",) and version == ("correct", NEW)
|
||||
stored = core.app.state.memory_task_store.outputs(output.task_id)[0]
|
||||
assert stored.memory_revision == 1 and stored.support_refs == output.support_refs
|
||||
# Inspect metadata only: no new policy that old refs authorize a manual correction.
|
||||
print("correction distinction:", corrected.revision, stored.memory_revision, operation)
|
||||
|
||||
asyncio.run(scenario())
|
||||
@@ -79,9 +79,3 @@ def test_default_distribution_avoids_per_launch_self_extraction(monkeypatch):
|
||||
calls, _, _ = evaluate_spec(monkeypatch, None)
|
||||
assert calls["exe"][1]["exclude_binaries"] is True
|
||||
assert calls["collect"][1]["name"] == "one-memory-runtime"
|
||||
|
||||
|
||||
@pytest.mark.parametrize("layout", ["folder", "single-file"])
|
||||
def test_runtime_permission_factory_is_explicitly_bundled(monkeypatch, layout):
|
||||
calls, _, _ = evaluate_spec(monkeypatch, layout)
|
||||
assert "one_memory.hy_permission_context" in calls["analysis"][1]["hiddenimports"]
|
||||
@@ -1,201 +0,0 @@
|
||||
"""D04 authority and workspace scope through the real continuous permission guard.
|
||||
|
||||
Only HY/model transport is controlled. These assert the documented boundary,
|
||||
not that a confirmed machine projection constitutes human factual acceptance.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
from datetime import UTC, datetime
|
||||
|
||||
import pytest
|
||||
|
||||
from one_memory.hy_memory_bridge import HyMemoryIngestResult, HyMemoryOutput
|
||||
from one_memory.imports.models import EvidenceInput, EvidenceOrigin, OriginMetadata, WrittenBy
|
||||
from one_memory.memory_governance import MemoryDraft, MemoryKind, MemoryOrigin, MemoryStatus
|
||||
from one_memory.memory_tasks import MemoryTaskStatus
|
||||
from tests.test_continuous_processing_integration import IntegrationCore
|
||||
|
||||
WORKSPACE = "workspace:v1:" + "a" * 64
|
||||
OTHER_WORKSPACE = "workspace:v1:" + "b" * 64
|
||||
AGENT_TEXT = "authorityneedle assistant proposes project delivery without user acceptance"
|
||||
PROJECT_TEXT = "projectneedle this project requires a staging approval before deployment"
|
||||
|
||||
|
||||
def envelope(scopes, tools=("om_recall",)):
|
||||
# Simulated authenticated Broker envelope, not a real host installation/grant.
|
||||
return {
|
||||
"x-one-memory-read-context": "external",
|
||||
"x-one-memory-installation-id": "00000000-0000-0000-0000-000000000001",
|
||||
"x-one-memory-installation-revision": "1",
|
||||
"x-one-memory-allowed-scopes": json.dumps(scopes),
|
||||
"x-one-memory-allowed-tools": json.dumps(tools),
|
||||
}
|
||||
|
||||
|
||||
async def search(core, query, scopes, tools=("om_recall",)):
|
||||
return await core.client.post(
|
||||
"/api/v1/context/search",
|
||||
json={"query": query, "kinds": ["memory"]},
|
||||
headers=envelope(scopes, tools),
|
||||
)
|
||||
|
||||
|
||||
def put(core, key, text, author):
|
||||
return core.app.state.evidence_store.put(
|
||||
EvidenceInput(
|
||||
user_id="local",
|
||||
source_id="cursor",
|
||||
conversation_id="authority-review",
|
||||
message_id=key,
|
||||
workspace=WORKSPACE,
|
||||
role="assistant" if author is WrittenBy.AGENT else "user",
|
||||
written_by=author,
|
||||
created_at=datetime.now(UTC),
|
||||
verbatim=text,
|
||||
origin_meta={EvidenceOrigin.FILE: OriginMetadata()},
|
||||
)
|
||||
).record
|
||||
|
||||
|
||||
async def prepare(core):
|
||||
core.grant_source()
|
||||
core.app.state.config_store.set_sync("cursor", True)
|
||||
await core.save_and_probe()
|
||||
await core.enable_api()
|
||||
runner = core.app.state.memory_processing_supervisor._create_runner()
|
||||
assert runner._permission_guard is not None
|
||||
|
||||
|
||||
async def process(core, record, outputs):
|
||||
async def ingest(request):
|
||||
core.bridge.requests.append(request)
|
||||
assert record.evidence_id in request.evidence_ids
|
||||
return HyMemoryIngestResult(success=True, memories=tuple(outputs))
|
||||
|
||||
core.bridge.ingest = ingest
|
||||
results = await core.run_prepared(record)
|
||||
assert results and all(task.status == MemoryTaskStatus.COMPLETED for task in results)
|
||||
assert core.bridge.requests and core.bridge.leases
|
||||
provenance = [
|
||||
output
|
||||
for task in results
|
||||
for output in core.app.state.memory_task_store.outputs(task.task_id)
|
||||
]
|
||||
assert len(provenance) == len(outputs)
|
||||
return provenance
|
||||
|
||||
|
||||
@pytest.mark.parametrize("existing", [False, True])
|
||||
def test_agent_output_lands_usable_without_demoting_an_existing_memory(
|
||||
tmp_path, monkeypatch, existing
|
||||
):
|
||||
async def scenario():
|
||||
core = IntegrationCore(tmp_path, monkeypatch)
|
||||
async with core.running():
|
||||
await prepare(core)
|
||||
person = put(core, "person", "Please review the project proposal", WrittenBy.PERSON)
|
||||
agent = put(core, "agent", AGENT_TEXT, WrittenBy.AGENT)
|
||||
governance = core.app.state.memory_governance_store
|
||||
if existing:
|
||||
previous = core.app.state.person_writes.create(
|
||||
MemoryDraft(
|
||||
user_id="local",
|
||||
kind=MemoryKind.CHAT,
|
||||
content=AGENT_TEXT,
|
||||
scopes=(WORKSPACE,),
|
||||
evidence_ids=(agent.evidence_id,),
|
||||
origin=MemoryOrigin.AGENT,
|
||||
),
|
||||
request_id="fixture-prior",
|
||||
memory_id="prior-agent-memory",
|
||||
)
|
||||
await process(
|
||||
core,
|
||||
person,
|
||||
[
|
||||
HyMemoryOutput(
|
||||
memory_id="hy-agent-proposal",
|
||||
content=AGENT_TEXT,
|
||||
layer="L2_FACT",
|
||||
owner="agent",
|
||||
evidence_ids=(agent.evidence_id,),
|
||||
)
|
||||
],
|
||||
)
|
||||
# HY's id is mirrored as its own row; the ledger does not fold it
|
||||
# into an older row that happens to share the words.
|
||||
memory = governance.get("hy-agent-proposal", user_id="local")
|
||||
assert memory is not None and memory.origin is MemoryOrigin.AGENT
|
||||
assert memory.evidence_ids == (agent.evidence_id,)
|
||||
assert memory.status is MemoryStatus.CONFIRMED
|
||||
prior_visible = 0
|
||||
if existing:
|
||||
prior = governance.get("prior-agent-memory", user_id="local")
|
||||
assert prior.status is MemoryStatus.CONFIRMED
|
||||
assert prior.revision == previous.revision
|
||||
prior_visible = 1
|
||||
matching = await search(core, "authorityneedle", [WORKSPACE])
|
||||
assert matching.status_code == 200, matching.text
|
||||
assert len(matching.json()["items"]) == 1 + prior_visible
|
||||
assert memory.scopes == (WORKSPACE,) or WORKSPACE in memory.scopes
|
||||
denied = await search(core, "authorityneedle", [WORKSPACE], ("om_conversations",))
|
||||
assert denied.status_code == 403
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_project_fact_is_shared_across_apps_and_leaves_with_its_grant(tmp_path, monkeypatch):
|
||||
async def scenario():
|
||||
core = IntegrationCore(tmp_path, monkeypatch)
|
||||
async with core.running():
|
||||
await prepare(core)
|
||||
record = put(core, "project", PROJECT_TEXT, WrittenBy.PERSON)
|
||||
put(core, "project-reply", "Acknowledged.", WrittenBy.AGENT)
|
||||
await process(
|
||||
core,
|
||||
record,
|
||||
[
|
||||
HyMemoryOutput(
|
||||
memory_id="hy-project-rule",
|
||||
content=PROJECT_TEXT,
|
||||
layer="L2_FACT",
|
||||
owner="user",
|
||||
evidence_ids=(record.evidence_id,),
|
||||
),
|
||||
*[
|
||||
HyMemoryOutput(
|
||||
memory_id="hy-profile-" + layer,
|
||||
content="profileneedle " + layer + " user prefers concise explanations",
|
||||
layer=layer,
|
||||
owner="user",
|
||||
evidence_ids=(record.evidence_id,),
|
||||
)
|
||||
for layer in ("L0_BASIC_INFO", "L6_SCHEMA")
|
||||
],
|
||||
],
|
||||
)
|
||||
memory = core.app.state.memory_governance_store.get("hy-project-rule", user_id="local")
|
||||
assert memory is not None and memory.status is MemoryStatus.CONFIRMED
|
||||
assert memory.origin is MemoryOrigin.EXTRACTED
|
||||
workspace = await search(core, "projectneedle", [WORKSPACE])
|
||||
assert workspace.status_code == 200 and len(workspace.json()["items"]) == 1
|
||||
unrelated = await search(core, "projectneedle", [OTHER_WORKSPACE])
|
||||
assert unrelated.status_code == 200 and unrelated.json()["items"] == []
|
||||
global_read = await search(core, "projectneedle", ["all-apps", OTHER_WORKSPACE])
|
||||
assert global_read.status_code == 200 and len(global_read.json()["items"]) == 1
|
||||
assert memory.scopes == ("all-apps", WORKSPACE)
|
||||
profiles = await search(core, "profileneedle", ["all-apps"])
|
||||
assert profiles.status_code == 200 and len(profiles.json()["items"]) == 2
|
||||
denied = await search(core, "projectneedle", [WORKSPACE], ("om_conversations",))
|
||||
assert denied.status_code == 403
|
||||
core.app.state.source_grant_store.revoke(user_id="local", source_id="cursor")
|
||||
revoked = await search(core, "projectneedle", [WORKSPACE])
|
||||
assert revoked.status_code == 200 and revoked.json()["items"] == []
|
||||
revoked_global = await search(core, "projectneedle", ["all-apps"])
|
||||
assert revoked_global.status_code == 200 and revoked_global.json()["items"] == []
|
||||
revoked_profile = await search(core, "profileneedle", ["all-apps"])
|
||||
assert revoked_profile.status_code == 200 and revoked_profile.json()["items"] == []
|
||||
assert len(core.bridge.requests) == 1
|
||||
|
||||
asyncio.run(scenario())
|
||||
@@ -3,13 +3,10 @@ import logging
|
||||
import sqlite3
|
||||
import threading
|
||||
import traceback
|
||||
from collections.abc import Callable
|
||||
from contextlib import suppress
|
||||
|
||||
import pytest
|
||||
from httpx import ASGITransport, AsyncClient, Response
|
||||
from one_memory.model_content_permissions import ModelContentPermissionStore
|
||||
from pydantic import SecretStr
|
||||
|
||||
from one_memory.api import create_app
|
||||
from one_memory.config import Settings
|
||||
@@ -18,10 +15,10 @@ from one_memory.imports.models import EvidenceInput, EvidenceOrigin, OriginMetad
|
||||
from one_memory.imports.sources.registry import RootCandidateKind
|
||||
from one_memory.imports.storage import EvidenceStore
|
||||
from one_memory.memory_governance import MemoryGovernanceStore
|
||||
from one_memory.memory_processing import MemoryProcessingSupervisor
|
||||
from one_memory.memory_server.durable_queue import DurableWriteQueue
|
||||
from one_memory.memory_server.unified_entry import UnifiedWriteIntake
|
||||
from one_memory.model_profiles import ModelCapabilities, ModelProfileInput, ModelProfileStore
|
||||
from one_memory.secrets import InMemorySecretStore, SecretStoreUnavailableError
|
||||
from one_memory.secrets import InMemorySecretStore
|
||||
|
||||
|
||||
async def request(
|
||||
@@ -86,45 +83,6 @@ def test_core_refuses_to_start_a_database_written_by_a_newer_build(tmp_path, mon
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_core_stays_ready_when_legacy_policy_upgrade_cannot_read_keychain(
|
||||
tmp_path, monkeypatch
|
||||
) -> None:
|
||||
monkeypatch.setattr("one_memory.model_presets._preset_data", lambda: None)
|
||||
monkeypatch.setattr(
|
||||
"one_memory.model_policy_activation.ModelPolicyActivation.upgrade_legacy_none_policy",
|
||||
lambda _self: (_ for _ in ()).throw(
|
||||
SecretStoreUnavailableError("Operating system secure storage read backoff")
|
||||
),
|
||||
)
|
||||
settings = Settings(
|
||||
_env_file=None,
|
||||
data_dir=tmp_path / "data",
|
||||
source_home=tmp_path / "source",
|
||||
api_token=SecretStr("test"),
|
||||
imports_enabled=False,
|
||||
automatic_memory_processing=False,
|
||||
live_sync_enabled=False,
|
||||
)
|
||||
app = create_app(settings, secret_store=InMemorySecretStore())
|
||||
|
||||
async def scenario() -> None:
|
||||
async with (
|
||||
app.router.lifespan_context(app),
|
||||
AsyncClient(
|
||||
transport=ASGITransport(app=app),
|
||||
base_url="http://test",
|
||||
headers={"Authorization": "Bearer test"},
|
||||
) as client,
|
||||
):
|
||||
health = await client.get("/api/v1/health")
|
||||
recall = await client.post("/api/v1/recall", json={"query": "still ready"})
|
||||
assert health.status_code == 200
|
||||
assert health.json()["status"] == "ok"
|
||||
assert recall.status_code == 200
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_lifespan_starts_the_content_candidate_backfill_and_exposes_running(
|
||||
tmp_path, monkeypatch
|
||||
) -> None:
|
||||
@@ -211,7 +169,7 @@ def test_lifespan_drains_committed_pending_dispatch_through_enqueue(tmp_path, mo
|
||||
origin_meta={EvidenceOrigin.FILE: OriginMetadata()},
|
||||
)
|
||||
).record
|
||||
evidence.put(
|
||||
_ = evidence.put(
|
||||
EvidenceInput(
|
||||
source_id="fixture",
|
||||
conversation_id="conversation",
|
||||
@@ -360,7 +318,7 @@ def test_diagnostic_log_level_allows_desktop_put_preflight() -> None:
|
||||
assert "PUT" in response.headers["access-control-allow-methods"]
|
||||
|
||||
|
||||
def core(tmp_path, *, permission=True):
|
||||
def core(tmp_path):
|
||||
database = tmp_path / "data/one-memory.sqlite3"
|
||||
root = tmp_path / "source"
|
||||
root.mkdir()
|
||||
@@ -369,7 +327,7 @@ def core(tmp_path, *, permission=True):
|
||||
user_id="local", source_id="fixture", root=root, candidate_kind=RootCandidateKind.DEFAULT
|
||||
)
|
||||
evidence = EvidenceStore(database)
|
||||
record = evidence.put(
|
||||
_record = evidence.put(
|
||||
EvidenceInput(
|
||||
source_id="fixture",
|
||||
conversation_id="health",
|
||||
@@ -399,15 +357,6 @@ def core(tmp_path, *, permission=True):
|
||||
secrets=secrets,
|
||||
)
|
||||
profiles.activate(profile.profile_id, secrets)
|
||||
store = ModelContentPermissionStore(database)
|
||||
if permission:
|
||||
snapshot = store.snapshot(
|
||||
user_id="local",
|
||||
source_ids=("fixture",),
|
||||
profile=profiles.snapshot(profile.profile_id, profile.revision),
|
||||
evidence=(record,),
|
||||
)
|
||||
store.grant(snapshot, user_confirmed=True, expected_revision=0)
|
||||
return create_app(
|
||||
Settings(
|
||||
data_dir=tmp_path / "data",
|
||||
@@ -420,25 +369,6 @@ def core(tmp_path, *, permission=True):
|
||||
)
|
||||
|
||||
|
||||
def test_configured_model_without_content_consent_does_not_start_backfill(tmp_path, monkeypatch):
|
||||
started = []
|
||||
|
||||
def prepare(*args, **kwargs):
|
||||
started.append(True)
|
||||
yield []
|
||||
|
||||
monkeypatch.setattr(MemoryProcessingSupervisor, "_prepare_backfill_batches", prepare)
|
||||
|
||||
async def scenario():
|
||||
app = core(tmp_path, permission=False)
|
||||
async with app.router.lifespan_context(app):
|
||||
await app.state.memory_backfill
|
||||
assert started == []
|
||||
assert app.state.memory_task_store.list() == []
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_startup_finishes_before_the_memory_backfill_does(tmp_path, monkeypatch) -> None:
|
||||
"""Startup must not wait on work that grows with the corpus.
|
||||
|
||||
@@ -451,16 +381,16 @@ def test_startup_finishes_before_the_memory_backfill_does(tmp_path, monkeypatch)
|
||||
release = threading.Event()
|
||||
|
||||
def blocking(
|
||||
_: MemoryProcessingSupervisor,
|
||||
should_stop: Callable[[], bool] = lambda: False,
|
||||
_: DurableWriteQueue,
|
||||
*,
|
||||
expected_permission_revision: int | None = None,
|
||||
limit: int = 512,
|
||||
after_cursor: str | None = None,
|
||||
):
|
||||
walking.set()
|
||||
assert release.wait(10)
|
||||
yield []
|
||||
return ()
|
||||
|
||||
monkeypatch.setattr(MemoryProcessingSupervisor, "_prepare_backfill_batches", blocking)
|
||||
monkeypatch.setattr(DurableWriteQueue, "unsubmitted_evidence_ids", blocking)
|
||||
|
||||
async def scenario() -> None:
|
||||
app = core(tmp_path)
|
||||
@@ -489,15 +419,14 @@ def test_startup_finishes_before_the_memory_backfill_does(tmp_path, monkeypatch)
|
||||
|
||||
def test_a_backfill_that_fails_says_so_rather_than_disappearing(tmp_path, monkeypatch) -> None:
|
||||
def broken(
|
||||
_: MemoryProcessingSupervisor,
|
||||
should_stop: Callable[[], bool] = lambda: False,
|
||||
_: DurableWriteQueue,
|
||||
*,
|
||||
expected_permission_revision: int | None = None,
|
||||
limit: int = 512,
|
||||
after_cursor: str | None = None,
|
||||
):
|
||||
raise RuntimeError("the catch-up broke")
|
||||
yield []
|
||||
|
||||
monkeypatch.setattr(MemoryProcessingSupervisor, "_prepare_backfill_batches", broken)
|
||||
monkeypatch.setattr(DurableWriteQueue, "unsubmitted_evidence_ids", broken)
|
||||
|
||||
async def scenario() -> None:
|
||||
app = core(tmp_path)
|
||||
@@ -521,16 +450,16 @@ def test_shutting_down_mid_backfill_leaves_nothing_running(tmp_path, monkeypatch
|
||||
started = threading.Event()
|
||||
|
||||
def slow(
|
||||
_: MemoryProcessingSupervisor,
|
||||
should_stop: Callable[[], bool] = lambda: False,
|
||||
_: DurableWriteQueue,
|
||||
*,
|
||||
expected_permission_revision: int | None = None,
|
||||
limit: int = 512,
|
||||
after_cursor: str | None = None,
|
||||
):
|
||||
started.set()
|
||||
threading.Event().wait(0.2)
|
||||
yield []
|
||||
return ()
|
||||
|
||||
monkeypatch.setattr(MemoryProcessingSupervisor, "_prepare_backfill_batches", slow)
|
||||
monkeypatch.setattr(DurableWriteQueue, "unsubmitted_evidence_ids", slow)
|
||||
|
||||
async def scenario() -> None:
|
||||
app = core(tmp_path)
|
||||
@@ -553,15 +482,14 @@ def test_a_failed_catch_up_leaves_a_traceback_behind(tmp_path, monkeypatch, capl
|
||||
"""
|
||||
|
||||
def failing(
|
||||
_: MemoryProcessingSupervisor,
|
||||
should_stop: Callable[[], bool] = lambda: False,
|
||||
_: DurableWriteQueue,
|
||||
*,
|
||||
expected_permission_revision: int | None = None,
|
||||
limit: int = 512,
|
||||
after_cursor: str | None = None,
|
||||
):
|
||||
raise sqlite3.OperationalError("database is locked")
|
||||
yield []
|
||||
|
||||
monkeypatch.setattr(MemoryProcessingSupervisor, "_prepare_backfill_batches", failing)
|
||||
monkeypatch.setattr(DurableWriteQueue, "unsubmitted_evidence_ids", failing)
|
||||
|
||||
async def scenario() -> dict:
|
||||
app = core(tmp_path)
|
||||
@@ -591,7 +519,7 @@ def test_a_failed_catch_up_leaves_a_traceback_behind(tmp_path, monkeypatch, capl
|
||||
if record.exc_info is not None
|
||||
for frame in traceback.extract_tb(record.exc_info[2])
|
||||
]
|
||||
assert any(frame.name == "backfill_off_thread" for frame in frames)
|
||||
assert any(frame.name == "_run_backfill" for frame in frames)
|
||||
assert any(frame.name == "failing" and frame.filename == __file__ for frame in frames), (
|
||||
"栈里没有出事的那个调用"
|
||||
)
|
||||
@@ -1,384 +0,0 @@
|
||||
"""HY's own coding/working route under the permission lease, on the pinned SDK.
|
||||
|
||||
HY decides the route; scripted HTTP stands in for the model, not for HY.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import sqlite3
|
||||
import traceback
|
||||
from collections.abc import Mapping
|
||||
|
||||
import httpx
|
||||
from hy_memory.coding.types import CodingMemory
|
||||
|
||||
from one_memory.memory_governance import MemoryKind, MemoryStatus
|
||||
from tests.test_hy_permission_raw_context import (
|
||||
bind_oracle,
|
||||
grant_records,
|
||||
native_client,
|
||||
on_native_loop,
|
||||
put_turn,
|
||||
raw_client,
|
||||
)
|
||||
from tests.test_hy_permission_store import Lease
|
||||
|
||||
TOOL_TEXT = "[Bash] pytest -q\n12 passed"
|
||||
TASK = "Run the synthetic test suite"
|
||||
SEARCH_KEY = "pytest quiet run"
|
||||
SOLUTION = "Run `pytest -q` from the repository root."
|
||||
STATEMENT = "The agent ran pytest -q in the synthetic repository and 12 tests passed."
|
||||
DENIED_EXPERIENCE = "denied-coding-experience"
|
||||
DENIED_TASK = "DENIED_CODING_TASK_5d2e run the suite"
|
||||
DENIED_SOLUTION = "DENIED_CODING_SOLUTION_7f10: skip the tests"
|
||||
|
||||
JUDGE = "You are a single-label scene classifier."
|
||||
EXTRACT = "You extract durable WORKING memories"
|
||||
RECONCILE = "You decide how to merge new coding memory drafts"
|
||||
FACT_RECONCILE = "You maintain session-level WORKING FACTS"
|
||||
|
||||
|
||||
def coding_vector():
|
||||
# One shared direction: every stored experience and fact is a candidate.
|
||||
return [float(index == 5) for index in range(384)]
|
||||
|
||||
|
||||
class FixedEmbeddings:
|
||||
async def embed_batch(self, texts):
|
||||
return [coding_vector() for _ in texts]
|
||||
|
||||
async def embed(self, text):
|
||||
return coding_vector()
|
||||
|
||||
|
||||
class CodingOracle:
|
||||
def __init__(
|
||||
self,
|
||||
records,
|
||||
*,
|
||||
solution=SOLUTION,
|
||||
statement=STATEMENT,
|
||||
update_target=None,
|
||||
forbidden=(),
|
||||
after_response=None,
|
||||
):
|
||||
self.records = records
|
||||
self.solution, self.statement = solution, statement
|
||||
self.update_target = update_target
|
||||
self.forbidden = forbidden
|
||||
self.after_response = after_response
|
||||
self.requests = []
|
||||
self.stages = []
|
||||
self.reconcile_prompts = []
|
||||
self.errors = []
|
||||
self.revoked_at = None
|
||||
|
||||
def __call__(self, request):
|
||||
try:
|
||||
return self.respond(request)
|
||||
except Exception:
|
||||
self.errors.append(traceback.format_exc())
|
||||
raise
|
||||
|
||||
def respond(self, request):
|
||||
body = json.loads(request.content)
|
||||
self.requests.append(body)
|
||||
assert self.revoked_at is None, "native child request escaped after revocation"
|
||||
serialized = json.dumps(body, ensure_ascii=False)
|
||||
for forbidden in self.forbidden:
|
||||
assert forbidden not in serialized, f"unapproved coding row reached HTTP: {forbidden}"
|
||||
if request.url.path == "/v1/embeddings":
|
||||
texts = body["input"] if isinstance(body["input"], list) else [body["input"]]
|
||||
self.stages.append("embedding")
|
||||
return httpx.Response(
|
||||
200,
|
||||
json={
|
||||
"object": "list",
|
||||
"model": body["model"],
|
||||
"data": [
|
||||
{"index": i, "object": "embedding", "embedding": coding_vector()}
|
||||
for i in range(len(texts))
|
||||
],
|
||||
"usage": {"prompt_tokens": 1, "total_tokens": 1},
|
||||
},
|
||||
)
|
||||
prompt = body["messages"][-1]["content"]
|
||||
if prompt.startswith(JUDGE):
|
||||
result = {"is_coding": True, "reason": "synthetic tool run"}
|
||||
stage = "judge"
|
||||
elif prompt.startswith(EXTRACT):
|
||||
for record in self.records:
|
||||
assert record.verbatim in prompt, record.role
|
||||
result = {
|
||||
"experiences": [
|
||||
{
|
||||
"task": TASK,
|
||||
"search_keys": [SEARCH_KEY],
|
||||
"solution": self.solution,
|
||||
"boundary_envs": "",
|
||||
"confidence": 0.9,
|
||||
}
|
||||
],
|
||||
"fact": {
|
||||
"subject": "synthetic repository",
|
||||
"statement": self.statement,
|
||||
"files": [],
|
||||
"details": "",
|
||||
},
|
||||
}
|
||||
stage = "extract"
|
||||
elif prompt.startswith(RECONCILE):
|
||||
self.reconcile_prompts.append(prompt)
|
||||
op = (
|
||||
{"action": "UPDATE", "draft_idx": 0, "target_experience_id": self.update_target}
|
||||
if self.update_target
|
||||
else {"action": "ADD", "draft_idx": 0}
|
||||
)
|
||||
result = {"ops": [op]}
|
||||
stage = "reconcile"
|
||||
elif prompt.startswith(FACT_RECONCILE):
|
||||
result = {"ops": []}
|
||||
stage = "fact_reconcile"
|
||||
else:
|
||||
raise AssertionError(f"unexpected native prompt: {prompt[:80]!r}")
|
||||
self.stages.append(stage)
|
||||
response = httpx.Response(
|
||||
200,
|
||||
json={
|
||||
"id": "synthetic-coding-response",
|
||||
"object": "chat.completion",
|
||||
"created": 1,
|
||||
"model": body["model"],
|
||||
"choices": [
|
||||
{
|
||||
"index": 0,
|
||||
"finish_reason": "stop",
|
||||
"message": {"role": "assistant", "content": json.dumps(result)},
|
||||
}
|
||||
],
|
||||
"usage": {"prompt_tokens": 1, "completion_tokens": 1, "total_tokens": 2},
|
||||
},
|
||||
)
|
||||
if self.after_response is not None and self.after_response(stage):
|
||||
self.revoked_at = len(self.requests)
|
||||
return response
|
||||
|
||||
|
||||
def proof_ids(supervisor):
|
||||
with sqlite3.connect(supervisor.data_dir / "one-memory.sqlite3") as db:
|
||||
return {row[0] for row in db.execute("SELECT node_id FROM hy_node_permissions")}
|
||||
|
||||
|
||||
async def run_coding_turn(supervisor, seed, oracle):
|
||||
# Renewed consent also re-queues earlier turns; run only the seed's turn.
|
||||
(task,) = (
|
||||
task
|
||||
for task in supervisor.prepare_conversation(seed)
|
||||
if seed.evidence_id in {ref["evidence_id"] for ref in supervisor.tasks.inputs(task.task_id)}
|
||||
)
|
||||
bind_oracle(supervisor, oracle)
|
||||
result = await supervisor._create_runner().run(task.task_id)
|
||||
assert not oracle.errors, oracle.errors
|
||||
return task, result
|
||||
|
||||
|
||||
def by_kind(supervisor):
|
||||
memories = supervisor.governance.list(user_id="local")
|
||||
grouped = {}
|
||||
for memory in memories:
|
||||
grouped.setdefault(memory.kind, []).append(memory)
|
||||
return grouped
|
||||
|
||||
|
||||
def test_hy_routes_a_tool_turn_to_coding_and_both_layers_are_kept(tmp_path):
|
||||
async def scenario():
|
||||
async with raw_client(tmp_path, tool=TOOL_TEXT) as (supervisor, records, _):
|
||||
oracle = CodingOracle(records)
|
||||
task, result = await run_coding_turn(supervisor, records[0], oracle)
|
||||
assert result.status == "completed", (result.last_error_code, oracle.stages)
|
||||
assert oracle.stages[:2] == ["judge", "extract"]
|
||||
assert "reconcile" in oracle.stages
|
||||
kinds = by_kind(supervisor)
|
||||
assert set(kinds) == {MemoryKind.CODING, MemoryKind.WORKING}
|
||||
(coding,), (working,) = kinds[MemoryKind.CODING], kinds[MemoryKind.WORKING]
|
||||
assert coding.content == f"{TASK}\n{SOLUTION}"
|
||||
assert working.content == STATEMENT
|
||||
every_record = {record.evidence_id for record in records}
|
||||
for memory in (coding, working):
|
||||
assert set(memory.evidence_ids) == every_record
|
||||
assert memory.status is MemoryStatus.CONFIRMED
|
||||
outputs = supervisor.tasks.outputs(task.task_id)
|
||||
assert {output.layer for output in outputs} == {"coding_experience", "working_fact"}
|
||||
assert {output.hy_memory_id for output in outputs} <= proof_ids(supervisor)
|
||||
# HY's pipeline log copies every coding prompt it is handed a cache for.
|
||||
(log_dir,) = tmp_path.rglob("hy-memory/*/logs")
|
||||
for path in log_dir.rglob("*"):
|
||||
if path.is_file():
|
||||
for text in (records[0].verbatim, TASK):
|
||||
assert text.encode() not in path.read_bytes(), (path.name, text)
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_an_unapproved_experience_never_reaches_the_model_or_changes(tmp_path):
|
||||
async def scenario():
|
||||
async with raw_client(tmp_path, tool=TOOL_TEXT) as (supervisor, records, _):
|
||||
task = supervisor.prepare_conversation(records[0])[0]
|
||||
_, request, _ = supervisor._create_runner()._prepare_permission_lease(task)
|
||||
client = await native_client(supervisor, task)
|
||||
|
||||
async def seed():
|
||||
await client._get_coding_writer()
|
||||
store = client._coding_store
|
||||
native_embeddings = store.embed_service
|
||||
store.embed_service = FixedEmbeddings()
|
||||
try:
|
||||
await store.insert(
|
||||
CodingMemory(
|
||||
experience_id=DENIED_EXPERIENCE,
|
||||
user_id=request.user_id,
|
||||
agent_id=request.agent_id,
|
||||
session_id=request.session_id,
|
||||
task=DENIED_TASK,
|
||||
solution=DENIED_SOLUTION,
|
||||
)
|
||||
)
|
||||
finally:
|
||||
store.embed_service = native_embeddings
|
||||
found = await store.search_by_query_embedding(
|
||||
coding_vector(), user_id=request.user_id
|
||||
)
|
||||
assert [hit["experience_id"] for hit in found] == [DENIED_EXPERIENCE]
|
||||
return (await store.get_by_id(DENIED_EXPERIENCE)).to_dict()
|
||||
|
||||
before = await on_native_loop(client, seed())
|
||||
oracle = CodingOracle(
|
||||
records, forbidden=(DENIED_EXPERIENCE, DENIED_TASK, DENIED_SOLUTION)
|
||||
)
|
||||
bind_oracle(supervisor, oracle)
|
||||
result = await supervisor._create_runner().run(task.task_id)
|
||||
assert not oracle.errors, oracle.errors
|
||||
assert result.status == "completed", (result.last_error_code, oracle.stages)
|
||||
assert oracle.reconcile_prompts
|
||||
after = await on_native_loop(client, client._coding_store.get_by_id(DENIED_EXPERIENCE))
|
||||
assert after.to_dict() == before
|
||||
assert DENIED_EXPERIENCE not in proof_ids(supervisor)
|
||||
contents = {memory.content for memory in supervisor.governance.list(user_id="local")}
|
||||
assert contents == {f"{TASK}\n{SOLUTION}", STATEMENT}
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_a_later_turn_updating_an_experience_updates_the_same_memory(tmp_path):
|
||||
async def scenario():
|
||||
async with raw_client(tmp_path, tool=TOOL_TEXT) as (supervisor, records, permission):
|
||||
_, first = await run_coding_turn(supervisor, records[0], CodingOracle(records))
|
||||
assert first.status == "completed", first.last_error_code
|
||||
(original,) = by_kind(supervisor)[MemoryKind.CODING]
|
||||
question = put_turn(
|
||||
supervisor.evidence, "user-again", "user", "Run it again verbosely.", 4
|
||||
)
|
||||
tool = put_turn(
|
||||
supervisor.evidence, "tool-again", "tool", "[Bash] pytest -v\n12 passed", 5
|
||||
)
|
||||
everything = (*records, question, tool)
|
||||
grant_records(supervisor, everything, permission.revision)
|
||||
verbose = "Run `pytest -v` from the repository root for per-test output."
|
||||
oracle = CodingOracle(
|
||||
(question, tool),
|
||||
solution=verbose,
|
||||
statement="The agent re-ran the suite verbosely; 12 passed.",
|
||||
update_target=original.memory_id,
|
||||
)
|
||||
_, second = await run_coding_turn(supervisor, question, oracle)
|
||||
assert second.status == "completed", (second.last_error_code, oracle.stages)
|
||||
assert original.memory_id in oracle.reconcile_prompts[0]
|
||||
kinds = by_kind(supervisor)
|
||||
(updated,) = kinds[MemoryKind.CODING]
|
||||
assert updated.memory_id == original.memory_id
|
||||
assert updated.content == f"{TASK}\n{verbose}"
|
||||
assert updated.revision > original.revision
|
||||
assert {question.evidence_id, tool.evidence_id} <= set(updated.evidence_ids)
|
||||
assert len(kinds[MemoryKind.WORKING]) == 2
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_revoking_during_coding_extraction_stops_the_route(tmp_path):
|
||||
async def scenario():
|
||||
async with raw_client(tmp_path, tool=TOOL_TEXT) as (supervisor, records, permission):
|
||||
|
||||
def revoke(stage):
|
||||
if stage != "extract":
|
||||
return False
|
||||
supervisor.model_content_permissions.revoke(
|
||||
user_id="local", expected_revision=permission.revision, user_confirmed=True
|
||||
)
|
||||
return True
|
||||
|
||||
oracle = CodingOracle(records, after_response=revoke)
|
||||
task, result = await run_coding_turn(supervisor, records[0], oracle)
|
||||
assert result.status == "cancelled"
|
||||
assert result.last_error_code == "model_content_permission_revoked", result
|
||||
assert oracle.revoked_at is not None and len(oracle.requests) == oracle.revoked_at
|
||||
assert not supervisor.governance.list(user_id="local")
|
||||
assert not supervisor.tasks.outputs(task.task_id)
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_exposed_coding_config_is_a_redacted_projection_not_the_native_handle(tmp_path):
|
||||
"""Public SDK 2.0.1 reads `config` off the coding store; it must not hand out secrets.
|
||||
|
||||
The wrapper answers `config` so the SDK's routing works, but the native
|
||||
MemoryConfig is a live backend handle carrying api keys and passwords. This
|
||||
asserts callers get a plain projection: not the native object, and no
|
||||
credential value survives anywhere inside it.
|
||||
"""
|
||||
|
||||
async def scenario():
|
||||
async with raw_client(tmp_path, tool=TOOL_TEXT) as (supervisor, records, _):
|
||||
task = supervisor.prepare_conversation(records[0])[0]
|
||||
client = await native_client(supervisor, task)
|
||||
await on_native_loop(client, client._get_coding_writer())
|
||||
return client._coding_store
|
||||
|
||||
native = asyncio.run(scenario())
|
||||
from one_memory.hy_permission_coding_store import HyPermissionCodingStore
|
||||
|
||||
native.config.llm.api_key = "DENIED_LLM_KEY_2f31"
|
||||
native.config.embedder.api_key = "DENIED_EMBED_KEY_9c04"
|
||||
native.config.graph_store.password = "DENIED_GRAPH_PW_77aa"
|
||||
store = HyPermissionCodingStore(
|
||||
native,
|
||||
Lease(),
|
||||
agent_id="agent",
|
||||
session_id="session",
|
||||
)
|
||||
|
||||
exposed = store.config
|
||||
assert exposed is not native.config
|
||||
assert not isinstance(exposed, type(native.config))
|
||||
# Attribute access still works: that is how the public SDK reads it.
|
||||
assert isinstance(exposed["llm"], Mapping)
|
||||
assert exposed["llm"]["model"] == native.config.llm.model
|
||||
assert exposed.llm.model == native.config.llm.model
|
||||
|
||||
def contains_secret(value, depth=0):
|
||||
if depth > 5:
|
||||
return False
|
||||
if isinstance(value, str):
|
||||
return "DENIED_" in value
|
||||
if isinstance(value, Mapping):
|
||||
return any(contains_secret(item, depth + 1) for item in value.values())
|
||||
if isinstance(value, (list, tuple)):
|
||||
return any(contains_secret(item, depth + 1) for item in value)
|
||||
return False
|
||||
|
||||
assert not contains_secret(exposed)
|
||||
assert exposed["llm"]["api_key"] is None
|
||||
assert exposed["embedder"]["api_key"] is None
|
||||
assert exposed["graph_store"]["password"] is None
|
||||
# No native handle, connection or callable escapes the projection.
|
||||
assert not hasattr(exposed, "_native")
|
||||
assert "native" not in exposed
|
||||
@@ -1,250 +0,0 @@
|
||||
"""Blocked lease checks, local failing doubles only; no network or credentials."""
|
||||
|
||||
import asyncio
|
||||
import threading
|
||||
from contextlib import asynccontextmanager
|
||||
from types import SimpleNamespace
|
||||
from unittest.mock import AsyncMock
|
||||
|
||||
import pytest
|
||||
|
||||
from one_memory.hy_memory_bridge import HyMemoryBridge, HyMemoryBridgeError
|
||||
from one_memory.hy_permission_context import _LeasedBridge, permission_bridge_factory
|
||||
from one_memory.hy_permission_transport import HyPermissionTransportError
|
||||
|
||||
|
||||
class ControlledLease:
|
||||
def __init__(self, *, blocked_call=1, error=None):
|
||||
self.receipt = SimpleNamespace(user_id="local")
|
||||
self.profile = object()
|
||||
self.task_id = "synthetic-task"
|
||||
self.records = (SimpleNamespace(evidence_id="synthetic-evidence"),)
|
||||
self.blocked_call = blocked_call
|
||||
self.error = error
|
||||
self.calls = []
|
||||
self.entered = threading.Event()
|
||||
self.release = threading.Event()
|
||||
self.exited = threading.Event()
|
||||
self.revoked = False
|
||||
|
||||
def recheck(self):
|
||||
self.calls.append(threading.get_ident())
|
||||
if len(self.calls) == self.blocked_call:
|
||||
self.entered.set()
|
||||
try:
|
||||
if not self.release.wait(2):
|
||||
raise HyMemoryBridgeError("test_check_was_not_released")
|
||||
if self.revoked:
|
||||
raise HyMemoryBridgeError("model_content_permission_revoked")
|
||||
if self.error is not None:
|
||||
raise self.error
|
||||
finally:
|
||||
self.exited.set()
|
||||
return self.receipt
|
||||
|
||||
|
||||
class ControlledLoop:
|
||||
def __init__(self):
|
||||
self.calls = []
|
||||
|
||||
def run(self, coroutine):
|
||||
self.calls.append(threading.get_ident())
|
||||
return asyncio.run(coroutine)
|
||||
|
||||
|
||||
class ControlledWiring:
|
||||
def __init__(self, *, error=None):
|
||||
self.client = SimpleNamespace(_loop_thread=ControlledLoop())
|
||||
self.error = error
|
||||
self.finished = 0
|
||||
self.closed = 0
|
||||
|
||||
async def finish(self):
|
||||
self.finished += 1
|
||||
if self.error is not None:
|
||||
raise self.error
|
||||
return {}
|
||||
|
||||
async def exit(self):
|
||||
self.closed += 1
|
||||
|
||||
|
||||
REQUEST = SimpleNamespace(
|
||||
user_id="local",
|
||||
evidence_ids=("synthetic-evidence",),
|
||||
agent_id="agent",
|
||||
session_id="session",
|
||||
request_id="synthetic-request",
|
||||
)
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
async def blocked_operation(lease, operation):
|
||||
main_thread = threading.get_ident()
|
||||
pending = asyncio.create_task(operation())
|
||||
try:
|
||||
# A timer on this loop must run while the synchronous check is held.
|
||||
async with asyncio.timeout(0.5):
|
||||
while not lease.entered.is_set():
|
||||
await asyncio.sleep(0.005)
|
||||
ticked = asyncio.Event()
|
||||
asyncio.get_running_loop().call_later(0.02, ticked.set)
|
||||
await ticked.wait()
|
||||
assert not lease.exited.is_set(), "lease check blocked the main event loop"
|
||||
assert main_thread not in lease.calls, "lease check executed on the main thread"
|
||||
yield pending
|
||||
finally:
|
||||
lease.release.set()
|
||||
if not pending.done():
|
||||
pending.cancel()
|
||||
await asyncio.gather(pending, return_exceptions=True)
|
||||
assert lease.exited.is_set()
|
||||
|
||||
|
||||
@pytest.mark.parametrize("operation", ["ingest", "ingest_extracted"])
|
||||
def test_cancelled_start_joins_check_without_later_sdk_entry(operation):
|
||||
async def scenario():
|
||||
lease = ControlledLease()
|
||||
bridge = SimpleNamespace(
|
||||
_get_client=AsyncMock(), ingest=AsyncMock(), ingest_extracted=AsyncMock()
|
||||
)
|
||||
bound = _LeasedBridge(bridge, lease, object(), None)
|
||||
|
||||
async def start():
|
||||
try:
|
||||
if operation == "ingest":
|
||||
await bound.ingest(REQUEST)
|
||||
else:
|
||||
await bound.ingest_extracted(REQUEST, ("Synthetic original",))
|
||||
finally:
|
||||
await bound.close()
|
||||
|
||||
async with blocked_operation(lease, start) as pending:
|
||||
pending.cancel()
|
||||
await asyncio.sleep(0.02)
|
||||
pending.cancel()
|
||||
await asyncio.sleep(0.02)
|
||||
assert not pending.done(), "cancel abandoned the active lease check"
|
||||
assert not bridge._get_client.called
|
||||
lease.release.set()
|
||||
with pytest.raises(asyncio.CancelledError):
|
||||
await pending
|
||||
assert not bridge._get_client.called
|
||||
assert not bridge.ingest.called and not bridge.ingest_extracted.called
|
||||
assert bound.wiring is None
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_revoked_start_preserves_error_and_never_opens_sdk():
|
||||
async def scenario():
|
||||
lease = ControlledLease()
|
||||
bridge = SimpleNamespace(_get_client=AsyncMock())
|
||||
bound = _LeasedBridge(bridge, lease, object(), None)
|
||||
async with blocked_operation(lease, lambda: bound._start(REQUEST)) as pending:
|
||||
lease.revoked = True
|
||||
lease.release.set()
|
||||
with pytest.raises(HyMemoryBridgeError, match="^model_content_permission_revoked$"):
|
||||
await pending
|
||||
await bound.close()
|
||||
assert not bridge._get_client.called and bound.wiring is None
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_released_start_reaches_client_once_and_preserves_client_error():
|
||||
async def scenario():
|
||||
lease = ControlledLease()
|
||||
expected = HyMemoryBridgeError("hy_memory_init_failed")
|
||||
bridge = SimpleNamespace(_get_client=AsyncMock(side_effect=expected))
|
||||
bound = _LeasedBridge(bridge, lease, object(), None)
|
||||
async with blocked_operation(lease, lambda: bound._start(REQUEST)) as pending:
|
||||
assert not bridge._get_client.called
|
||||
lease.release.set()
|
||||
with pytest.raises(HyMemoryBridgeError) as raised:
|
||||
await pending
|
||||
assert raised.value is expected
|
||||
bridge._get_client.assert_awaited_once_with()
|
||||
await bound.close()
|
||||
assert len(lease.calls) == 1 and bound.wiring is None
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
@pytest.mark.parametrize("transport_failure", [False, True])
|
||||
@pytest.mark.parametrize("cancel", [False, True])
|
||||
def test_postcheck_and_transport_error_recheck_are_joined(transport_failure, cancel):
|
||||
async def scenario():
|
||||
expected = HyMemoryBridgeError("model_content_evidence_changed")
|
||||
lease = ControlledLease(blocked_call=2 if transport_failure else 1, error=expected)
|
||||
bound = _LeasedBridge(object(), lease, object(), None)
|
||||
wiring = ControlledWiring(
|
||||
error=HyPermissionTransportError("controlled_transport_failure")
|
||||
if transport_failure
|
||||
else None
|
||||
)
|
||||
bound.wiring = wiring
|
||||
|
||||
async def checked():
|
||||
try:
|
||||
await bound._check()
|
||||
finally:
|
||||
await bound.close()
|
||||
|
||||
async with blocked_operation(lease, checked) as pending:
|
||||
if cancel:
|
||||
pending.cancel()
|
||||
await asyncio.sleep(0.02)
|
||||
assert not pending.done() and wiring.closed == 0
|
||||
lease.release.set()
|
||||
if cancel:
|
||||
with pytest.raises(asyncio.CancelledError):
|
||||
await pending
|
||||
else:
|
||||
with pytest.raises(HyMemoryBridgeError) as raised:
|
||||
await pending
|
||||
assert raised.value is expected
|
||||
assert wiring.finished == int(transport_failure)
|
||||
assert wiring.closed == 1 and bound.finished
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
@pytest.mark.parametrize("revoked", [False, True])
|
||||
@pytest.mark.parametrize("cancel", [False, True])
|
||||
def test_factory_error_check_holds_exclusive_access_until_joined(tmp_path, revoked, cancel):
|
||||
async def scenario():
|
||||
lease = ControlledLease()
|
||||
bridge = HyMemoryBridge(
|
||||
profile=lease.profile,
|
||||
api_key=None,
|
||||
data_dir=tmp_path,
|
||||
client=SimpleNamespace(close=lambda: None),
|
||||
)
|
||||
factory = permission_bridge_factory(lambda *_: bridge)
|
||||
expected = HyPermissionTransportError("controlled_transport_failure")
|
||||
|
||||
async def failed_context():
|
||||
async with factory(lease, None, tmp_path):
|
||||
raise expected
|
||||
|
||||
async with blocked_operation(lease, failed_context) as pending:
|
||||
assert bridge._access_lock.locked()
|
||||
lease.revoked = revoked
|
||||
if cancel:
|
||||
pending.cancel()
|
||||
await asyncio.sleep(0.02)
|
||||
assert not pending.done() and bridge._access_lock.locked()
|
||||
lease.release.set()
|
||||
if cancel:
|
||||
with pytest.raises(asyncio.CancelledError):
|
||||
await pending
|
||||
else:
|
||||
code = "model_content_permission_revoked" if revoked else expected.code
|
||||
with pytest.raises(HyMemoryBridgeError, match=f"^{code}$"):
|
||||
await pending
|
||||
assert bridge._access_owner is None and not bridge._access_lock.locked()
|
||||
await bridge.close()
|
||||
assert bridge._client is None
|
||||
|
||||
asyncio.run(scenario())
|
||||
@@ -1,145 +0,0 @@
|
||||
"""HY decides deletions; the ledger mirrors them. Pinned SDK, scripted HTTP."""
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
|
||||
import httpx
|
||||
from hy_memory.models.memory import MemoryStatus as HyStatus
|
||||
|
||||
from one_memory.memory_governance import DeletionStatus, MemoryKind, MemoryStatus
|
||||
from tests.test_hy_coding_lease import (
|
||||
TOOL_TEXT,
|
||||
CodingOracle,
|
||||
by_kind,
|
||||
run_coding_turn,
|
||||
)
|
||||
from tests.test_hy_permission_raw_context import (
|
||||
AGENT_TEXT,
|
||||
USER_TEXT,
|
||||
ScriptedOracle,
|
||||
assert_outputs,
|
||||
bind_oracle,
|
||||
grant_records,
|
||||
native_client,
|
||||
on_native_loop,
|
||||
put_turn,
|
||||
raw_client,
|
||||
)
|
||||
|
||||
|
||||
class DeletingOracle(ScriptedOracle):
|
||||
"""HY's reconciler withdraws the user's claim and keeps the agent's."""
|
||||
|
||||
def respond(self, request):
|
||||
response = super().respond(request)
|
||||
if self.stages[-1] != "reconcile":
|
||||
return response
|
||||
body = json.loads(response.content)
|
||||
ops = json.loads(body["choices"][0]["message"]["content"])
|
||||
for op in ops:
|
||||
if op["content"] == USER_TEXT and "memory_id" in op:
|
||||
op.clear()
|
||||
op.update({"op": "DELETE", "memory_id": self.withdrawn})
|
||||
body["choices"][0]["message"]["content"] = json.dumps(ops)
|
||||
return httpx.Response(200, json=body)
|
||||
|
||||
|
||||
def test_a_memory_hy_deletes_is_deleted_in_the_ledger_and_the_rest_stay(tmp_path):
|
||||
async def scenario():
|
||||
async with raw_client(tmp_path) as (supervisor, records, permission):
|
||||
first = supervisor.prepare_conversation(records[0])[0]
|
||||
client = await native_client(supervisor, first)
|
||||
bind_oracle(supervisor, ScriptedOracle(records))
|
||||
assert (await supervisor._create_runner().run(first.task_id)).status == "completed"
|
||||
await assert_outputs(supervisor, first, records, client)
|
||||
by_content = {m.content: m for m in supervisor.governance.list(user_id="local")}
|
||||
withdrawn, kept = by_content[USER_TEXT], by_content[AGENT_TEXT]
|
||||
|
||||
added = put_turn(supervisor.evidence, "user-withdraw", "user", USER_TEXT, 3)
|
||||
records = (*records, added)
|
||||
grant_records(supervisor, records, permission.revision)
|
||||
second = supervisor.prepare_conversation(added)[0]
|
||||
oracle = DeletingOracle(records)
|
||||
oracle.withdrawn = withdrawn.memory_id
|
||||
bind_oracle(supervisor, oracle)
|
||||
result = await supervisor._create_runner().run(second.task_id)
|
||||
assert not oracle.errors, oracle.errors
|
||||
assert result.status == "completed", (result.last_error_code, oracle.stages)
|
||||
|
||||
native = await on_native_loop(
|
||||
client, client._vector_store.get_by_id(withdrawn.memory_id)
|
||||
)
|
||||
assert native is not None and native.status == HyStatus.SHADOW
|
||||
mirrored = supervisor.governance.get(withdrawn.memory_id, user_id="local")
|
||||
assert mirrored.status is MemoryStatus.DELETED
|
||||
deletion = supervisor.governance.deletion(withdrawn.memory_id, user_id="local")
|
||||
assert deletion is not None and deletion.status is DeletionStatus.COMPLETED
|
||||
untouched = supervisor.governance.get(kept.memory_id, user_id="local")
|
||||
assert untouched.status is MemoryStatus.CONFIRMED
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
class RetiringCodingOracle(CodingOracle):
|
||||
def __init__(self, records, *, retire, **kwargs):
|
||||
super().__init__(records, **kwargs)
|
||||
self.retire = retire
|
||||
|
||||
def respond(self, request):
|
||||
response = super().respond(request)
|
||||
if self.stages[-1] != "reconcile":
|
||||
return response
|
||||
body = json.loads(response.content)
|
||||
body["choices"][0]["message"]["content"] = json.dumps(
|
||||
{
|
||||
"ops": [
|
||||
{"action": "ADD", "draft_idx": 0},
|
||||
{
|
||||
"action": "DELETE",
|
||||
"draft_idx": 0,
|
||||
"target_experience_id": self.retire,
|
||||
"reason": "the new run explicitly replaces the old procedure",
|
||||
},
|
||||
]
|
||||
}
|
||||
)
|
||||
return httpx.Response(200, json=body)
|
||||
|
||||
|
||||
def test_an_experience_hy_deletes_is_deleted_in_the_ledger(tmp_path):
|
||||
async def scenario():
|
||||
async with raw_client(tmp_path, tool=TOOL_TEXT) as (supervisor, records, permission):
|
||||
_, first = await run_coding_turn(supervisor, records[0], CodingOracle(records))
|
||||
assert first.status == "completed", first.last_error_code
|
||||
(original,) = by_kind(supervisor)[MemoryKind.CODING]
|
||||
question = put_turn(supervisor.evidence, "user-again", "user", "Use tox now.", 4)
|
||||
tool = put_turn(supervisor.evidence, "tool-again", "tool", "[Bash] tox\nok", 5)
|
||||
grant_records(supervisor, (*records, question, tool), permission.revision)
|
||||
oracle = RetiringCodingOracle(
|
||||
(question, tool),
|
||||
retire=original.memory_id,
|
||||
solution="Run `tox` from the repository root.",
|
||||
statement="The agent switched the synthetic suite to tox.",
|
||||
)
|
||||
_, second = await run_coding_turn(supervisor, question, oracle)
|
||||
assert second.status == "completed", (second.last_error_code, oracle.stages)
|
||||
client = await native_client(supervisor, second)
|
||||
assert (
|
||||
await on_native_loop(client, client._coding_store.get_by_id(original.memory_id))
|
||||
is None
|
||||
)
|
||||
|
||||
assert (
|
||||
supervisor.governance.get(original.memory_id, user_id="local").status
|
||||
is MemoryStatus.DELETED
|
||||
)
|
||||
live = [
|
||||
m
|
||||
for m in by_kind(supervisor)[MemoryKind.CODING]
|
||||
if m.status is MemoryStatus.CONFIRMED
|
||||
]
|
||||
assert [m.content for m in live] == [
|
||||
"Run the synthetic test suite\nRun `tox` from the repository root."
|
||||
]
|
||||
|
||||
asyncio.run(scenario())
|
||||
@@ -18,10 +18,12 @@ from one_memory.hy_memory_bridge import (
|
||||
HyMemoryBridge,
|
||||
HyMemoryBridgeError,
|
||||
HyMemoryIngestRequest,
|
||||
HyMemoryIngestResult,
|
||||
HyMemoryTurn,
|
||||
UnsupportedHyMemoryProviderError,
|
||||
_basic_info_memories,
|
||||
_collection_name,
|
||||
_with_native_mutation_delta,
|
||||
build_hy_memory_config,
|
||||
configure_hy_memory_environment,
|
||||
reattach_legacy_vector_collection,
|
||||
@@ -1324,7 +1326,7 @@ def test_hy_runtime_matches_public_sdk_without_reranker_or_service_backends(
|
||||
config = build_hy_memory_config(profile, api_key=None, data_dir=tmp_path)
|
||||
|
||||
assert HY_MEMORY_DISTRIBUTION == "hy-memory"
|
||||
assert HY_MEMORY_VERSION == "2.0.1"
|
||||
assert HY_MEMORY_VERSION == "2.0.2"
|
||||
assert HY_MEMORY_MODE == "ultra"
|
||||
assert HY_MEMORY_READER == "hybrid"
|
||||
assert os.environ["MEMORY_MODE"] == "ultra"
|
||||
@@ -1507,3 +1509,58 @@ def test_the_client_mode_follows_the_graph_store_switch(tmp_path, monkeypatch) -
|
||||
hy_memory_bridge.create_hy_memory_client(profile, api_key=None, data_dir=tmp_path).close()
|
||||
|
||||
assert seen == ["pro", HY_MEMORY_MODE]
|
||||
|
||||
|
||||
def test_native_mutation_delta_reports_coding_deletes_and_supersedes() -> None:
|
||||
raw = {
|
||||
"ops": [
|
||||
{
|
||||
"action": "DELETE",
|
||||
"target_experience_id": "old-experience",
|
||||
}
|
||||
],
|
||||
"fact_ops": [
|
||||
{
|
||||
"action": "SUPERSEDE",
|
||||
"fact_id": "old-fact",
|
||||
"superseded_by": "new-fact",
|
||||
}
|
||||
],
|
||||
}
|
||||
|
||||
result = asyncio.run(
|
||||
_with_native_mutation_delta(
|
||||
SimpleNamespace(),
|
||||
HyMemoryIngestResult(success=True),
|
||||
raw,
|
||||
(),
|
||||
)
|
||||
)
|
||||
|
||||
assert result.retired_memory_ids == ("old-experience", "old-fact")
|
||||
assert result.superseded_memory_ids == {"old-fact": ("new-fact",)}
|
||||
|
||||
|
||||
def test_native_mutation_delta_reads_chat_status_from_the_sdk() -> None:
|
||||
class Client:
|
||||
def get(self, memory_id: str) -> Mapping[str, object]:
|
||||
assert memory_id == "old-chat"
|
||||
return {
|
||||
"memory_id": memory_id,
|
||||
"content": "old",
|
||||
"status": "superseded",
|
||||
"superseded_by": ["new-chat"],
|
||||
}
|
||||
|
||||
guard = SimpleNamespace(mutation_target_ids={"old-chat"})
|
||||
result = asyncio.run(
|
||||
_with_native_mutation_delta(
|
||||
Client(),
|
||||
HyMemoryIngestResult(success=True),
|
||||
{},
|
||||
(guard,),
|
||||
)
|
||||
)
|
||||
|
||||
assert result.retired_memory_ids == ("old-chat",)
|
||||
assert result.superseded_memory_ids == {"old-chat": ("new-chat",)}
|
||||
@@ -1,208 +0,0 @@
|
||||
"""HY writes with its own prompts and judgment; the product adds no audit of its own."""
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
from types import SimpleNamespace
|
||||
|
||||
import pytest
|
||||
from hy_memory.agent.reconciler import ReconcileOp, ReconcileResult
|
||||
|
||||
from one_memory.hy_response_guard import HyReconcileGuard
|
||||
from one_memory.hy_source_context import HySourceContext, HySupportedMemory, HyWriteBindings
|
||||
from one_memory.hy_write_verification import HyExtractionGuard
|
||||
from one_memory.imports.storage import EvidenceStore
|
||||
from tests.test_hy_permission_raw_context import (
|
||||
ScriptedOracle,
|
||||
assert_outputs,
|
||||
bind_oracle,
|
||||
native_client,
|
||||
raw_client,
|
||||
)
|
||||
from tests.test_hy_source_context import source
|
||||
|
||||
PRODUCT_RULE_MARKERS = ("# 来源与语气保真", "# 原始发言主体规则", "owner记录命题的陈述权威")
|
||||
|
||||
|
||||
def test_a_raw_turn_reaches_hy_with_its_native_prompts_and_no_product_audit(tmp_path):
|
||||
async def scenario():
|
||||
async with raw_client(tmp_path) as (supervisor, records, _):
|
||||
task = supervisor.prepare_conversation(records[0])[0]
|
||||
oracle = ScriptedOracle(records)
|
||||
bind_oracle(supervisor, oracle)
|
||||
result = await supervisor._create_runner().run(task.task_id)
|
||||
assert not oracle.errors, oracle.errors
|
||||
assert result.status == "completed", (result.last_error_code, oracle.stages)
|
||||
assert "extract" in oracle.stages
|
||||
assert "review" not in oracle.stages
|
||||
assert "citations" not in oracle.stages
|
||||
sent = json.dumps([body for _path, body in oracle.requests], ensure_ascii=False)
|
||||
for marker in PRODUCT_RULE_MARKERS:
|
||||
assert marker not in sent, marker
|
||||
# Each memory still points at the words of its own author in this turn.
|
||||
client = await native_client(supervisor, task)
|
||||
await assert_outputs(supervisor, task, records, client)
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_an_imported_memory_keeps_the_owner_hy_gives_it():
|
||||
class Native:
|
||||
received = None
|
||||
|
||||
async def reconcile(self, **kwargs):
|
||||
self.received = kwargs["new_memories_with_meta"]
|
||||
return ReconcileResult(
|
||||
ops=[ReconcileOp(op="ADD", content="用户偏好简洁表达。", owner="user")]
|
||||
)
|
||||
|
||||
native = Native()
|
||||
guard = HyReconcileGuard(native)
|
||||
guard.imported_contents = frozenset({"用户偏好简洁表达。"})
|
||||
result = asyncio.run(
|
||||
guard.reconcile(
|
||||
new_memories=["用户偏好简洁表达。"],
|
||||
new_memories_with_meta=[{"content": "用户偏好简洁表达。", "layer": "L2_FACT"}],
|
||||
)
|
||||
)
|
||||
assert "owner" not in native.received[0]
|
||||
assert result.ops[0].owner == "user"
|
||||
assert guard.failures == []
|
||||
|
||||
|
||||
def bound(tmp_path, *, known=False):
|
||||
store = EvidenceStore(tmp_path / "evidence.db")
|
||||
old = source(store, "old", "user", "Atlas用本地存储。")
|
||||
current = source(store, "now", "user", "Atlas也要能离线同步。")
|
||||
memories = (
|
||||
HySupportedMemory(
|
||||
hy_memory_id="prior",
|
||||
governed_memory_id="prior",
|
||||
revision=1,
|
||||
content="用户要求Atlas使用本地存储。",
|
||||
owner="user",
|
||||
evidence_ids=(old.evidence_id,),
|
||||
),
|
||||
)
|
||||
context = HySourceContext(
|
||||
current=(current,), history=(old,), known_memories=memories if known else ()
|
||||
)
|
||||
return HyWriteBindings(context), old, current
|
||||
|
||||
|
||||
def test_the_turn_hy_extracts_must_be_the_turn_that_was_sent(tmp_path):
|
||||
bindings, _old, current = bound(tmp_path)
|
||||
calls = []
|
||||
|
||||
class Native:
|
||||
async def extract(self, content, context, **kwargs):
|
||||
calls.append(content)
|
||||
return SimpleNamespace(success=True, info={"memory": []})
|
||||
|
||||
guard = HyExtractionGuard(Native())
|
||||
guard.bindings = bindings
|
||||
sent = json.dumps([{"role": "user", "content": current.verbatim}], ensure_ascii=False)
|
||||
assert asyncio.run(guard.extract(sent, {})).success and calls == [sent]
|
||||
with pytest.raises(ValueError, match="input_mismatch"):
|
||||
asyncio.run(guard.extract(json.dumps([{"role": "user", "content": "别的内容"}]), {}))
|
||||
assert calls == [sent]
|
||||
assert guard.failures == ["hy_memory_write_verification_input_mismatch"]
|
||||
|
||||
|
||||
class PriorStore:
|
||||
async def get_by_id(self, key):
|
||||
return SimpleNamespace(
|
||||
user_id="local", agent_id="atlas", owner="user", content="用户要求Atlas使用本地存储。"
|
||||
)
|
||||
|
||||
|
||||
def merge_into_prior(content):
|
||||
class Native:
|
||||
async def reconcile(self, **kwargs):
|
||||
return ReconcileResult(
|
||||
ops=[ReconcileOp(op="UPDATE", memory_id="prior", owner="user", content=content)]
|
||||
)
|
||||
|
||||
return Native()
|
||||
|
||||
|
||||
ARGS = {"user_id": "local", "agent_id": "atlas", "vector_store": PriorStore()}
|
||||
|
||||
|
||||
def test_hy_may_only_change_a_memory_this_write_was_allowed_to_see(tmp_path):
|
||||
allowed, _old, _current = bound(tmp_path / "allowed", known=True)
|
||||
guard = HyReconcileGuard(merge_into_prior("用户要求Atlas本地存储并支持离线同步。"))
|
||||
guard.bindings = allowed
|
||||
assert asyncio.run(guard.reconcile(**ARGS)).success
|
||||
|
||||
unseen, _old, _current = bound(tmp_path / "unseen")
|
||||
guard = HyReconcileGuard(merge_into_prior("用户要求Atlas本地存储并支持离线同步。"))
|
||||
guard.bindings = unseen
|
||||
with pytest.raises(ValueError, match="target_unbound"):
|
||||
asyncio.run(guard.reconcile(**ARGS))
|
||||
|
||||
|
||||
def test_a_memory_hy_merges_cites_this_turn_and_what_it_already_cited(tmp_path):
|
||||
bindings, old, current = bound(tmp_path, known=True)
|
||||
merged = "用户要求Atlas本地存储并支持离线同步。"
|
||||
guard = HyReconcileGuard(merge_into_prior(merged))
|
||||
guard.bindings = bindings
|
||||
assert asyncio.run(guard.reconcile(**ARGS)).success
|
||||
assert bindings.support_for(merged, "user") == (current.evidence_id, old.evidence_id)
|
||||
|
||||
|
||||
HY_USER_ONLY_LINE = "只提取 NEW user messages 中首次出现的新信息。"
|
||||
|
||||
|
||||
def _section(prompt, heading):
|
||||
_, found, rest = prompt.partition(heading)
|
||||
assert found, heading
|
||||
return rest.partition("\n---\n")[0]
|
||||
|
||||
|
||||
def _served_extract_prompts(tmp_path):
|
||||
from one_memory.hy_memory_bridge import build_hy_memory_config
|
||||
from tests.test_memory_language import _profile
|
||||
|
||||
config = build_hy_memory_config(_profile(), api_key=None, data_dir=tmp_path)
|
||||
return config.llm.extract_system_prompt_zh, config.llm.extract_system_prompt_en
|
||||
|
||||
|
||||
def test_hys_native_extraction_only_reads_what_the_user_said():
|
||||
from hy_memory.agent.prompts_zh import EXTRACT_SYSTEM_PROMPT_ZH
|
||||
|
||||
assert HY_USER_ONLY_LINE in EXTRACT_SYSTEM_PROMPT_ZH
|
||||
# HY already knows how to keep an assistant's words apart from the person's.
|
||||
assert '`"agent"` —— AI 助手' in _section(EXTRACT_SYSTEM_PROMPT_ZH, "# OWNER(归属)\n")
|
||||
|
||||
|
||||
def test_hy_extracts_both_speakers_and_keeps_its_own_authorship_rules(tmp_path):
|
||||
from hy_memory.agent.prompts_zh import EXTRACT_SYSTEM_PROMPT_ZH as native
|
||||
|
||||
for served in _served_extract_prompts(tmp_path):
|
||||
assert HY_USER_ONLY_LINE not in served
|
||||
principle = _section(served, "# 提取原则\n")
|
||||
assert "用户" in principle and "AI 助手" in principle
|
||||
assert "OWNER" in principle
|
||||
for heading in ("# OWNER(归属)\n", "# 主体规则\n"):
|
||||
assert _section(served, heading) == _section(native, heading), heading
|
||||
for marker in PRODUCT_RULE_MARKERS:
|
||||
assert marker not in served, marker
|
||||
|
||||
|
||||
def test_a_reworded_extraction_line_still_asks_for_both_speakers(tmp_path, monkeypatch, caplog):
|
||||
from hy_memory.agent.prompts_zh import EXTRACT_SYSTEM_PROMPT_ZH as native
|
||||
|
||||
from one_memory import hy_memory_bridge
|
||||
|
||||
class Reworded:
|
||||
EXTRACT_SYSTEM_PROMPT_ZH = native.replace(HY_USER_ONLY_LINE, "HY 换了提取原则的措辞。")
|
||||
|
||||
real_import = hy_memory_bridge.import_module
|
||||
monkeypatch.setattr(
|
||||
hy_memory_bridge,
|
||||
"import_module",
|
||||
lambda name: Reworded if name == "hy_memory.agent.prompts_zh" else real_import(name),
|
||||
)
|
||||
served, _ = _served_extract_prompts(tmp_path)
|
||||
assert "AI 助手提供的" in served
|
||||
assert "hy_extraction_scope_rule_not_found" in caplog.text
|
||||
@@ -1,192 +0,0 @@
|
||||
"""Whole native HY client with synthetic content and HTTP MockTransport, never cloud."""
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
from contextlib import contextmanager
|
||||
from pathlib import Path
|
||||
|
||||
import httpx
|
||||
import pytest
|
||||
|
||||
from one_memory.hy_permission_context import permission_bridge_factory
|
||||
from one_memory.imports.grants import SourceGrantStore
|
||||
from one_memory.imports.models import (
|
||||
EvidenceContentKind,
|
||||
EvidenceInput,
|
||||
EvidenceOrigin,
|
||||
OriginMetadata,
|
||||
WrittenBy,
|
||||
)
|
||||
from one_memory.imports.sources.registry import RootCandidateKind
|
||||
from one_memory.imports.storage import EvidenceStore
|
||||
from one_memory.memory_governance import MemoryGovernanceStore
|
||||
from one_memory.memory_processing import MemoryProcessingSupervisor
|
||||
from one_memory.memory_tasks import MemoryTaskStore
|
||||
from one_memory.model_content_permissions import ModelContentPermissionStore
|
||||
from one_memory.model_profiles import (
|
||||
ModelCapabilities,
|
||||
ModelProfileInput,
|
||||
ModelProfileStore,
|
||||
ProviderKind,
|
||||
)
|
||||
from one_memory.secrets import InMemorySecretStore
|
||||
|
||||
|
||||
@contextmanager
|
||||
def whole_client(tmp_path: Path):
|
||||
root = tmp_path / "source"
|
||||
root.mkdir()
|
||||
database = tmp_path / "one-memory.sqlite3"
|
||||
grants = SourceGrantStore(database, home=tmp_path)
|
||||
grants.grant(
|
||||
user_id="local", source_id="fixture", root=root, candidate_kind=RootCandidateKind.DEFAULT
|
||||
)
|
||||
evidence = EvidenceStore(database)
|
||||
text = "Synthetic supplied memory: review permissions before release."
|
||||
record = evidence.put(
|
||||
EvidenceInput(
|
||||
source_id="fixture",
|
||||
conversation_id="session",
|
||||
message_id="1",
|
||||
role="assistant",
|
||||
written_by=WrittenBy.UNKNOWN,
|
||||
content_kind=EvidenceContentKind.MEMORY_CONTENT,
|
||||
verbatim=text,
|
||||
origin_meta={EvidenceOrigin.MANUAL_IMPORT: OriginMetadata()},
|
||||
)
|
||||
).record
|
||||
profiles, secrets = ModelProfileStore(database), InMemorySecretStore()
|
||||
profile = profiles.create(
|
||||
ModelProfileInput(
|
||||
name="Stub only",
|
||||
provider_kind=ProviderKind.OLLAMA,
|
||||
base_url="http://127.0.0.1:1/v1",
|
||||
model="test-generation",
|
||||
embedding_provider_kind=ProviderKind.OLLAMA,
|
||||
embedding_base_url="http://127.0.0.1:2/v1",
|
||||
embedding_model="test-embedding",
|
||||
),
|
||||
secrets,
|
||||
)
|
||||
profiles.record_capabilities(
|
||||
profile.profile_id,
|
||||
expected_revision=profile.revision,
|
||||
capabilities=ModelCapabilities(
|
||||
endpoint_reachable=True, model_available=True, embedding_available=True
|
||||
),
|
||||
secrets=secrets,
|
||||
)
|
||||
profiles.activate(profile.profile_id, secrets)
|
||||
permissions = ModelContentPermissionStore(database)
|
||||
snapshot = permissions.snapshot(
|
||||
user_id="local",
|
||||
source_ids=("fixture",),
|
||||
profile=profiles.snapshot(profile.profile_id, profile.revision),
|
||||
evidence=(record,),
|
||||
)
|
||||
permission = permissions.grant(snapshot, user_confirmed=True, expected_revision=0)
|
||||
supervisor = MemoryProcessingSupervisor(
|
||||
tasks=MemoryTaskStore(database),
|
||||
profiles=profiles,
|
||||
secrets=secrets,
|
||||
evidence=evidence,
|
||||
governance=MemoryGovernanceStore(database),
|
||||
data_dir=tmp_path,
|
||||
model_content_permissions=permissions,
|
||||
)
|
||||
yield supervisor, record, permissions, permission
|
||||
|
||||
|
||||
def test_native_memory_content_uses_guarded_sdk_then_returns_real_native_memory(tmp_path):
|
||||
async def scenario():
|
||||
with whole_client(tmp_path) as (supervisor, record, _, _):
|
||||
sends = []
|
||||
|
||||
def handler(request: httpx.Request):
|
||||
body = json.loads(request.content)
|
||||
sends.append({"path": request.url.path, "body": body})
|
||||
if request.url.path.endswith("embeddings"):
|
||||
return httpx.Response(
|
||||
200,
|
||||
json={
|
||||
"object": "list",
|
||||
"model": "test-embedding",
|
||||
"data": [
|
||||
{
|
||||
"index": i,
|
||||
"object": "embedding",
|
||||
"embedding": [1.0] + [0.0] * 383,
|
||||
}
|
||||
for i, _ in enumerate(body["input"])
|
||||
],
|
||||
"usage": {"prompt_tokens": 1, "total_tokens": 1},
|
||||
},
|
||||
)
|
||||
return httpx.Response(
|
||||
200,
|
||||
json={
|
||||
"id": "fixture-response",
|
||||
"object": "chat.completion",
|
||||
"created": 1,
|
||||
"model": "test-generation",
|
||||
"choices": [
|
||||
{
|
||||
"index": 0,
|
||||
"finish_reason": "stop",
|
||||
"message": {
|
||||
"role": "assistant",
|
||||
"content": json.dumps(
|
||||
[
|
||||
{
|
||||
"op": "ADD",
|
||||
"content": record.verbatim,
|
||||
"layer": "L2_FACT",
|
||||
"owner": "agent",
|
||||
}
|
||||
]
|
||||
),
|
||||
},
|
||||
}
|
||||
],
|
||||
"usage": {"prompt_tokens": 1, "completion_tokens": 1, "total_tokens": 2},
|
||||
},
|
||||
)
|
||||
|
||||
supervisor.permission_bridge_factory = permission_bridge_factory(
|
||||
supervisor._bridge, http_transport=httpx.MockTransport(handler)
|
||||
)
|
||||
tasks = supervisor.prepare_conversation(record)
|
||||
try:
|
||||
result = await supervisor._create_runner().run(tasks[0].task_id)
|
||||
assert result.status == "completed", result.last_error_code
|
||||
assert sends and all(
|
||||
item["path"].endswith(("embeddings", "chat/completions")) for item in sends
|
||||
)
|
||||
memories = supervisor.governance.list(user_id="local")
|
||||
assert len(memories) == 1 and memories[0].content == record.verbatim
|
||||
assert memories[0].status == "confirmed"
|
||||
assert supervisor.evidence.get(record.evidence_id).verbatim == record.verbatim
|
||||
finally:
|
||||
await supervisor.shutdown()
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_revoked_lease_never_creates_an_sdk_client_or_sends(tmp_path):
|
||||
async def scenario():
|
||||
with whole_client(tmp_path) as (supervisor, record, permissions, permission):
|
||||
tasks = supervisor.prepare_conversation(record)
|
||||
permissions.revoke(
|
||||
user_id="local", expected_revision=permission.revision, user_confirmed=True
|
||||
)
|
||||
supervisor.permission_bridge_factory = permission_bridge_factory(
|
||||
supervisor._bridge,
|
||||
http_transport=httpx.MockTransport(lambda _: pytest.fail("unauthorized request")),
|
||||
)
|
||||
result = await supervisor._create_runner().run(tasks[0].task_id)
|
||||
assert result.last_error_code == "model_content_permission_revoked"
|
||||
assert supervisor._bridges == {}
|
||||
assert not supervisor.governance.list(user_id="local")
|
||||
await supervisor.shutdown()
|
||||
|
||||
asyncio.run(scenario())
|
||||
@@ -1,380 +0,0 @@
|
||||
"""Bounded factory races with a temporary native HY client and no model calls."""
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import threading
|
||||
|
||||
import httpx
|
||||
import pytest
|
||||
from hy_memory.models.memory import MemoryLayer, MemoryNode
|
||||
|
||||
from one_memory.hy_memory_bridge import HyMemoryBridgeError
|
||||
from one_memory.hy_permission_coding_store import HyPermissionCodingStore
|
||||
from one_memory.hy_permission_context import _Wiring, permission_bridge_factory
|
||||
from one_memory.hy_permission_store import HyPermissionVectorStore
|
||||
from one_memory.hy_permission_transport import HyPermissionTransportError
|
||||
from one_memory.memory_governance import MemoryDraft, MemoryKind
|
||||
from tests.person_door import as_person
|
||||
from tests.test_hy_permission_context import whole_client
|
||||
|
||||
|
||||
def test_cancelled_start_cannot_install_wiring_after_factory_releases_client(tmp_path):
|
||||
async def scenario():
|
||||
with whole_client(tmp_path) as (supervisor, record, _, _):
|
||||
task = supervisor.prepare_conversation(record)[0]
|
||||
lease, request, _ = supervisor._create_runner()._prepare_permission_lease(task)
|
||||
bridge = supervisor._bridge(lease.profile, None, tmp_path)
|
||||
client = await bridge._get_client()
|
||||
native_vector, native_graph = client._vector_store, client._graph_store
|
||||
original_coding = client._config.coding.enable
|
||||
run = client._loop_thread.run
|
||||
submitted, release, completed = (threading.Event() for _ in range(3))
|
||||
worker_errors = []
|
||||
bound_contexts = []
|
||||
sends = []
|
||||
|
||||
def mock_send(request):
|
||||
sends.append(request)
|
||||
pytest.fail("This startup-only race must not make any model request")
|
||||
|
||||
def delayed_run(coroutine):
|
||||
if coroutine.cr_code is not _Wiring.enter.__code__:
|
||||
return run(coroutine)
|
||||
# Model a worker being descheduled just before submitting to HY.
|
||||
submitted.set()
|
||||
try:
|
||||
if not release.wait(5):
|
||||
coroutine.close()
|
||||
raise TimeoutError("test did not release startup worker")
|
||||
return run(coroutine)
|
||||
except BaseException as error:
|
||||
worker_errors.append(error)
|
||||
raise
|
||||
finally:
|
||||
completed.set()
|
||||
|
||||
client._loop_thread.run = delayed_run
|
||||
factory = permission_bridge_factory(
|
||||
supervisor._bridge, http_transport=httpx.MockTransport(mock_send)
|
||||
)
|
||||
|
||||
async def ingest():
|
||||
async with factory(lease, None, tmp_path) as bound:
|
||||
bound_contexts.append(bound)
|
||||
await bound.ingest_extracted(request, (record.verbatim,))
|
||||
|
||||
pending = asyncio.create_task(ingest())
|
||||
try:
|
||||
assert await asyncio.to_thread(submitted.wait, 5)
|
||||
pending.cancel()
|
||||
# A fixed factory may keep cancellation pending until it has
|
||||
# joined startup. Let both that behavior and early exit proceed.
|
||||
await asyncio.wait({pending}, timeout=0.1)
|
||||
released_before_start = not bridge._access_lock.locked()
|
||||
release.set()
|
||||
assert await asyncio.to_thread(completed.wait, 5)
|
||||
with pytest.raises(asyncio.CancelledError):
|
||||
await asyncio.wait_for(pending, 5)
|
||||
assert not worker_errors
|
||||
assert not sends
|
||||
assert bridge._access_owner is None
|
||||
assert not bridge._access_lock.locked()
|
||||
assert not isinstance(client._vector_store, HyPermissionVectorStore), (
|
||||
"Cancelled startup installed a live permission wrapper after the "
|
||||
"factory exited and released exclusive ownership; "
|
||||
f"released_before_start={released_before_start}"
|
||||
)
|
||||
assert client._vector_store is native_vector
|
||||
assert client._graph_store is native_graph
|
||||
assert client._config.coding.enable == original_coding
|
||||
finally:
|
||||
release.set()
|
||||
if not pending.done():
|
||||
pending.cancel()
|
||||
await asyncio.gather(pending, return_exceptions=True)
|
||||
assert await asyncio.to_thread(completed.wait, 5)
|
||||
client._loop_thread.run = run
|
||||
# The regression leaves finished=True with live wiring; force
|
||||
# cleanup directly so the failing test leaks no SDK resources.
|
||||
for bound in bound_contexts:
|
||||
if bound.wiring is not None:
|
||||
await asyncio.to_thread(run, bound.wiring.exit())
|
||||
await supervisor.shutdown()
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_completed_start_restores_client_and_revokes_saved_vector_reference(tmp_path):
|
||||
async def scenario():
|
||||
with whole_client(tmp_path) as (supervisor, record, _, _):
|
||||
task = supervisor.prepare_conversation(record)[0]
|
||||
lease, request, _ = supervisor._create_runner()._prepare_permission_lease(task)
|
||||
bridge = supervisor._bridge(lease.profile, None, tmp_path)
|
||||
client = await bridge._get_client()
|
||||
vector, graph = client._vector_store, client._graph_store
|
||||
factory = permission_bridge_factory(
|
||||
supervisor._bridge,
|
||||
http_transport=httpx.MockTransport(lambda _: pytest.fail("unexpected send")),
|
||||
)
|
||||
try:
|
||||
async with factory(lease, None, tmp_path) as bound:
|
||||
await bound._start(request)
|
||||
wrapped = client._vector_store
|
||||
assert isinstance(wrapped, HyPermissionVectorStore)
|
||||
assert client._graph_store is None
|
||||
coding = client._coding_store
|
||||
assert isinstance(coding, HyPermissionCodingStore)
|
||||
assert client._coding_writer.reconciler.store is coding
|
||||
assert client._cache is None
|
||||
assert client._vector_store is vector
|
||||
assert client._graph_store is graph
|
||||
assert client._coding_writer.reconciler.store is client._coding_store
|
||||
assert not isinstance(client._coding_store, HyPermissionCodingStore)
|
||||
assert client._cache is not None
|
||||
with pytest.raises(HyMemoryBridgeError, match="hy_store_closed"):
|
||||
wrapped.recheck()
|
||||
with pytest.raises(HyMemoryBridgeError, match="hy_coding_store_closed"):
|
||||
coding.recheck()
|
||||
finally:
|
||||
await supervisor.shutdown()
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_graph_disabled_keeps_graph_absent_even_for_cross_evidence_operations(tmp_path):
|
||||
async def scenario():
|
||||
with whole_client(tmp_path) as (supervisor, record, _, _):
|
||||
task = supervisor.prepare_conversation(record)[0]
|
||||
lease, request, _ = supervisor._create_runner()._prepare_permission_lease(task)
|
||||
bridge = supervisor._bridge(lease.profile, None, tmp_path)
|
||||
client = await bridge._get_client()
|
||||
native_graph = client._graph_store
|
||||
assert native_graph is None
|
||||
factory = permission_bridge_factory(
|
||||
supervisor._bridge,
|
||||
http_transport=httpx.MockTransport(lambda _: pytest.fail("unexpected send")),
|
||||
)
|
||||
try:
|
||||
async with factory(lease, None, tmp_path) as bound:
|
||||
await bound._start(request)
|
||||
assert client._graph_store is None
|
||||
assert bound.wiring is not None and bound.wiring.graph is None
|
||||
assert client._graph_store is native_graph
|
||||
|
||||
async with factory(lease, None, tmp_path) as digest_bound:
|
||||
await digest_bound._start_operation(
|
||||
user_id=request.user_id,
|
||||
agent_id=request.agent_id,
|
||||
request_id="digest-test",
|
||||
session_id="digest:test",
|
||||
cross_evidence_scope=True,
|
||||
)
|
||||
assert client._graph_store is None
|
||||
assert digest_bound.wiring is not None
|
||||
assert digest_bound.wiring.graph is None
|
||||
assert client._graph_store is native_graph
|
||||
finally:
|
||||
await supervisor.shutdown()
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_persisted_native_proof_does_not_authorize_deleted_governed_memory(tmp_path):
|
||||
async def scenario():
|
||||
with whole_client(tmp_path) as (supervisor, record, permissions, permission):
|
||||
runner = supervisor._create_runner()
|
||||
task = supervisor.prepare_conversation(record)[0]
|
||||
lease, request, _ = runner._prepare_permission_lease(task)
|
||||
factory = permission_bridge_factory(
|
||||
supervisor._bridge,
|
||||
http_transport=httpx.MockTransport(lambda _: pytest.fail("unexpected send")),
|
||||
)
|
||||
bridge = supervisor._bridge(lease.profile, None, tmp_path)
|
||||
client = await bridge._get_client()
|
||||
run = client._loop_thread.run
|
||||
native = MemoryNode(
|
||||
node_id="synthetic-governed-memory",
|
||||
user_id=request.user_id,
|
||||
agent_id=request.agent_id,
|
||||
session_id=request.session_id,
|
||||
content=record.verbatim,
|
||||
owner="agent",
|
||||
layer=MemoryLayer.L2_FACT,
|
||||
embedding=[1.0] + [0.0] * 383,
|
||||
)
|
||||
try:
|
||||
# Seed only through the guarded native store; profile readback
|
||||
# persists the factory's own proof ledger, as in a completed run.
|
||||
async with factory(lease, None, tmp_path) as first:
|
||||
await first._start(request)
|
||||
store = first.wiring.vector
|
||||
assert await asyncio.to_thread(run, store.upsert(native)) == native.node_id
|
||||
positive = await asyncio.to_thread(run, store.get_by_id(native.node_id))
|
||||
assert positive.content == record.verbatim
|
||||
await first.profile_memories(user_id=request.user_id, agent_id=request.agent_id)
|
||||
governed = as_person(supervisor.governance).create(
|
||||
MemoryDraft(
|
||||
user_id=request.user_id,
|
||||
kind=MemoryKind.CHAT,
|
||||
content=record.verbatim,
|
||||
scopes=("synthetic-scope",),
|
||||
evidence_ids=(record.evidence_id,),
|
||||
),
|
||||
request_id="synthetic-publication",
|
||||
memory_id=native.node_id,
|
||||
)
|
||||
rejected = as_person(supervisor.governance).delete(
|
||||
governed.memory_id,
|
||||
user_id=request.user_id,
|
||||
request_id="synthetic-rejection",
|
||||
expected_revision=governed.revision,
|
||||
)
|
||||
assert rejected.status == "deleted"
|
||||
# Explicit renewed consent permits the unchanged source text,
|
||||
# but must not reinstate a deleted derived native memory.
|
||||
snapshot = permissions.snapshot(
|
||||
user_id=request.user_id,
|
||||
source_ids=(record.source_id,),
|
||||
profile=lease.profile,
|
||||
evidence=(record,),
|
||||
)
|
||||
permissions.grant(
|
||||
snapshot, user_confirmed=True, expected_revision=permission.revision
|
||||
)
|
||||
next_task = supervisor.prepare_conversation(record)[0]
|
||||
assert next_task.task_id != task.task_id
|
||||
next_lease, next_request, _ = runner._prepare_permission_lease(next_task)
|
||||
assert next_lease.source_context is None
|
||||
assert next_lease.recheck()
|
||||
async with factory(next_lease, None, tmp_path) as second:
|
||||
await second._start(next_request)
|
||||
reused = await asyncio.to_thread(
|
||||
run, second.wiring.vector.get_by_id(native.node_id)
|
||||
)
|
||||
assert reused is None, (
|
||||
"Evidence-only proof reload exposed a deleted governed memory "
|
||||
"as native context in a new permitted task"
|
||||
)
|
||||
finally:
|
||||
await supervisor.shutdown()
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
@pytest.mark.parametrize("rejection_point", ["after_start", "after_proof_load"])
|
||||
def test_deletion_during_proof_binding_blocks_next_physical_send(tmp_path, rejection_point):
|
||||
async def scenario():
|
||||
with whole_client(tmp_path) as (supervisor, record, _, _):
|
||||
task = supervisor.prepare_conversation(record)[0]
|
||||
lease, request, _ = supervisor._create_runner()._prepare_permission_lease(task)
|
||||
bridge = supervisor._bridge(lease.profile, None, tmp_path)
|
||||
client = await bridge._get_client()
|
||||
run = client._loop_thread.run
|
||||
sends = []
|
||||
|
||||
def handler(outbound):
|
||||
body = json.loads(outbound.content)
|
||||
sends.append(body)
|
||||
return httpx.Response(
|
||||
200,
|
||||
json={
|
||||
"object": "list",
|
||||
"model": "test-embedding",
|
||||
"data": [
|
||||
{
|
||||
"object": "embedding",
|
||||
"index": i,
|
||||
"embedding": [1.0] + [0.0] * 383,
|
||||
}
|
||||
for i, _ in enumerate(body["input"])
|
||||
],
|
||||
"usage": {"prompt_tokens": 1, "total_tokens": 1},
|
||||
},
|
||||
)
|
||||
|
||||
factory = permission_bridge_factory(
|
||||
supervisor._bridge, http_transport=httpx.MockTransport(handler)
|
||||
)
|
||||
native = MemoryNode(
|
||||
node_id="synthetic-rejection-race",
|
||||
user_id=request.user_id,
|
||||
agent_id=request.agent_id,
|
||||
session_id=request.session_id,
|
||||
content=record.verbatim,
|
||||
owner="agent",
|
||||
layer=MemoryLayer.L2_FACT,
|
||||
embedding=[1.0] + [0.0] * 383,
|
||||
)
|
||||
try:
|
||||
async with factory(lease, None, tmp_path) as first:
|
||||
await first._start(request)
|
||||
await asyncio.to_thread(run, first.wiring.vector.upsert(native))
|
||||
permitted = await asyncio.to_thread(
|
||||
run, first.wiring.vector.get_by_id(native.node_id)
|
||||
)
|
||||
assert permitted is not None
|
||||
await asyncio.to_thread(
|
||||
run, client._embed_service.embed(permitted.content, use_cache=False)
|
||||
)
|
||||
assert len(sends) == 1 and sends[0]["input"] == [record.verbatim]
|
||||
await first.profile_memories(user_id=request.user_id, agent_id=request.agent_id)
|
||||
|
||||
governed = as_person(supervisor.governance).create(
|
||||
MemoryDraft(
|
||||
user_id=request.user_id,
|
||||
kind=MemoryKind.CHAT,
|
||||
content=record.verbatim,
|
||||
scopes=("synthetic-scope",),
|
||||
evidence_ids=(record.evidence_id,),
|
||||
),
|
||||
request_id="synthetic-race-publication",
|
||||
memory_id=native.node_id,
|
||||
)
|
||||
|
||||
def reject():
|
||||
result = as_person(supervisor.governance).delete(
|
||||
governed.memory_id,
|
||||
user_id=request.user_id,
|
||||
request_id="synthetic-race-rejection",
|
||||
expected_revision=governed.revision,
|
||||
)
|
||||
assert result.status == "deleted"
|
||||
|
||||
try:
|
||||
async with factory(lease, None, tmp_path) as second:
|
||||
if rejection_point == "after_proof_load":
|
||||
load = second.proofs.load
|
||||
|
||||
def reject_after_load(current_lease):
|
||||
approved = load(current_lease)
|
||||
assert native.node_id in approved
|
||||
# A governance write can commit between the two
|
||||
# separately awaited load/guard database reads.
|
||||
reject()
|
||||
return approved
|
||||
|
||||
second.proofs.load = reject_after_load
|
||||
await second._start(request)
|
||||
if rejection_point == "after_start":
|
||||
reject()
|
||||
candidate = await asyncio.to_thread(
|
||||
run, second.wiring.vector.get_by_id(native.node_id)
|
||||
)
|
||||
if candidate is not None:
|
||||
await asyncio.to_thread(
|
||||
run,
|
||||
client._embed_service.embed(candidate.content, use_cache=False),
|
||||
)
|
||||
except (HyMemoryBridgeError, HyPermissionTransportError) as error:
|
||||
assert str(error).startswith("model_content_")
|
||||
assert (
|
||||
supervisor.governance.get(native.node_id, user_id="local").status == "deleted"
|
||||
)
|
||||
assert len(sends) == 1, (
|
||||
f"Deleted proof context reached the physical transport at {rejection_point}; "
|
||||
f"expected one approved control send, observed {len(sends)} sends"
|
||||
)
|
||||
finally:
|
||||
await supervisor.shutdown()
|
||||
|
||||
asyncio.run(scenario())
|
||||
@@ -1,834 +0,0 @@
|
||||
"""Real pinned HY SDK/Chroma and guards; scripted HTTP is not model acceptance."""
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import sqlite3
|
||||
import traceback
|
||||
from contextlib import asynccontextmanager
|
||||
from datetime import UTC, datetime, timedelta
|
||||
|
||||
import httpx
|
||||
import pytest
|
||||
from hy_memory.client import HyMemoryClient
|
||||
from hy_memory.data.vector_store_chroma import ChromaVectorStore
|
||||
from hy_memory.models.memory import MemoryLayer, MemoryNode
|
||||
from hy_memory.pipelines._retrieval.reconcile_retrieval import ReconcileHybridRetriever
|
||||
|
||||
from one_memory.hy_permission_context import permission_bridge_factory
|
||||
from one_memory.hy_response_guard import HyReconcileGuard, HyResponseGuard
|
||||
from one_memory.hy_runtime_config import hy_memory_version
|
||||
from one_memory.hy_write_verification import HyExtractionGuard
|
||||
from one_memory.imports.grants import SourceGrantStore
|
||||
from one_memory.imports.models import (
|
||||
EvidenceContentKind,
|
||||
EvidenceInput,
|
||||
EvidenceOrigin,
|
||||
OriginMetadata,
|
||||
WrittenBy,
|
||||
)
|
||||
from one_memory.imports.sources.registry import RootCandidateKind
|
||||
from one_memory.imports.storage import EvidenceStore
|
||||
from one_memory.memory_governance import MemoryGovernanceStore
|
||||
from one_memory.memory_processing import MemoryProcessingSupervisor
|
||||
from one_memory.memory_tasks import MemoryTaskStore, evidence_ref
|
||||
from one_memory.model_content_permissions import ModelContentPermissionStore
|
||||
from one_memory.model_profiles import (
|
||||
ModelCapabilities,
|
||||
ModelProfileInput,
|
||||
ModelProfileStore,
|
||||
ProviderKind,
|
||||
)
|
||||
from one_memory.secrets import InMemorySecretStore
|
||||
|
||||
USER_TEXT = "The user requires a permission review before the synthetic release."
|
||||
AGENT_TEXT = "The agent proposes a checklist for the synthetic release."
|
||||
CLAIMS = {USER_TEXT: "user", AGENT_TEXT: "agent"}
|
||||
DENIED_FACT = "DENIED_NATIVE_FACT_63a1: omit the permission review."
|
||||
DENIED_HISTORY = "DENIED_NATIVE_HISTORY_86b2: publish without asking."
|
||||
DENIED_TAG = "DENIED_NATIVE_TAG_91c3"
|
||||
DENIED_IDS = ("denied-native-fact", "denied-native-history")
|
||||
NOW = datetime(2026, 9, 13, 8, tzinfo=UTC)
|
||||
|
||||
|
||||
def vector(text):
|
||||
# Orthogonal synthetic vectors keep the two claims distinct under native dedup.
|
||||
axis = 1 if text == USER_TEXT else 2 if text == AGENT_TEXT else 0
|
||||
return [float(index == axis) for index in range(384)]
|
||||
|
||||
|
||||
def prompt_json(prompt, heading):
|
||||
payload = prompt_section(prompt, heading)
|
||||
starts = [index for marker in ("[", "{") if (index := payload.find(marker)) >= 0]
|
||||
assert starts, (heading, payload)
|
||||
payload = payload[min(starts) :]
|
||||
try:
|
||||
return json.JSONDecoder().raw_decode(payload)[0]
|
||||
except json.JSONDecodeError as error:
|
||||
raise AssertionError(f"expected JSON after {heading!r}, got {payload[:500]!r}") from error
|
||||
|
||||
|
||||
def prompt_section(prompt: str, heading: str) -> str:
|
||||
assert prompt.count(heading) == 1, heading
|
||||
section = prompt.split(heading, 1)[1]
|
||||
return section.split("\n---\n", 1)[0].strip()
|
||||
|
||||
|
||||
class ScriptedOracle:
|
||||
"""Only the two fixture claims receive support, with exact supplied Evidence IDs."""
|
||||
|
||||
def __init__(self, records, *, after_response=None):
|
||||
self.records = {record.evidence_id: record for record in records}
|
||||
self.requests = []
|
||||
self.stages = []
|
||||
self.reconciles = []
|
||||
self.errors = []
|
||||
self.after_response = after_response
|
||||
self.revoked_at = None
|
||||
self.limit = 24
|
||||
|
||||
def __call__(self, request):
|
||||
try:
|
||||
return self.respond(request)
|
||||
except Exception:
|
||||
self.errors.append(traceback.format_exc())
|
||||
raise
|
||||
|
||||
def respond(self, request):
|
||||
body = json.loads(request.content)
|
||||
self.requests.append((request.url.path, body))
|
||||
assert len(self.requests) <= self.limit, "scripted HTTP response budget exceeded"
|
||||
assert self.revoked_at is None, "native child request escaped after revocation"
|
||||
assert request.method == "POST"
|
||||
serialized = json.dumps(body)
|
||||
for forbidden in (DENIED_FACT, DENIED_HISTORY, DENIED_TAG, *DENIED_IDS):
|
||||
assert forbidden not in serialized, f"denied native context reached HTTP: {forbidden}"
|
||||
if request.url.path == "/v1/embeddings":
|
||||
assert str(request.url) == "http://127.0.0.1:2/v1/embeddings"
|
||||
assert body["model"] == "raw-fixture-embedding"
|
||||
texts = body["input"]
|
||||
assert isinstance(texts, list) and texts
|
||||
lines = [f"[{record.role}]: {record.verbatim}" for record in self.records.values()]
|
||||
# HY stores one raw copy per add, and each add is one turn.
|
||||
turns = {
|
||||
"\n".join(lines[start:end])
|
||||
for start in range(len(lines))
|
||||
for end in range(start + 1, len(lines) + 1)
|
||||
}
|
||||
for text in texts:
|
||||
if text not in CLAIMS:
|
||||
assert text in turns
|
||||
self.stages.append("embedding")
|
||||
return httpx.Response(
|
||||
200,
|
||||
json={
|
||||
"object": "list",
|
||||
"model": body["model"],
|
||||
"data": [
|
||||
{"index": i, "object": "embedding", "embedding": vector(text)}
|
||||
for i, text in enumerate(texts)
|
||||
],
|
||||
"usage": {"prompt_tokens": 1, "total_tokens": 1},
|
||||
},
|
||||
)
|
||||
assert str(request.url) == "http://127.0.0.1:1/v1/chat/completions"
|
||||
assert body["model"] == "raw-fixture-generation"
|
||||
prompt = body["messages"][-1]["content"]
|
||||
if prompt.startswith("You are a single-label scene classifier."):
|
||||
result = {"is_coding": False, "reason": "synthetic chat"}
|
||||
stage = "judge"
|
||||
elif "## New Messages\n" in prompt:
|
||||
self.check_turns(prompt_json(prompt, "## New Messages\n"))
|
||||
history_section = prompt_section(prompt, "## Last k Messages\n")
|
||||
history = (
|
||||
[] if history_section == "(none)" else prompt_json(prompt, "## Last k Messages\n")
|
||||
)
|
||||
for source in history:
|
||||
record = self.records[source["evidence_id"]]
|
||||
assert source["content"] == record.verbatim
|
||||
assert source["role"] == record.role
|
||||
assert source["revision"] == record.revision
|
||||
assert '"memory"' in prompt
|
||||
assert ('"intentions"' in prompt) is ("never" not in prompt)
|
||||
result = {
|
||||
"memory": [
|
||||
{"content": text, "owner": owner, "tags": []} for text, owner in CLAIMS.items()
|
||||
],
|
||||
"basic_info": {},
|
||||
}
|
||||
if '"intentions"' in prompt:
|
||||
result["intentions"] = []
|
||||
stage = "extract"
|
||||
elif "## Existing memories\n" in prompt:
|
||||
existing = prompt_json(prompt, "## Existing memories\n")
|
||||
incoming = prompt_json(prompt, "## New memories\n")
|
||||
assert existing and incoming
|
||||
for item in (*existing, *incoming):
|
||||
assert CLAIMS[item["content"]] == item["owner"]
|
||||
assert not item.get("tags")
|
||||
self.reconciles.append(existing)
|
||||
by_content = {item["content"]: item for item in existing}
|
||||
result = [
|
||||
{
|
||||
"op": "UPDATE" if item["content"] in by_content else "ADD",
|
||||
"content": item["content"],
|
||||
"owner": item["owner"],
|
||||
"layer": "L2_FACT",
|
||||
"tags": [],
|
||||
**(
|
||||
{"memory_id": by_content[item["content"]]["memory_id"]}
|
||||
if item["content"] in by_content
|
||||
else {}
|
||||
),
|
||||
}
|
||||
for item in incoming
|
||||
]
|
||||
stage = "reconcile"
|
||||
else:
|
||||
if not prompt.lstrip().startswith("{"):
|
||||
raise AssertionError(f"unknown native child prompt: {prompt[:300]!r}")
|
||||
data = json.loads(prompt)
|
||||
if set(data) == {"items", "sources", "before"}:
|
||||
result = self.review(data)
|
||||
stage = "review"
|
||||
else:
|
||||
assert set(data) == {"proofs"}, "unknown native child prompt"
|
||||
result = self.cited_review(data)
|
||||
stage = "citations"
|
||||
self.stages.append(stage)
|
||||
response = httpx.Response(
|
||||
200,
|
||||
json={
|
||||
"id": "synthetic-raw-response",
|
||||
"object": "chat.completion",
|
||||
"created": 1,
|
||||
"model": body["model"],
|
||||
"choices": [
|
||||
{
|
||||
"index": 0,
|
||||
"finish_reason": "stop",
|
||||
"message": {"role": "assistant", "content": json.dumps(result)},
|
||||
}
|
||||
],
|
||||
"usage": {"prompt_tokens": 1, "completion_tokens": 1, "total_tokens": 2},
|
||||
},
|
||||
)
|
||||
if self.after_response is not None and self.after_response(stage):
|
||||
self.revoked_at = len(self.requests)
|
||||
return response
|
||||
|
||||
def check_turns(self, turns):
|
||||
assert turns and isinstance(turns, list)
|
||||
allowed = {(record.role, record.verbatim) for record in self.records.values()}
|
||||
assert all(set(turn) == {"role", "content"} for turn in turns)
|
||||
assert {(turn["role"], turn["content"]) for turn in turns} <= allowed
|
||||
|
||||
def review(self, data):
|
||||
assert data["items"] and data["sources"]
|
||||
for source in data["sources"]:
|
||||
record = self.records[source["id"]]
|
||||
assert source["role"] == record.role
|
||||
assert source["revision"] == record.revision
|
||||
assert source["created_at"] == record.created_at.isoformat()
|
||||
assert "".join(span["text"] for span in source["spans"]) == record.verbatim
|
||||
reviews = []
|
||||
for item in data["items"]:
|
||||
assert CLAIMS[item["content"]] == item["owner"]
|
||||
role = "user" if item["owner"] == "user" else "assistant"
|
||||
source = next(
|
||||
source
|
||||
for source in data["sources"]
|
||||
if source["role"] == role and self.records[source["id"]].verbatim == item["content"]
|
||||
)
|
||||
reviews.append(
|
||||
{
|
||||
"id": item["id"],
|
||||
"clauses": [
|
||||
{
|
||||
"text": item["content"],
|
||||
"verdict": "supported",
|
||||
"authority": item["owner"],
|
||||
"citations": [
|
||||
{
|
||||
"source_id": source["id"],
|
||||
"span_ids": [span["id"] for span in source["spans"]],
|
||||
}
|
||||
],
|
||||
"reason": "exact synthetic fixture claim",
|
||||
}
|
||||
],
|
||||
}
|
||||
)
|
||||
for before in data["before"]:
|
||||
assert any(
|
||||
(item["content"], item["owner"]) == (before["content"], before["owner"])
|
||||
for item in data["items"]
|
||||
)
|
||||
return {
|
||||
"reviews": reviews,
|
||||
"coverage": [
|
||||
{"id": item["id"], "verdict": "preserved", "reason": "exact fixture match"}
|
||||
for item in data["before"]
|
||||
],
|
||||
}
|
||||
|
||||
def cited_review(self, data):
|
||||
assert data["proofs"]
|
||||
for proof in data["proofs"]:
|
||||
assert CLAIMS[proof["claim"]] == proof["owner"]
|
||||
assert proof["citations"] and not proof["context_citations"]
|
||||
for citation in proof["citations"]:
|
||||
record = self.records[citation["source_id"]]
|
||||
assert citation["text"] == record.verbatim == proof["claim"]
|
||||
assert citation["role"] == record.role
|
||||
assert citation["revision"] == record.revision
|
||||
assert citation["created_at"] == record.created_at.isoformat()
|
||||
return {
|
||||
"proofs": [
|
||||
{"id": proof["id"], "verdict": "supported", "reason": "exact fixture match"}
|
||||
for proof in data["proofs"]
|
||||
]
|
||||
}
|
||||
|
||||
|
||||
def put_turn(evidence, message_id, role, text, ordinal, *, conversation_id="raw-session"):
|
||||
return evidence.put(
|
||||
EvidenceInput(
|
||||
user_id="local",
|
||||
source_id="raw-fixture",
|
||||
conversation_id=conversation_id,
|
||||
message_id=message_id,
|
||||
source_ordinal=str(ordinal),
|
||||
role=role,
|
||||
written_by={"user": WrittenBy.PERSON, "tool": WrittenBy.TOOL}.get(
|
||||
role, WrittenBy.AGENT
|
||||
),
|
||||
content_kind=EvidenceContentKind.RAW_CONVERSATION,
|
||||
created_at=NOW + timedelta(seconds=ordinal),
|
||||
verbatim=text,
|
||||
origin_meta={EvidenceOrigin.MANUAL_IMPORT: OriginMetadata()},
|
||||
)
|
||||
).record
|
||||
|
||||
|
||||
def grant_records(supervisor, records, expected_revision=0):
|
||||
profile = supervisor.profiles.active(supervisor.secrets)
|
||||
snapshot = supervisor.model_content_permissions.snapshot(
|
||||
user_id="local",
|
||||
source_ids=("raw-fixture",),
|
||||
profile=supervisor.profiles.snapshot(profile.profile_id, profile.revision),
|
||||
evidence=records,
|
||||
)
|
||||
return supervisor.model_content_permissions.grant(
|
||||
snapshot, user_confirmed=True, expected_revision=expected_revision
|
||||
)
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
async def raw_client(tmp_path, *, tool=None, conversation_id="raw-session"):
|
||||
assert hy_memory_version() == "2.0.1"
|
||||
root = tmp_path / "source"
|
||||
root.mkdir()
|
||||
database = tmp_path / "one-memory.sqlite3"
|
||||
grants = SourceGrantStore(database, home=tmp_path)
|
||||
grants.grant(
|
||||
user_id="local",
|
||||
source_id="raw-fixture",
|
||||
root=root,
|
||||
candidate_kind=RootCandidateKind.DEFAULT,
|
||||
)
|
||||
evidence = EvidenceStore(database)
|
||||
records = (
|
||||
put_turn(
|
||||
evidence,
|
||||
"user-original",
|
||||
"user",
|
||||
USER_TEXT,
|
||||
1,
|
||||
conversation_id=conversation_id,
|
||||
),
|
||||
put_turn(
|
||||
evidence,
|
||||
"agent-original",
|
||||
"assistant",
|
||||
AGENT_TEXT,
|
||||
2,
|
||||
conversation_id=conversation_id,
|
||||
),
|
||||
*(
|
||||
(
|
||||
put_turn(
|
||||
evidence,
|
||||
"tool-original",
|
||||
"tool",
|
||||
tool,
|
||||
3,
|
||||
conversation_id=conversation_id,
|
||||
),
|
||||
)
|
||||
if tool
|
||||
else ()
|
||||
),
|
||||
)
|
||||
profiles, secrets = ModelProfileStore(database), InMemorySecretStore()
|
||||
profile = profiles.create(
|
||||
ModelProfileInput(
|
||||
name="Synthetic HTTP only",
|
||||
provider_kind=ProviderKind.OLLAMA,
|
||||
base_url="http://127.0.0.1:1/v1",
|
||||
model="raw-fixture-generation",
|
||||
embedding_provider_kind=ProviderKind.OLLAMA,
|
||||
embedding_base_url="http://127.0.0.1:2/v1",
|
||||
embedding_model="raw-fixture-embedding",
|
||||
embedding_dimensions=384,
|
||||
),
|
||||
secrets,
|
||||
)
|
||||
profiles.record_capabilities(
|
||||
profile.profile_id,
|
||||
expected_revision=profile.revision,
|
||||
capabilities=ModelCapabilities(
|
||||
endpoint_reachable=True, model_available=True, embedding_available=True
|
||||
),
|
||||
secrets=secrets,
|
||||
)
|
||||
profiles.activate(profile.profile_id, secrets)
|
||||
supervisor = MemoryProcessingSupervisor(
|
||||
tasks=MemoryTaskStore(database),
|
||||
profiles=profiles,
|
||||
secrets=secrets,
|
||||
evidence=evidence,
|
||||
governance=MemoryGovernanceStore(database),
|
||||
data_dir=tmp_path,
|
||||
model_content_permissions=ModelContentPermissionStore(database),
|
||||
)
|
||||
try:
|
||||
permission = grant_records(supervisor, records)
|
||||
assert supervisor.governance.list(user_id="local") == []
|
||||
yield supervisor, records, permission
|
||||
finally:
|
||||
await supervisor.shutdown()
|
||||
|
||||
|
||||
def test_search_after_first_wb20_task_does_not_poison_the_second_task(tmp_path):
|
||||
async def scenario():
|
||||
async with raw_client(tmp_path, conversation_id="wb-20") as (
|
||||
supervisor,
|
||||
records,
|
||||
permission,
|
||||
):
|
||||
first = supervisor.prepare_conversation(records[0])[0]
|
||||
_, request, _ = supervisor._create_runner()._prepare_permission_lease(first)
|
||||
client = await native_client(supervisor, first)
|
||||
duplicate = MemoryNode(
|
||||
node_id="older-identical-memory",
|
||||
user_id=request.user_id,
|
||||
agent_id=request.agent_id,
|
||||
session_id="wb-19",
|
||||
layer=MemoryLayer.L2_FACT,
|
||||
content=USER_TEXT,
|
||||
owner="user",
|
||||
embedding=vector(USER_TEXT),
|
||||
gmt_created=NOW - timedelta(days=1),
|
||||
gmt_modified=NOW - timedelta(days=1),
|
||||
)
|
||||
await on_native_loop(client, client._vector_store.upsert(duplicate))
|
||||
|
||||
first_oracle = ScriptedOracle(records)
|
||||
bind_oracle(supervisor, first_oracle)
|
||||
first_result = await supervisor._create_runner().run(first.task_id)
|
||||
assert first_result.status == "completed", first_result.last_error_code
|
||||
|
||||
current = next(
|
||||
memory
|
||||
for memory in supervisor.governance.list(user_id="local")
|
||||
if memory.content == USER_TEXT
|
||||
)
|
||||
client._spawn_search_dedup(
|
||||
[
|
||||
{
|
||||
"memory_id": duplicate.node_id,
|
||||
"content": duplicate.content,
|
||||
"layer": duplicate.layer.value,
|
||||
"gmt_created": int(duplicate.gmt_created.timestamp()),
|
||||
},
|
||||
{
|
||||
"memory_id": current.memory_id,
|
||||
"content": current.content,
|
||||
"layer": MemoryLayer.L2_FACT.value,
|
||||
"gmt_created": int(NOW.timestamp()),
|
||||
},
|
||||
],
|
||||
request_id="wb-20-search",
|
||||
user_id=request.user_id,
|
||||
agent_id=request.agent_id,
|
||||
)
|
||||
await asyncio.sleep(0.2)
|
||||
|
||||
added = put_turn(
|
||||
supervisor.evidence,
|
||||
"wb-20-second",
|
||||
"user",
|
||||
USER_TEXT,
|
||||
3,
|
||||
conversation_id="wb-20",
|
||||
)
|
||||
records = (*records, added)
|
||||
grant_records(supervisor, records, permission.revision)
|
||||
second = supervisor.prepare_conversation(added)[0]
|
||||
second_oracle = ScriptedOracle(records)
|
||||
bind_oracle(supervisor, second_oracle)
|
||||
second_result = await supervisor._create_runner().run(second.task_id)
|
||||
|
||||
assert second_result.status == "completed", second_result.last_error_code
|
||||
assert second_result.last_error_code is None
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
async def native_client(supervisor, task):
|
||||
bridge = supervisor._bridge(task.profile_snapshot, None, supervisor.data_dir)
|
||||
client = await bridge._get_client()
|
||||
assert isinstance(client, HyMemoryClient)
|
||||
assert isinstance(client._vector_store, ChromaVectorStore)
|
||||
guards = client._one_memory_write_guards
|
||||
assert isinstance(guards[0], HyResponseGuard)
|
||||
assert isinstance(guards[2], HyReconcileGuard)
|
||||
assert isinstance(guards[3], HyExtractionGuard)
|
||||
return client
|
||||
|
||||
|
||||
def test_governance_write_runs_after_shared_hy_bridge_is_released(tmp_path):
|
||||
async def scenario():
|
||||
async with raw_client(tmp_path) as (supervisor, records, _permission):
|
||||
task = supervisor.prepare_conversation(records[0])[0]
|
||||
bind_oracle(supervisor, ScriptedOracle(records))
|
||||
await native_client(supervisor, task)
|
||||
bridge = next(
|
||||
value
|
||||
for key, value in supervisor._bridges.items()
|
||||
if key[:2] == (task.profile_id, task.profile_revision)
|
||||
)
|
||||
runner = supervisor._create_runner()
|
||||
original = runner._record_outputs
|
||||
observed = []
|
||||
|
||||
def checked(*args, **kwargs):
|
||||
observed.append(bridge._access_owner)
|
||||
return original(*args, **kwargs)
|
||||
|
||||
runner._record_outputs = checked
|
||||
result = await runner.run(task.task_id)
|
||||
|
||||
assert result.status == "completed"
|
||||
assert observed == [None], "governance writes must not hold the shared HY bridge"
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
async def on_native_loop(client, operation):
|
||||
return await asyncio.to_thread(client._loop_thread.run, operation)
|
||||
|
||||
|
||||
async def seed_denied(client, request):
|
||||
store = client._vector_store
|
||||
nodes = (
|
||||
MemoryNode(
|
||||
node_id=DENIED_IDS[0],
|
||||
user_id=request.user_id,
|
||||
agent_id=request.agent_id,
|
||||
session_id=request.session_id,
|
||||
layer=MemoryLayer.L2_FACT,
|
||||
content=DENIED_FACT,
|
||||
owner="user",
|
||||
embedding=vector(USER_TEXT),
|
||||
tags=[DENIED_TAG],
|
||||
),
|
||||
MemoryNode(
|
||||
node_id=DENIED_IDS[1],
|
||||
user_id=request.user_id,
|
||||
agent_id=request.agent_id,
|
||||
session_id=request.session_id,
|
||||
layer=MemoryLayer.L1_RAW,
|
||||
content=f"[user]: {DENIED_HISTORY}",
|
||||
owner="user",
|
||||
embedding=vector("raw"),
|
||||
tags=[DENIED_TAG],
|
||||
),
|
||||
)
|
||||
await on_native_loop(client, store.upsert_batch(list(nodes)))
|
||||
retrieved = await on_native_loop(
|
||||
client,
|
||||
store.search(
|
||||
query_embedding=vector(USER_TEXT),
|
||||
user_id=request.user_id,
|
||||
agent_ids=[request.agent_id],
|
||||
layers=[MemoryLayer.L2_FACT],
|
||||
),
|
||||
)
|
||||
assert {hit["node_id"] for hit in retrieved} == {DENIED_IDS[0]}
|
||||
history = await on_native_loop(
|
||||
client,
|
||||
store.list_by_user(
|
||||
user_id=request.user_id,
|
||||
agent_id=request.agent_id,
|
||||
layers=[MemoryLayer.L1_RAW],
|
||||
allow_raw_layer=True,
|
||||
),
|
||||
)
|
||||
assert [node.node_id for node in history] == [DENIED_IDS[1]]
|
||||
candidates = await on_native_loop(
|
||||
client,
|
||||
ReconcileHybridRetriever().search_candidates(
|
||||
USER_TEXT,
|
||||
vector(USER_TEXT),
|
||||
vector_store=store,
|
||||
user_id=request.user_id,
|
||||
agent_id=request.agent_id,
|
||||
layers=[MemoryLayer.L2_FACT],
|
||||
),
|
||||
)
|
||||
assert [hit["node"].node_id for hit in candidates] == [DENIED_IDS[0]]
|
||||
pipeline = await on_native_loop(client, client._registry.get_system2_writer())
|
||||
native_history = await on_native_loop(
|
||||
client,
|
||||
pipeline._writer._get_recent_history(store, request.user_id, request.agent_id),
|
||||
)
|
||||
native_tags = await on_native_loop(
|
||||
client,
|
||||
pipeline._writer._collect_existing_tags(store, request.user_id, request.agent_id),
|
||||
)
|
||||
assert DENIED_HISTORY in native_history
|
||||
assert DENIED_TAG in native_tags
|
||||
return await node_snapshots(client, DENIED_IDS)
|
||||
|
||||
|
||||
async def node_snapshots(client, identifiers):
|
||||
result = {}
|
||||
for identifier in identifiers:
|
||||
node = await on_native_loop(client, client._vector_store.get_by_id(identifier))
|
||||
assert node is not None, identifier
|
||||
result[identifier] = node.to_dict()
|
||||
return result
|
||||
|
||||
|
||||
def bind_oracle(supervisor, oracle):
|
||||
supervisor.permission_bridge_factory = permission_bridge_factory(
|
||||
supervisor._bridge, http_transport=httpx.MockTransport(oracle)
|
||||
)
|
||||
|
||||
|
||||
async def assert_outputs(supervisor, task, records, client, *, raw_records=None):
|
||||
outputs = supervisor.tasks.outputs(task.task_id)
|
||||
assert len(outputs) == 2
|
||||
memories = supervisor.governance.list(user_id="local")
|
||||
assert {memory.content for memory in memories} == set(CLAIMS)
|
||||
by_content = {memory.content: memory for memory in memories}
|
||||
for record in records[:2]:
|
||||
memory = by_content[record.verbatim]
|
||||
output = next(output for output in outputs if output.memory_id == memory.memory_id)
|
||||
assert output.owner == CLAIMS[record.verbatim]
|
||||
assert output.engine == "hy-memory" and output.engine_version == "2.0.1"
|
||||
assert output.hy_memory_id == memory.memory_id
|
||||
assert output.memory_revision == memory.revision
|
||||
assert output.support_refs == (evidence_ref(record),)
|
||||
assert memory.evidence_ids == (record.evidence_id,)
|
||||
# Usable either way; origin is what keeps an agent report from passing as the person.
|
||||
assert memory.status == "confirmed"
|
||||
assert memory.origin == ("agent" if output.owner == "agent" else "extracted")
|
||||
stored = await on_native_loop(client, client._vector_store.get_by_id(output.hy_memory_id))
|
||||
assert stored is not None
|
||||
assert stored.content == record.verbatim and stored.owner == output.owner
|
||||
for record in records:
|
||||
assert supervisor.evidence.get(record.evidence_id) == record
|
||||
raw = await on_native_loop(
|
||||
client,
|
||||
client._vector_store.list_by_user(
|
||||
user_id="local",
|
||||
layers=[MemoryLayer.L1_RAW],
|
||||
allow_raw_layer=True,
|
||||
),
|
||||
)
|
||||
allowed_raw = [node for node in raw if node.node_id not in DENIED_IDS]
|
||||
assert allowed_raw
|
||||
# One raw copy per turn; together they hold every record.
|
||||
for record in records if raw_records is None else raw_records:
|
||||
line = f"[{record.role}]: {record.verbatim}"
|
||||
assert any(line in node.content for node in allowed_raw), line
|
||||
with sqlite3.connect(supervisor.data_dir / "one-memory.sqlite3") as db:
|
||||
proofs = db.execute("SELECT node_id, evidence_json FROM hy_node_permissions").fetchall()
|
||||
assert {output.hy_memory_id for output in outputs} <= {key for key, _ in proofs}
|
||||
assert not set(DENIED_IDS) & {key for key, _ in proofs}
|
||||
allowed_refs = [evidence_ref(record) for record in records]
|
||||
for _, refs in proofs:
|
||||
assert json.loads(refs) and all(ref in allowed_refs for ref in json.loads(refs))
|
||||
return {output.hy_memory_id for output in outputs}
|
||||
|
||||
|
||||
def record_receipt(test_node, supervisor, task, oracle, *, phase, hidden):
|
||||
test_node.user_properties.append(
|
||||
(
|
||||
phase,
|
||||
json.dumps(
|
||||
{
|
||||
"sdk": hy_memory_version(),
|
||||
"transport": "scripted-http-oracle-not-real-model",
|
||||
"request_count": len(oracle.requests),
|
||||
"response_limit": oracle.limit,
|
||||
"stages": oracle.stages,
|
||||
"oracle_errors": oracle.errors,
|
||||
"requests_after_revocation": len(oracle.requests) - oracle.revoked_at
|
||||
if oracle.revoked_at is not None
|
||||
else None,
|
||||
"hidden_native_fixture_checked": hidden,
|
||||
"status": task.status,
|
||||
"error": task.last_error_code,
|
||||
"outputs": [
|
||||
output.model_dump(mode="json")
|
||||
for output in supervisor.tasks.outputs(task.task_id)
|
||||
],
|
||||
"governance_count": len(supervisor.governance.list(user_id="local")),
|
||||
}
|
||||
),
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("hidden", [False, True], ids=["fresh-raw", "denied-native-context"])
|
||||
def test_native_raw_input_runs_extraction_guards_and_governance(tmp_path, hidden, request):
|
||||
test_node = request.node
|
||||
|
||||
async def scenario():
|
||||
async with raw_client(tmp_path) as (supervisor, records, _):
|
||||
tasks = supervisor.prepare_conversation(records[0])
|
||||
assert len(tasks) == 1
|
||||
task = tasks[0]
|
||||
runner = supervisor._create_runner()
|
||||
_, request, _ = runner._prepare_permission_lease(task)
|
||||
client = await native_client(supervisor, task)
|
||||
before = await seed_denied(client, request) if hidden else {}
|
||||
oracle = ScriptedOracle(records)
|
||||
bind_oracle(supervisor, oracle)
|
||||
result = await supervisor._create_runner().run(task.task_id)
|
||||
assert not oracle.errors, oracle.errors
|
||||
if hidden:
|
||||
assert await node_snapshots(client, DENIED_IDS) == before
|
||||
assert result.status == "completed", (result.last_error_code, oracle.stages)
|
||||
assert oracle.stages.count("extract") == 1
|
||||
assert "review" not in oracle.stages and "citations" not in oracle.stages
|
||||
assert "embedding" in oracle.stages
|
||||
assert "reconcile" not in oracle.stages
|
||||
await assert_outputs(supervisor, task, records, client)
|
||||
record_receipt(test_node, supervisor, result, oracle, phase="raw", hidden=hidden)
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_a_tool_turn_hy_judges_as_chat_is_extracted_without_its_tool_messages(tmp_path):
|
||||
async def scenario():
|
||||
async with raw_client(tmp_path, tool="[Bash] pytest -q\n12 passed") as (
|
||||
supervisor,
|
||||
records,
|
||||
_,
|
||||
):
|
||||
tasks = supervisor.prepare_conversation(records[0])
|
||||
assert len(tasks) == 1
|
||||
oracle = ScriptedOracle(records)
|
||||
bind_oracle(supervisor, oracle)
|
||||
result = await supervisor._create_runner().run(tasks[0].task_id)
|
||||
assert not oracle.errors, oracle.errors
|
||||
assert result.status == "completed", (result.last_error_code, oracle.stages)
|
||||
assert oracle.stages[0] == "judge"
|
||||
prompt = next(
|
||||
body["messages"][-1]["content"]
|
||||
for _path, body in oracle.requests
|
||||
if "messages" in body and "## New Messages\n" in body["messages"][-1]["content"]
|
||||
)
|
||||
turns = prompt_json(prompt, "## New Messages\n")
|
||||
assert [(turn["role"], turn["content"]) for turn in turns] == [
|
||||
(record.role, record.verbatim) for record in records[:2]
|
||||
]
|
||||
client = await native_client(supervisor, tasks[0])
|
||||
await assert_outputs(supervisor, tasks[0], records, client, raw_records=records[:2])
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_revoke_during_native_extractor_response_stops_children_and_governance(tmp_path, request):
|
||||
test_node = request.node
|
||||
|
||||
async def scenario():
|
||||
async with raw_client(tmp_path) as (supervisor, records, permission):
|
||||
task = supervisor.prepare_conversation(records[0])[0]
|
||||
_, request, _ = supervisor._create_runner()._prepare_permission_lease(task)
|
||||
client = await native_client(supervisor, task)
|
||||
before = await seed_denied(client, request)
|
||||
|
||||
def revoke(stage):
|
||||
if stage != "extract":
|
||||
return False
|
||||
supervisor.model_content_permissions.revoke(
|
||||
user_id="local", expected_revision=permission.revision, user_confirmed=True
|
||||
)
|
||||
return True
|
||||
|
||||
oracle = ScriptedOracle(records, after_response=revoke)
|
||||
bind_oracle(supervisor, oracle)
|
||||
result = await supervisor._create_runner().run(task.task_id)
|
||||
assert not oracle.errors, oracle.errors
|
||||
assert result.status == "cancelled"
|
||||
assert oracle.revoked_at is not None and len(oracle.requests) == oracle.revoked_at
|
||||
assert oracle.stages.count("extract") == 1
|
||||
assert "review" not in oracle.stages and "citations" not in oracle.stages
|
||||
assert not supervisor.governance.list(user_id="local")
|
||||
assert not supervisor.tasks.outputs(task.task_id)
|
||||
assert await node_snapshots(client, DENIED_IDS) == before
|
||||
for record in records:
|
||||
assert supervisor.evidence.get(record.evidence_id) == record
|
||||
record_receipt(test_node, supervisor, result, oracle, phase="revoked", hidden=True)
|
||||
assert result.last_error_code == "model_content_permission_revoked", result
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_approved_native_candidates_are_reused_without_disclosing_denied_nodes(tmp_path, request):
|
||||
test_node = request.node
|
||||
|
||||
async def scenario():
|
||||
async with raw_client(tmp_path) as (supervisor, records, permission):
|
||||
first = supervisor.prepare_conversation(records[0])[0]
|
||||
_, request, _ = supervisor._create_runner()._prepare_permission_lease(first)
|
||||
client = await native_client(supervisor, first)
|
||||
before = await seed_denied(client, request)
|
||||
first_oracle = ScriptedOracle(records)
|
||||
bind_oracle(supervisor, first_oracle)
|
||||
first_result = await supervisor._create_runner().run(first.task_id)
|
||||
assert not first_oracle.errors, first_oracle.errors
|
||||
assert first_result.status == "completed", first_result.last_error_code
|
||||
ids = await assert_outputs(supervisor, first, records, client)
|
||||
record_receipt(
|
||||
test_node, supervisor, first_result, first_oracle, phase="initial", hidden=True
|
||||
)
|
||||
added = put_turn(supervisor.evidence, "user-repeat", "user", USER_TEXT, 3)
|
||||
records = (*records, added)
|
||||
grant_records(supervisor, records, permission.revision)
|
||||
second = supervisor.prepare_conversation(added)[0]
|
||||
assert second.task_id != first.task_id
|
||||
oracle = ScriptedOracle(records)
|
||||
bind_oracle(supervisor, oracle)
|
||||
result = await supervisor._create_runner().run(second.task_id)
|
||||
assert not oracle.errors, oracle.errors
|
||||
assert await node_snapshots(client, DENIED_IDS) == before
|
||||
assert result.status == "completed", (result.last_error_code, oracle.stages)
|
||||
# SDK 2.0.1 keeps the `limit=1` pre-check removed, so
|
||||
# skip reconcile, so the second write now compares against the
|
||||
# already-approved candidates instead of blindly ADDing them. The
|
||||
# contract is the reuse itself, not the absence of the call: one
|
||||
# reconcile pass, the returning ids are the ones already governed,
|
||||
# and no third/fourth memory appears.
|
||||
assert oracle.stages.count("reconcile") == 1
|
||||
assert {item["memory_id"] for item in oracle.reconciles[0]} == ids
|
||||
assert await assert_outputs(supervisor, second, records, client) == ids
|
||||
assert len(supervisor.governance.list(user_id="local")) == 2
|
||||
record_receipt(test_node, supervisor, result, oracle, phase="reuse", hidden=True)
|
||||
|
||||
asyncio.run(scenario())
|
||||
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
@@ -62,5 +62,10 @@ def test_the_installed_sdk_is_the_pinned_one():
|
||||
|
||||
def test_the_pin_matches_the_lock_file_and_its_wheel():
|
||||
lock = (DOCUMENT.parent / "apps/core/uv.lock").read_text(encoding="utf-8")
|
||||
assert f"{HY_MEMORY_DISTRIBUTION}/{HY_MEMORY_VERSION}/" in lock
|
||||
assert re.search(
|
||||
rf'\[\[package\]\]\nname = "{HY_MEMORY_DISTRIBUTION}"\n'
|
||||
rf'version = "{HY_MEMORY_VERSION}"',
|
||||
lock,
|
||||
)
|
||||
assert f"hy_memory-{HY_MEMORY_VERSION}-" in lock
|
||||
assert f'specifier = "=={HY_MEMORY_VERSION}"' in lock
|
||||
@@ -69,7 +69,7 @@ def test_no_idempotency_key_means_unknown_must_remain() -> None:
|
||||
|
||||
|
||||
def test_measured_sdk_verdict_is_recorded() -> None:
|
||||
"""Pin the measured answer for `hy_memory-internal==2.0.1`.
|
||||
"""Pin the measured answer for `hy-memory==2.0.2`.
|
||||
|
||||
The finding: `add()` takes a `request_id` but nothing de-duplicates on it,
|
||||
while `MemoryInspector.get_pipeline_logs(request_id=...)` can look the
|
||||
|
||||
@@ -1,547 +0,0 @@
|
||||
"""RC4:记忆生命周期(有效 / 被取代 / 去重隐藏 / 过期)直接读 HY,产品账本不再逐类镜像。
|
||||
|
||||
产品只保留 HY 没有的三样:使用范围、来源撤权后的隐藏过滤、用户删除记录。
|
||||
这里钉住:HY 自己改了状态(导入写回从未见过的也算),读路径立刻按 HY 走;
|
||||
HY 读不到时读路径退回账本,而不是把"没读到"当成"全部有效"。
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
from hy_memory.models.memory import MemoryStatus as HyStatus
|
||||
|
||||
from one_memory.hy_lifecycle import HyLifecycle, HyLifecycleUnreadable
|
||||
from one_memory.hy_memory_bridge import HyMemoryBridge
|
||||
from one_memory.imports.grants import SourceGrantStore
|
||||
from one_memory.imports.models import EvidenceInput, EvidenceOrigin, OriginMetadata, WrittenBy
|
||||
from one_memory.imports.sources.registry import RootCandidateKind
|
||||
from one_memory.imports.storage import EvidenceStore
|
||||
from one_memory.memory_governance import (
|
||||
HyCreate,
|
||||
MemoryDraft,
|
||||
MemoryGovernanceStore,
|
||||
MemoryKind,
|
||||
MemoryMutationConflictError,
|
||||
MemoryStatus,
|
||||
)
|
||||
from one_memory.memory_library import read_memory_library
|
||||
from tests.person_door import as_person
|
||||
from tests.test_hy_permission_raw_context import (
|
||||
AGENT_TEXT,
|
||||
USER_TEXT,
|
||||
ScriptedOracle,
|
||||
assert_outputs,
|
||||
bind_oracle,
|
||||
native_client,
|
||||
on_native_loop,
|
||||
raw_client,
|
||||
)
|
||||
|
||||
FACT = "用户的发布前检查要求一次权限评审"
|
||||
|
||||
|
||||
def _evidence(database: Path, message_id: str, *, conversation: str = "conversation") -> str:
|
||||
return (
|
||||
EvidenceStore(database)
|
||||
.put(
|
||||
EvidenceInput(
|
||||
source_id="cursor",
|
||||
conversation_id=conversation,
|
||||
message_id=message_id,
|
||||
correlation_id=message_id,
|
||||
role="user",
|
||||
written_by=WrittenBy.PERSON,
|
||||
verbatim=f"原话 {message_id}",
|
||||
origin_meta={EvidenceOrigin.FILE: OriginMetadata()},
|
||||
)
|
||||
)
|
||||
.record.evidence_id
|
||||
)
|
||||
|
||||
|
||||
def _library(tmp_path: Path) -> tuple[Path, MemoryGovernanceStore, dict[str, str]]:
|
||||
"""Four memories, newest last: a, b, c, d. Each rests on its own evidence."""
|
||||
database = tmp_path / "one-memory.sqlite3"
|
||||
SourceGrantStore(database, home=tmp_path)
|
||||
store = MemoryGovernanceStore(database)
|
||||
ids: dict[str, str] = {}
|
||||
for key in ("a", "b", "c", "d"):
|
||||
evidence_id = _evidence(database, key)
|
||||
as_person(store).create(
|
||||
MemoryDraft(
|
||||
user_id="local",
|
||||
kind=MemoryKind.CHAT,
|
||||
content=f"{FACT} {key}",
|
||||
scopes=("all-apps",),
|
||||
evidence_ids=(evidence_id,),
|
||||
),
|
||||
request_id=f"create-{key}",
|
||||
memory_id=key,
|
||||
)
|
||||
ids[key] = evidence_id
|
||||
return database, store, ids
|
||||
|
||||
|
||||
def _ids(memories) -> list[str]:
|
||||
return [memory.memory_id for memory in memories]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- 账本读路径
|
||||
|
||||
|
||||
def test_what_hy_hides_leaves_every_current_read_and_shows_hy_state_by_id(tmp_path) -> None:
|
||||
_, store, _ = _library(tmp_path)
|
||||
# HY deduplicated d into b during a search, and replaced a with c.
|
||||
lifecycle = HyLifecycle(hidden={"d": ("b",), "a": ("c",)})
|
||||
|
||||
assert _ids(store.list(user_id="local")) == ["d", "c", "b", "a"]
|
||||
assert _ids(store.list(user_id="local", lifecycle=lifecycle)) == ["c", "b"]
|
||||
found, more = store.find_confirmed_matching_terms(
|
||||
user_id="local", terms=(FACT,), lifecycle=lifecycle
|
||||
)
|
||||
assert _ids(found) == ["c", "b"] and more is False
|
||||
|
||||
hidden = store.get("d", user_id="local", lifecycle=lifecycle)
|
||||
assert hidden.status is MemoryStatus.SUPERSEDED
|
||||
assert hidden.superseded_by == ("b",) and hidden.deleted_at is None
|
||||
kept = store.get("b", user_id="local", lifecycle=lifecycle)
|
||||
assert kept.status is MemoryStatus.CONFIRMED and kept.supersedes == ("d",)
|
||||
assert _ids(
|
||||
store.list(user_id="local", status=MemoryStatus.SUPERSEDED, lifecycle=lifecycle)
|
||||
) == [
|
||||
"d",
|
||||
"a",
|
||||
]
|
||||
assert _ids(
|
||||
store.list(user_id="local", status=MemoryStatus.CONFIRMED, lifecycle=lifecycle)
|
||||
) == [
|
||||
"c",
|
||||
"b",
|
||||
]
|
||||
|
||||
|
||||
def test_hidden_rows_are_excluded_before_the_page_is_cut(tmp_path) -> None:
|
||||
_, store, _ = _library(tmp_path)
|
||||
lifecycle = HyLifecycle(hidden={"d": ("b",), "c": ("b",)})
|
||||
|
||||
assert _ids(store.list(user_id="local", limit=1, lifecycle=lifecycle)) == ["b"]
|
||||
assert _ids(store.list(user_id="local", limit=1, offset=1, lifecycle=lifecycle)) == ["a"]
|
||||
found, more = store.find_confirmed_matching_terms(
|
||||
user_id="local", terms=(FACT,), limit=1, lifecycle=lifecycle
|
||||
)
|
||||
assert _ids(found) == ["b"] and more is True
|
||||
|
||||
|
||||
def test_a_user_deletion_stays_a_deletion_whatever_hy_says(tmp_path) -> None:
|
||||
_, store, _ = _library(tmp_path)
|
||||
b = store.get("b", user_id="local")
|
||||
as_person(store).delete(
|
||||
"b", user_id="local", request_id="delete-b", expected_revision=b.revision
|
||||
)
|
||||
# HY still holds b as active until the deletion outbox reaches it.
|
||||
lifecycle = HyLifecycle(hidden={})
|
||||
|
||||
assert "b" not in _ids(store.list(user_id="local", lifecycle=lifecycle))
|
||||
assert store.get("b", user_id="local", lifecycle=lifecycle).status is MemoryStatus.DELETED
|
||||
assert _ids(store.list(user_id="local", status=MemoryStatus.DELETED, lifecycle=lifecycle)) == [
|
||||
"b"
|
||||
]
|
||||
|
||||
|
||||
def test_hy_restoring_a_memory_it_had_replaced_brings_it_back(tmp_path) -> None:
|
||||
"""A row an older build mirrored as superseded is current again once HY says so.
|
||||
|
||||
HY restores the predecessor when the head of a chain is deleted; no write
|
||||
reaches the ledger for that, so only reading HY can see it.
|
||||
"""
|
||||
from one_memory.memory_governance import HySupersede
|
||||
|
||||
_, store, _ = _library(tmp_path)
|
||||
a = store.get("a", user_id="local")
|
||||
store.write_from_hy(
|
||||
HySupersede(
|
||||
memory_id="a",
|
||||
request_id="hy-supersede:t:a",
|
||||
expected_revision=a.revision,
|
||||
superseded_by=("c",),
|
||||
),
|
||||
user_id="local",
|
||||
derivation=store.hy_derivation(),
|
||||
)
|
||||
assert "a" not in _ids(store.list(user_id="local"))
|
||||
|
||||
restored = HyLifecycle(hidden={})
|
||||
assert "a" in _ids(store.list(user_id="local", lifecycle=restored))
|
||||
memory = store.get("a", user_id="local", lifecycle=restored)
|
||||
assert memory.status is MemoryStatus.CONFIRMED and memory.superseded_by == ()
|
||||
assert store.get("c", user_id="local", lifecycle=restored).supersedes == ()
|
||||
|
||||
|
||||
def test_without_a_snapshot_the_ledger_reads_as_before(tmp_path) -> None:
|
||||
_, store, _ = _library(tmp_path)
|
||||
|
||||
assert _ids(store.list(user_id="local", lifecycle=None)) == ["d", "c", "b", "a"]
|
||||
assert store.get("d", user_id="local", lifecycle=None).status is MemoryStatus.CONFIRMED
|
||||
|
||||
|
||||
def test_a_superseded_memory_does_not_come_back_after_reopen(tmp_path) -> None:
|
||||
"""Reopening the library may not resurrect what HY already replaced.
|
||||
|
||||
The ledger's own mirrored supersession is the floor that holds while HY is
|
||||
not open; a second read path must not treat "no snapshot yet" as "active".
|
||||
"""
|
||||
from one_memory.memory_governance import HySupersede
|
||||
|
||||
database, store, _ = _library(tmp_path)
|
||||
a = store.get("a", user_id="local")
|
||||
store.write_from_hy(
|
||||
HySupersede(
|
||||
memory_id="a",
|
||||
request_id="hy-supersede:reopen:a",
|
||||
expected_revision=a.revision,
|
||||
superseded_by=("c",),
|
||||
),
|
||||
user_id="local",
|
||||
derivation=store.hy_derivation(),
|
||||
)
|
||||
reopened = MemoryGovernanceStore(database)
|
||||
assert "a" not in _ids(reopened.list(user_id="local"))
|
||||
page = read_memory_library(database, limit=100)
|
||||
assert "a" not in [item.memory_id for item in page.items]
|
||||
|
||||
|
||||
def test_revoked_source_stays_hidden_even_with_a_stale_lifecycle_snapshot(tmp_path) -> None:
|
||||
"""Authority outranks an old HY snapshot: no snapshot may re-expose it."""
|
||||
database, store, _ = _library(tmp_path)
|
||||
stale = HyLifecycle(hidden={})
|
||||
assert _ids(store.list(user_id="local", lifecycle=stale)) == ["d", "c", "b", "a"]
|
||||
|
||||
grants = SourceGrantStore(database, home=tmp_path)
|
||||
root = tmp_path / "cursor-root"
|
||||
root.mkdir()
|
||||
grants.grant(
|
||||
user_id="local",
|
||||
source_id="cursor",
|
||||
root=root,
|
||||
candidate_kind=RootCandidateKind.DEFAULT,
|
||||
)
|
||||
grants.revoke(user_id="local", source_id="cursor")
|
||||
# Documented contract: the ledger itself still lists the rows it owns.
|
||||
# Revocation is enforced by the read projections and external authority,
|
||||
# so a stale lifecycle snapshot must not re-expose them there.
|
||||
assert _ids(store.list(user_id="local", lifecycle=stale)) == ["d", "c", "b", "a"]
|
||||
page = read_memory_library(database, source="cursor", limit=100, lifecycle=stale)
|
||||
assert page.total == 0
|
||||
|
||||
|
||||
def test_revocation_survives_reopen_and_keeps_the_ledger_intact(tmp_path) -> None:
|
||||
"""Revocation hides reads; it does not delete, and it must outlive a restart.
|
||||
|
||||
The governance store keeps its row by design, so a downstream consumer has
|
||||
to read through authority and must not treat the surviving row as current.
|
||||
"""
|
||||
database, _, _ = _library(tmp_path)
|
||||
grants = SourceGrantStore(database, home=tmp_path)
|
||||
root = tmp_path / "cursor-root"
|
||||
root.mkdir()
|
||||
grants.grant(
|
||||
user_id="local",
|
||||
source_id="cursor",
|
||||
root=root,
|
||||
candidate_kind=RootCandidateKind.DEFAULT,
|
||||
)
|
||||
grants.revoke(user_id="local", source_id="cursor")
|
||||
|
||||
reopened = MemoryGovernanceStore(database)
|
||||
receipt = SourceGrantStore(database, home=tmp_path).receipt(user_id="local", source_id="cursor")
|
||||
|
||||
assert receipt is not None and receipt.revoked_at is not None
|
||||
# Rows survive; they are simply not readable through this source.
|
||||
assert len(reopened.list(user_id="local")) == 4
|
||||
assert read_memory_library(database, source="cursor", limit=100).total == 0
|
||||
|
||||
|
||||
def test_the_memory_library_page_and_total_follow_hy(tmp_path) -> None:
|
||||
database, _, _ = _library(tmp_path)
|
||||
lifecycle = HyLifecycle(hidden={"d": ("b",)})
|
||||
|
||||
page = read_memory_library(database, lifecycle=lifecycle)
|
||||
assert _ids(page.items) == ["c", "b", "a"] and page.total == 3
|
||||
replaced = read_memory_library(database, status=MemoryStatus.SUPERSEDED, lifecycle=lifecycle)
|
||||
assert _ids(replaced.items) == ["d"] and replaced.total == 1
|
||||
assert replaced.items[0].superseded_by == ("b",)
|
||||
active = read_memory_library(database, active_only=True, lifecycle=lifecycle, limit=1)
|
||||
assert _ids(active.items) == ["c"] and active.total == 3
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- 删除语义
|
||||
|
||||
|
||||
def test_a_new_conversation_saying_a_deleted_fact_is_remembered_again(tmp_path) -> None:
|
||||
"""Deleting removes that memory, not the fact: no content tombstone.
|
||||
|
||||
Only the exact old evidence stays guarded (RC10); a different conversation
|
||||
saying the same thing later is a new statement.
|
||||
"""
|
||||
database, store, _ = _library(tmp_path)
|
||||
a = store.get("a", user_id="local")
|
||||
as_person(store).delete(
|
||||
"a", user_id="local", request_id="delete-a", expected_revision=a.revision
|
||||
)
|
||||
again = _evidence(database, "again", conversation="later-conversation")
|
||||
|
||||
created = store.write_from_hy(
|
||||
HyCreate(
|
||||
draft=MemoryDraft(
|
||||
user_id="local",
|
||||
kind=MemoryKind.CHAT,
|
||||
content=a.content,
|
||||
scopes=("all-apps",),
|
||||
evidence_ids=(again,),
|
||||
),
|
||||
request_id="extract:later:new-a",
|
||||
memory_id="new-a",
|
||||
),
|
||||
user_id="local",
|
||||
derivation=store.hy_derivation((again,)),
|
||||
)
|
||||
|
||||
assert created.status is MemoryStatus.CONFIRMED and created.content == a.content
|
||||
assert "new-a" in _ids(store.list(user_id="local"))
|
||||
assert store.blocked_source_ids(user_id="local", evidence_ids=(again,)) == set()
|
||||
|
||||
|
||||
def test_the_store_uses_its_lifecycle_source_for_reads_and_person_edits(tmp_path) -> None:
|
||||
_, store, _ = _library(tmp_path)
|
||||
hidden = HyLifecycle(hidden={"d": ("b",)})
|
||||
calls: list[tuple[str, tuple[str, ...] | None]] = []
|
||||
|
||||
def source(user_id: str, ids: tuple[str, ...] | None = None) -> HyLifecycle:
|
||||
calls.append((user_id, ids))
|
||||
return hidden
|
||||
|
||||
store.read_lifecycle_from(source)
|
||||
assert _ids(store.list(user_id="local")) == ["c", "b", "a"]
|
||||
current = store.get("d", user_id="local")
|
||||
assert current is not None and current.status is MemoryStatus.SUPERSEDED
|
||||
assert calls == [("local", None), ("local", ("d",))]
|
||||
# Editing what HY has set aside would fork the chain it already replaced.
|
||||
with pytest.raises(MemoryMutationConflictError, match="memory_superseded"):
|
||||
as_person(store).correct(
|
||||
"d",
|
||||
user_id="local",
|
||||
request_id="correct-hidden",
|
||||
expected_revision=current.revision,
|
||||
content=f"{FACT} corrected",
|
||||
)
|
||||
|
||||
|
||||
def test_a_rejected_or_missing_source_answers_degraded_not_all_active(tmp_path) -> None:
|
||||
"""Wiring contract for C0/C4: unreadable means unavailable, never 'active'.
|
||||
|
||||
A source that raises must be visible as an error to the composition root; a
|
||||
source that answers `None` keeps the ledger floor rather than inventing
|
||||
freshness. Neither may be read as 'HY says everything is current'.
|
||||
"""
|
||||
_, store, _ = _library(tmp_path)
|
||||
|
||||
def broken(_user_id: str, _ids: tuple[str, ...] | None = None) -> HyLifecycle:
|
||||
raise HyLifecycleUnreadable("hy_lifecycle_unreadable")
|
||||
|
||||
store.read_lifecycle_from(broken)
|
||||
with pytest.raises(HyLifecycleUnreadable):
|
||||
store.list(user_id="local")
|
||||
with pytest.raises(HyLifecycleUnreadable):
|
||||
store.get("d", user_id="local")
|
||||
|
||||
# Before HY is open, `None` keeps the ledger's own supersession floor:
|
||||
# a memory HY replaced stays hidden because the write was observed here.
|
||||
from one_memory.memory_governance import HySupersede
|
||||
|
||||
_, store, _ = _library(tmp_path)
|
||||
a = store.get("a", user_id="local")
|
||||
store.write_from_hy(
|
||||
HySupersede(
|
||||
memory_id="a",
|
||||
request_id="hy-supersede:floor:a",
|
||||
expected_revision=a.revision,
|
||||
superseded_by=("c",),
|
||||
),
|
||||
user_id="local",
|
||||
derivation=store.hy_derivation(),
|
||||
)
|
||||
store.read_lifecycle_from(lambda _user_id, _ids=None: None)
|
||||
assert "a" not in _ids(store.list(user_id="local"))
|
||||
|
||||
|
||||
def test_one_snapshot_per_page_not_one_per_row(tmp_path) -> None:
|
||||
"""Wiring contract for C4: a page read asks HY once, not once per row."""
|
||||
_, store, _ = _library(tmp_path)
|
||||
calls: list[tuple[str, tuple[str, ...] | None]] = []
|
||||
|
||||
def source(user_id: str, ids: tuple[str, ...] | None = None) -> HyLifecycle:
|
||||
calls.append((user_id, ids))
|
||||
return HyLifecycle(hidden={"d": ("b",)})
|
||||
|
||||
store.read_lifecycle_from(source)
|
||||
assert len(store.list(user_id="local")) == 3
|
||||
assert calls == [("local", None)]
|
||||
|
||||
|
||||
def test_an_unreadable_lifecycle_blocks_edits_but_not_a_local_delete(tmp_path) -> None:
|
||||
_, store, _ = _library(tmp_path)
|
||||
current = store.get("d", user_id="local", lifecycle=None)
|
||||
assert current is not None
|
||||
|
||||
def unreadable(_user_id: str, _ids: tuple[str, ...] | None = None) -> HyLifecycle:
|
||||
raise HyLifecycleUnreadable("hy_lifecycle_unreadable")
|
||||
|
||||
store.read_lifecycle_from(unreadable)
|
||||
with pytest.raises(HyLifecycleUnreadable):
|
||||
as_person(store).correct(
|
||||
"d",
|
||||
user_id="local",
|
||||
request_id="correct-unreadable",
|
||||
expected_revision=current.revision,
|
||||
content=f"{FACT} corrected",
|
||||
)
|
||||
|
||||
deleted = as_person(store).delete(
|
||||
"d",
|
||||
user_id="local",
|
||||
request_id="delete-unreadable",
|
||||
expected_revision=current.revision,
|
||||
)
|
||||
assert deleted.status is MemoryStatus.DELETED
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- 真 HY
|
||||
|
||||
|
||||
async def _first_turn(supervisor, records):
|
||||
first = supervisor.prepare_conversation(records[0])[0]
|
||||
client = await native_client(supervisor, first)
|
||||
bind_oracle(supervisor, ScriptedOracle(records))
|
||||
assert (await supervisor._create_runner().run(first.task_id)).status == "completed"
|
||||
await assert_outputs(supervisor, first, records, client)
|
||||
by_content = {m.content: m for m in supervisor.governance.list(user_id="local")}
|
||||
bridge = supervisor._bridge(first.profile_snapshot, None, supervisor.data_dir)
|
||||
return client, bridge, by_content[USER_TEXT], by_content[AGENT_TEXT]
|
||||
|
||||
|
||||
def test_hy_search_dedup_is_read_straight_from_hy(tmp_path) -> None:
|
||||
"""HY shadows a duplicate while searching; no ingest write-back ever sees it."""
|
||||
|
||||
async def scenario():
|
||||
async with raw_client(tmp_path) as (supervisor, records, _):
|
||||
client, bridge, dup, kept = await _first_turn(supervisor, records)
|
||||
assert isinstance(bridge, HyMemoryBridge)
|
||||
assert bridge.lifecycle_now("local") == HyLifecycle(hidden={})
|
||||
|
||||
await on_native_loop(
|
||||
client,
|
||||
client._vector_store.update_payload(
|
||||
dup.memory_id,
|
||||
{
|
||||
"status": HyStatus.SHADOW.value,
|
||||
"is_latest": False,
|
||||
"shadow_reason": "dedup:search",
|
||||
"dedup_kept_id": kept.memory_id,
|
||||
},
|
||||
),
|
||||
)
|
||||
|
||||
lifecycle = bridge.lifecycle_now("local")
|
||||
assert lifecycle == HyLifecycle(hidden={dup.memory_id: (kept.memory_id,)})
|
||||
assert _ids(supervisor.governance.list(user_id="local", lifecycle=lifecycle)) == [
|
||||
kept.memory_id
|
||||
]
|
||||
assert bridge.lifecycle_now("someone-else") == HyLifecycle(hidden={})
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_a_replacement_hy_records_on_the_old_node_is_read_from_hy(tmp_path) -> None:
|
||||
async def scenario():
|
||||
async with raw_client(tmp_path) as (supervisor, records, _):
|
||||
client, bridge, old, head = await _first_turn(supervisor, records)
|
||||
await on_native_loop(
|
||||
client,
|
||||
client._vector_store.update_payload(
|
||||
old.memory_id,
|
||||
{
|
||||
"status": HyStatus.SUPERSEDED.value,
|
||||
"is_latest": False,
|
||||
"superseded_by": [head.memory_id],
|
||||
},
|
||||
),
|
||||
)
|
||||
|
||||
lifecycle = bridge.lifecycle_now("local")
|
||||
assert lifecycle == HyLifecycle(hidden={old.memory_id: (head.memory_id,)})
|
||||
memory = supervisor.governance.get(old.memory_id, user_id="local", lifecycle=lifecycle)
|
||||
assert memory.status is MemoryStatus.SUPERSEDED
|
||||
assert memory.superseded_by == (head.memory_id,)
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_an_unreadable_hy_store_raises_instead_of_reading_as_all_active(tmp_path) -> None:
|
||||
async def scenario():
|
||||
async with raw_client(tmp_path) as (supervisor, records, _):
|
||||
client, bridge, dup, kept = await _first_turn(supervisor, records)
|
||||
await on_native_loop(
|
||||
client,
|
||||
client._vector_store.update_payload(
|
||||
dup.memory_id,
|
||||
{"status": HyStatus.SHADOW.value, "is_latest": False},
|
||||
),
|
||||
)
|
||||
# Positive first: this store does hold something hidden.
|
||||
assert bridge.lifecycle_now("local") == HyLifecycle(hidden={dup.memory_id: ()})
|
||||
|
||||
collection = client._vector_store._collection
|
||||
|
||||
def broken(*args, **kwargs):
|
||||
raise RuntimeError("chroma is gone")
|
||||
|
||||
original = collection.get
|
||||
collection.get = broken
|
||||
try:
|
||||
with pytest.raises(HyLifecycleUnreadable):
|
||||
bridge.lifecycle_now("local")
|
||||
finally:
|
||||
collection.get = original
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_hy_not_opened_yet_is_only_safe_when_no_store_exists(tmp_path) -> None:
|
||||
async def scenario():
|
||||
async with raw_client(tmp_path) as (supervisor, records, _):
|
||||
first = supervisor.prepare_conversation(records[0])[0]
|
||||
bridge = supervisor._bridge(first.profile_snapshot, None, supervisor.data_dir)
|
||||
assert bridge.lifecycle_now("local") is None
|
||||
|
||||
(supervisor.data_dir / "hy-memory" / "governed-v2" / "graph").mkdir(parents=True)
|
||||
with pytest.raises(HyLifecycleUnreadable, match="hy_lifecycle_not_ready"):
|
||||
bridge.lifecycle_now("local")
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_malformed_hy_lifecycle_rows_fail_closed(tmp_path) -> None:
|
||||
async def scenario():
|
||||
async with raw_client(tmp_path) as (supervisor, records, _):
|
||||
client, bridge, _dup, _kept = await _first_turn(supervisor, records)
|
||||
collection = client._vector_store._collection
|
||||
original = collection.get
|
||||
collection.get = lambda **_kwargs: {"ids": ["missing-metadata"], "metadatas": []}
|
||||
try:
|
||||
with pytest.raises(HyLifecycleUnreadable, match="hy_lifecycle_store_mismatch"):
|
||||
bridge.lifecycle_now("local")
|
||||
finally:
|
||||
collection.get = original
|
||||
|
||||
asyncio.run(scenario())
|
||||
@@ -1,145 +0,0 @@
|
||||
"""Where an HY write spends the time that is not a model call."""
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import time
|
||||
|
||||
import httpx
|
||||
|
||||
from one_memory.diagnostic_log import configure_diagnostics, reset_diagnostics
|
||||
from one_memory.hy_permission_context import permission_bridge_factory
|
||||
from one_memory.hy_step_timing import HyStepTimer
|
||||
from tests.test_hy_permission_context import whole_client
|
||||
|
||||
|
||||
def _handler(record):
|
||||
def handler(request: httpx.Request):
|
||||
body = json.loads(request.content)
|
||||
if request.url.path.endswith("embeddings"):
|
||||
return httpx.Response(
|
||||
200,
|
||||
json={
|
||||
"object": "list",
|
||||
"model": "test-embedding",
|
||||
"data": [
|
||||
{"index": i, "object": "embedding", "embedding": [1.0] + [0.0] * 383}
|
||||
for i, _ in enumerate(body["input"])
|
||||
],
|
||||
"usage": {"prompt_tokens": 1, "total_tokens": 1},
|
||||
},
|
||||
)
|
||||
return httpx.Response(
|
||||
200,
|
||||
json={
|
||||
"id": "fixture-response",
|
||||
"object": "chat.completion",
|
||||
"created": 1,
|
||||
"model": "test-generation",
|
||||
"choices": [
|
||||
{
|
||||
"index": 0,
|
||||
"finish_reason": "stop",
|
||||
"message": {
|
||||
"role": "assistant",
|
||||
"content": json.dumps(
|
||||
[
|
||||
{
|
||||
"op": "ADD",
|
||||
"content": record.verbatim,
|
||||
"layer": "L2_FACT",
|
||||
"owner": "agent",
|
||||
}
|
||||
]
|
||||
),
|
||||
},
|
||||
}
|
||||
],
|
||||
"usage": {"prompt_tokens": 1, "completion_tokens": 1, "total_tokens": 2},
|
||||
},
|
||||
)
|
||||
|
||||
return handler
|
||||
|
||||
|
||||
def test_leased_ingest_logs_each_hy_step_with_duration_and_count(tmp_path, monkeypatch):
|
||||
# Per-step tracing is its own tier on top of the master switch, so a test
|
||||
# that reads the step events has to ask for both.
|
||||
monkeypatch.setenv("ONE_MEMORY_DIAGNOSTICS", "1")
|
||||
monkeypatch.setenv("ONE_MEMORY_HY_STEP_TRACE", "1")
|
||||
receipt = configure_diagnostics(tmp_path / "diagnostics")
|
||||
|
||||
async def scenario():
|
||||
with whole_client(tmp_path) as (supervisor, record, _, _):
|
||||
supervisor.permission_bridge_factory = permission_bridge_factory(
|
||||
supervisor._bridge, http_transport=httpx.MockTransport(_handler(record))
|
||||
)
|
||||
tasks = supervisor.prepare_conversation(record)
|
||||
try:
|
||||
result = await supervisor._create_runner().run(tasks[0].task_id)
|
||||
assert result.status == "completed", result.last_error_code
|
||||
finally:
|
||||
await supervisor.shutdown()
|
||||
return record, tasks[0].task_id
|
||||
|
||||
try:
|
||||
record, task_id = asyncio.run(scenario())
|
||||
finally:
|
||||
reset_diagnostics()
|
||||
written = receipt.path.read_text()
|
||||
events = [json.loads(line) for line in written.splitlines()]
|
||||
steps = {
|
||||
event["details"]["phase"]: event
|
||||
for event in events
|
||||
if event["event"] == "hy.step.completed"
|
||||
}
|
||||
# Memory content with no prior candidate takes HY's no-model ADD path.
|
||||
assert "model" not in steps
|
||||
expected = {
|
||||
"embedding": "hy.embedding",
|
||||
"vector_store_read": "hy.vector_store",
|
||||
"vector_store_write": "hy.vector_store",
|
||||
"permission_check": "hy.permission_store",
|
||||
"permission_store_read": "hy.permission_store",
|
||||
"permission_store_write": "hy.permission_store",
|
||||
"write_verification": "hy.write_verification",
|
||||
"reconcile": "hy.reconcile",
|
||||
}
|
||||
assert expected.items() <= {name: event["component"] for name, event in steps.items()}.items()
|
||||
for event in steps.values():
|
||||
assert event["task_id"] == task_id
|
||||
assert event["request_id"]
|
||||
assert isinstance(event["duration_ms"], int)
|
||||
assert event["details"]["call_count"] >= 1
|
||||
assert event["details"]["summed_ms"] + 1 >= event["duration_ms"]
|
||||
summaries = [event for event in events if event["event"] == "hy.steps.summary"]
|
||||
assert len(summaries) == 1
|
||||
summary = summaries[0]
|
||||
assert summary["component"] == "hy.bridge"
|
||||
assert summary["task_id"] == task_id
|
||||
details = summary["details"]
|
||||
assert details["model_ms"] == 0
|
||||
assert details["accounted_ms"] + details["unaccounted_ms"] == summary["duration_ms"]
|
||||
assert details["accounted_ms"] + 1 >= max(event["duration_ms"] for event in steps.values())
|
||||
assert record.verbatim not in written
|
||||
|
||||
|
||||
def test_nested_same_step_counts_once_and_overlap_counts_wall_clock_once():
|
||||
timer = HyStepTimer()
|
||||
with timer.step("permission_check"), timer.step("permission_check"):
|
||||
time.sleep(0.02)
|
||||
|
||||
async def overlapping():
|
||||
async def one():
|
||||
with timer.step("embedding", items=2):
|
||||
await asyncio.sleep(0.05)
|
||||
|
||||
await asyncio.gather(one(), one())
|
||||
|
||||
asyncio.run(overlapping())
|
||||
measured = timer.measurements()
|
||||
assert measured["permission_check"].call_count == 1
|
||||
embedding = measured["embedding"]
|
||||
assert embedding.call_count == 2
|
||||
assert embedding.item_count == 4
|
||||
assert embedding.summed_ms >= 95
|
||||
assert 45 <= embedding.busy_ms < 90
|
||||
@@ -1,453 +0,0 @@
|
||||
"""HY decides supersessions; the ledger mirrors them. Pinned SDK, scripted HTTP."""
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
from copy import deepcopy
|
||||
|
||||
import httpx
|
||||
import pytest
|
||||
from hy_memory.agent.tools.basic_profile import upsert_basic_profile
|
||||
from hy_memory.coding.types import WorkingFact
|
||||
from hy_memory.models.memory import MemoryLayer
|
||||
from hy_memory.models.memory import MemoryStatus as HyStatus
|
||||
|
||||
import tests.test_hy_permission_raw_context as raw_context
|
||||
from one_memory.hy_memory_bridge import HyMemoryBridgeError
|
||||
from one_memory.hy_permission_coding_store import HyPermissionCodingStore, hy_coding_fingerprint
|
||||
from one_memory.hy_permission_store import HyNodePermission
|
||||
from one_memory.memory_governance import MemoryKind, MemoryStatus
|
||||
from tests.test_hy_coding_lease import TOOL_TEXT, CodingOracle, by_kind, run_coding_turn
|
||||
from tests.test_hy_permission_raw_context import (
|
||||
AGENT_TEXT,
|
||||
USER_TEXT,
|
||||
ScriptedOracle,
|
||||
assert_outputs,
|
||||
bind_oracle,
|
||||
grant_records,
|
||||
native_client,
|
||||
on_native_loop,
|
||||
put_turn,
|
||||
raw_client,
|
||||
)
|
||||
from tests.test_hy_permission_store import Backend, Lease, node, wrapper
|
||||
|
||||
NEW_TEXT = "The user now requires two permission reviews before the synthetic release."
|
||||
|
||||
|
||||
def _vector(text):
|
||||
axis = {USER_TEXT: 1, AGENT_TEXT: 2, NEW_TEXT: 3}.get(text, 0)
|
||||
return [float(index == axis) for index in range(384)]
|
||||
|
||||
|
||||
class SupersedingOracle(ScriptedOracle):
|
||||
"""HY's reconciler replaces existing claims with the new one instead of updating them."""
|
||||
|
||||
def __init__(self, records, *, chain):
|
||||
super().__init__(records)
|
||||
self.chain = chain
|
||||
|
||||
def respond(self, request):
|
||||
response = super().respond(request)
|
||||
if self.stages[-1] != "reconcile":
|
||||
return response
|
||||
body = json.loads(response.content)
|
||||
ops = json.loads(body["choices"][0]["message"]["content"])
|
||||
by_content = {op["content"]: op for op in ops}
|
||||
supersede = {
|
||||
"op": "SUPERSEDE",
|
||||
"content": NEW_TEXT,
|
||||
"owner": "user",
|
||||
"layer": "L2_FACT",
|
||||
"tags": [],
|
||||
"supersede_reason": "the user raised the requirement",
|
||||
}
|
||||
if self.chain:
|
||||
supersede["memory_ids"] = [self.chain[0], self.chain[1]]
|
||||
replaced = {USER_TEXT, AGENT_TEXT}
|
||||
else:
|
||||
supersede["memory_id"] = by_content[USER_TEXT]["memory_id"]
|
||||
replaced = {USER_TEXT}
|
||||
ops = [op for op in ops if op["content"] not in replaced | {NEW_TEXT}] + [supersede]
|
||||
body["choices"][0]["message"]["content"] = json.dumps(ops)
|
||||
return httpx.Response(200, json=body)
|
||||
|
||||
|
||||
async def _first_turn(supervisor, records):
|
||||
first = supervisor.prepare_conversation(records[0])[0]
|
||||
client = await native_client(supervisor, first)
|
||||
bind_oracle(supervisor, ScriptedOracle(records))
|
||||
assert (await supervisor._create_runner().run(first.task_id)).status == "completed"
|
||||
await assert_outputs(supervisor, first, records, client)
|
||||
by_content = {m.content: m for m in supervisor.governance.list(user_id="local")}
|
||||
return client, by_content[USER_TEXT], by_content[AGENT_TEXT]
|
||||
|
||||
|
||||
async def _superseding_turn(supervisor, records, permission, *, chain=()):
|
||||
added = put_turn(supervisor.evidence, "user-raise", "user", NEW_TEXT, 3)
|
||||
records = (*records, added)
|
||||
grant_records(supervisor, records, permission.revision)
|
||||
second = supervisor.prepare_conversation(added)[0]
|
||||
oracle = SupersedingOracle(records, chain=chain)
|
||||
bind_oracle(supervisor, oracle)
|
||||
result = await supervisor._create_runner().run(second.task_id)
|
||||
assert not oracle.errors, oracle.errors
|
||||
assert result.status == "completed", (result.last_error_code, oracle.stages)
|
||||
return oracle
|
||||
|
||||
|
||||
def _head(supervisor):
|
||||
(head,) = [
|
||||
m for m in supervisor.governance.list(user_id="local", limit=500) if m.content == NEW_TEXT
|
||||
]
|
||||
return head
|
||||
|
||||
|
||||
def test_a_memory_hy_supersedes_is_superseded_in_the_ledger_with_its_replacement(
|
||||
tmp_path, monkeypatch
|
||||
):
|
||||
monkeypatch.setattr(raw_context, "vector", _vector)
|
||||
|
||||
async def scenario():
|
||||
async with raw_client(tmp_path) as (supervisor, records, permission):
|
||||
client, replaced, kept = await _first_turn(supervisor, records)
|
||||
monkeypatch.setitem(raw_context.CLAIMS, NEW_TEXT, "user")
|
||||
await _superseding_turn(supervisor, records, permission)
|
||||
|
||||
native = await on_native_loop(
|
||||
client, client._vector_store.get_by_id(replaced.memory_id)
|
||||
)
|
||||
assert native.status == HyStatus.SUPERSEDED
|
||||
head = _head(supervisor)
|
||||
assert native.superseded_by == [head.memory_id]
|
||||
|
||||
old = supervisor.governance.get(replaced.memory_id, user_id="local")
|
||||
assert old.status is MemoryStatus.SUPERSEDED
|
||||
assert old.deleted_at is None and old.content == USER_TEXT
|
||||
assert old.superseded_by == (head.memory_id,)
|
||||
assert supervisor.governance.deletion(old.memory_id, user_id="local") is None
|
||||
assert head.status is MemoryStatus.CONFIRMED
|
||||
assert head.supersedes == (old.memory_id,)
|
||||
untouched = supervisor.governance.get(kept.memory_id, user_id="local")
|
||||
assert untouched.status is MemoryStatus.CONFIRMED
|
||||
assert untouched.supersedes == untouched.superseded_by == ()
|
||||
assert {m.memory_id for m in supervisor.governance.list(user_id="local")} == {
|
||||
head.memory_id,
|
||||
kept.memory_id,
|
||||
}
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_a_chain_fold_supersedes_every_folded_memory(tmp_path, monkeypatch):
|
||||
monkeypatch.setattr(raw_context, "vector", _vector)
|
||||
|
||||
async def scenario():
|
||||
async with raw_client(tmp_path) as (supervisor, records, permission):
|
||||
client, user, agent = await _first_turn(supervisor, records)
|
||||
monkeypatch.setitem(raw_context.CLAIMS, NEW_TEXT, "user")
|
||||
await _superseding_turn(
|
||||
supervisor, records, permission, chain=(agent.memory_id, user.memory_id)
|
||||
)
|
||||
|
||||
head = _head(supervisor)
|
||||
oldest = supervisor.governance.get(agent.memory_id, user_id="local")
|
||||
newer = supervisor.governance.get(user.memory_id, user_id="local")
|
||||
assert oldest.status is newer.status is MemoryStatus.SUPERSEDED
|
||||
assert oldest.superseded_by == (user.memory_id,)
|
||||
assert newer.superseded_by == (head.memory_id,)
|
||||
assert newer.supersedes == (agent.memory_id,)
|
||||
assert head.supersedes == (user.memory_id,)
|
||||
assert head.status is MemoryStatus.CONFIRMED
|
||||
assert [m.memory_id for m in supervisor.governance.list(user_id="local")] == [
|
||||
head.memory_id
|
||||
]
|
||||
for key in (agent.memory_id, user.memory_id):
|
||||
native = await on_native_loop(client, client._vector_store.get_by_id(key))
|
||||
assert native.status == HyStatus.SUPERSEDED
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
async def _turn_after_supersede(supervisor, records, permission):
|
||||
"""The user repeats the new requirement one turn after HY superseded the old one."""
|
||||
added = next(
|
||||
r
|
||||
for r in supervisor.evidence.list_conversation(records[0].evidence_id)
|
||||
if r.verbatim == NEW_TEXT
|
||||
)
|
||||
again = put_turn(supervisor.evidence, "user-again", "user", NEW_TEXT, 4)
|
||||
records = (*records, added, again)
|
||||
grant_records(supervisor, records, permission.revision + 1)
|
||||
third = supervisor.prepare_conversation(again)[0]
|
||||
oracle = ScriptedOracle(records)
|
||||
bind_oracle(supervisor, oracle)
|
||||
result = await supervisor._create_runner().run(third.task_id)
|
||||
assert not oracle.errors, oracle.errors
|
||||
return result, oracle
|
||||
|
||||
|
||||
def test_the_turn_after_a_supersede_still_completes(tmp_path, monkeypatch):
|
||||
monkeypatch.setattr(raw_context, "vector", _vector)
|
||||
|
||||
async def scenario():
|
||||
async with raw_client(tmp_path) as (supervisor, records, permission):
|
||||
_, replaced, _ = await _first_turn(supervisor, records)
|
||||
monkeypatch.setitem(raw_context.CLAIMS, NEW_TEXT, "user")
|
||||
await _superseding_turn(supervisor, records, permission)
|
||||
|
||||
# The old memory's evidence is in this turn's window; being
|
||||
# superseded by HY is not the user changing what was said.
|
||||
result, oracle = await _turn_after_supersede(supervisor, records, permission)
|
||||
|
||||
assert result.status == "completed", (result.last_error_code, oracle.stages)
|
||||
old = supervisor.governance.get(replaced.memory_id, user_id="local")
|
||||
assert old.status is MemoryStatus.SUPERSEDED
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_hy_still_sees_the_head_of_a_supersede_chain_on_the_next_turn(tmp_path, monkeypatch):
|
||||
monkeypatch.setattr(raw_context, "vector", _vector)
|
||||
|
||||
async def scenario():
|
||||
async with raw_client(tmp_path) as (supervisor, records, permission):
|
||||
await _first_turn(supervisor, records)
|
||||
monkeypatch.setitem(raw_context.CLAIMS, NEW_TEXT, "user")
|
||||
await _superseding_turn(supervisor, records, permission)
|
||||
head = _head(supervisor)
|
||||
|
||||
result, oracle = await _turn_after_supersede(supervisor, records, permission)
|
||||
|
||||
assert result.status == "completed", (result.last_error_code, oracle.stages)
|
||||
existing = {item["memory_id"] for batch in oracle.reconciles for item in batch}
|
||||
assert head.memory_id in existing, (head.memory_id, oracle.reconciles)
|
||||
repeated = [
|
||||
m
|
||||
for m in supervisor.governance.list(user_id="local", limit=500)
|
||||
if m.content == NEW_TEXT
|
||||
]
|
||||
assert [m.memory_id for m in repeated] == [head.memory_id]
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
class SupersedingFactOracle(CodingOracle):
|
||||
"""HY's fact reconciler judges the new session to continue the old task."""
|
||||
|
||||
def __init__(self, records, *, old_fact, **kwargs):
|
||||
super().__init__(records, **kwargs)
|
||||
self.old_fact = old_fact
|
||||
|
||||
def respond(self, request):
|
||||
body = json.loads(request.content)
|
||||
if request.url.path != "/v1/embeddings":
|
||||
prompt = body["messages"][-1]["content"]
|
||||
if prompt.startswith("A user task in a working-memory store has progressed"):
|
||||
self.requests.append(body)
|
||||
self.stages.append("chain_summary")
|
||||
return _completion(body, "The suite moved from pytest to tox.")
|
||||
response = super().respond(request)
|
||||
if self.stages[-1] != "fact_reconcile":
|
||||
return response
|
||||
return _completion(
|
||||
body,
|
||||
json.dumps(
|
||||
{
|
||||
"ops": [
|
||||
{
|
||||
"draft_idx": 0,
|
||||
"supersedes": [self.old_fact],
|
||||
"reason": "continues the same test-suite task",
|
||||
}
|
||||
]
|
||||
}
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
def _completion(body, content):
|
||||
return httpx.Response(
|
||||
200,
|
||||
json={
|
||||
"id": "synthetic-coding-response",
|
||||
"object": "chat.completion",
|
||||
"created": 1,
|
||||
"model": body["model"],
|
||||
"choices": [
|
||||
{
|
||||
"index": 0,
|
||||
"finish_reason": "stop",
|
||||
"message": {"role": "assistant", "content": content},
|
||||
}
|
||||
],
|
||||
"usage": {"prompt_tokens": 1, "completion_tokens": 1, "total_tokens": 2},
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
def test_a_working_fact_hy_supersedes_is_superseded_in_the_ledger(tmp_path):
|
||||
async def scenario():
|
||||
async with raw_client(tmp_path, tool=TOOL_TEXT) as (supervisor, records, permission):
|
||||
_, first = await run_coding_turn(supervisor, records[0], CodingOracle(records))
|
||||
assert first.status == "completed", first.last_error_code
|
||||
(old_fact,) = by_kind(supervisor)[MemoryKind.WORKING]
|
||||
question = put_turn(supervisor.evidence, "user-again", "user", "Use tox now.", 4)
|
||||
tool = put_turn(supervisor.evidence, "tool-again", "tool", "[Bash] tox\nok", 5)
|
||||
grant_records(supervisor, (*records, question, tool), permission.revision)
|
||||
oracle = SupersedingFactOracle(
|
||||
(question, tool),
|
||||
old_fact=old_fact.memory_id,
|
||||
solution="Run `tox` from the repository root.",
|
||||
statement="The agent switched the synthetic suite to tox.",
|
||||
)
|
||||
_, second = await run_coding_turn(supervisor, question, oracle)
|
||||
assert second.status == "completed", (second.last_error_code, oracle.stages)
|
||||
assert "fact_reconcile" in oracle.stages
|
||||
client = await native_client(supervisor, second)
|
||||
(native_old,) = await on_native_loop(
|
||||
client, client._coding_store.get_facts([old_fact.memory_id], user_id="local")
|
||||
)
|
||||
assert native_old.status == "superseded"
|
||||
|
||||
old = supervisor.governance.get(old_fact.memory_id, user_id="local")
|
||||
assert old.status is MemoryStatus.SUPERSEDED
|
||||
assert old.superseded_by == (native_old.fact_link,)
|
||||
new = supervisor.governance.get(native_old.fact_link, user_id="local")
|
||||
assert new is not None and new.status is MemoryStatus.CONFIRMED
|
||||
assert new.kind is MemoryKind.WORKING
|
||||
assert new.supersedes == (old.memory_id,)
|
||||
working = [
|
||||
m.memory_id
|
||||
for m in supervisor.governance.list(user_id="local")
|
||||
if m.kind is MemoryKind.WORKING
|
||||
]
|
||||
assert working == [new.memory_id]
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_the_chat_store_reports_what_hy_superseded_and_nothing_else():
|
||||
head = node("head", supersedes=["old"])
|
||||
old = node("old")
|
||||
kept = node("kept")
|
||||
backend = Backend(old, kept)
|
||||
store = wrapper(backend, approved=(old, kept))
|
||||
|
||||
async def scenario():
|
||||
# Positive control and negative in one store: an ordinary update is not reported.
|
||||
await store.update_payload("kept", {"content": "Synthetic kept, refined"})
|
||||
assert store.superseded_nodes == {}
|
||||
await store.update_payload("old", {"is_latest": False, "status": "superseded"})
|
||||
await store.upsert(head)
|
||||
await store.update_payload("old", {"superseded_by": ["head"]})
|
||||
assert store.superseded_nodes == {"old": ("head",)}
|
||||
assert "old" not in store.retired_node_ids
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_a_superseded_link_written_only_on_the_head_is_still_reported():
|
||||
old = node("old")
|
||||
backend = Backend(old)
|
||||
store = wrapper(backend, approved=(old,))
|
||||
|
||||
async def scenario():
|
||||
await store.update_payload("old", {"is_latest": False, "status": "superseded"})
|
||||
await store.upsert(node("head", supersedes=["old"]))
|
||||
assert store.superseded_nodes == {"old": ("head",)}
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_a_basic_profile_supersede_is_reported_for_the_ledger():
|
||||
old = node(
|
||||
"old-profile",
|
||||
agent_id="",
|
||||
session_id="",
|
||||
owner=None,
|
||||
layer=MemoryLayer.L0_BASIC_INFO,
|
||||
content="name: Before",
|
||||
custom={"basic_info_kv": {"name": "Before"}},
|
||||
)
|
||||
backend = Backend(old)
|
||||
store = wrapper(backend, approved=(old,))
|
||||
|
||||
class Embedder:
|
||||
async def embed_queued(self, text):
|
||||
return [1.0, 0.0]
|
||||
|
||||
async def scenario():
|
||||
result = await upsert_basic_profile(
|
||||
user_id="local",
|
||||
kv={"name": "After"},
|
||||
vector_store=store,
|
||||
embed_service=Embedder(),
|
||||
allowed_fields=["name"],
|
||||
)
|
||||
assert result.success and result.changed
|
||||
assert store.superseded_nodes == {"old-profile": (result.node_id,)}
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def _fact(key, **kwargs):
|
||||
return WorkingFact(
|
||||
fact_id=key, user_id="local", agent_id="agent", session_id="session", **kwargs
|
||||
)
|
||||
|
||||
|
||||
class _NativeFacts:
|
||||
def __init__(self, *facts):
|
||||
self.facts = {fact.fact_id: deepcopy(fact) for fact in facts}
|
||||
|
||||
async def get_facts(self, keys, user_id=None):
|
||||
return [deepcopy(self.facts[key]) for key in keys if key in self.facts]
|
||||
|
||||
async def supersede_fact(self, old_id, new_id):
|
||||
row = self.facts[old_id]
|
||||
row.status, row.fact_link = "superseded", new_id
|
||||
return True
|
||||
|
||||
async def insert_fact(self, fact):
|
||||
self.facts[fact.fact_id] = deepcopy(fact)
|
||||
|
||||
|
||||
def _coding_store(native, *approved):
|
||||
return HyPermissionCodingStore(
|
||||
native,
|
||||
Lease(),
|
||||
agent_id="agent",
|
||||
session_id="session",
|
||||
node_permissions={
|
||||
fact.fact_id: HyNodePermission(hy_coding_fingerprint(fact), ("current", "support"))
|
||||
for fact in approved
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
def test_the_coding_store_lets_hy_supersede_a_fact_before_inserting_its_successor():
|
||||
old = _fact("old-fact")
|
||||
store = _coding_store(_NativeFacts(old), old)
|
||||
|
||||
async def scenario():
|
||||
assert store.superseded_ids == {}
|
||||
assert await store.supersede_fact("old-fact", "new-fact")
|
||||
await store.insert_fact(_fact("new-fact"))
|
||||
store.recheck()
|
||||
assert store.superseded_ids == {"old-fact": ("new-fact",)}
|
||||
assert "old-fact" not in store.retired_ids
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_the_coding_store_never_links_a_fact_to_an_existing_unapproved_one():
|
||||
old, foreign = _fact("old-fact"), _fact("foreign-fact", statement="DENIED")
|
||||
native = _NativeFacts(old, foreign)
|
||||
store = _coding_store(native, old)
|
||||
|
||||
async def scenario():
|
||||
with pytest.raises(HyMemoryBridgeError, match="target_denied"):
|
||||
await store.supersede_fact("old-fact", "foreign-fact")
|
||||
|
||||
asyncio.run(scenario())
|
||||
assert native.facts["old-fact"].status == "active"
|
||||
@@ -1,433 +0,0 @@
|
||||
"""HY gets one turn at a time, exactly as it happened, and picks its own route.
|
||||
|
||||
A turn is one question from the person plus everything that answered it:
|
||||
assistant replies and the tool calls made along the way. HY's `add` decides
|
||||
chat or coding itself when it sees tool messages (`scene=None`); forcing
|
||||
`scene="chat"` made it strip them before it ever looked.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import sqlite3
|
||||
from datetime import UTC, datetime
|
||||
|
||||
from one_memory.hy_memory_bridge import (
|
||||
HyMemoryBridge,
|
||||
HyMemoryIngestRequest,
|
||||
HyMemoryOutput,
|
||||
HyMemoryTurn,
|
||||
)
|
||||
from one_memory.hy_source_context import HySourceContext, HyWriteBindings
|
||||
from one_memory.imports.grants import GRANT_SCHEMA
|
||||
from one_memory.imports.models import WrittenBy
|
||||
from one_memory.memory_processing import MemoryProcessingSupervisor, _conversation_chunks
|
||||
from one_memory.memory_tasks import MemoryTaskKind, MemoryTaskStatus
|
||||
from one_memory.model_profiles import ProviderKind
|
||||
from tests.test_hy_memory_bridge import FakeHyMemoryClient, snapshot
|
||||
from tests.test_memory_processing import FakeBridge, _put, _runner, _setup, _task
|
||||
from tests.test_model_processing_permissions import Harness
|
||||
|
||||
|
||||
def _turn(evidence, index: int, *, tool: str | None = None):
|
||||
records = [
|
||||
_put(
|
||||
evidence,
|
||||
message_id=f"u{index}",
|
||||
role="user",
|
||||
written_by=WrittenBy.PERSON,
|
||||
content=f"第{index}个问题。",
|
||||
),
|
||||
_put(
|
||||
evidence,
|
||||
message_id=f"a{index}",
|
||||
role="assistant",
|
||||
written_by=WrittenBy.AGENT,
|
||||
content=f"第{index}个回答。",
|
||||
),
|
||||
]
|
||||
if tool is not None:
|
||||
records.append(
|
||||
_put(
|
||||
evidence,
|
||||
message_id=f"t{index}",
|
||||
role="tool",
|
||||
written_by=WrittenBy.TOOL,
|
||||
content=tool,
|
||||
)
|
||||
)
|
||||
records.append(
|
||||
_put(
|
||||
evidence,
|
||||
message_id=f"a{index}-after-tool",
|
||||
role="assistant",
|
||||
written_by=WrittenBy.AGENT,
|
||||
content=f"第{index}个结论。",
|
||||
)
|
||||
)
|
||||
return records
|
||||
|
||||
|
||||
def _supervisor(tmp_path, stores, bridge=None):
|
||||
evidence, profiles, tasks, governance, secrets, _profile = stores
|
||||
return MemoryProcessingSupervisor(
|
||||
tasks=tasks,
|
||||
profiles=profiles,
|
||||
secrets=secrets,
|
||||
evidence=evidence,
|
||||
governance=governance,
|
||||
data_dir=tmp_path,
|
||||
runner_factory=lambda: _runner(tmp_path, stores, bridge or FakeBridge(outputs=())),
|
||||
)
|
||||
|
||||
|
||||
def _sent_turns(tmp_path, stores, created):
|
||||
bridge = FakeBridge(outputs=())
|
||||
runner = _runner(tmp_path, stores, bridge)
|
||||
for task in created:
|
||||
asyncio.run(runner.run(task.task_id))
|
||||
return [[turn.content for turn in request.turns] for request in bridge.requests]
|
||||
|
||||
|
||||
def test_each_turn_goes_to_hy_on_its_own(tmp_path) -> None:
|
||||
evidence = _setup(tmp_path)[0]
|
||||
first = _turn(evidence, 1, tool="[Bash] ls")
|
||||
second = _turn(evidence, 2)
|
||||
|
||||
chunks = _conversation_chunks(evidence.list_conversation(first[0].evidence_id))
|
||||
|
||||
assert [[r.evidence_id for r in chunk] for chunk in chunks] == [
|
||||
[r.evidence_id for r in first],
|
||||
[r.evidence_id for r in second],
|
||||
]
|
||||
|
||||
|
||||
def test_what_came_before_the_first_question_joins_the_turn_that_answers_it(tmp_path) -> None:
|
||||
"""An opener with no reply yet is not a turn of its own.
|
||||
|
||||
C3's real-authority drift showed splitting `assistant` proposal from the
|
||||
user turn that adopts it sends HY an answer without what it answers, so an
|
||||
unanswered assistant prefix joins the following turn. The prefix is still
|
||||
sent, once, and in order — it is just no longer a window of its own.
|
||||
"""
|
||||
evidence = _setup(tmp_path)[0]
|
||||
opener = _put(
|
||||
evidence,
|
||||
message_id="a0",
|
||||
role="assistant",
|
||||
written_by=WrittenBy.AGENT,
|
||||
content="你好,我能帮你做什么?",
|
||||
)
|
||||
first = _turn(evidence, 1)
|
||||
|
||||
chunks = _conversation_chunks(evidence.list_conversation(opener.evidence_id))
|
||||
|
||||
assert [[r.evidence_id for r in chunk] for chunk in chunks] == [
|
||||
[opener.evidence_id, *[r.evidence_id for r in first]]
|
||||
]
|
||||
|
||||
|
||||
def test_a_turn_too_long_for_one_window_is_split_but_every_window_keeps_its_question(
|
||||
tmp_path, monkeypatch
|
||||
) -> None:
|
||||
# D74 sends this turn whole; the invariant still holds on the split path.
|
||||
monkeypatch.setenv("ONE_MEMORY_SPLIT_TURNS", "1")
|
||||
evidence = _setup(tmp_path)[0]
|
||||
question = _put(
|
||||
evidence,
|
||||
message_id="u1",
|
||||
role="user",
|
||||
written_by=WrittenBy.PERSON,
|
||||
content="把这个仓库的测试跑一遍。",
|
||||
)
|
||||
calls = [
|
||||
_put(
|
||||
evidence,
|
||||
message_id=f"t{index}",
|
||||
role="tool",
|
||||
written_by=WrittenBy.TOOL,
|
||||
content=f"[Bash] {index} " + "x" * 1500,
|
||||
)
|
||||
for index in range(12)
|
||||
]
|
||||
|
||||
chunks = _conversation_chunks(evidence.list_conversation(question.evidence_id))
|
||||
|
||||
assert len(chunks) > 1
|
||||
assert all(chunk[0].evidence_id == question.evidence_id for chunk in chunks)
|
||||
assert all(sum(len(r.verbatim) for r in chunk) <= 12_000 for chunk in chunks)
|
||||
assert {r.evidence_id for chunk in chunks for r in chunk} == {
|
||||
question.evidence_id,
|
||||
*(call.evidence_id for call in calls),
|
||||
}
|
||||
|
||||
|
||||
def test_a_split_turn_never_sends_a_tool_only_window(tmp_path, monkeypatch) -> None:
|
||||
"""WorkBuddy/Codex can answer, then emit many tool calls.
|
||||
|
||||
If the answer fills the first window, the remaining calls used to become a
|
||||
tool-only window; HY rejected it with ``content or messages is required``.
|
||||
The tool calls must stay with the preceding non-tool window instead.
|
||||
D74 sends this turn whole; the invariant still holds on the split path.
|
||||
"""
|
||||
monkeypatch.setenv("ONE_MEMORY_SPLIT_TURNS", "1")
|
||||
evidence = _setup(tmp_path)[0]
|
||||
question = _put(
|
||||
evidence,
|
||||
message_id="u1",
|
||||
role="user",
|
||||
written_by=WrittenBy.PERSON,
|
||||
content="先回答,再调用很多工具。",
|
||||
)
|
||||
answer = _put(
|
||||
evidence,
|
||||
message_id="a1",
|
||||
role="assistant",
|
||||
written_by=WrittenBy.AGENT,
|
||||
content="好。" + "z" * 7_000,
|
||||
)
|
||||
calls = [
|
||||
_put(
|
||||
evidence,
|
||||
message_id=f"t{index}",
|
||||
role="tool",
|
||||
written_by=WrittenBy.TOOL,
|
||||
content=f"[Bash] {index} " + "x" * 1_500,
|
||||
)
|
||||
for index in range(4)
|
||||
]
|
||||
|
||||
chunks = _conversation_chunks(evidence.list_conversation(question.evidence_id))
|
||||
|
||||
assert chunks
|
||||
assert all(any(record.role != "tool" for record in chunk) for chunk in chunks)
|
||||
assert all(sum(len(record.verbatim) for record in chunk) <= 12_000 for chunk in chunks)
|
||||
assert list(dict.fromkeys(record.evidence_id for chunk in chunks for record in chunk)) == [
|
||||
question.evidence_id,
|
||||
answer.evidence_id,
|
||||
*(call.evidence_id for call in calls),
|
||||
]
|
||||
assert any(record.role == "tool" for chunk in chunks for record in chunk)
|
||||
|
||||
|
||||
def test_tool_calls_travel_with_their_turn_redacted_and_as_tool_messages(tmp_path) -> None:
|
||||
stores = _setup(tmp_path)
|
||||
evidence, _profiles, tasks, _governance, _secrets, profile = stores
|
||||
records = _turn(
|
||||
evidence, 1, tool="[Bash] docker login --username kristinhe --password exposed-token-value"
|
||||
)
|
||||
bridge = FakeBridge()
|
||||
|
||||
asyncio.run(_runner(tmp_path, stores, bridge).run(_task(tasks, profile, records[0]).task_id))
|
||||
|
||||
request = bridge.requests[0]
|
||||
assert [turn.role for turn in request.turns] == ["user", "assistant", "tool", "assistant"]
|
||||
assert "exposed-token-value" not in request.turns[2].content
|
||||
assert "<redacted-secret>" in request.turns[2].content
|
||||
assert records[2].evidence_id in request.evidence_ids
|
||||
|
||||
|
||||
def test_the_next_turn_reaches_hy_with_the_previous_turn_as_history(tmp_path) -> None:
|
||||
stores = _setup(tmp_path)
|
||||
evidence, _profiles, _tasks, _governance, _secrets, profile = stores
|
||||
with sqlite3.connect(tmp_path / "one-memory.sqlite3") as db:
|
||||
db.executescript(GRANT_SCHEMA)
|
||||
db.execute(
|
||||
"INSERT INTO source_grants VALUES ('local','cursor','/authorized','default',?,NULL)",
|
||||
(datetime.now(UTC).isoformat(),),
|
||||
)
|
||||
first = _turn(evidence, 1, tool="[Bash] 12 passed")
|
||||
second = _turn(evidence, 2)
|
||||
supervisor = _supervisor(tmp_path, stores)
|
||||
bridge = FakeBridge(outputs=())
|
||||
runner = _runner(tmp_path, stores, bridge)
|
||||
|
||||
for turn in (first, second):
|
||||
task = supervisor._enqueue_chunk(tuple(turn), profile, freeze_inputs=True)
|
||||
asyncio.run(runner.run(task.task_id))
|
||||
|
||||
context = bridge.requests[1].source_context
|
||||
assert context is not None
|
||||
assert [r.evidence_id for r in context.current] == [r.evidence_id for r in second]
|
||||
assert [r.evidence_id for r in context.history] == [r.evidence_id for r in first]
|
||||
|
||||
|
||||
def test_each_later_turn_carries_the_earlier_turns_as_history(tmp_path) -> None:
|
||||
"""Later windows in one conversation must not arrive at HY alone.
|
||||
|
||||
The frozen live pair expects later batches to grow because a window is sent
|
||||
with the turns behind it as history. A run where every request holds only the
|
||||
current turn loses the context the authority/adoption gate reads, which is
|
||||
what the cross-window real run showed.
|
||||
"""
|
||||
stores = _setup(tmp_path)
|
||||
evidence, _profiles, _tasks, _governance, _secrets, profile = stores
|
||||
with sqlite3.connect(tmp_path / "one-memory.sqlite3") as db:
|
||||
db.executescript(GRANT_SCHEMA)
|
||||
db.execute(
|
||||
"INSERT INTO source_grants VALUES ('local','cursor','/authorized','default',?,NULL)",
|
||||
(datetime.now(UTC).isoformat(),),
|
||||
)
|
||||
turns = [_turn(evidence, index) for index in (1, 2, 3)]
|
||||
supervisor = _supervisor(tmp_path, stores)
|
||||
bridge = FakeBridge(outputs=())
|
||||
runner = _runner(tmp_path, stores, bridge)
|
||||
|
||||
for turn in turns:
|
||||
task = supervisor._enqueue_chunk(tuple(turn), profile, freeze_inputs=True)
|
||||
asyncio.run(runner.run(task.task_id))
|
||||
|
||||
assert len(bridge.requests) == 3
|
||||
for index, request in enumerate(bridge.requests):
|
||||
context = request.source_context
|
||||
assert context is not None
|
||||
assert [r.evidence_id for r in context.current] == [r.evidence_id for r in turns[index]]
|
||||
# Every turn after the first is sent with the ones behind it.
|
||||
expected = [r.evidence_id for turn in turns[:index] for r in turn]
|
||||
assert [r.evidence_id for r in context.history] == expected
|
||||
|
||||
|
||||
def test_the_bridge_hands_hy_the_turn_and_lets_hy_choose_the_route(tmp_path) -> None:
|
||||
client = FakeHyMemoryClient({"success": True, "memory_ids": []})
|
||||
bridge = HyMemoryBridge(
|
||||
profile=snapshot(ProviderKind.OLLAMA, base_url="http://127.0.0.1:11434", has_api_key=False),
|
||||
api_key=None,
|
||||
data_dir=tmp_path,
|
||||
client=client,
|
||||
)
|
||||
request = HyMemoryIngestRequest(
|
||||
request_id="turn-1",
|
||||
user_id="local",
|
||||
source_id="codex",
|
||||
session_id="conversation-1",
|
||||
turns=(
|
||||
HyMemoryTurn(role="user", content="跑一下测试"),
|
||||
HyMemoryTurn(role="tool", content="[Bash] 12 passed"),
|
||||
HyMemoryTurn(role="assistant", content="全部通过。"),
|
||||
),
|
||||
evidence_ids=("u", "t", "a"),
|
||||
)
|
||||
|
||||
asyncio.run(bridge.ingest(request))
|
||||
asyncio.run(bridge.close())
|
||||
|
||||
assert client.calls[0]["scene"] is None
|
||||
assert [item["role"] for item in client.calls[0]["data"]] == ["user", "tool", "assistant"]
|
||||
|
||||
|
||||
def test_write_verification_accepts_the_tool_messages_hy_extracts_from(tmp_path) -> None:
|
||||
evidence = _setup(tmp_path)[0]
|
||||
question, _reply, call, _conclusion = _turn(evidence, 1, tool="[Bash] 12 passed")
|
||||
bindings = HyWriteBindings(HySourceContext(current=(question, call)))
|
||||
|
||||
bindings.check_content(
|
||||
json.dumps(
|
||||
[
|
||||
{"role": "user", "content": question.verbatim},
|
||||
{"role": "tool", "content": call.verbatim},
|
||||
],
|
||||
ensure_ascii=False,
|
||||
)
|
||||
)
|
||||
|
||||
assert [source["role"] for source in bindings.sources] == ["user", "tool"]
|
||||
|
||||
|
||||
def test_every_turn_gets_its_own_task_when_nothing_was_queued_before(tmp_path) -> None:
|
||||
stores = _setup(tmp_path)
|
||||
evidence = stores[0]
|
||||
turns = [_turn(evidence, index) for index in (1, 2, 3)]
|
||||
|
||||
created = _supervisor(tmp_path, stores).prepare_conversation(turns[2][0])
|
||||
|
||||
assert _sent_turns(tmp_path, stores, created) == [[r.verbatim for r in turn] for turn in turns]
|
||||
|
||||
|
||||
def _earlier_policy_window(stores, records, *, succeeded: bool):
|
||||
_evidence, _profiles, tasks, _governance, _secrets, profile = stores
|
||||
task = tasks.create(
|
||||
request_id=f"conversation:hy-native-v22:cursor:{records[0].evidence_id}",
|
||||
user_id=records[0].user_id,
|
||||
kind=MemoryTaskKind.EXTRACT,
|
||||
input_ref=records[0].evidence_id,
|
||||
profile=profile,
|
||||
input_records=tuple(records),
|
||||
)
|
||||
_claimed, token = tasks.claim(task.task_id)
|
||||
if succeeded:
|
||||
tasks.complete(task.task_id, runner_token=token)
|
||||
else:
|
||||
tasks.fail(task.task_id, runner_token=token, error_code="hy_memory_unavailable")
|
||||
return tasks.require(task.task_id)
|
||||
|
||||
|
||||
def test_turns_an_earlier_policy_already_extracted_are_not_sent_again(tmp_path) -> None:
|
||||
stores = _setup(tmp_path)
|
||||
evidence = stores[0]
|
||||
old = [*_turn(evidence, 1), *_turn(evidence, 2)]
|
||||
assert _earlier_policy_window(stores, old, succeeded=True).status is (
|
||||
MemoryTaskStatus.COMPLETED
|
||||
)
|
||||
new = _turn(evidence, 3)
|
||||
|
||||
created = _supervisor(tmp_path, stores).prepare_conversation(new[0])
|
||||
|
||||
assert _sent_turns(tmp_path, stores, created) == [[r.verbatim for r in new]]
|
||||
|
||||
|
||||
def test_renewed_consent_still_prepares_a_turn_the_current_policy_completed(tmp_path) -> None:
|
||||
h = Harness(tmp_path)
|
||||
h.bridge.fail = False
|
||||
turn = h.put_closed_turn("selected", conversation="renewed-consent")
|
||||
h.grant(turn)
|
||||
supervisor = h.supervisor()
|
||||
|
||||
async def scenario():
|
||||
try:
|
||||
first = supervisor.prepare_conversation(turn[0])[0]
|
||||
done = await supervisor._create_runner().run(first.task_id)
|
||||
assert done.status is MemoryTaskStatus.COMPLETED
|
||||
h.grant(turn)
|
||||
return first, supervisor.prepare_conversation(turn[0])
|
||||
finally:
|
||||
await supervisor.shutdown()
|
||||
|
||||
first, again = asyncio.run(scenario())
|
||||
|
||||
assert len(again) == 1 and again[0].task_id != first.task_id
|
||||
|
||||
|
||||
def test_a_failed_earlier_window_does_not_count_as_already_sent(tmp_path) -> None:
|
||||
stores = _setup(tmp_path)
|
||||
old = _turn(stores[0], 1)
|
||||
assert _earlier_policy_window(stores, old, succeeded=False).status is MemoryTaskStatus.FAILED
|
||||
|
||||
created = _supervisor(tmp_path, stores).prepare_conversation(old[0])
|
||||
|
||||
assert _sent_turns(tmp_path, stores, created) == [[r.verbatim for r in old]]
|
||||
|
||||
|
||||
def test_a_tool_heavy_turn_reaches_hy_once_and_still_becomes_a_memory(tmp_path) -> None:
|
||||
"""D74's full criterion: one call to HY, and a memory comes out of it.
|
||||
|
||||
The half that was only ever checked by hand is the second one. On 17:14
|
||||
the acceptance run for this ended with `completed=false` and zero
|
||||
evidence, so nothing proved the memory half either way; the window count
|
||||
alone leaves it open.
|
||||
"""
|
||||
stores = _setup(tmp_path)
|
||||
evidence, _profiles, tasks, governance, _secrets, profile = stores
|
||||
records = _turn(evidence, 1, tool="x" * 50_000)
|
||||
task = _task(tasks, profile, records[0])
|
||||
bridge = FakeBridge(
|
||||
outputs=(HyMemoryOutput(memory_id="hy-tool-1", content="用户跑过一次全量测试。"),)
|
||||
)
|
||||
|
||||
asyncio.run(_runner(tmp_path, stores, bridge).run(task.task_id))
|
||||
|
||||
# One window, so one call.
|
||||
assert len(_conversation_chunks(records)) == 1
|
||||
assert len(bridge.requests) == 1
|
||||
# And the memory that call produced actually landed.
|
||||
assert [item.content for item in governance.list(user_id="local")] == ["用户跑过一次全量测试。"]
|
||||
@@ -1,672 +0,0 @@
|
||||
import asyncio
|
||||
import threading
|
||||
from contextlib import suppress
|
||||
|
||||
import pytest
|
||||
|
||||
from one_memory.imports.models import EvidenceContentKind, WrittenBy
|
||||
from one_memory.memory_processing import MemoryProcessingSupervisor, _merge_historical_chunks
|
||||
from one_memory.memory_tasks import MemoryTaskStatus
|
||||
from one_memory.model_profiles import ModelProfilePatch
|
||||
from tests.test_memory_processing import _put, _setup
|
||||
from tests.test_model_processing_permissions import Harness
|
||||
|
||||
|
||||
def setup(tmp_path, count=3):
|
||||
evidence, profiles, tasks, governance, secrets, _profile = _setup(tmp_path)
|
||||
records = [
|
||||
_put(
|
||||
evidence,
|
||||
message_id=f"message-{index}",
|
||||
conversation_id=f"conversation-{index}",
|
||||
role="user",
|
||||
written_by=WrittenBy.PERSON,
|
||||
content=f"Synthetic conversation {index}",
|
||||
content_kind=EvidenceContentKind.MEMORY_CONTENT,
|
||||
)
|
||||
for index in range(count)
|
||||
]
|
||||
supervisor = MemoryProcessingSupervisor(
|
||||
tasks=tasks,
|
||||
profiles=profiles,
|
||||
secrets=secrets,
|
||||
evidence=evidence,
|
||||
governance=governance,
|
||||
data_dir=tmp_path,
|
||||
)
|
||||
return supervisor, records
|
||||
|
||||
|
||||
def test_accept_backfill_returns_after_first_batch_while_later_history_is_still_read(
|
||||
tmp_path, monkeypatch
|
||||
):
|
||||
monkeypatch.setattr("one_memory.memory_processing._BACKFILL_START_GRACE_SECONDS", 0)
|
||||
|
||||
async def scenario():
|
||||
supervisor, records = setup(tmp_path)
|
||||
later_entered, release = threading.Event(), threading.Event()
|
||||
scheduled = []
|
||||
reads = []
|
||||
original = supervisor.evidence.list_conversation
|
||||
|
||||
def blocked(evidence_id):
|
||||
reads.append(evidence_id)
|
||||
if len(reads) == 2:
|
||||
later_entered.set()
|
||||
assert release.wait(5)
|
||||
return original(evidence_id)
|
||||
|
||||
monkeypatch.setattr(supervisor.evidence, "list_conversation", blocked)
|
||||
monkeypatch.setattr(
|
||||
supervisor, "_queue_validated_work", lambda work: scheduled.extend(work)
|
||||
)
|
||||
try:
|
||||
accepted = asyncio.create_task(supervisor.accept_backfill_off_thread())
|
||||
assert await asyncio.to_thread(later_entered.wait, 3)
|
||||
await asyncio.wait_for(accepted, 1)
|
||||
assert len(scheduled) == 1
|
||||
assert accepted.done()
|
||||
assert supervisor._live_backfill is not None
|
||||
assert not supervisor._live_backfill.done()
|
||||
finally:
|
||||
release.set()
|
||||
if supervisor._live_backfill is not None:
|
||||
await supervisor._live_backfill
|
||||
await supervisor.shutdown()
|
||||
assert len(scheduled) == len(records)
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_first_task_is_dispatched_before_later_conversation_is_prepared(tmp_path, monkeypatch):
|
||||
async def scenario():
|
||||
supervisor, records = setup(tmp_path)
|
||||
later_entered, release = threading.Event(), threading.Event()
|
||||
scheduled = []
|
||||
reads = []
|
||||
original = supervisor.evidence.list_conversation
|
||||
|
||||
def blocked(evidence_id):
|
||||
reads.append(evidence_id)
|
||||
if len(reads) == 2:
|
||||
later_entered.set()
|
||||
assert release.wait(5)
|
||||
return original(evidence_id)
|
||||
|
||||
monkeypatch.setattr(supervisor.evidence, "list_conversation", blocked)
|
||||
monkeypatch.setattr(
|
||||
supervisor, "_queue_validated_work", lambda work: scheduled.extend(work)
|
||||
)
|
||||
task = asyncio.create_task(supervisor.backfill_off_thread())
|
||||
try:
|
||||
assert await asyncio.to_thread(later_entered.wait, 3)
|
||||
assert len(scheduled) == 1, "first task still waits behind later conversation reads"
|
||||
assert supervisor.tasks.require(scheduled[0].task_id).input_ref in {
|
||||
record.evidence_id for record in records
|
||||
}
|
||||
finally:
|
||||
release.set()
|
||||
await task
|
||||
await supervisor.shutdown()
|
||||
assert len(scheduled) == len(records)
|
||||
assert {supervisor.tasks.require(item.task_id).input_ref for item in scheduled} == {
|
||||
record.evidence_id for record in records
|
||||
}
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_cancellation_joins_the_preparation_thread_before_return(tmp_path, monkeypatch):
|
||||
async def scenario():
|
||||
supervisor, _records = setup(tmp_path)
|
||||
entered, release, exited = threading.Event(), threading.Event(), threading.Event()
|
||||
original = supervisor.evidence.list_conversation
|
||||
scheduled = []
|
||||
|
||||
def blocked(evidence_id):
|
||||
entered.set()
|
||||
assert release.wait(5)
|
||||
result = original(evidence_id)
|
||||
exited.set()
|
||||
return result
|
||||
|
||||
monkeypatch.setattr(supervisor.evidence, "list_conversation", blocked)
|
||||
monkeypatch.setattr(
|
||||
supervisor, "_queue_validated_work", lambda work: scheduled.extend(work)
|
||||
)
|
||||
task = asyncio.create_task(supervisor.backfill_off_thread())
|
||||
try:
|
||||
assert await asyncio.to_thread(entered.wait, 3)
|
||||
task.cancel()
|
||||
await asyncio.sleep(0.03)
|
||||
task.cancel()
|
||||
await asyncio.sleep(0.03)
|
||||
assert not task.done(), "backfill abandoned a thread that can still write tasks"
|
||||
finally:
|
||||
release.set()
|
||||
with suppress(asyncio.CancelledError):
|
||||
await task
|
||||
await supervisor.shutdown()
|
||||
assert exited.is_set()
|
||||
assert scheduled == []
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_later_failure_keeps_the_first_committed_batch_visible(tmp_path, monkeypatch):
|
||||
async def scenario():
|
||||
supervisor, _records = setup(tmp_path)
|
||||
scheduled, reads = [], []
|
||||
original = supervisor.evidence.list_conversation
|
||||
|
||||
def broken(evidence_id):
|
||||
reads.append(evidence_id)
|
||||
if len(reads) == 2:
|
||||
raise RuntimeError("synthetic later read failure")
|
||||
return original(evidence_id)
|
||||
|
||||
monkeypatch.setattr(supervisor.evidence, "list_conversation", broken)
|
||||
monkeypatch.setattr(
|
||||
supervisor, "_queue_validated_work", lambda work: scheduled.extend(work)
|
||||
)
|
||||
try:
|
||||
with pytest.raises(RuntimeError, match="synthetic later"):
|
||||
await supervisor.backfill_off_thread()
|
||||
assert len(scheduled) == 1
|
||||
assert len(supervisor.tasks.list()) == 1
|
||||
finally:
|
||||
await supervisor.shutdown()
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
@pytest.mark.parametrize("change", ["revoke", "regrant"])
|
||||
def test_permission_change_between_batches_stops_without_new_tasks(tmp_path, monkeypatch, change):
|
||||
async def scenario():
|
||||
h = Harness(tmp_path)
|
||||
records = [
|
||||
h.put("first", conversation="first", memory_content=True),
|
||||
h.put("second", conversation="second", memory_content=True),
|
||||
]
|
||||
receipt = h.grant(records)
|
||||
supervisor = h.supervisor()
|
||||
scheduled = []
|
||||
|
||||
def dispatch(work):
|
||||
scheduled.extend(work)
|
||||
if len(scheduled) == 1:
|
||||
if change == "revoke":
|
||||
h.permissions.revoke(
|
||||
user_id="local", user_confirmed=True, expected_revision=receipt.revision
|
||||
)
|
||||
else:
|
||||
h.grant(records)
|
||||
|
||||
monkeypatch.setattr(supervisor, "_queue_validated_work", dispatch)
|
||||
try:
|
||||
with pytest.raises(RuntimeError, match="model_content_permission_"):
|
||||
await supervisor.backfill_off_thread(expected_permission_revision=receipt.revision)
|
||||
assert len(h.tasks.list()) == len(scheduled) == 1
|
||||
assert h.factory_calls == [] and h.bridge.calls == []
|
||||
finally:
|
||||
await supervisor.shutdown()
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_profile_change_after_first_batch_preserves_old_task_identity(tmp_path, monkeypatch):
|
||||
async def scenario():
|
||||
h = Harness(tmp_path)
|
||||
h.grant(
|
||||
[
|
||||
h.put("first", conversation="first", memory_content=True),
|
||||
h.put("second", conversation="second", memory_content=True),
|
||||
]
|
||||
)
|
||||
supervisor = h.supervisor()
|
||||
scheduled = []
|
||||
|
||||
def dispatch(work):
|
||||
scheduled.extend(work)
|
||||
h.profiles.update(
|
||||
h.profile.profile_id,
|
||||
ModelProfilePatch(model="changed-model", expected_revision=h.profile.revision),
|
||||
h.secrets,
|
||||
)
|
||||
|
||||
monkeypatch.setattr(supervisor, "_queue_validated_work", dispatch)
|
||||
try:
|
||||
with pytest.raises(RuntimeError, match="model_content_profile_changed"):
|
||||
await supervisor.backfill_off_thread()
|
||||
assert len(h.tasks.list()) == len(scheduled) == 1
|
||||
assert h.tasks.list()[0].profile_revision == h.profile.revision
|
||||
assert h.factory_calls == []
|
||||
finally:
|
||||
await supervisor.shutdown()
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_batched_and_sync_preparation_keep_all_task_inputs_and_idempotency(tmp_path, monkeypatch):
|
||||
async def scenario():
|
||||
supervisor, records = setup(tmp_path, count=21)
|
||||
scheduled = []
|
||||
monkeypatch.setattr(
|
||||
supervisor, "_queue_validated_work", lambda batch: scheduled.append(batch)
|
||||
)
|
||||
try:
|
||||
expected = supervisor.prepare_backfill()
|
||||
expected_ids = {work.task_id for work in expected}
|
||||
assert len(expected_ids) == 21
|
||||
before = {
|
||||
record.evidence_id: supervisor.evidence.get(record.evidence_id)
|
||||
for record in records
|
||||
}
|
||||
await supervisor.backfill_off_thread()
|
||||
assert len(scheduled[0]) == 1
|
||||
assert all(len(batch) <= 8 for batch in scheduled)
|
||||
assert {work.task_id for batch in scheduled for work in batch} == expected_ids
|
||||
assert len(supervisor.tasks.list()) == 21
|
||||
assert {
|
||||
record.evidence_id: supervisor.evidence.get(record.evidence_id)
|
||||
for record in records
|
||||
} == before
|
||||
finally:
|
||||
await supervisor.shutdown()
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_history_backfill_merges_adjacent_unsent_turns_in_one_conversation(tmp_path):
|
||||
supervisor, records = setup(tmp_path, count=0)
|
||||
profile = supervisor.profiles.active(supervisor.secrets)
|
||||
assert profile is not None
|
||||
conversation = []
|
||||
for index in range(2):
|
||||
conversation.extend(
|
||||
(
|
||||
_put(
|
||||
supervisor.evidence,
|
||||
message_id=f"user-{index}",
|
||||
role="user",
|
||||
written_by=WrittenBy.PERSON,
|
||||
content=f"问题 {index}",
|
||||
conversation_id="shared-history",
|
||||
),
|
||||
_put(
|
||||
supervisor.evidence,
|
||||
message_id=f"assistant-{index}",
|
||||
role="assistant",
|
||||
written_by=WrittenBy.AGENT,
|
||||
content=f"回答 {index}",
|
||||
conversation_id="shared-history",
|
||||
),
|
||||
)
|
||||
)
|
||||
|
||||
chunks = supervisor._prepare_conversation_chunks(conversation, profile, None)
|
||||
merged = _merge_historical_chunks(chunks)
|
||||
|
||||
assert len(chunks) == 2
|
||||
assert len(merged) == 1
|
||||
assert {record.evidence_id for record in merged[0]} == {
|
||||
record.evidence_id for record in conversation
|
||||
}
|
||||
|
||||
|
||||
def test_backfill_holds_an_unanswered_raw_turn_until_its_reply_arrives(tmp_path):
|
||||
supervisor, _records = setup(tmp_path, count=0)
|
||||
question = _put(
|
||||
supervisor.evidence,
|
||||
message_id="open-question",
|
||||
role="user",
|
||||
written_by=WrittenBy.PERSON,
|
||||
content="发布包上传了吗?",
|
||||
conversation_id="open-turn",
|
||||
)
|
||||
|
||||
assert supervisor.prepare_conversation(question) == []
|
||||
assert supervisor.prepare_backfill() == []
|
||||
|
||||
answer = _put(
|
||||
supervisor.evidence,
|
||||
message_id="closing-answer",
|
||||
role="assistant",
|
||||
written_by=WrittenBy.AGENT,
|
||||
content="已经上传完成。",
|
||||
conversation_id="open-turn",
|
||||
)
|
||||
|
||||
prepared = supervisor.prepare_conversation(answer)
|
||||
|
||||
assert len(prepared) == 1
|
||||
assert prepared[0].input_ref == question.evidence_id
|
||||
|
||||
|
||||
def test_history_merge_supersedes_an_already_queued_shorter_session_task(tmp_path):
|
||||
supervisor, _records = setup(tmp_path, count=0)
|
||||
first = _put(
|
||||
supervisor.evidence,
|
||||
message_id="merge-user-1",
|
||||
role="user",
|
||||
written_by=WrittenBy.PERSON,
|
||||
content="第一问",
|
||||
conversation_id="growing-history",
|
||||
)
|
||||
first_reply = _put(
|
||||
supervisor.evidence,
|
||||
message_id="merge-assistant-1",
|
||||
role="assistant",
|
||||
written_by=WrittenBy.AGENT,
|
||||
content="第一答",
|
||||
conversation_id="growing-history",
|
||||
)
|
||||
profile = supervisor.profiles.active(supervisor.secrets)
|
||||
assert profile is not None
|
||||
old = supervisor._enqueue_chunk((first, first_reply), profile, freeze_inputs=True)
|
||||
second = _put(
|
||||
supervisor.evidence,
|
||||
message_id="merge-user-2",
|
||||
role="user",
|
||||
written_by=WrittenBy.PERSON,
|
||||
content="第二问",
|
||||
conversation_id="growing-history",
|
||||
)
|
||||
second_reply = _put(
|
||||
supervisor.evidence,
|
||||
message_id="merge-assistant-2",
|
||||
role="assistant",
|
||||
written_by=WrittenBy.AGENT,
|
||||
content="第二答",
|
||||
conversation_id="growing-history",
|
||||
)
|
||||
|
||||
chunks = supervisor._prepare_conversation_chunks(
|
||||
supervisor.evidence.list_conversation(second.evidence_id), profile, None
|
||||
)
|
||||
merged_chunk = _merge_historical_chunks(chunks)[0]
|
||||
merged_task = supervisor._enqueue_chunk(merged_chunk, profile, freeze_inputs=True)
|
||||
|
||||
pending = supervisor.tasks.list(statuses=(MemoryTaskStatus.PENDING,))
|
||||
merged = next(task for task in supervisor.tasks.list() if task.task_id != old.task_id)
|
||||
assert supervisor.tasks.require(old.task_id).status.value == "cancelled"
|
||||
assert [task.task_id for task in pending] == [merged.task_id]
|
||||
assert {ref["evidence_id"] for ref in supervisor.tasks.inputs(merged.task_id)} == {
|
||||
first.evidence_id,
|
||||
first_reply.evidence_id,
|
||||
second.evidence_id,
|
||||
second_reply.evidence_id,
|
||||
}
|
||||
assert merged_task.task_id == merged.task_id
|
||||
|
||||
|
||||
def test_backfill_budget_yields_between_rounds_and_resumes_same_cursor(tmp_path, monkeypatch):
|
||||
async def scenario():
|
||||
supervisor, records = setup(tmp_path, count=3)
|
||||
scheduled = []
|
||||
clock = iter((0.0, 0.0, 331.0, 331.0, 331.0, 662.0, 662.0, 662.0))
|
||||
supervisor._backfill_clock = lambda: next(clock)
|
||||
supervisor._backfill_round_budget_seconds = 330.0
|
||||
releases = []
|
||||
|
||||
async def schedule(work):
|
||||
assert supervisor._backfill_lock.locked()
|
||||
scheduled.extend(work)
|
||||
|
||||
original_sleep = asyncio.sleep
|
||||
|
||||
async def observed_sleep(delay):
|
||||
if delay == 0:
|
||||
releases.append(not supervisor._backfill_lock.locked())
|
||||
await original_sleep(0)
|
||||
|
||||
monkeypatch.setattr(supervisor, "_schedule_tasks_off_thread", schedule)
|
||||
monkeypatch.setattr("one_memory.memory_processing.asyncio.sleep", observed_sleep)
|
||||
try:
|
||||
await supervisor.backfill_off_thread()
|
||||
finally:
|
||||
await supervisor.shutdown()
|
||||
|
||||
assert releases == [True, True]
|
||||
assert len(scheduled) == len(records)
|
||||
assert len({work.task_id for work in scheduled}) == len(records)
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_shutdown_waits_for_backfill_preparation_and_never_dispatches_tail(tmp_path, monkeypatch):
|
||||
async def scenario():
|
||||
supervisor, _records = setup(tmp_path)
|
||||
entered, release = threading.Event(), threading.Event()
|
||||
scheduled = []
|
||||
original = supervisor.evidence.list_conversation
|
||||
|
||||
def blocked(evidence_id):
|
||||
entered.set()
|
||||
assert release.wait(5)
|
||||
return original(evidence_id)
|
||||
|
||||
monkeypatch.setattr(supervisor.evidence, "list_conversation", blocked)
|
||||
monkeypatch.setattr(
|
||||
supervisor, "_queue_validated_work", lambda work: scheduled.extend(work)
|
||||
)
|
||||
task = asyncio.create_task(supervisor.backfill_off_thread())
|
||||
try:
|
||||
assert await asyncio.to_thread(entered.wait, 3)
|
||||
stopping = asyncio.create_task(supervisor.shutdown())
|
||||
await asyncio.sleep(0.03)
|
||||
assert not stopping.done()
|
||||
finally:
|
||||
release.set()
|
||||
await task
|
||||
await stopping
|
||||
assert scheduled == []
|
||||
assert supervisor._backfill_preparations == set()
|
||||
assert supervisor.tasks.list() == []
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
@pytest.mark.parametrize("count", [0, 1, 8, 9, 18])
|
||||
def test_empty_and_batch_boundaries_preserve_all_inputs(tmp_path, monkeypatch, count):
|
||||
async def scenario():
|
||||
supervisor, records = setup(tmp_path, count=count)
|
||||
scheduled = []
|
||||
monkeypatch.setattr(
|
||||
supervisor, "_queue_validated_work", lambda work: scheduled.extend(work)
|
||||
)
|
||||
active_calls = []
|
||||
active = supervisor.profiles.active
|
||||
|
||||
def counted(secrets):
|
||||
active_calls.append(True)
|
||||
return active(secrets)
|
||||
|
||||
monkeypatch.setattr(supervisor.profiles, "active", counted)
|
||||
try:
|
||||
await supervisor.backfill_off_thread()
|
||||
assert len(active_calls) == 1, "batch identity checks should not reaccess credentials"
|
||||
assert {supervisor.tasks.require(work.task_id).input_ref for work in scheduled} == {
|
||||
record.evidence_id for record in records
|
||||
}
|
||||
assert len(supervisor.tasks.list()) == count
|
||||
finally:
|
||||
await supervisor.shutdown()
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_empty_denied_batches_do_not_end_the_remaining_traversal(tmp_path, monkeypatch):
|
||||
async def scenario():
|
||||
supervisor, records = setup(tmp_path, count=19)
|
||||
scheduled = []
|
||||
original = supervisor.evidence.list_conversation
|
||||
earliest = records[0].evidence_id
|
||||
monkeypatch.setattr(
|
||||
supervisor.evidence,
|
||||
"list_conversation",
|
||||
lambda identity: original(identity) if identity == earliest else [],
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
supervisor, "_queue_validated_work", lambda work: scheduled.append(work)
|
||||
)
|
||||
try:
|
||||
await supervisor.backfill_off_thread()
|
||||
assert scheduled[:2] == [[], []]
|
||||
useful = [work for batch in scheduled for work in batch]
|
||||
assert len(useful) == 1
|
||||
assert supervisor.tasks.require(useful[0].task_id).input_ref == earliest
|
||||
finally:
|
||||
await supervisor.shutdown()
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_replacement_backfill_waits_for_cancelled_writer_and_remains_idempotent(
|
||||
tmp_path, monkeypatch
|
||||
):
|
||||
async def scenario():
|
||||
supervisor, records = setup(tmp_path)
|
||||
entered, release = threading.Event(), threading.Event()
|
||||
reads, scheduled = [], []
|
||||
original = supervisor.evidence.list_conversation
|
||||
|
||||
def blocked(evidence_id):
|
||||
reads.append(evidence_id)
|
||||
if len(reads) == 1:
|
||||
entered.set()
|
||||
assert release.wait(5)
|
||||
return original(evidence_id)
|
||||
|
||||
monkeypatch.setattr(supervisor.evidence, "list_conversation", blocked)
|
||||
monkeypatch.setattr(
|
||||
supervisor, "_queue_validated_work", lambda work: scheduled.extend(work)
|
||||
)
|
||||
old = asyncio.create_task(supervisor.backfill_off_thread())
|
||||
try:
|
||||
assert await asyncio.to_thread(entered.wait, 3)
|
||||
old.cancel()
|
||||
replacement = asyncio.create_task(supervisor.backfill_off_thread())
|
||||
await asyncio.sleep(0.03)
|
||||
assert not old.done() and not replacement.done()
|
||||
assert len(reads) == 1
|
||||
finally:
|
||||
release.set()
|
||||
with suppress(asyncio.CancelledError):
|
||||
await old
|
||||
await replacement
|
||||
await supervisor.shutdown()
|
||||
assert len(supervisor.tasks.list()) == len(records)
|
||||
assert len({work.task_id for work in scheduled}) == len(scheduled) == len(records)
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
@pytest.mark.parametrize("unclaimable", ["completed", "deferred"])
|
||||
def test_unclaimable_first_seed_does_not_delay_first_useful_batch(
|
||||
tmp_path, monkeypatch, unclaimable
|
||||
):
|
||||
async def scenario():
|
||||
supervisor, records = setup(tmp_path)
|
||||
first = supervisor.prepare_conversation(records[-1])[0]
|
||||
if unclaimable == "completed":
|
||||
_, token = supervisor.tasks.claim(first.task_id)
|
||||
supervisor.tasks.complete(first.task_id, runner_token=token)
|
||||
else:
|
||||
supervisor.tasks.defer_outside_window(first.task_id)
|
||||
scheduled, reads = [], []
|
||||
entered, release = threading.Event(), threading.Event()
|
||||
original = supervisor.evidence.list_conversation
|
||||
|
||||
def blocked(identity):
|
||||
reads.append(identity)
|
||||
if len(reads) == 3:
|
||||
entered.set()
|
||||
assert release.wait(5)
|
||||
return original(identity)
|
||||
|
||||
monkeypatch.setattr(supervisor.evidence, "list_conversation", blocked)
|
||||
monkeypatch.setattr(
|
||||
supervisor, "_queue_validated_work", lambda work: scheduled.extend(work)
|
||||
)
|
||||
task = asyncio.create_task(supervisor.backfill_off_thread())
|
||||
try:
|
||||
assert await asyncio.to_thread(entered.wait, 3)
|
||||
assert len(scheduled) == 1
|
||||
assert scheduled[0].task_id != first.task_id
|
||||
finally:
|
||||
release.set()
|
||||
await task
|
||||
await supervisor.shutdown()
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_cancelled_shutdown_waits_for_inflight_generator_and_preserves_cleanup(
|
||||
tmp_path, monkeypatch
|
||||
):
|
||||
async def scenario():
|
||||
supervisor, _records = setup(tmp_path)
|
||||
entered, release = threading.Event(), threading.Event()
|
||||
original = supervisor.evidence.list_conversation
|
||||
|
||||
def blocked(identity):
|
||||
entered.set()
|
||||
assert release.wait(5)
|
||||
return original(identity)
|
||||
|
||||
monkeypatch.setattr(supervisor.evidence, "list_conversation", blocked)
|
||||
backfill = asyncio.create_task(supervisor.backfill_off_thread())
|
||||
try:
|
||||
assert await asyncio.to_thread(entered.wait, 3)
|
||||
stopping = asyncio.create_task(supervisor.shutdown())
|
||||
await asyncio.sleep(0.02)
|
||||
stopping.cancel()
|
||||
await asyncio.sleep(0.02)
|
||||
stopping.cancel()
|
||||
await asyncio.sleep(0.02)
|
||||
assert not stopping.done() and not backfill.done()
|
||||
assert len(supervisor._backfill_preparations) == 1
|
||||
finally:
|
||||
release.set()
|
||||
await asyncio.gather(stopping, backfill, return_exceptions=True)
|
||||
assert not backfill.cancelled() and backfill.exception() is None
|
||||
assert supervisor._backfill_preparations == set()
|
||||
assert supervisor.tasks.list() == []
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_exact_backfill_keeps_granted_prefix_when_same_conversation_grows(tmp_path):
|
||||
async def scenario():
|
||||
h = Harness(tmp_path)
|
||||
first = h.put("first", memory_content=True)
|
||||
second = h.put("second", memory_content=True)
|
||||
receipt = h.grant([first, second])
|
||||
later = h.put("later", memory_content=True)
|
||||
other = h.put("other", conversation="other", memory_content=True)
|
||||
supervisor = h.supervisor()
|
||||
try:
|
||||
await supervisor.backfill_off_thread(expected_permission_revision=receipt.revision)
|
||||
tasks = supervisor.tasks.list()
|
||||
# One task per turn; both granted turns, nothing after them.
|
||||
assert len(tasks) == 1
|
||||
assert {task.input_ref for task in tasks} == {second.evidence_id}
|
||||
refs = {
|
||||
item["evidence_id"]
|
||||
for task in tasks
|
||||
for item in supervisor.tasks.inputs(task.task_id)
|
||||
}
|
||||
assert {first.evidence_id, second.evidence_id} <= refs
|
||||
assert later.evidence_id not in refs
|
||||
assert other.evidence_id not in refs
|
||||
# The first turn may already be running; whatever reached the model was granted.
|
||||
assert all(
|
||||
{record.evidence_id for record in lease.records}
|
||||
<= {first.evidence_id, second.evidence_id}
|
||||
for lease, _data_dir in h.factory_calls
|
||||
)
|
||||
finally:
|
||||
await supervisor.shutdown()
|
||||
|
||||
asyncio.run(scenario())
|
||||
@@ -1,460 +0,0 @@
|
||||
"""T04: what the person explicitly asks to remember is written through HY's import path.
|
||||
|
||||
Whole native HY client behind the production permission context; only the model
|
||||
endpoints are scripted through an httpx MockTransport, never a network.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import hashlib
|
||||
import json
|
||||
import sqlite3
|
||||
from contextlib import asynccontextmanager, closing
|
||||
from datetime import UTC, datetime
|
||||
from uuid import uuid4
|
||||
|
||||
import httpx
|
||||
import pytest
|
||||
from httpx import ASGITransport, AsyncClient
|
||||
from pydantic import SecretStr
|
||||
|
||||
from one_memory import continuous_policy, ingest_journal
|
||||
from one_memory.api import create_app
|
||||
from one_memory.config import Settings
|
||||
from one_memory.hy_memory_bridge import HyMemoryIngestResult, HyMemoryOutput
|
||||
from one_memory.hy_permission_context import permission_bridge_factory
|
||||
from one_memory.imports.models import EvidenceInput, EvidenceOrigin, OriginMetadata, WrittenBy
|
||||
from one_memory.matters.selection import select_evidence
|
||||
from one_memory.memory_notes import SOURCE_ID
|
||||
from one_memory.model_profiles import ModelCapabilities, ModelProfileInput, ProviderKind
|
||||
from one_memory.secrets import InMemorySecretStore
|
||||
from tests.source_consent import confirm_model_content
|
||||
from tests.test_memory_processing import FakeBridge, _runner, _setup, _task
|
||||
|
||||
TEXT = "请记住:项目周会固定不排在周五。"
|
||||
|
||||
|
||||
def _vector(text: str) -> list[float]:
|
||||
digest = hashlib.sha256(text.encode()).digest()
|
||||
values = [float(byte) + 1.0 for byte in digest * 12]
|
||||
norm = sum(value * value for value in values) ** 0.5
|
||||
return [value / norm for value in values]
|
||||
|
||||
|
||||
class ScriptedModel:
|
||||
"""The model endpoints HY reaches. Reconcile says "nothing to change" unless told."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
self.reconcile_prompts: list[str] = []
|
||||
self.other_prompts: list[str] = []
|
||||
self.embeddings_down = False
|
||||
|
||||
def __call__(self, request: httpx.Request) -> httpx.Response:
|
||||
body = json.loads(request.content)
|
||||
if request.url.path.endswith("embeddings"):
|
||||
if self.embeddings_down:
|
||||
return httpx.Response(503, json={"error": "down"})
|
||||
return httpx.Response(
|
||||
200,
|
||||
json={
|
||||
"object": "list",
|
||||
"model": "test-embedding",
|
||||
"data": [
|
||||
{"index": i, "object": "embedding", "embedding": _vector(text)}
|
||||
for i, text in enumerate(body["input"])
|
||||
],
|
||||
"usage": {"prompt_tokens": 1, "total_tokens": 1},
|
||||
},
|
||||
)
|
||||
prompt = "\n".join(str(message.get("content")) for message in body["messages"])
|
||||
reconcile = "Existing memories" in prompt
|
||||
(self.reconcile_prompts if reconcile else self.other_prompts).append(prompt)
|
||||
# Extraction or reconcile: the model finds nothing worth writing.
|
||||
content = "[]" if reconcile else '{"memory": [], "identity": [], "intentions": []}'
|
||||
return httpx.Response(
|
||||
200,
|
||||
json={
|
||||
"id": "scripted",
|
||||
"object": "chat.completion",
|
||||
"created": 1,
|
||||
"model": "test-generation",
|
||||
"choices": [
|
||||
{
|
||||
"index": 0,
|
||||
"finish_reason": "stop",
|
||||
"message": {"role": "assistant", "content": content},
|
||||
}
|
||||
],
|
||||
"usage": {"prompt_tokens": 1, "completion_tokens": 1, "total_tokens": 2},
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def hy_calls(monkeypatch):
|
||||
from hy_memory.client import HyMemoryClient
|
||||
|
||||
calls: list[tuple[str, object]] = []
|
||||
original_add = HyMemoryClient.add
|
||||
original_extracted = HyMemoryClient.add_extracted
|
||||
|
||||
def add(self, data, **kwargs):
|
||||
calls.append(("add", [turn["content"] for turn in data]))
|
||||
return original_add(self, data, **kwargs)
|
||||
|
||||
def add_extracted(self, contents, **kwargs):
|
||||
raw = original_extracted(self, contents, **kwargs)
|
||||
calls.append(("add_extracted", {"contents": list(contents), "raw": raw}))
|
||||
return raw
|
||||
|
||||
monkeypatch.setattr(HyMemoryClient, "add", add)
|
||||
monkeypatch.setattr(HyMemoryClient, "add_extracted", add_extracted)
|
||||
return calls
|
||||
|
||||
|
||||
def _application(tmp_path):
|
||||
return create_app(
|
||||
Settings(
|
||||
data_dir=tmp_path / "data",
|
||||
source_home=tmp_path,
|
||||
api_token=SecretStr("t04"),
|
||||
imports_enabled=False,
|
||||
automatic_memory_processing=False,
|
||||
live_sync_enabled=False,
|
||||
),
|
||||
secret_store=InMemorySecretStore(),
|
||||
)
|
||||
|
||||
|
||||
async def _configure(app, model: ScriptedModel) -> None:
|
||||
profiles, secrets = app.state.model_profile_store, app.state.secret_store
|
||||
profile = profiles.create(
|
||||
ModelProfileInput(
|
||||
name="Scripted",
|
||||
provider_kind=ProviderKind.OLLAMA,
|
||||
base_url="http://127.0.0.1:1/v1",
|
||||
model="test-generation",
|
||||
embedding_provider_kind=ProviderKind.OLLAMA,
|
||||
embedding_base_url="http://127.0.0.1:2/v1",
|
||||
embedding_model="test-embedding",
|
||||
),
|
||||
secrets,
|
||||
)
|
||||
profiles.record_capabilities(
|
||||
profile.profile_id,
|
||||
expected_revision=profile.revision,
|
||||
capabilities=ModelCapabilities(
|
||||
endpoint_reachable=True, model_available=True, embedding_available=True
|
||||
),
|
||||
secrets=secrets,
|
||||
)
|
||||
profiles.activate(profile.profile_id, secrets)
|
||||
processor = app.state.memory_processing_supervisor
|
||||
processor.permission_bridge_factory = permission_bridge_factory(
|
||||
processor._bridge, http_transport=httpx.MockTransport(model)
|
||||
)
|
||||
|
||||
|
||||
async def _remember(app, client, text: str) -> dict:
|
||||
saved = (
|
||||
await client.post(
|
||||
"/api/v1/memory-notes",
|
||||
json={"request_id": str(uuid4()), "text": text, "scope": "all-apps"},
|
||||
)
|
||||
).json()
|
||||
await confirm_model_content(
|
||||
client,
|
||||
source_ids=("one-memory-manual",),
|
||||
evidence_ids=(saved["evidence_id"],),
|
||||
future_additions=False,
|
||||
)
|
||||
started = await client.post(f"/api/v1/memory-notes/{saved['evidence_id']}/process")
|
||||
assert started.status_code == 200, started.text
|
||||
await _wait_for_note_job(app, saved["evidence_id"])
|
||||
return (await client.get(f"/api/v1/memory-notes/{saved['evidence_id']}")).json()
|
||||
|
||||
|
||||
async def _wait_for_note_job(app, evidence_id: str, *, timeout: float = 10) -> str:
|
||||
"""Wait for the production durable note job, not the legacy worker map."""
|
||||
deadline = asyncio.get_running_loop().time() + timeout
|
||||
key = f"entry:{evidence_id}"
|
||||
while asyncio.get_running_loop().time() < deadline:
|
||||
with sqlite3.connect(app.state.evidence_store.database_path) as db:
|
||||
row = db.execute(
|
||||
"SELECT state FROM memory_write_jobs WHERE idempotency_key=?",
|
||||
(key,),
|
||||
).fetchone()
|
||||
if row is not None and row[0] not in {"pending", "running"}:
|
||||
return str(row[0])
|
||||
await asyncio.sleep(0.05)
|
||||
raise AssertionError(f"note job for {evidence_id} never settled")
|
||||
|
||||
|
||||
def _client(app):
|
||||
return AsyncClient(
|
||||
transport=ASGITransport(app=app),
|
||||
base_url="http://test",
|
||||
headers={"Authorization": "Bearer t04"},
|
||||
)
|
||||
|
||||
|
||||
def test_note_is_written_through_hy_import_even_when_the_model_would_extract_nothing(
|
||||
tmp_path, hy_calls
|
||||
):
|
||||
async def scenario():
|
||||
app, model = _application(tmp_path), ScriptedModel()
|
||||
async with app.router.lifespan_context(app), _client(app) as client:
|
||||
await _configure(app, model)
|
||||
receipt = await _remember(app, client, TEXT)
|
||||
|
||||
assert receipt["state"] == "ready", (receipt, [name for name, _ in hy_calls])
|
||||
assert [name for name, _ in hy_calls] == ["add_extracted"]
|
||||
assert hy_calls[0][1]["contents"] == [TEXT]
|
||||
assert len(receipt["memory_ids"]) == 1
|
||||
memory = app.state.memory_governance_store.get(
|
||||
receipt["memory_ids"][0], user_id="local"
|
||||
)
|
||||
assert memory is not None and memory.content == TEXT
|
||||
assert memory.status == "confirmed" and memory.scopes == ("all-apps",)
|
||||
assert receipt["evidence_id"] in memory.evidence_ids
|
||||
assert memory.memory_id in hy_calls[0][1]["raw"]["memory_ids"]
|
||||
assert app.state.evidence_store.get(receipt["evidence_id"]).verbatim == TEXT
|
||||
# Still the person's own words: matter discovery keeps reading the note.
|
||||
selected = select_evidence(app.state.evidence_store.database_path)
|
||||
assert receipt["evidence_id"] in {record.evidence_id for record in selected.records}
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_enabled_continuous_policy_processes_a_saved_note_without_separate_consent(
|
||||
tmp_path, hy_calls
|
||||
):
|
||||
async def scenario():
|
||||
app, model = _application(tmp_path), ScriptedModel()
|
||||
async with app.router.lifespan_context(app), _client(app) as client:
|
||||
await _configure(app, model)
|
||||
app.state.memory_note_store.intake = None
|
||||
profile = app.state.model_profile_store.active(app.state.secret_store)
|
||||
assert profile is not None
|
||||
with app.state.evidence_store._connect(writing=True) as db:
|
||||
ingest_journal.initialize(db)
|
||||
continuous_policy.initialize(db)
|
||||
continuous_policy.enable(
|
||||
db,
|
||||
profile=app.state.model_profile_store.snapshot(
|
||||
profile.profile_id, profile.revision
|
||||
),
|
||||
expected_revision=0,
|
||||
expected_legacy_revision=0,
|
||||
operation_id=str(uuid4()),
|
||||
backlog_mode="none",
|
||||
user_confirmed=True,
|
||||
)
|
||||
|
||||
saved = await client.post(
|
||||
"/api/v1/memory-notes",
|
||||
json={"request_id": str(uuid4()), "text": TEXT, "scope": "all-apps"},
|
||||
)
|
||||
assert saved.status_code == 201, saved.text
|
||||
processor = app.state.memory_processing_supervisor
|
||||
await asyncio.gather(*tuple(processor._workers.values()))
|
||||
receipt = (
|
||||
await client.get(f"/api/v1/memory-notes/{saved.json()['evidence_id']}")
|
||||
).json()
|
||||
|
||||
assert receipt["state"] == "ready"
|
||||
assert [name for name, _ in hy_calls] == ["add_extracted"]
|
||||
with app.state.evidence_store._connect() as db:
|
||||
assert (
|
||||
db.execute("SELECT COUNT(*) FROM model_content_permissions").fetchone()[0] == 0
|
||||
)
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
@pytest.mark.xfail(
|
||||
strict=True,
|
||||
reason=(
|
||||
"T04 wait-user: a note's permission lease covers only that note's evidence, so HY "
|
||||
"cannot see the memory an earlier note produced and adds a second one. Letting it "
|
||||
"see earlier notes means sending them to the model with this one: a consent-scope "
|
||||
"decision. The bridge's no-ids fallback also cannot find it (decorated search "
|
||||
"content; the leased vector store has no keyword_search)."
|
||||
),
|
||||
)
|
||||
def test_identical_second_note_points_at_the_memory_hy_already_holds(tmp_path, hy_calls):
|
||||
async def scenario():
|
||||
app, model = _application(tmp_path), ScriptedModel()
|
||||
async with app.router.lifespan_context(app), _client(app) as client:
|
||||
await _configure(app, model)
|
||||
first = await _remember(app, client, TEXT)
|
||||
assert first["state"] == "ready", first
|
||||
second = await _remember(app, client, TEXT)
|
||||
|
||||
# HY really deduplicated: reconcile saw the first memory and wrote nothing new.
|
||||
assert len(model.reconcile_prompts) == 1 and TEXT in model.reconcile_prompts[0]
|
||||
assert first["memory_ids"][0] in model.reconcile_prompts[0]
|
||||
assert not hy_calls[-1][1]["raw"].get("memory_ids")
|
||||
assert second["state"] == "ready", second
|
||||
assert second["memory_ids"] == first["memory_ids"]
|
||||
memories = app.state.memory_governance_store.list(user_id="local")
|
||||
assert [item.memory_id for item in memories] == first["memory_ids"]
|
||||
assert {first["evidence_id"], second["evidence_id"]} <= set(memories[0].evidence_ids)
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_note_saved_before_t04_is_imported_too_and_keeps_all_apps_scope(tmp_path):
|
||||
"""Notes stored before T04 are not migrated; retrying one must still write it as said."""
|
||||
calls = []
|
||||
|
||||
class Bridge:
|
||||
async def ingest(self, request):
|
||||
pytest.fail("A note the person asked to keep must be imported, not re-extracted")
|
||||
|
||||
async def ingest_extracted(self, request, contents):
|
||||
calls.append(list(contents))
|
||||
return HyMemoryIngestResult(
|
||||
success=True,
|
||||
memories=(
|
||||
HyMemoryOutput(
|
||||
memory_id="legacy-note-memory",
|
||||
content=TEXT,
|
||||
owner="user",
|
||||
layer="L2_FACT",
|
||||
),
|
||||
),
|
||||
)
|
||||
|
||||
async def profile_memories(self, **kwargs):
|
||||
return ()
|
||||
|
||||
async def close(self):
|
||||
pass
|
||||
|
||||
async def scenario():
|
||||
app, model = _application(tmp_path), ScriptedModel()
|
||||
async with app.router.lifespan_context(app), _client(app) as client:
|
||||
await _configure(app, model)
|
||||
request_id = str(uuid4())
|
||||
record = app.state.evidence_store.put(
|
||||
EvidenceInput(
|
||||
source_id=SOURCE_ID,
|
||||
conversation_id=f"note:{request_id}",
|
||||
message_id=request_id,
|
||||
role="user",
|
||||
written_by=WrittenBy.PERSON,
|
||||
created_at=datetime.now(UTC),
|
||||
verbatim=TEXT,
|
||||
origin_meta={
|
||||
EvidenceOrigin.MANUAL_IMPORT: OriginMetadata(
|
||||
raw_meta={
|
||||
"input_mode": "direct_memory_note",
|
||||
"recall_scope": "all-apps",
|
||||
}
|
||||
)
|
||||
},
|
||||
)
|
||||
).record
|
||||
database = app.state.evidence_store.database_path
|
||||
with closing(sqlite3.connect(database)) as db, db:
|
||||
db.execute(
|
||||
"INSERT INTO source_grants VALUES ('local',?,?,'default',?,NULL)",
|
||||
(SOURCE_ID, str(database.parent), datetime.now(UTC).isoformat()),
|
||||
)
|
||||
await confirm_model_content(
|
||||
client,
|
||||
source_ids=(SOURCE_ID,),
|
||||
evidence_ids=(record.evidence_id,),
|
||||
future_additions=False,
|
||||
)
|
||||
processor = app.state.memory_processing_supervisor
|
||||
|
||||
@asynccontextmanager
|
||||
async def isolated(lease, _secret, _data):
|
||||
lease.recheck()
|
||||
yield Bridge()
|
||||
|
||||
processor.permission_bridge_factory = isolated
|
||||
app.state.memory_write_drain.stop()
|
||||
await app.state.memory_write_drain_task
|
||||
started = await client.post(f"/api/v1/memory-notes/{record.evidence_id}/process")
|
||||
assert started.status_code == 200, started.text
|
||||
assert await app.state.memory_write_drain.run_one()
|
||||
deadline = asyncio.get_running_loop().time() + 5
|
||||
while True:
|
||||
await asyncio.gather(*tuple(processor._workers.values()))
|
||||
receipt = (await client.get(f"/api/v1/memory-notes/{record.evidence_id}")).json()
|
||||
if receipt["state"] != "running" or asyncio.get_running_loop().time() >= deadline:
|
||||
break
|
||||
await asyncio.sleep(0.01)
|
||||
assert calls == [[TEXT]]
|
||||
assert receipt["state"] == "ready" and receipt["memory_ids"] == ["legacy-note-memory"]
|
||||
memory = app.state.memory_governance_store.get("legacy-note-memory", user_id="local")
|
||||
assert memory.scopes == ("all-apps",) and memory.content == TEXT
|
||||
assert record.evidence_id in memory.evidence_ids
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_model_outage_keeps_the_original_words_and_a_retry_writes_them(tmp_path, hy_calls):
|
||||
async def scenario():
|
||||
app, model = _application(tmp_path), ScriptedModel()
|
||||
async with app.router.lifespan_context(app), _client(app) as client:
|
||||
await _configure(app, model)
|
||||
model.embeddings_down = True
|
||||
failed = await _remember(app, client, TEXT)
|
||||
assert failed["state"] in {"waiting_for_model", "failed"}, failed
|
||||
assert failed["memory_ids"] == []
|
||||
assert app.state.evidence_store.get(failed["evidence_id"]).verbatim == TEXT
|
||||
assert app.state.memory_governance_store.list(user_id="local") == []
|
||||
|
||||
model.embeddings_down = False
|
||||
retried = await client.post(
|
||||
"/api/v1/memory-processing/retry-unknown", json={"confirm": True}
|
||||
)
|
||||
assert retried.status_code == 200, retried.text
|
||||
await _wait_for_note_job(app, failed["evidence_id"])
|
||||
receipt = (await client.get(f"/api/v1/memory-notes/{failed['evidence_id']}")).json()
|
||||
assert receipt["state"] == "ready", receipt
|
||||
assert hy_calls[-1][0] == "add_extracted" and hy_calls[-1][1]["contents"] == [TEXT]
|
||||
memory = app.state.memory_governance_store.get(
|
||||
receipt["memory_ids"][0], user_id="local"
|
||||
)
|
||||
assert memory.content == TEXT and failed["evidence_id"] in memory.evidence_ids
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("raw_meta", "imported"),
|
||||
[
|
||||
({"input_mode": "direct_memory_note", "recall_scope": "all-apps"}, True),
|
||||
({"matter_id": "matter-1"}, False),
|
||||
],
|
||||
ids=["memory-note", "matter-note"],
|
||||
)
|
||||
def test_only_a_note_saved_as_memory_skips_extraction(tmp_path, raw_meta, imported):
|
||||
"""Matter notes share the manual source but are context for a matter, not a memory."""
|
||||
stores = _setup(tmp_path)
|
||||
evidence, _profiles, tasks, governance, _secrets, profile = stores
|
||||
record = evidence.put(
|
||||
EvidenceInput(
|
||||
source_id=SOURCE_ID,
|
||||
conversation_id=f"note:{uuid4()}",
|
||||
message_id=str(uuid4()),
|
||||
role="user",
|
||||
written_by=WrittenBy.PERSON,
|
||||
created_at=datetime.now(UTC),
|
||||
verbatim=TEXT,
|
||||
origin_meta={EvidenceOrigin.MANUAL_IMPORT: OriginMetadata(raw_meta=raw_meta)},
|
||||
)
|
||||
).record
|
||||
bridge = FakeBridge()
|
||||
|
||||
asyncio.run(_runner(tmp_path, stores, bridge).run(_task(tasks, profile, record).task_id))
|
||||
|
||||
assert len(bridge.requests) == 1
|
||||
assert bridge.extracted_contents == ([(TEXT,)] if imported else [])
|
||||
if imported:
|
||||
(memory,) = governance.list(user_id="local")
|
||||
assert memory.content == TEXT and record.evidence_id in memory.evidence_ids
|
||||
@@ -1,6 +1,5 @@
|
||||
import asyncio
|
||||
import sqlite3
|
||||
from contextlib import asynccontextmanager
|
||||
from uuid import uuid4
|
||||
|
||||
import pytest
|
||||
@@ -9,9 +8,7 @@ from pydantic import SecretStr
|
||||
|
||||
from one_memory.api import create_app
|
||||
from one_memory.config import Settings
|
||||
from one_memory.hy_memory_bridge import HyMemoryBridgeError, HyMemoryIngestResult, HyMemoryOutput
|
||||
from one_memory.imports.models import WrittenBy
|
||||
from one_memory.model_profiles import ModelCapabilities, ModelProfileInput, ProviderKind
|
||||
from one_memory.secrets import InMemorySecretStore
|
||||
|
||||
|
||||
@@ -37,30 +34,6 @@ def client_for(app):
|
||||
)
|
||||
|
||||
|
||||
async def wait_for_note_job(app, evidence_id, *, timeout=10.0):
|
||||
"""Wait for the durable note job to leave pending/running.
|
||||
|
||||
The note now reaches HY through the shared durable queue
|
||||
(`entry:<evidence_id>`), so the drain - not the supervisor's in-memory
|
||||
workers - is what completes it. Waiting on `processor._workers` directly
|
||||
races that drain and can observe zero requests.
|
||||
"""
|
||||
import time as _time
|
||||
|
||||
deadline = _time.monotonic() + timeout
|
||||
key = f"entry:{evidence_id}"
|
||||
while _time.monotonic() < deadline:
|
||||
with sqlite3.connect(app.state.evidence_store.database_path) as db:
|
||||
row = db.execute(
|
||||
"SELECT state FROM memory_write_jobs WHERE idempotency_key=?",
|
||||
(key,),
|
||||
).fetchone()
|
||||
if row is not None and row[0] not in {"pending", "running"}:
|
||||
return row[0]
|
||||
await asyncio.sleep(0.05)
|
||||
raise AssertionError(f"note job for {evidence_id} never settled")
|
||||
|
||||
|
||||
def test_empty_library_can_save_a_note_without_a_matter_or_configured_model(tmp_path):
|
||||
async def scenario():
|
||||
app = application(tmp_path)
|
||||
@@ -153,12 +126,6 @@ def test_saved_note_evidence_returns_raw_and_consent_reference_from_same_record(
|
||||
assert value["evidence_id"] == saved["evidence_id"] and value["revision"] == 1
|
||||
assert value["role"] == "user" and value["written_by"] == "person"
|
||||
assert value["available"] is True and value["source_id"] == "one-memory-manual"
|
||||
from one_memory.model_content_permissions import model_content_evidence_snapshot
|
||||
|
||||
record = app.state.evidence_store.get(saved["evidence_id"])
|
||||
reference = model_content_evidence_snapshot(record).model_dump(mode="json")
|
||||
assert all(value[key] == expected for key, expected in reference.items())
|
||||
assert app.state.model_content_permissions.receipt(user_id="local") is None
|
||||
assert app.state.memory_task_store.list() == []
|
||||
assert (await client.get("/api/v1/memory-notes/missing/evidence")).status_code == 404
|
||||
app.state.source_grant_store.revoke(user_id="local", source_id="one-memory-manual")
|
||||
@@ -210,128 +177,3 @@ def test_revoked_note_source_is_not_silently_reauthorized(tmp_path):
|
||||
assert (await client.get("/api/v1/memory-notes")).json() == []
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
@pytest.mark.parametrize("fail", [False, True])
|
||||
def test_note_reuses_hy_queue_and_never_reports_saved_raw_text_as_extracted(tmp_path, fail):
|
||||
async def scenario():
|
||||
app = application(tmp_path)
|
||||
requests = []
|
||||
|
||||
class Bridge:
|
||||
async def ingest(self, request):
|
||||
pytest.fail("A note the person asked to keep must be imported, not re-extracted")
|
||||
|
||||
async def ingest_extracted(self, request, contents):
|
||||
assert list(contents) == [turn.content for turn in request.turns]
|
||||
requests.append(request)
|
||||
if fail:
|
||||
raise HyMemoryBridgeError("hy_memory_unavailable")
|
||||
return HyMemoryIngestResult(
|
||||
success=True,
|
||||
memories=(
|
||||
HyMemoryOutput(
|
||||
memory_id="note-memory",
|
||||
content="用户不在周五安排项目周会。",
|
||||
owner="user",
|
||||
layer="L2_FACT",
|
||||
),
|
||||
),
|
||||
)
|
||||
|
||||
async def profile_memories(self, **kwargs):
|
||||
return ()
|
||||
|
||||
async def close(self):
|
||||
pass
|
||||
|
||||
async with app.router.lifespan_context(app), client_for(app) as client:
|
||||
profiles, secrets = app.state.model_profile_store, app.state.secret_store
|
||||
profile = profiles.create(
|
||||
ModelProfileInput(name="Test", provider_kind=ProviderKind.OLLAMA, model="fixture"),
|
||||
secrets,
|
||||
)
|
||||
profiles.record_capabilities(
|
||||
profile.profile_id,
|
||||
expected_revision=profile.revision,
|
||||
capabilities=ModelCapabilities(
|
||||
endpoint_reachable=True, model_available=True, embedding_available=True
|
||||
),
|
||||
secrets=secrets,
|
||||
)
|
||||
profiles.activate(profile.profile_id, secrets)
|
||||
processor = app.state.memory_processing_supervisor
|
||||
|
||||
@asynccontextmanager
|
||||
async def isolated_bridge(lease, _secret, _data):
|
||||
lease.recheck()
|
||||
yield Bridge()
|
||||
|
||||
processor.permission_bridge_factory = isolated_bridge
|
||||
response = await client.post(
|
||||
"/api/v1/memory-notes",
|
||||
json={
|
||||
"request_id": str(uuid4()),
|
||||
"text": "请记住:项目周会不排在周五。",
|
||||
"scope": "all-apps",
|
||||
},
|
||||
)
|
||||
assert response.status_code == 201
|
||||
saved = response.json()
|
||||
assert saved["state"] == "waiting_for_permission"
|
||||
assert requests == []
|
||||
# Reproduce E5: the source is revoked while this unsent durable
|
||||
# note is parked, so the queue records a terminal cancellation.
|
||||
# Explicit processing after a fresh grant must safely revive it.
|
||||
with sqlite3.connect(app.state.evidence_store.database_path) as db:
|
||||
db.execute(
|
||||
"UPDATE memory_write_jobs SET state='cancelled',"
|
||||
"last_error='model_content_permission_revoked',hy_invoked=0 "
|
||||
"WHERE idempotency_key=?",
|
||||
(f"entry:{saved['evidence_id']}",),
|
||||
)
|
||||
preview = await client.post(
|
||||
"/api/v1/model-content-permission/preview",
|
||||
json={
|
||||
"profile_id": profile.profile_id,
|
||||
"profile_revision": profile.revision,
|
||||
"source_ids": ["one-memory-manual"],
|
||||
"evidence_ids": [saved["evidence_id"]],
|
||||
},
|
||||
)
|
||||
assert preview.status_code == 200
|
||||
consent = await client.post(
|
||||
"/api/v1/model-content-permission",
|
||||
json={
|
||||
"snapshot": preview.json(),
|
||||
"expected_revision": 0,
|
||||
"user_confirmed": True,
|
||||
},
|
||||
)
|
||||
assert consent.status_code == 200
|
||||
started = await client.post(f"/api/v1/memory-notes/{saved['evidence_id']}/process")
|
||||
assert started.status_code == 200
|
||||
# The note is completed by the shared durable queue now; wait on its
|
||||
# job rather than the supervisor's in-memory workers.
|
||||
await wait_for_note_job(app, saved["evidence_id"])
|
||||
receipt = (await client.get(f"/api/v1/memory-notes/{saved['evidence_id']}")).json()
|
||||
assert (
|
||||
len(requests) == 1
|
||||
and requests[0].turns[0].content == "请记住:项目周会不排在周五。"
|
||||
)
|
||||
assert app.state.evidence_store.count() == 1
|
||||
if fail:
|
||||
assert receipt["state"] == "waiting_for_model" and not receipt["memory_ids"]
|
||||
assert receipt["error"] == "hy_memory_unavailable"
|
||||
else:
|
||||
assert receipt["state"] == "ready" and receipt["memory_ids"] == ["note-memory"]
|
||||
memory = app.state.memory_governance_store.get("note-memory", user_id="local")
|
||||
assert memory.scopes == ("all-apps",)
|
||||
recall = await client.post("/api/v1/recall", json={"query": "周五 项目周会"})
|
||||
assert recall.status_code == 200
|
||||
assert any(hit["memory_id"] == "note-memory" for hit in recall.json()["hits"])
|
||||
await client.post(f"/api/v1/memory-notes/{saved['evidence_id']}/process", json={})
|
||||
await wait_for_note_job(app, saved["evidence_id"])
|
||||
assert len(requests) == 1
|
||||
|
||||
asyncio.run(scenario())
|
||||
@@ -315,6 +315,12 @@ def test_memory_task_logs_six_step_durations_without_content(tmp_path) -> None:
|
||||
events = [json.loads(line) for line in written.splitlines()]
|
||||
steps = [event for event in events if event["event"] == "memory_task.step_completed"]
|
||||
assert [event["details"]["phase"] for event in steps] == ["1", "2", "3", "4", "5", "6"]
|
||||
finished = next(event for event in events if event["event"] == "memory_task.finished")
|
||||
assert finished["details"] == {
|
||||
"attempt": 1,
|
||||
"pending_count": 0,
|
||||
"result_count": 1,
|
||||
}
|
||||
assert tasks.processing_metrics()["hy_calls"] == 1
|
||||
assert all(event["duration_ms"] >= 0 for event in steps)
|
||||
assert marker not in written
|
||||
@@ -2925,7 +2931,7 @@ def test_frozen_harness_refuses_new_processing_before_a_task_or_model_call(tmp_p
|
||||
supervisor._bridges[(profile.profile_id, profile.revision, None)] = bridge
|
||||
|
||||
assert supervisor._source_may_write([user]) is False
|
||||
with pytest.raises(HyMemoryBridgeError, match="model_content_source_disabled"):
|
||||
with pytest.raises(HyMemoryBridgeError, match="hy_memory_source_disabled"):
|
||||
asyncio.run(supervisor.process_records((user, reply)))
|
||||
assert asyncio.run(supervisor.enqueue_conversation_off_thread(user)) == []
|
||||
assert supervisor.prepare_backfill() == []
|
||||
@@ -2999,7 +3005,7 @@ def test_revoked_grant_refuses_new_processing(tmp_path) -> None:
|
||||
supervisor._bridges[(profile.profile_id, profile.revision, None)] = bridge
|
||||
|
||||
assert supervisor._source_may_write([user]) is False
|
||||
with pytest.raises(HyMemoryBridgeError, match="model_content_source_disabled"):
|
||||
with pytest.raises(HyMemoryBridgeError, match="hy_memory_source_disabled"):
|
||||
asyncio.run(supervisor.process_records((user, reply)))
|
||||
assert asyncio.run(supervisor.enqueue_conversation_off_thread(user)) == []
|
||||
assert supervisor.prepare_backfill() == []
|
||||
|
||||
@@ -13,7 +13,6 @@ def test_sync_config_defaults_off_and_survives_restart(tmp_path):
|
||||
|
||||
saved = json.loads((tmp_path / "config.json").read_text())
|
||||
assert saved == {
|
||||
"model_content_allowed": False,
|
||||
"schema_version": 1,
|
||||
"sources": {"cursor": {"sync_enabled": True}},
|
||||
}
|
||||
@@ -31,19 +30,27 @@ def test_sync_config_updates_one_source_without_losing_another(tmp_path):
|
||||
assert reopened.sync_enabled("claude-code")
|
||||
|
||||
|
||||
def test_model_content_permission_is_one_global_final_state(tmp_path):
|
||||
def test_legacy_model_content_permission_is_ignored_on_upgrade(tmp_path):
|
||||
(tmp_path / "config.json").write_text(
|
||||
json.dumps(
|
||||
{
|
||||
"schema_version": 1,
|
||||
"model_content_allowed": False,
|
||||
"sources": {"cursor": {"sync_enabled": True}},
|
||||
}
|
||||
)
|
||||
)
|
||||
|
||||
store = ConfigStore(tmp_path)
|
||||
assert not store.model_content_allowed()
|
||||
|
||||
store.set_model_content_allowed(True)
|
||||
store.set_sync("cursor", True)
|
||||
reopened = ConfigStore(tmp_path)
|
||||
assert reopened.model_content_allowed()
|
||||
assert reopened.sync_enabled("cursor")
|
||||
|
||||
reopened.set_model_content_allowed(False)
|
||||
assert not ConfigStore(tmp_path).model_content_allowed()
|
||||
assert ConfigStore(tmp_path).sync_enabled("cursor")
|
||||
assert store.sync_enabled("cursor")
|
||||
store.set_sync("codex", True)
|
||||
assert json.loads((tmp_path / "config.json").read_text()) == {
|
||||
"schema_version": 1,
|
||||
"sources": {
|
||||
"codex": {"sync_enabled": True},
|
||||
"cursor": {"sync_enabled": True},
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def test_invalid_config_fails_loudly_instead_of_silently_disabling_sources(tmp_path):
|
||||
|
||||
@@ -1097,6 +1097,52 @@ def test_reconciled_absent_timeout_is_requeued_once_without_resetting_attempts(
|
||||
)
|
||||
|
||||
|
||||
def test_unresolved_unknown_is_automatically_retried_only_once(tmp_path) -> None:
|
||||
path = tmp_path / "one-memory.sqlite3"
|
||||
queue = DurableWriteQueue(path)
|
||||
record = queue.submit(_request())
|
||||
with sqlite3.connect(path) as connection, connection:
|
||||
connection.execute(
|
||||
"UPDATE memory_write_jobs SET state='unknown',attempts=1,hy_invoked=1,"
|
||||
"last_error='model_content_outcome_unknown',updated_at=? WHERE job_id=?",
|
||||
("2026-01-01T00:00:00+00:00", record.job_id),
|
||||
)
|
||||
|
||||
cutoff = datetime(2026, 1, 1, 0, 5, tzinfo=UTC)
|
||||
assert queue.retry_first_unknown_after_reconcile(older_than=cutoff) == 1
|
||||
first_retry = queue.get(record.request_id)
|
||||
assert first_retry.state == PENDING
|
||||
with sqlite3.connect(path) as connection:
|
||||
assert connection.execute(
|
||||
"SELECT attempts FROM memory_write_jobs WHERE job_id=?",
|
||||
(record.job_id,),
|
||||
).fetchone()[0] == 1
|
||||
|
||||
drain = DurableWriteDrain(queue, execute=_never_execute, owner="retry-owner")
|
||||
assert drain.claim(limit=1) == (record.job_id,)
|
||||
assert queue.mark_invoked(record.job_id, owner="retry-owner")
|
||||
queue.settle(
|
||||
record.job_id,
|
||||
state=UNKNOWN,
|
||||
owner="retry-owner",
|
||||
error_code="model_content_outcome_unknown",
|
||||
hy_invoked=True,
|
||||
)
|
||||
|
||||
with sqlite3.connect(path) as connection:
|
||||
assert connection.execute(
|
||||
"SELECT attempts FROM memory_write_jobs WHERE job_id=?",
|
||||
(record.job_id,),
|
||||
).fetchone()[0] == 2
|
||||
assert (
|
||||
queue.retry_first_unknown_after_reconcile(
|
||||
older_than=datetime.now(UTC) + timedelta(minutes=1)
|
||||
)
|
||||
== 0
|
||||
)
|
||||
assert queue.get(record.request_id).state == UNKNOWN
|
||||
|
||||
|
||||
def test_drain_runs_unknown_reconciliation_during_busy_safe_maintenance(tmp_path) -> None:
|
||||
queue = DurableWriteQueue(tmp_path / "one-memory.sqlite3")
|
||||
calls = 0
|
||||
@@ -1116,6 +1162,20 @@ def test_drain_runs_unknown_reconciliation_during_busy_safe_maintenance(tmp_path
|
||||
assert calls == 1
|
||||
|
||||
|
||||
def test_unknown_reconciliation_runs_on_a_ten_second_ceiling(tmp_path) -> None:
|
||||
queue = DurableWriteQueue(tmp_path / "one-memory.sqlite3")
|
||||
|
||||
production = DurableWriteDrain(queue, execute=_never_execute)
|
||||
immediate = DurableWriteDrain(
|
||||
queue,
|
||||
execute=_never_execute,
|
||||
reap_interval=timedelta(0),
|
||||
)
|
||||
|
||||
assert production._unknown_reconcile_interval == timedelta(seconds=10)
|
||||
assert immediate._unknown_reconcile_interval == timedelta(0)
|
||||
|
||||
|
||||
def test_a_reclaimed_uninvoked_job_with_attempts_left_is_retryable(tmp_path) -> None:
|
||||
"""Recovery of a never-invoked lease is the retry path, not `unknown`.
|
||||
|
||||
@@ -2063,6 +2123,19 @@ def test_sqlite_claim_keeps_fifo_order_when_later_work_is_live(tmp_path) -> None
|
||||
assert drain.claim(limit=1) == (normal.job_id,)
|
||||
|
||||
|
||||
def test_unknown_source_head_does_not_block_later_pending_work(tmp_path) -> None:
|
||||
path = tmp_path / "one-memory.sqlite3"
|
||||
queue = DurableWriteQueue(path)
|
||||
uncertain = queue.submit(_request(evidence=("e1",), key="unknown"))
|
||||
later = queue.submit(_request(evidence=("e2",), key="later"))
|
||||
queue.settle(uncertain.job_id, state=UNKNOWN, hy_invoked=True)
|
||||
|
||||
drain = DurableWriteDrain(queue, execute=_never_execute, owner="next-owner")
|
||||
|
||||
assert drain.claim(limit=1) == (later.job_id,)
|
||||
assert queue.get("unknown").state == UNKNOWN
|
||||
|
||||
|
||||
def test_sqlite_claim_orders_different_sources_by_enqueue_sequence(tmp_path) -> None:
|
||||
path = tmp_path / "one-memory.sqlite3"
|
||||
queue = DurableWriteQueue(path)
|
||||
@@ -3267,103 +3340,6 @@ def test_idle_maintenance_database_failure_is_sampled_without_database_error(
|
||||
assert events[-1]["error_type"] == "RuntimeError"
|
||||
|
||||
|
||||
@pytest.mark.slow
|
||||
@pytest.mark.integration
|
||||
@pytest.mark.serial
|
||||
def test_no_model_permission_parks_without_sending(tmp_path) -> None:
|
||||
"""Model egress permission is separate from source read permission.
|
||||
|
||||
With a permission guard installed but no grant, the durable job must park
|
||||
in `waiting_for_permission` and never reach HY: the Evidence is committed
|
||||
locally, but no private verbatim is sent to a model. It must not read as
|
||||
`failed` (a user switch is not a pipeline break), and a later grant must be
|
||||
able to re-admit it.
|
||||
"""
|
||||
from datetime import UTC, datetime
|
||||
|
||||
from one_memory.imports.models import (
|
||||
EvidenceInput,
|
||||
EvidenceOrigin,
|
||||
OriginMetadata,
|
||||
WrittenBy,
|
||||
)
|
||||
from one_memory.imports.storage import EvidenceStore
|
||||
from one_memory.memory_governance import MemoryGovernanceStore
|
||||
from one_memory.memory_processing import MemoryProcessingSupervisor
|
||||
from one_memory.memory_tasks import MemoryTaskStore
|
||||
from one_memory.memory_server.config_state import ConfigStore
|
||||
from one_memory.model_profiles import (
|
||||
ModelCapabilities,
|
||||
ModelProfileInput,
|
||||
ModelProfileStore,
|
||||
ProviderKind,
|
||||
)
|
||||
from one_memory.secrets import InMemorySecretStore
|
||||
|
||||
database = tmp_path / "one-memory.sqlite3"
|
||||
secrets = InMemorySecretStore()
|
||||
store = EvidenceStore(database)
|
||||
record = store.put(
|
||||
EvidenceInput(
|
||||
source_id="cursor",
|
||||
conversation_id="c",
|
||||
message_id="m-private",
|
||||
role="user",
|
||||
written_by=WrittenBy.PERSON,
|
||||
verbatim="private verbatim that must not be sent",
|
||||
created_at=datetime.now(UTC),
|
||||
origin_meta={EvidenceOrigin.FILE: OriginMetadata()},
|
||||
)
|
||||
).record
|
||||
|
||||
profiles = ModelProfileStore(database)
|
||||
profile = profiles.create(
|
||||
ModelProfileInput(
|
||||
name="Local",
|
||||
provider_kind=ProviderKind.OLLAMA,
|
||||
model="hy-bot-phi4",
|
||||
embedding_model="nomic-embed-text",
|
||||
),
|
||||
secrets,
|
||||
)
|
||||
profiles.record_capabilities(
|
||||
profile.profile_id,
|
||||
expected_revision=profile.revision,
|
||||
capabilities=ModelCapabilities(
|
||||
endpoint_reachable=True,
|
||||
model_available=True,
|
||||
structured_output=True,
|
||||
embedding_available=True,
|
||||
),
|
||||
secrets=secrets,
|
||||
)
|
||||
profiles.activate(profile.profile_id, secrets)
|
||||
|
||||
config = ConfigStore(tmp_path)
|
||||
config.set_model_content_allowed(False)
|
||||
supervisor = MemoryProcessingSupervisor(
|
||||
tasks=MemoryTaskStore(database),
|
||||
profiles=profiles,
|
||||
secrets=secrets,
|
||||
evidence=store,
|
||||
governance=MemoryGovernanceStore(database),
|
||||
data_dir=tmp_path,
|
||||
model_content_config=config,
|
||||
)
|
||||
|
||||
queue = DurableWriteQueue(database)
|
||||
submitted = queue.submit(_request(evidence=(record.evidence_id,)))
|
||||
before = queue.get(submitted.request_id)
|
||||
assert before.state == "pending"
|
||||
_claim_through_drain(queue, submitted.job_id)
|
||||
|
||||
asyncio.run(execute_job(submitted.job_id, queue=queue, supervisor=supervisor, evidence=store))
|
||||
|
||||
settled = queue.get(submitted.request_id)
|
||||
assert settled.state == "waiting_for_permission", settled.state
|
||||
assert "permission" in settled.detail.lower()
|
||||
|
||||
|
||||
def test_an_unknown_is_retired_only_by_an_inquiry(tmp_path) -> None:
|
||||
"""`unknown` must be resolvable, but never by a blind retry.
|
||||
|
||||
|
||||
@@ -2103,7 +2103,47 @@ def test_migration_49_resets_only_cursor_history_when_conversation_time_is_missi
|
||||
|
||||
result = asyncio.run(migrate(SqliteMigrationStore(path), MIGRATIONS))
|
||||
|
||||
assert result.applied == (49, 50)
|
||||
assert result.applied == (49, 50, 51)
|
||||
with main_database(path).read() as connection:
|
||||
assert connection.execute("SELECT COUNT(*) FROM source_cursors").fetchone()[0] == 0
|
||||
assert connection.execute("SELECT COUNT(*) FROM live_source_cursors").fetchone()[0] == 1
|
||||
|
||||
|
||||
def test_upgrade_drops_obsolete_hy_node_permissions_without_touching_memories(
|
||||
tmp_path,
|
||||
) -> None:
|
||||
import sqlite3
|
||||
|
||||
from one_memory.memory_server.migration_runner import SqliteMigrationStore
|
||||
|
||||
path = tmp_path / "existing.sqlite3"
|
||||
asyncio.run(
|
||||
migrate(
|
||||
SqliteMigrationStore(path),
|
||||
tuple(migration for migration in MIGRATIONS if migration.version <= 50),
|
||||
)
|
||||
)
|
||||
with sqlite3.connect(path) as connection:
|
||||
connection.execute(
|
||||
"CREATE TABLE hy_node_permissions ("
|
||||
"node_id TEXT PRIMARY KEY, fingerprint TEXT NOT NULL, evidence_json TEXT NOT NULL)"
|
||||
)
|
||||
connection.execute(
|
||||
"INSERT INTO hy_node_permissions(node_id,fingerprint,evidence_json) "
|
||||
"VALUES ('legacy-node', ?, '[]')",
|
||||
("a" * 64,),
|
||||
)
|
||||
|
||||
result = asyncio.run(migrate(SqliteMigrationStore(path), MIGRATIONS))
|
||||
|
||||
assert result.applied == (51,)
|
||||
with sqlite3.connect(path) as connection:
|
||||
tables = {
|
||||
row[0]
|
||||
for row in connection.execute(
|
||||
"SELECT name FROM sqlite_master WHERE type='table'"
|
||||
)
|
||||
}
|
||||
assert "hy_node_permissions" not in tables
|
||||
assert "governed_memories" in tables
|
||||
assert "evidence" in tables
|
||||
@@ -1070,6 +1070,49 @@ def test_confirmed_retry_requeues_task_only_unknown_without_a_durable_job(tmp_pa
|
||||
assert store.retry_unknown_confirmed() == 0
|
||||
|
||||
|
||||
def test_task_only_unknown_is_automatically_retried_only_once(tmp_path) -> None:
|
||||
store, profile = _profile_and_tasks(tmp_path)
|
||||
task = store.create(
|
||||
request_id="auto-retry-unknown",
|
||||
user_id="local",
|
||||
kind=MemoryTaskKind.EXTRACT,
|
||||
input_ref="orphan-evidence",
|
||||
profile=profile,
|
||||
)
|
||||
with sqlite3.connect(store.database_path) as connection, connection:
|
||||
connection.execute(
|
||||
"UPDATE memory_tasks SET status=?,attempts=1,last_error_code=?,updated_at=? "
|
||||
"WHERE task_id=?",
|
||||
(
|
||||
MemoryTaskStatus.UNKNOWN,
|
||||
"model_content_outcome_unknown",
|
||||
"2026-01-01T00:00:00+00:00",
|
||||
task.task_id,
|
||||
),
|
||||
)
|
||||
|
||||
cutoff = datetime(2026, 1, 1, 0, 5, tzinfo=UTC)
|
||||
assert store.retry_first_unknown_after_reconcile(older_than=cutoff) == 1
|
||||
retried = store.require(task.task_id)
|
||||
assert retried.status is MemoryTaskStatus.PENDING
|
||||
assert retried.attempts == 1
|
||||
|
||||
running, token = store.claim(task.task_id)
|
||||
assert running.attempts == 2
|
||||
store.mark_unknown(
|
||||
task.task_id,
|
||||
runner_token=token,
|
||||
error_code="model_content_outcome_unknown",
|
||||
)
|
||||
assert (
|
||||
store.retry_first_unknown_after_reconcile(
|
||||
older_than=datetime.now(UTC) + timedelta(minutes=1)
|
||||
)
|
||||
== 0
|
||||
)
|
||||
assert store.require(task.task_id).status is MemoryTaskStatus.UNKNOWN
|
||||
|
||||
|
||||
def test_orphan_recovery_preserves_active_running_tasks(tmp_path) -> None:
|
||||
store, profile = _profile_and_tasks(tmp_path)
|
||||
active = store.create(
|
||||
|
||||
@@ -1,303 +0,0 @@
|
||||
"""Bulk consent reads preserve exact identities and SQLite snapshot boundaries."""
|
||||
|
||||
import asyncio
|
||||
import sqlite3
|
||||
|
||||
import pytest
|
||||
from fastapi import FastAPI
|
||||
from httpx import ASGITransport, AsyncClient
|
||||
|
||||
from one_memory.database_access import close_main_databases
|
||||
from one_memory.imports.models import EvidenceOrigin, OriginMetadata, WrittenBy
|
||||
from one_memory.model_content_permissions import (
|
||||
ModelContentPermissionError,
|
||||
ModelContentPermissionSnapshot,
|
||||
model_content_evidence_snapshot,
|
||||
)
|
||||
from one_memory.model_content_routes import model_content_router
|
||||
from tests.test_model_content_permissions import (
|
||||
SOURCE,
|
||||
USER,
|
||||
evidence_value,
|
||||
)
|
||||
from tests.test_model_content_permissions import (
|
||||
env as environment_fixture,
|
||||
)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def env(tmp_path):
|
||||
return environment_fixture.__wrapped__(tmp_path)
|
||||
|
||||
|
||||
def many_records(environment, count):
|
||||
records = [environment.record]
|
||||
with environment.evidence._connect(writing=True) as connection:
|
||||
for index in range(1, count):
|
||||
value = evidence_value(str(index), text=f"Synthetic {index}: " + "x" * 128)
|
||||
if index % 2:
|
||||
value = value.model_copy(
|
||||
update={
|
||||
"role": "assistant",
|
||||
"written_by": WrittenBy.AGENT,
|
||||
"origin_meta": {
|
||||
EvidenceOrigin.FILE: OriginMetadata(raw_meta={"nested": [1, "text"]}),
|
||||
EvidenceOrigin.HOOK: OriginMetadata(source_version="fixture-v1"),
|
||||
},
|
||||
}
|
||||
)
|
||||
records.append(environment.evidence.put(value, connection=connection).record)
|
||||
return records
|
||||
|
||||
|
||||
def app_for(environment):
|
||||
app = FastAPI()
|
||||
app.state.model_profile_store = environment.profiles
|
||||
app.state.evidence_store = environment.evidence
|
||||
app.state.model_content_permissions = environment.permissions
|
||||
app.include_router(model_content_router([]))
|
||||
return app
|
||||
|
||||
|
||||
def preview(environment, **changes):
|
||||
async def request():
|
||||
async with AsyncClient(
|
||||
transport=ASGITransport(app=app_for(environment)), base_url="http://test"
|
||||
) as client:
|
||||
return await client.post(
|
||||
"/api/v1/model-content-permission/preview",
|
||||
json={
|
||||
"profile_id": environment.profile.profile_id,
|
||||
"profile_revision": environment.profile.revision,
|
||||
"source_ids": [SOURCE],
|
||||
"all_imported_from_selected_sources": True,
|
||||
**changes,
|
||||
},
|
||||
)
|
||||
|
||||
return asyncio.run(request())
|
||||
|
||||
|
||||
def observe_connections(monkeypatch):
|
||||
connects, statements = [], []
|
||||
original = sqlite3.connect
|
||||
|
||||
def observed(*args, **kwargs):
|
||||
connection = original(*args, **kwargs)
|
||||
connects.append(connection)
|
||||
connection.set_trace_callback(lambda sql: statements.append(sql))
|
||||
return connection
|
||||
|
||||
close_main_databases() # pooled connections would skip the patch
|
||||
|
||||
monkeypatch.setattr(sqlite3, "connect", observed)
|
||||
return connects, statements
|
||||
|
||||
|
||||
def test_preview_uses_bounded_bulk_reads_without_per_id_connections(env, monkeypatch):
|
||||
records = many_records(env, 1000)
|
||||
expected = env.snapshot(records=tuple(records))
|
||||
connects, statements = observe_connections(monkeypatch)
|
||||
response = preview(env)
|
||||
assert response.status_code == 200
|
||||
actual = ModelContentPermissionSnapshot.model_validate(response.json())
|
||||
assert actual == expected
|
||||
assert len(connects) <= 2, "preview still opens one connection per Evidence ID"
|
||||
selects = [sql for sql in statements if sql.lstrip().upper().startswith("SELECT")]
|
||||
assert len(selects) <= 20, "preview still rereads each record separately"
|
||||
assert env.permissions.receipt(user_id=USER) is None
|
||||
|
||||
|
||||
def test_hot_check_is_bulk_and_reuses_same_transaction_verified_refs(env, monkeypatch):
|
||||
records = many_records(env, 1000)
|
||||
receipt = env.permissions.grant(
|
||||
env.snapshot(records=tuple(records)), expected_revision=0, user_confirmed=True
|
||||
)
|
||||
connects, statements = observe_connections(monkeypatch)
|
||||
assert env.check(receipt, records=(records[0],)) == receipt
|
||||
assert len(connects) == 1
|
||||
selects = [sql for sql in statements if sql.lstrip().upper().startswith("SELECT")]
|
||||
assert len(selects) <= 20, (
|
||||
"checking one input still performs per-record SQL for the entire scope"
|
||||
)
|
||||
|
||||
|
||||
def test_all_source_preview_and_future_boundary_share_one_read_snapshot(env, monkeypatch):
|
||||
records = many_records(env, 4)
|
||||
expected = env.snapshot(records=tuple(records), future=True)
|
||||
source = env.permissions._source
|
||||
changed = False
|
||||
|
||||
def concurrent_insert(connection, user_id, source_id):
|
||||
nonlocal changed
|
||||
result = source(connection, user_id, source_id)
|
||||
if not changed:
|
||||
changed = True
|
||||
env.evidence.put(evidence_value("arrived-during-preview"))
|
||||
return result
|
||||
|
||||
monkeypatch.setattr(env.permissions, "_source", concurrent_insert)
|
||||
response = preview(env, include_future_additions=True)
|
||||
assert response.status_code == 200
|
||||
actual = ModelContentPermissionSnapshot.model_validate(response.json())
|
||||
assert changed and actual == expected, "preview mixed old IDs with a newer future watermark"
|
||||
receipt = env.permissions.grant(actual, expected_revision=0, user_confirmed=True)
|
||||
assert receipt.snapshot == expected
|
||||
added = env.evidence.put(evidence_value("arrived-during-preview")).record
|
||||
assert added.evidence_id not in {ref.evidence_id for ref in actual.evidence}
|
||||
assert env.check(receipt, records=(added,)) == receipt
|
||||
|
||||
|
||||
@pytest.mark.parametrize("mutation", ["body", "author", "origins", "delete"])
|
||||
def test_hot_check_still_rejects_changed_unrequested_scope_record(env, mutation):
|
||||
records = many_records(env, 4)
|
||||
receipt = env.permissions.grant(
|
||||
env.snapshot(records=tuple(records)), expected_revision=0, user_confirmed=True
|
||||
)
|
||||
other = records[-1]
|
||||
with sqlite3.connect(env.database) as db:
|
||||
sql = {
|
||||
"body": (
|
||||
"UPDATE evidence SET verbatim='changed without revision or hash' "
|
||||
"WHERE evidence_id=?"
|
||||
),
|
||||
"author": "UPDATE evidence SET written_by='unknown' WHERE evidence_id=?",
|
||||
"origins": "UPDATE evidence_origins SET source_version='changed' WHERE evidence_id=?",
|
||||
"delete": "DELETE FROM evidence WHERE evidence_id=?",
|
||||
}[mutation]
|
||||
db.execute(sql, (other.evidence_id,))
|
||||
with pytest.raises(ModelContentPermissionError, match="evidence_changed"):
|
||||
env.check(receipt, records=(records[0],))
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"selection,code",
|
||||
[
|
||||
("missing", "evidence_not_found"),
|
||||
("other_user", "evidence_not_found"),
|
||||
("other_source", "invalid_content_scope"),
|
||||
("duplicate", "invalid_content_scope"),
|
||||
("ambiguous", "ambiguous_content_scope"),
|
||||
],
|
||||
)
|
||||
def test_bulk_preview_keeps_exact_selection_failures(env, selection, code):
|
||||
other_user = env.evidence.put(evidence_value("other-user", user_id="other")).record
|
||||
other_source = env.evidence.put(evidence_value("other-source", source_id="cursor")).record
|
||||
identifiers = {
|
||||
"missing": ["not-present"],
|
||||
"other_user": [other_user.evidence_id],
|
||||
"other_source": [other_source.evidence_id],
|
||||
"duplicate": [env.record.evidence_id, env.record.evidence_id],
|
||||
"ambiguous": [env.record.evidence_id],
|
||||
}[selection]
|
||||
response = preview(
|
||||
env, evidence_ids=identifiers, all_imported_from_selected_sources=selection == "ambiguous"
|
||||
)
|
||||
assert response.status_code == 409
|
||||
assert response.json()["detail"] == f"model_content_{code}"
|
||||
assert env.permissions.receipt(user_id=USER) is None
|
||||
|
||||
|
||||
def test_bulk_snapshot_preserves_every_existing_hash_field(env):
|
||||
records = many_records(env, 8)
|
||||
expected = tuple(
|
||||
sorted(
|
||||
(model_content_evidence_snapshot(record) for record in records),
|
||||
key=lambda ref: ref.evidence_id,
|
||||
)
|
||||
)
|
||||
actual = ModelContentPermissionSnapshot.model_validate(preview(env).json())
|
||||
assert actual.evidence == expected
|
||||
assert not any(record.verbatim in actual.model_dump_json() for record in records)
|
||||
|
||||
|
||||
def test_identifier_selection_does_not_mix_a_newer_future_watermark(env, monkeypatch):
|
||||
records = many_records(env, 4)
|
||||
expected = env.snapshot(records=tuple(records), future=True)
|
||||
original_connect = sqlite3.connect
|
||||
inserted = False
|
||||
|
||||
class ConcurrentInsert(sqlite3.Connection):
|
||||
def execute(self, sql, parameters=()):
|
||||
nonlocal inserted
|
||||
cursor = super().execute(sql, parameters)
|
||||
if sql.startswith("SELECT evidence_id FROM evidence WHERE user_id=") and not inserted:
|
||||
inserted = True
|
||||
env.evidence.put(evidence_value("after-identifier-selection"))
|
||||
return cursor
|
||||
|
||||
def connect(*args, **kwargs):
|
||||
return original_connect(*args, **kwargs, factory=ConcurrentInsert)
|
||||
|
||||
close_main_databases() # pooled connections would skip the patch
|
||||
|
||||
monkeypatch.setattr(sqlite3, "connect", connect)
|
||||
response = preview(env, include_future_additions=True)
|
||||
assert inserted and response.status_code == 200
|
||||
snapshot = ModelContentPermissionSnapshot.model_validate(response.json())
|
||||
assert snapshot == expected
|
||||
receipt = env.permissions.grant(snapshot, expected_revision=0, user_confirmed=True)
|
||||
assert receipt.snapshot == expected
|
||||
added = env.evidence.put(evidence_value("after-identifier-selection")).record
|
||||
assert added.evidence_id not in {ref.evidence_id for ref in snapshot.evidence}
|
||||
assert env.check(receipt, records=(added,)) == receipt
|
||||
|
||||
|
||||
def test_bulk_batches_keep_one_snapshot_and_next_check_observes_an_edit(env, monkeypatch):
|
||||
records = many_records(env, 1001)
|
||||
snapshot = env.snapshot(records=tuple(records))
|
||||
receipt = env.permissions.grant(snapshot, expected_revision=0, user_confirmed=True)
|
||||
# Large grants now build a bounded scope seal. Invalidate it explicitly to
|
||||
# exercise the full-snapshot fallback and its single read transaction.
|
||||
with sqlite3.connect(env.database) as db:
|
||||
db.execute(
|
||||
"UPDATE model_content_scope_seals SET valid=0 WHERE user_id=? AND revision=?",
|
||||
(receipt.user_id, receipt.revision),
|
||||
)
|
||||
ordered = sorted(records, key=lambda record: record.evidence_id)
|
||||
later = ordered[-1]
|
||||
original_batch = env.permissions._evidence_batch
|
||||
calls = []
|
||||
|
||||
def edit_after_first_batch(connection, identifiers):
|
||||
yield from original_batch(connection, identifiers)
|
||||
calls.append(len(identifiers))
|
||||
if len(calls) == 1:
|
||||
with sqlite3.connect(env.database) as db:
|
||||
db.execute(
|
||||
"UPDATE evidence SET verbatim='edited but same stored hash' "
|
||||
"WHERE evidence_id=?",
|
||||
(later.evidence_id,),
|
||||
)
|
||||
|
||||
with monkeypatch.context() as scoped:
|
||||
scoped.setattr(env.permissions, "_evidence_batch", edit_after_first_batch)
|
||||
assert env.check(receipt, records=(ordered[0],)) == receipt
|
||||
assert calls == [500, 500, 1]
|
||||
with pytest.raises(ModelContentPermissionError, match="evidence_changed"):
|
||||
env.check(receipt, records=(ordered[0],))
|
||||
|
||||
|
||||
def test_same_transaction_reuse_does_not_trust_forged_request_metadata(env):
|
||||
receipt = env.grant()
|
||||
forged = env.record.model_copy(update={"written_by": WrittenBy.AGENT})
|
||||
with pytest.raises(ModelContentPermissionError, match="evidence_changed"):
|
||||
env.check(receipt, records=(forged,))
|
||||
with pytest.raises(ModelContentPermissionError, match="invalid_content_scope"):
|
||||
env.check(receipt, records=(env.record, env.record))
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"column,value",
|
||||
[
|
||||
("raw_meta_json", "not-json"),
|
||||
("origin", "unknown-origin"),
|
||||
("source_version", "x" * 129),
|
||||
],
|
||||
)
|
||||
def test_bulk_corrupt_origins_fail_closed(env, column, value):
|
||||
receipt = env.grant()
|
||||
with sqlite3.connect(env.database) as db:
|
||||
db.execute(f"UPDATE evidence_origins SET {column}=?", (value,))
|
||||
with pytest.raises(ModelContentPermissionError, match="evidence_changed"):
|
||||
env.check(receipt)
|
||||
Loaded 100 of 202 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user