Compare commits

..
Author SHA1 Message Date
alvinfeiandCursor 1dc7e957c0 fix(memhub): harden SDK migration inventory
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-23 17:34:59 +08:00
alvinfeiandCursor 790d6d3d5e fix(memhub): repair SDK authority migration
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-23 17:26:40 +08:00
feitianxiang 4c311b2e93 Merge pull request #149 from Tencent-Hunyuan/fix/memhub-1.0.5-release-doc-test
test(memhub): update 1.0.5 release doc expectations
2026-10-22 21:31:04 +08:00
alvinfeiandCursor 801a2af36b test(memhub): update release documentation expectations
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-22 21:30:31 +08:00
feitianxiang 0a8e43e570 Merge pull request #148 from Tencent-Hunyuan/release/memhub-1.0.5
release(memhub): bump to 1.0.5 and pin hy-memory 2.0.5
2026-10-22 21:22:40 +08:00
alvinfeiandCursor f5a38c38b3 release(memhub): pin hy-memory 2.0.5
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-22 21:20:40 +08:00
feitianxiang d08e070816 Merge pull request #147 from Tencent-Hunyuan/release/hy-memory-2.0.5
release: hy-memory 2.0.5
2026-10-22 20:58:49 +08:00
alvinfeiandCursor 29bd9146a4 release: hy-memory 2.0.5
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-22 20:52:21 +08:00
26c33ee40f refactor(sdk): internalize legacy data migration (#146)
* refactor(sdk): keep legacy migration out of client APIs

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(migration): stage SDK data before cutover

Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: alvinfei <alvinfei@tencent.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-22 20:49:23 +08:00
38 changed files with 1489 additions and 284 deletions

No files matched your search

+84 -61
View File
@@ -2,6 +2,73 @@
Hy Memory and MemHub are versioned and released independently.
## Unreleased
- Fixed legacy-to-SDK startup migration to adopt native Coding Experience and
Working Fact IDs instead of importing them as Chat layers. Raw migration now
rebuilds bounded user/assistant turns from the latest three days, excludes
tools, and enforces the same rolling retention after cutover.
- Paused host capture writes with a live Desktop migration fence; Cursor
History catch-up resumes from its existing checkpoint after cutover.
- Removed MemHub's private `hy_memory_bridge` implementation. MemHub now calls
public `HyMemoryClient` methods at runtime; prompt/reconcile/response
monkey-patches and private VDB/Coding runtime access were deleted. Kuzu Graph
projection/admin and stopped-writer App migration are the documented product
exceptions.
- Added a MemHub-managed native Hermes MemoryProvider with automatic prefetch,
persistent per-session memory-ID deduplication, authorized Broker reads, and
recall-context stripping to prevent memory recursion.
- Added an extensible MemHub interface-localization layer with Simplified
Chinese, English, and Spanish catalogs shared by Web and Electron-native
surfaces. Interface language remains independent from memory content and
model processing language.
- Fixed governed-memory revisions and repeated evidence links failing when the
visibility dirty-marker trigger encounters the same memory twice. Startup
now replaces the affected legacy trigger without changing schema versions
or governed and visibility data.
- Kept post-write authority verification failures available for read-only
reconciliation while preventing automatic or generic confirmed retries from
invoking the model again. Automatic reconciliation skips those sticky rows
before applying its batch limit, and linked durable jobs exclusively own
retry scheduling for their memory tasks.
- Removed MemHub's task-scoped HyPermission store wrappers and global
model-content consent switch. The trusted embedded Hy Memory SDK now uses
its native local stores directly.
- Existing installations ignore the retired config flag, readmit previously
parked pre-invocation work, and drop only the obsolete
`hy_node_permissions` proof table. Evidence, governed memories, model
profiles, SDK stores, and source/host grants are preserved.
## Hy Memory 2.0.5 — 2026-10-22
- Published
[`hy-memory==2.0.5`](https://pypi.org/project/hy-memory/2.0.5/).
- Extended `get()` across Chat, Coding Experience, Working Fact, and raw IDs;
the returned `layer`, `kind`, and `variant` identify the actual record.
- Added explicit raw retrieval through the existing `search()` and
`list_memories()` contracts, including user, agent, session, and source
filters.
- Added public embedding-service injection and corrected extraction language
detection for structured OpenAI-style messages.
- Added lifecycle status filters, durable idempotent write outcomes, and
quiesced local data-plane snapshot/restore.
- Kept legacy migration mechanics out of the public Client API; separate
migration-only raw/import methods from development previews were removed.
- Physical delete now reports partial VDB/Graph cleanup as failure.
- Coding search no longer injects the version-evolution diff block by default;
opt in with `MEMORY_CODING_CHAIN_BLOCK_ENABLED=true`.
- Serialized SDK SQLite cache/history connection lifecycle and tightened
Coding reconciliation to its agent/session/workspace authority.
- Clarified persistent caller context through `meta`, while `metadata` remains
legacy processing configuration.
## Hy Memory 2.0.5.dev4 — 2026-10-21
- Development preview used for SDK-authority migration validation; not the
final `2.0.5` release.
- Published as
[`hy-memory==2.0.5.dev4`](https://pypi.org/project/hy-memory/2.0.5.dev4/).
## Hy Memory 2.0.5.dev3 — 2026-10-21
- Development preview; not the final `2.0.5` release.
@@ -28,6 +95,12 @@ Hy Memory and MemHub are versioned and released independently.
APIs, offline local data-plane snapshot/restore, and fail-closed physical
delete receipts.
## Hy Memory 2.0.4 — 2026-10-17
- Published
[`hy-memory==2.0.4`](https://pypi.org/project/hy-memory/2.0.4/) as a
maintenance release.
## Hy Memory 2.0.3 — 2026-10-17
- Added an optional runtime HTTP transport factory so embedded products can
@@ -38,66 +111,6 @@ Hy Memory and MemHub are versioned and released independently.
unchanged memory extraction, reconciliation, storage, and Coding/Working
behavior.
## Unreleased
- Removed MemHub's private `hy_memory_bridge` implementation. MemHub now calls
public `HyMemoryClient` methods at runtime; prompt/reconcile/response
monkey-patches and private VDB/Coding runtime access were deleted. Kuzu Graph
projection/admin and stopped-writer App migration are the documented product
exceptions.
- Extended public SDK `get()` to resolve Chat, Coding Experience, and Working
Fact IDs, including raw records, owner/session fields, and explicit ambiguity
handling; added constructor-time public embedding-service injection.
- Added a MemHub-managed native Hermes MemoryProvider with automatic prefetch,
persistent per-session memory-ID deduplication, authorized Broker reads, and
recall-context stripping to prevent memory recursion.
- Added an extensible MemHub interface-localization layer with Simplified
Chinese, English, and Spanish catalogs shared by Web and Electron-native
surfaces. Interface language remains independent from memory content and
model processing language.
- `search(..., layers=["l1_raw"])` now explicitly searches SDK raw records in
pro/ultra mode and keeps existing user/agent/session/source filters; default
search continues to hide raw.
- Added SDK-authoritative lifecycle listing with explicit status filters,
idempotent durable write outcomes, and quiesced local data-plane
snapshot/restore contracts for the MemHub authority migration. Exact legacy
backfill is now an App-owned privileged offline operation rather than a
public `HyMemoryClient` API. Backfill runs against a complete staging copy;
cutover atomically swaps SDK roots and restores the previous root on failure.
- Physical delete now reports incomplete VDB/Graph cleanup as a partial failure
instead of claiming success after only one store was changed.
- Coding search no longer injects the version-evolution diff block
(`chain_block`) by default. Enable with
`MEMORY_CODING_CHAIN_BLOCK_ENABLED=true` or
`CodingConfig.chain_block_enabled=True` (write-side observe unchanged).
- Fixed governed-memory revisions and repeated evidence links failing when the
visibility dirty-marker trigger encounters the same memory twice. Startup
now replaces the affected legacy trigger without changing schema versions
or governed and visibility data.
- Serialized each SDK SQLite store's shared connection lifecycle so concurrent
history/cache operations cannot race with `close()` and crash the process.
- Scoped Coding reconciliation targets to the current HY authority,
conversation, and workspace, preventing near-duplicate sessions from
mutating an experience or working fact owned by another evidence context.
- Kept post-write authority verification failures available for read-only
reconciliation while preventing automatic or generic confirmed retries from
invoking the model again. Automatic reconciliation skips those sticky rows
before applying its batch limit, and linked durable jobs exclusively own
retry scheduling for their memory tasks.
- Clarified that `add(metadata=...)` carries legacy SDK processing options,
while persistent caller business context belongs in `meta`.
- Unified exact reads and lists across memory layers: `get(memory_id)` returns
the matched layer, while `list_memories(layers=["l1_raw"])` explicitly lists
raw records. Separate raw get/list methods and the `include_raw` switch were
removed.
- Removed MemHub's task-scoped HyPermission store wrappers and global
model-content consent switch. The trusted embedded Hy Memory SDK now uses
its native local stores directly.
- Existing installations ignore the retired config flag, readmit previously
parked pre-invocation work, and drop only the obsolete
`hy_node_permissions` proof table. Evidence, governed memories, model
profiles, SDK stores, and source/host grants are preserved.
## Hy Memory 2.0.2 — 2026-10-17
- Published
@@ -112,7 +125,17 @@ Hy Memory and MemHub are versioned and released independently.
Because PyPI release metadata is immutable, the next SDK patch release must
carry the updated license metadata; `2.0.1` retains its published metadata.
## MemHub 1.0.3 — Unreleased
## MemHub 1.0.5 — Unreleased
- Updated the source version to `1.0.5` (build 14).
- Pinned the public SDK to the stable `hy-memory==2.0.5` release.
## MemHub 1.0.4 — 2026-10-18
- Updated the source version to `1.0.4` (build 13).
- Published the `v1.0.4` source and release metadata.
## MemHub 1.0.3 — 2026-10-17
- Updated the source version to `1.0.3` (build 7).
- Pinned the public SDK to `hy-memory==2.0.3`.
+2
View File
@@ -13,6 +13,8 @@
接口先补 SDK。Runtime writers 停止后的 App 启动迁移可直接操作底层存储,
但不得把迁移专用方法加入公共 Client。
3. MemHub **不存 memory 实体**;**可以存 `memory_id`** 做组织/回执。原文进入 **SDK raw**。
4. SDK raw 只保留滚动三天;一条 raw 对应一个完整 turn,只含真人 user 与
assistant,tool 不持久化。提取后的高层 memory 不受该窗口影响。
生产入口通过共享 Runtime 租约取得公开 `HyMemoryClient`;MemHub 只在调用前执行
模型配置、密钥和 harness/model 门禁,不维护第二套 memory 正文权威。
+5 -5
View File
@@ -18,13 +18,13 @@ If you run into a problem, please open an Issue with the version, platform,repro
## 发行与平台合同
源码当前固定使用公开 PyPI 开发预览 `hy-memory==2.0.5.dev3`。MemHub 源码版本为 `1.0.4`
(build 13),GitHub 仓库为
源码当前固定使用公开 PyPI 正式版 `hy-memory==2.0.5`。MemHub 源码版本为 `1.0.5`
(build 14),GitHub 仓库为
[`Tencent-Hunyuan/Hy-Memory`](https://github.com/Tencent-Hunyuan/Hy-Memory),npm 包名为
`memhub-cli`。当前代码架构见
[`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md)。
`v1.0.4` 的发行目标是 Linux x64/arm64 Runtime 与 macOS arm64/x64 App 和 Runtime。
`v1.0.5` 的发行目标是 Linux x64/arm64 Runtime 与 macOS arm64/x64 App 和 Runtime。
Linux 制品由 GitHub Actions 在原生架构 runner 上构建并完成生命周期验收;macOS
制品由发行流水线完成 Developer ID 签名、公证和验收后手动上传。Windows 不支持。
只有同时出现在 GitHub Release 与 `release-manifest.json` 中、且通过公开校验的制品才属于
@@ -45,7 +45,7 @@ Linux 制品由 GitHub Actions 在原生架构 runner 上构建并完成生命
## 仓库结构
- `apps/core`:Python / FastAPI Kernel,负责授权、模型、队列、SDK bridge 与 API。
- `apps/core`:Python / FastAPI Kernel,负责授权、模型、队列、SDK Runtime 边界与 API。
- `apps/cli`:安装、控制和诊断共享 Runtime 的 `one-memory` CLI。
- `apps/web`:React / TypeScript 管理界面。
- `apps/desktop`:Electron 桌面壳与图形安装入口。
@@ -117,7 +117,7 @@ MemHub 的 SQLite 数据默认位于 `~/.one-memory`,不在 npm 包或 Runtime
## Hy Memory
SDK 依赖固定为公开 PyPI 开发预览 `hy-memory==2.0.5.dev3`。SDK 配置、权限边界及 search
SDK 依赖固定为公开 PyPI 正式版 `hy-memory==2.0.5`。SDK 配置、权限边界及 search
返回格式见 [`MEMORY.md`](MEMORY.md)。HY search 返回的 `content` 已包含 SDK 统一生成的
逻辑时间和演化上下文;调用方不应再次转换时间戳或重复拼接 `When`。
+3 -3
View File
@@ -3,9 +3,9 @@
"dataLicense": "CC0-1.0",
"SPDXID": "SPDXRef-DOCUMENT",
"name": "one-memory",
"documentNamespace": "https://one-memory.local/spdx/2026-10-22T08:09:31Z",
"documentNamespace": "https://one-memory.local/spdx/2026-10-22T13:18:16Z",
"creationInfo": {
"created": "2026-10-22T08:09:31Z",
"created": "2026-10-22T13:18:16Z",
"creators": [
"Tool: one-memory-generate-sbom"
]
@@ -2352,7 +2352,7 @@
{
"SPDXID": "SPDXRef-Package-hy-memory",
"name": "hy-memory",
"versionInfo": "2.0.5.dev3",
"versionInfo": "2.0.5",
"downloadLocation": "NOASSERTION",
"filesAnalyzed": false,
"licenseConcluded": "Apache-2.0",
+2 -2
View File
@@ -2,14 +2,14 @@
MemHub itself is licensed under Apache-2.0 (see `LICENSE`).
It bundles the runtime dependencies listed below.
Generated from the installed dependency tree on 2026-10-22T08:09:31Z
Generated from the installed dependency tree on 2026-10-22T13:18:16Z
by `python3 scripts/generate_sbom.py`. Do not edit by hand.
Per-package license texts are not reproduced here yet; the license
column is filled in as part of release review, and the SBOM records
`NOASSERTION` until then rather than guessing.
## hy-memory 2.0.5.dev3
## hy-memory 2.0.5
License: Apache-2.0
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@one-memory/cli",
"version": "1.0.4",
"version": "1.0.5",
"description": "MemHub bootstrap CLI for installing and managing the local-first shared memory runtime",
"license": "Apache-2.0",
"repository": {
+2 -2
View File
@@ -4,7 +4,7 @@ build-backend = "uv_build"
[project]
name = "one-memory"
version = "1.0.4"
version = "1.0.5"
description = "MemHub local-first memory core"
readme = "README.md"
requires-python = ">=3.12"
@@ -12,7 +12,7 @@ dependencies = [
"fastapi>=0.141.1",
"filelock>=3.32.4",
# Public SDK version validated against MemHub's permission and recall contracts.
"hy-memory==2.0.5.dev3",
"hy-memory==2.0.5",
# Retrieval's derived vector index directly consumes these locked versions.
"chromadb==1.5.9",
"numpy==2.5.2",
+2
View File
@@ -1151,6 +1151,8 @@ def create_app(
extraction_settings=app.state.extraction_settings_store,
data_dir=runtime_settings.data_dir,
)
if app.state.memory_authority is MemoryAuthority.SDK:
app.state.sdk_memory_runtime.start_raw_retention()
# C2 owns the single production budget (handle 5s + reserve 250ms,
# total 5.25s, lease 10s, renew 1s, claim 250ms). The composition
# root only consumes it: no local formula, and the 250ms claim
+20 -1
View File
@@ -744,11 +744,30 @@ def sdk_migration_status(options: Sequence[str], stream: TextIO) -> int:
else 0
),
}
active_migration = (
database.execute(
"SELECT migration_id,backup_bundle_id "
"FROM sdk_data_plane_migrations "
"WHERE state IN ('running','blocked','ready') "
"ORDER BY created_at DESC LIMIT 1"
).fetchone()
if "sdk_data_plane_migrations" in tables
else None
)
document = {
"schema_version": 1,
"authority": "legacy",
"state": "required" if any(counts.values()) else "empty",
"migration_id": None,
"migration_id": (
str(active_migration["migration_id"])
if active_migration is not None
else None
),
"bundle_id": (
str(active_migration["backup_bundle_id"])
if active_migration is not None
else None
),
"counts": counts,
"generation": generation,
}
@@ -14,7 +14,7 @@ from pathlib import Path
from typing import Literal, cast
HY_MEMORY_DISTRIBUTION = "hy-memory"
HY_MEMORY_VERSION = "2.0.5.dev3"
HY_MEMORY_VERSION = "2.0.5"
HY_MEMORY_NAMESPACE = "governed-v2"
DiagnosticLogLevel = Literal["error", "warning", "info", "debug"]
@@ -58,6 +58,16 @@ class MigrationClient(Protocol):
def get_write_outcome(self, request_id: str) -> Mapping[str, object] | None: ...
def get(
self,
memory_id: str,
*,
kind: str | None = None,
variant: str | None = None,
) -> Mapping[str, object] | None: ...
async def _coding_store_for_exact_get(self, variant: str) -> Any: ...
class HySdkMigration:
"""Typed offline adapter over one initialized SDK data plane."""
@@ -91,6 +101,138 @@ class HySdkMigration:
)
return _memory_page(raw, limit=limit, offset=offset)
async def list_productivity_memories(self) -> tuple[dict[str, object], ...]:
raw = await _sdk_call(
"hy_sdk_productivity_inventory_failed",
self._loop_thread.run,
self._list_productivity_memories(),
)
return tuple(raw)
async def _list_productivity_memories(self) -> list[dict[str, object]]:
records: list[dict[str, object]] = []
for variant in ("working", "coding"):
store = await self.client._coding_store_for_exact_get(variant)
if store is None:
continue
with store._lock:
experiences = store._conn.execute(
"SELECT experience_id,user_id,agent_id,solution "
"FROM coding_memory_meta ORDER BY experience_id"
).fetchall()
facts = store._conn.execute(
"SELECT fact_id,user_id,agent_id,statement,status "
"FROM working_fact ORDER BY fact_id"
).fetchall()
records.extend(
{
"memory_id": str(row["experience_id"]),
"content": str(row["solution"]),
"status": "active",
"user_id": str(row["user_id"]),
"agent_id": str(row["agent_id"]),
"kind": "experience",
"variant": variant,
}
for row in experiences
)
records.extend(
{
"memory_id": str(row["fact_id"]),
"content": str(row["statement"]),
"status": str(row["status"]),
"user_id": str(row["user_id"]),
"agent_id": str(row["agent_id"]),
"kind": "fact",
"variant": variant,
}
for row in facts
)
return records
async def read_productivity_memory(
self,
memory_id: str,
*,
kind: str,
variant: str | None,
) -> tuple[HySdkMemoryRecord, str] | None:
raw = await _sdk_call(
"hy_sdk_productivity_get_failed",
self.client.get,
memory_id,
kind=kind,
variant=variant,
)
if raw is None:
return None
if not isinstance(raw, Mapping) or raw.get("variant") not in {
"working",
"coding",
}:
raise HySdkMigrationError("hy_sdk_productivity_variant_invalid")
return HySdkMemoryRecord.from_sdk(raw), str(raw["variant"])
async def delete_productivity_memory(
self,
memory_id: str,
*,
kind: str,
variant: str | None,
user_id: str,
) -> dict[str, object]:
raw = await _sdk_call(
"hy_sdk_productivity_delete_failed",
self._loop_thread.run,
self._delete_productivity_memory(
memory_id,
kind=kind,
variant=variant,
user_id=user_id,
),
)
return _successful_result(raw, "hy_sdk_productivity_delete_failed")
async def _delete_productivity_memory(
self,
memory_id: str,
*,
kind: str,
variant: str | None,
user_id: str,
) -> dict[str, object]:
if kind not in {"experience", "fact"}:
raise ValueError("productivity kind must be experience or fact")
matches: list[tuple[Any, str]] = []
for candidate in ((variant,) if variant is not None else ("working", "coding")):
store = await self.client._coding_store_for_exact_get(candidate)
if store is None:
continue
existing = (
await store.get_by_id(memory_id, user_id=user_id)
if kind == "experience"
else await store.get_fact(memory_id, user_id=user_id)
)
if existing is not None:
matches.append((store, candidate))
if len(matches) > 1:
raise ValueError("productivity memory id is ambiguous across variants")
if not matches:
return {"success": True, "deleted_count": 0}
store, _actual_variant = matches[0]
if kind == "experience":
deleted = await store.delete(memory_id, user_id=user_id)
elif kind == "fact":
await store._fact_vec.delete(memory_id)
with store._lock:
store._conn.execute(
"DELETE FROM working_fact WHERE fact_id=? AND user_id=?",
(memory_id, user_id),
)
store._conn.commit()
deleted = True
return {"success": True, "deleted_count": int(deleted)}
async def write_legacy_memory(
self,
*,
@@ -267,7 +409,7 @@ class HySdkMigration:
nodes = await self._vector_store.list_by_user(
user_id=user_id,
agent_id=agent_id,
status_filter=[MemoryStatus.ACTIVE],
status_filter=list(MemoryStatus),
layers=[MemoryLayer.L1_RAW],
include_sources=[source] if source else None,
limit=100_000,
@@ -318,6 +460,21 @@ class HySdkMigration:
)
return _successful_result(raw, "hy_sdk_raw_import_failed")
async def delete_raw(self, raw_id: str) -> dict[str, object]:
raw = await _sdk_call(
"hy_sdk_raw_delete_failed",
self._loop_thread.run,
self._delete_raw(raw_id),
)
return _successful_result(raw, "hy_sdk_raw_delete_failed")
async def _delete_raw(self, raw_id: str) -> dict[str, object]:
existing = await self._vector_store.get_by_id(raw_id)
if existing is None or existing.layer is not MemoryLayer.L1_RAW:
return {"success": True, "deleted_count": 0}
deleted = await self._vector_store.delete(raw_id)
return {"success": True, "deleted_count": int(bool(deleted))}
async def _write_legacy_raw(
self,
*,
@@ -1,6 +1,7 @@
"""Write visible hook text into Evidence. Tool output stays out."""
import json
import os
import time
from datetime import UTC, datetime
from enum import StrEnum
@@ -74,6 +75,7 @@ _FILE_FALLBACK = {
}
_FALLBACK_EVENTS = frozenset({"Stop", "stop", "SubagentStop", "StopFailure"})
_TEXT_BLOCKS = frozenset({"text"})
SDK_MIGRATION_CAPTURE_FENCE = ".sdk-migration-capture-fence.json"
def database_path() -> Path:
@@ -88,17 +90,42 @@ def capture_stdio(source_id: str, stdin: TextIO, output: TextIO) -> int:
del output
if source_id not in CAPTURE_SOURCES:
return 2
database = database_path()
if sdk_migration_capture_fenced(database.parent):
return 0
try:
payload = json.loads(stdin.read() or "")
except json.JSONDecodeError:
return 0
try:
capture_hook(source_id, payload, database_path())
capture_hook(source_id, payload, database)
except Exception:
return 0
return 0
def sdk_migration_capture_fenced(data_dir: Path) -> bool:
"""Ignore stale fences; only a live Desktop migration pauses hook writes."""
marker = data_dir / SDK_MIGRATION_CAPTURE_FENCE
try:
document = json.loads(marker.read_text(encoding="utf-8"))
pid = int(document["pid"])
if pid < 1:
raise ValueError("invalid migration fence pid")
os.kill(pid, 0)
return True
except FileNotFoundError:
return False
except ProcessLookupError:
marker.unlink(missing_ok=True)
return False
except PermissionError:
return True
except (KeyError, TypeError, ValueError, json.JSONDecodeError, OSError):
marker.unlink(missing_ok=True)
return False
def capture_hook(source_id: str, payload: object, database: Path) -> bool:
if source_id not in CAPTURE_SOURCES or not isinstance(payload, dict):
return False
@@ -3,6 +3,8 @@
from __future__ import annotations
import asyncio
import logging
from contextlib import suppress
from datetime import UTC, datetime, timedelta
from pathlib import Path
from typing import Any
@@ -25,6 +27,18 @@ from one_memory.memory_server.durable_queue import COMPLETED, DurableWriteQueue
from one_memory.sdk_write_processor import SdkWriteCoordinator
from one_memory.semantic_deadline import SemanticMemoryHit
logger = logging.getLogger(__name__)
RAW_RETENTION_DAYS = 3
RAW_RETENTION_CHECK_SECONDS = 6 * 60 * 60
_RAW_STATUSES = (
"active",
"superseded",
"shadow",
"negated",
"conflicted",
"archived",
)
class SdkMemoryRuntime:
"""Product services that remain after legacy task/governance writes retire."""
@@ -43,6 +57,68 @@ class SdkMemoryRuntime:
self.write_queue = write_queue
self.extraction_settings = extraction_settings
self.data_dir = data_dir
self._raw_retention_task: asyncio.Task[None] | None = None
def start_raw_retention(self) -> None:
if self._raw_retention_task is None or self._raw_retention_task.done():
self._raw_retention_task = asyncio.create_task(
self._raw_retention_loop(),
name="one-memory-sdk-raw-retention",
)
async def prune_expired_raw(self, *, now: datetime | None = None) -> int:
cutoff = (now or datetime.now(UTC)) - timedelta(days=RAW_RETENTION_DAYS)
cutoff_timestamp = int(cutoff.timestamp())
client = await self.read_handle.client()
deleted = 0
while True:
page = await asyncio.to_thread(
client.list_memories,
user_id="local",
layers=["l1_raw"],
statuses=list(_RAW_STATUSES),
order="asc",
limit=500,
offset=0,
)
bucket = page.get("vdb") if isinstance(page, dict) else None
rows = bucket.get("memories") if isinstance(bucket, dict) else None
if not isinstance(rows, list) or not rows:
return deleted
expired = [
row
for row in rows
if isinstance(row, dict)
and isinstance(row.get("memory_id"), str)
and int(row.get("memory_at") or row.get("gmt_created") or 0)
< cutoff_timestamp
]
if not expired:
return deleted
removed_page = 0
for row in expired:
result = await asyncio.to_thread(client.delete, row["memory_id"])
if isinstance(result, dict) and result.get("success") is True:
count = int(result.get("deleted_count") or 0)
deleted += count
removed_page += count
if removed_page == 0:
return deleted
async def _raw_retention_loop(self) -> None:
while True:
await asyncio.sleep(RAW_RETENTION_CHECK_SECONDS)
try:
removed = await self.prune_expired_raw()
if removed:
logger.info("sdk_raw_retention_deleted count=%d", removed)
except asyncio.CancelledError:
raise
except Exception as error:
logger.warning(
"sdk_raw_retention_failed error_type=%s",
type(error).__name__,
)
async def semantic_recall(
self,
@@ -279,7 +355,12 @@ class SdkMemoryRuntime:
return None
async def shutdown(self) -> None:
return None
if self._raw_retention_task is None:
return
self._raw_retention_task.cancel()
with suppress(asyncio.CancelledError):
await self._raw_retention_task
self._raw_retention_task = None
def _agent_ids(raw: object) -> tuple[str, ...]:
@@ -7,12 +7,13 @@ import json
import sqlite3
from collections.abc import Awaitable, Mapping
from dataclasses import dataclass
from datetime import datetime
from datetime import UTC, datetime
from pathlib import Path
from typing import Protocol
from one_memory.database_access import main_database
from one_memory.hy_sdk_types import HySdkMemoryPage, HySdkRawPage
from one_memory.hy_sdk_types import HySdkMemoryPage, HySdkMemoryRecord, HySdkRawPage
from one_memory.input_sanitization import sanitize_model_input
from one_memory.sdk_data_plane_migration import SdkDataPlaneMigrationStore
@@ -35,6 +36,8 @@ class LegacyMemoryInventory:
tags: tuple[str, ...]
supersedes: tuple[str, ...]
superseded_by: tuple[str, ...]
productivity_kind: str | None = None
productivity_variant: str | None = None
@dataclass(frozen=True)
@@ -44,6 +47,8 @@ class SdkMemoryInventory:
agent_id: str
content_hash: str
status: str
productivity_kind: str | None = None
productivity_variant: str | None = None
@dataclass(frozen=True)
@@ -92,6 +97,17 @@ _KIND_LAYER = {
"schema": "l6_schema",
"intention": "l7_intention",
}
_PRODUCTIVITY_LAYER = {
"coding_experience": ("experience", "coding"),
"working_experience": ("experience", "working"),
"coding_fact": ("fact", "coding"),
# Historical receipts called every productivity fact "working_fact",
# including facts stored beside a coding-variant experience. Resolve the
# actual variant from the SDK exact lookup instead of trusting this label.
"working_fact": ("fact", None),
}
_RAW_TURN_MAX_CHARACTERS = 60_000
_RAW_USER_MAX_CHARACTERS = 30_000
SDK_LIFECYCLE_STATUSES = (
"active",
@@ -161,6 +177,29 @@ class SdkMigrationClient(Protocol):
def delete(self, memory_id: str) -> Awaitable[dict[str, object]]: ...
def read_productivity_memory(
self,
memory_id: str,
*,
kind: str,
variant: str | None,
) -> Awaitable[tuple[HySdkMemoryRecord, str] | None]: ...
def list_productivity_memories(
self,
) -> Awaitable[tuple[dict[str, object], ...]]: ...
def delete_productivity_memory(
self,
memory_id: str,
*,
kind: str,
variant: str | None,
user_id: str,
) -> Awaitable[dict[str, object]]: ...
def delete_raw(self, raw_id: str) -> Awaitable[dict[str, object]]: ...
def read_legacy_memory_inventory(database_path: Path) -> tuple[LegacyMemoryInventory, ...]:
"""Read legacy memory intent without constructing a Governance writer."""
@@ -204,6 +243,7 @@ def read_legacy_memory_inventory(database_path: Path) -> tuple[LegacyMemoryInven
tags = tuple(value for value in tags_value if isinstance(value, str))
kind = str(row["kind"])
status = str(row["status"])
productivity = _PRODUCTIVITY_LAYER.get(mapped_layer or "")
records.append(
LegacyMemoryInventory(
legacy_memory_id=legacy_id,
@@ -225,6 +265,8 @@ def read_legacy_memory_inventory(database_path: Path) -> tuple[LegacyMemoryInven
tags=tags,
supersedes=tuple(supersedes.get(legacy_id, ())),
superseded_by=tuple(superseded_by.get(legacy_id, ())),
productivity_kind=productivity[0] if productivity else None,
productivity_variant=productivity[1] if productivity else None,
)
)
return tuple(records)
@@ -239,43 +281,116 @@ def _parse_datetime(value: object) -> datetime | None:
return None
def read_legacy_raw_inventory(database_path: Path) -> tuple[LegacyRawInventory, ...]:
def read_legacy_raw_inventory(
database_path: Path,
*,
cutoff: datetime,
) -> tuple[LegacyRawInventory, ...]:
"""Rebuild recent legacy messages into one user/assistant raw per turn."""
if cutoff.tzinfo is None:
cutoff = cutoff.replace(tzinfo=UTC)
with main_database(database_path).read(row_factory=sqlite3.Row) as connection:
rows = connection.execute(
"""
SELECT evidence_id,user_id,source_id,workspace,project,
conversation_id,message_id,source_ordinal,content_kind,
role,written_by,created_at,imported_at,verbatim,content_hash,revision
FROM evidence ORDER BY COALESCE(ingest_seq,rowid),evidence_id
FROM evidence
WHERE content_kind='raw_conversation'
AND (
(role='user' AND written_by='person')
OR role='assistant'
)
ORDER BY source_id,conversation_id,workspace,
COALESCE(created_at,imported_at),
COALESCE(ingest_seq,rowid),evidence_id
"""
).fetchall()
return tuple(
LegacyRawInventory(
evidence_id=str(row["evidence_id"]),
user_id=str(row["user_id"]),
agent_id="one-memory",
session_id=str(row["conversation_id"]),
source=str(row["source_id"]),
content=str(row["verbatim"]),
content_hash=str(row["content_hash"]),
memory_at=_parse_datetime(row["created_at"] or row["imported_at"]),
metadata={
key: row[key]
for key in (
"workspace",
"project",
"message_id",
"source_ordinal",
"content_kind",
"role",
"written_by",
"revision",
)
if row[key] is not None
},
turns: list[LegacyRawInventory] = []
current_key: tuple[str, str, str, str] | None = None
current: list[sqlite3.Row] = []
def finish() -> None:
if not current or str(current[0]["role"]) != "user":
return
assistants = [row for row in current[1:] if str(row["role"]) == "assistant"]
if not assistants:
return
rows_for_turn = [current[0], *assistants]
completed_at = _parse_datetime(
assistants[-1]["created_at"] or assistants[-1]["imported_at"]
)
for row in rows
)
if completed_at is not None and completed_at.tzinfo is None:
completed_at = completed_at.replace(tzinfo=UTC)
if completed_at is None or completed_at.astimezone(UTC) < cutoff.astimezone(UTC):
return
evidence_ids = [str(row["evidence_id"]) for row in rows_for_turn]
source = str(current[0]["source_id"])
session_id = str(current[0]["conversation_id"])
digest = hashlib.sha256(
"\0".join((source, session_id, *evidence_ids)).encode()
).hexdigest()
raw_user_text = str(rows_for_turn[0]["verbatim"] or "")
user_text = sanitize_model_input(raw_user_text)[:_RAW_USER_MAX_CHARACTERS]
content_parts = [f"[user]: {user_text}"]
truncated = len(raw_user_text) > len(user_text)
for row in assistants:
sanitized = sanitize_model_input(str(row["verbatim"] or ""))
text = sanitized
prefix = "[assistant]: "
remaining = _RAW_TURN_MAX_CHARACTERS - len("\n".join(content_parts)) - 1
if remaining <= len(prefix):
truncated = True
break
available = remaining - len(prefix)
if len(sanitized) > available:
text = text[:available]
truncated = True
content_parts.append(f"{prefix}{text}")
if len(text) < len(sanitized):
break
content = "\n".join(content_parts)
if not content:
return
turns.append(
LegacyRawInventory(
evidence_id=f"turn:{digest}",
user_id=str(current[0]["user_id"]),
agent_id="all-apps",
session_id=session_id,
source=source,
content=content,
content_hash=content_hash(content),
memory_at=_parse_datetime(
current[0]["created_at"] or current[0]["imported_at"]
),
metadata={
"evidence_ids": evidence_ids,
"workspace": current[0]["workspace"],
"project": current[0]["project"],
"input_kind": "legacy_turn",
"truncated": truncated,
},
)
)
for row in rows:
key = (
str(row["source_id"]),
str(row["conversation_id"]),
str(row["user_id"]),
str(row["workspace"] or ""),
)
if key != current_key:
finish()
current_key, current = key, []
if str(row["role"]) == "user":
finish()
current = [row]
elif current:
current.append(row)
finish()
return tuple(turns)
def classify_memory_inventory(
@@ -283,13 +398,39 @@ def classify_memory_inventory(
sdk: tuple[SdkMemoryInventory, ...],
) -> tuple[MemoryMigrationDecision, ...]:
"""Classify every legacy and SDK memory; no item may silently disappear."""
sdk_by_id = {item.memory_id: item for item in sdk}
claimed_sdk_ids: set[str] = set()
sdk_by_id: dict[str, list[SdkMemoryInventory]] = {}
for sdk_item in sdk:
sdk_by_id.setdefault(sdk_item.memory_id, []).append(sdk_item)
claimed_sdk_items: set[tuple[str, str | None, str | None]] = set()
decisions: list[MemoryMigrationDecision] = []
for item in legacy:
current = sdk_by_id.get(item.canonical_memory_id)
candidates = [
candidate
for candidate in sdk_by_id.get(item.canonical_memory_id, ())
if (
candidate.productivity_kind == item.productivity_kind
and (
item.productivity_kind is None
or item.productivity_variant is None
or candidate.productivity_variant == item.productivity_variant
)
)
]
if len(candidates) > 1:
claimed_sdk_items.update(_sdk_inventory_key(candidate) for candidate in candidates)
decisions.append(
MemoryMigrationDecision(
item_key=f"memory:{item.legacy_memory_id}",
action="quarantine",
legacy_memory_id=item.legacy_memory_id,
canonical_memory_id=item.canonical_memory_id,
reason="sdk_memory_id_ambiguous",
)
)
continue
current = candidates[0] if candidates else None
if current is not None:
claimed_sdk_ids.add(current.memory_id)
claimed_sdk_items.add(_sdk_inventory_key(current))
if item.status == "deleted":
decisions.append(
MemoryMigrationDecision(
@@ -313,6 +454,17 @@ def classify_memory_inventory(
)
continue
if current is None:
if item.productivity_kind is not None:
decisions.append(
MemoryMigrationDecision(
item_key=f"memory:{item.legacy_memory_id}",
action="quarantine",
legacy_memory_id=item.legacy_memory_id,
canonical_memory_id=item.canonical_memory_id,
reason="mapped_productivity_memory_missing",
)
)
continue
decisions.append(
MemoryMigrationDecision(
item_key=f"memory:{item.legacy_memory_id}",
@@ -323,7 +475,19 @@ def classify_memory_inventory(
)
)
continue
if (
if item.productivity_kind is not None and (
current.user_id == item.user_id
and current.productivity_kind == item.productivity_kind
and (
item.productivity_variant is None
or current.productivity_variant == item.productivity_variant
)
):
action = (
"matched" if item.legacy_memory_id == item.canonical_memory_id else "record_alias"
)
reason = "existing_productivity_identity"
elif (
current.content_hash == item.content_hash
and current.status == item.status
and current.user_id == item.user_id
@@ -346,10 +510,10 @@ def classify_memory_inventory(
)
for sdk_item in sdk:
if sdk_item.memory_id not in claimed_sdk_ids:
if _sdk_inventory_key(sdk_item) not in claimed_sdk_items:
decisions.append(
MemoryMigrationDecision(
item_key=f"sdk:{sdk_item.memory_id}",
item_key=_sdk_item_key(sdk_item),
action="adopt",
legacy_memory_id=None,
canonical_memory_id=sdk_item.memory_id,
@@ -359,13 +523,33 @@ def classify_memory_inventory(
return tuple(decisions)
def _sdk_inventory_key(
item: SdkMemoryInventory,
) -> tuple[str, str | None, str | None]:
return (
item.memory_id,
item.productivity_kind,
item.productivity_variant,
)
def _sdk_item_key(item: SdkMemoryInventory) -> str:
namespace = (
f"{item.productivity_variant}:{item.productivity_kind}"
if item.productivity_kind is not None
else "chat"
)
return f"sdk:{namespace}:{item.memory_id}"
async def read_sdk_memory_inventory(
client: SdkMigrationClient,
*,
legacy: tuple[LegacyMemoryInventory, ...] = (),
user_id: str = "local",
page_size: int = 500,
) -> tuple[SdkMemoryInventory, ...]:
records: dict[str, SdkMemoryInventory] = {}
records: list[SdkMemoryInventory] = []
offset = 0
while True:
page = await client.list_memories(
@@ -376,18 +560,45 @@ async def read_sdk_memory_inventory(
order="asc",
)
for memory in page.memories:
records[memory.memory_id] = SdkMemoryInventory(
memory_id=memory.memory_id,
user_id=memory.user_id,
agent_id=memory.agent_id,
content_hash=content_hash(memory.content),
status=memory.status,
records.append(
SdkMemoryInventory(
memory_id=memory.memory_id,
user_id=memory.user_id,
agent_id=memory.agent_id,
content_hash=content_hash(memory.content),
status=memory.status,
)
)
consumed = len(page.memories)
offset += consumed
if consumed == 0 or offset >= page.total:
break
return tuple(records[key] for key in sorted(records))
del legacy
for memory in await client.list_productivity_memories():
try:
records.append(
SdkMemoryInventory(
memory_id=str(memory["memory_id"]),
user_id=str(memory["user_id"]),
agent_id=str(memory["agent_id"]),
content_hash=content_hash(str(memory["content"])),
status=str(memory["status"]),
productivity_kind=str(memory["kind"]),
productivity_variant=str(memory["variant"]),
)
)
except KeyError as error:
raise RuntimeError("invalid productivity inventory row") from error
return tuple(
sorted(
records,
key=lambda item: (
item.memory_id,
item.productivity_kind or "",
item.productivity_variant or "",
),
)
)
def record_memory_migration_plan(
@@ -399,12 +610,12 @@ def record_memory_migration_plan(
) -> tuple[MemoryMigrationDecision, ...]:
decisions = classify_memory_inventory(legacy, sdk)
legacy_by_id = {item.legacy_memory_id: item for item in legacy}
sdk_by_id = {item.memory_id: item for item in sdk}
sdk_by_item_key = {_sdk_item_key(item): item for item in sdk}
for decision in decisions:
source_hash = (
legacy_by_id[decision.legacy_memory_id].content_hash
if decision.legacy_memory_id is not None
else sdk_by_id[decision.canonical_memory_id].content_hash
else sdk_by_item_key[decision.item_key].content_hash
)
operation_key = hashlib.sha256(
f"{migration_id}:{decision.item_key}:{decision.action}:{source_hash}".encode()
@@ -457,7 +668,15 @@ async def execute_memory_migration_plan(
continue
if decision.action in {"delete", "verified_deleted"}:
if decision.action == "delete":
await client.delete(decision.canonical_memory_id)
if item is not None and item.productivity_kind is not None:
await client.delete_productivity_memory(
decision.canonical_memory_id,
kind=item.productivity_kind,
variant=item.productivity_variant,
user_id=item.user_id,
)
else:
await client.delete(decision.canonical_memory_id)
store.settle_item(
migration_id=migration_id,
item_key=decision.item_key,
@@ -595,10 +814,10 @@ def classify_raw_inventory(
decisions.append(
RawMigrationDecision(
item_key=f"sdk-raw:{raw.raw_id}",
action="adopt",
action="delete",
evidence_id=None,
raw_id=raw.raw_id,
reason="sdk_only",
reason="outside_canonical_three_day_turns",
)
)
return tuple(decisions)
@@ -644,11 +863,12 @@ async def execute_raw_migration_plan(
) -> None:
legacy_by_id = {item.evidence_id: item for item in legacy}
for decision in decisions:
if decision.action == "adopt":
if decision.action == "delete":
await client.delete_raw(decision.raw_id)
store.settle_item(
migration_id=migration_id,
item_key=decision.item_key,
state="adopted",
state="deleted",
target_memory_id=decision.raw_id,
)
continue
@@ -677,8 +897,15 @@ async def execute_raw_migration_plan(
target_memory_id=decision.raw_id,
target_hash=item.content_hash,
)
store.record_raw_alias(
migration_id=migration_id,
legacy_evidence_id=item.evidence_id,
canonical_raw_id=decision.raw_id,
)
evidence_ids = item.metadata.get("evidence_ids")
for evidence_id in (
evidence_ids
if isinstance(evidence_ids, list)
else [item.evidence_id]
):
if isinstance(evidence_id, str) and evidence_id:
store.record_raw_alias(
migration_id=migration_id,
legacy_evidence_id=evidence_id,
canonical_raw_id=decision.raw_id,
)
@@ -9,6 +9,7 @@ import sqlite3
from collections.abc import Callable
from contextlib import suppress
from dataclasses import dataclass
from datetime import UTC, datetime, timedelta
from pathlib import Path
from one_memory.database_access import main_database
@@ -406,14 +407,27 @@ async def run_sdk_data_plane_backfill(
store.set_phase(migration_id, phase="inventory", state="running")
legacy_memories = read_legacy_memory_inventory(database_path)
sdk_memories = await read_sdk_memory_inventory(client)
sdk_memories = await read_sdk_memory_inventory(client, legacy=legacy_memories)
memory_decisions = record_memory_migration_plan(
store,
migration_id=migration_id,
legacy=legacy_memories,
sdk=sdk_memories,
)
legacy_raw = read_legacy_raw_inventory(database_path)
with main_database(database_path).read(row_factory=sqlite3.Row) as database:
started = database.execute(
"SELECT created_at FROM sdk_data_plane_migrations WHERE migration_id=?",
(migration_id,),
).fetchone()
if started is None:
raise DataPlaneMigrationError("sdk_data_plane_migration_start_missing")
started_at = datetime.fromisoformat(str(started["created_at"]))
if started_at.tzinfo is None:
started_at = started_at.replace(tzinfo=UTC)
legacy_raw = read_legacy_raw_inventory(
database_path,
cutoff=started_at.astimezone(UTC) - timedelta(days=3),
)
sdk_raw = await read_sdk_raw_inventory(client)
raw_decisions = record_raw_migration_plan(
store,
+45 -2
View File
@@ -1,10 +1,15 @@
import asyncio
import io
import json
import sqlite3
from types import SimpleNamespace
from one_memory import cli
from one_memory.config import Settings
from one_memory.memory_server.migration_runner import SqliteMigrationStore, migrate
from one_memory.memory_server.migrations import MIGRATIONS
from one_memory.runtime_client import RuntimeStatus
from one_memory.sdk_data_plane_migration import SdkDataPlaneMigrationStore
def test_restore_requires_a_source(tmp_path, monkeypatch) -> None:
@@ -363,7 +368,7 @@ def test_sdk_cutover_is_offline_confirmed_and_returns_marker(tmp_path, monkeypat
lambda **_kwargs: SimpleNamespace(
migration_id="migration-1",
backup_bundle_id="bundle-1",
sdk_version="2.0.5.dev3",
sdk_version="2.0.5",
completed_at="2026-10-21T00:00:00Z",
),
)
@@ -393,7 +398,7 @@ def test_sdk_cutover_is_offline_confirmed_and_returns_marker(tmp_path, monkeypat
assert code == 0
document = output.getvalue().replace(" ", "")
assert '"state":"cutover"' in document
assert '"sdk_version":"2.0.5.dev3"' in document
assert '"sdk_version":"2.0.5"' in document
def test_sdk_cutover_never_runs_while_runtime_is_live(tmp_path, monkeypatch) -> None:
@@ -436,3 +441,41 @@ def test_sdk_migration_status_distinguishes_empty_and_legacy_data(tmp_path, monk
assert cli.run(["sdk-migration-status", "--json"], output=required) == 0
assert '"state": "required"' in required.getvalue()
assert '"raw": 1' in required.getvalue()
def test_sdk_migration_status_exposes_a_resumable_active_identity(tmp_path, monkeypatch) -> None:
monkeypatch.setattr(
cli,
"load_settings",
lambda: Settings(_env_file=None, data_dir=tmp_path),
)
database = tmp_path / "one-memory.sqlite3"
asyncio.run(migrate(SqliteMigrationStore(database), MIGRATIONS))
with sqlite3.connect(database) as connection:
connection.execute(
"INSERT INTO evidence ("
"evidence_id,idempotency_key,user_id,source_id,conversation_id,"
"content_kind,role,written_by,verbatim,content_hash,revision,"
"imported_at,updated_at"
") VALUES ('e1','k','local','cursor','s','raw_conversation',"
"'user','person','hello','hash',1,'now','now')"
)
SdkDataPlaneMigrationStore(database).start(
migration_id="sdk-authority-v2-0123456789abcdef",
source_runtime_build="17",
target_sdk_version="2.0.5",
source_schema_version=MIGRATIONS[-1].version,
namespace_fingerprint="namespace",
backend_fingerprint="backend",
backup_bundle_id=(
"sdk-authority-v2-0123456789abcdef-pre-cutover"
),
)
output = io.StringIO()
assert cli.run(["sdk-migration-status", "--json"], output=output) == 0
document = json.loads(output.getvalue())
assert document["migration_id"] == "sdk-authority-v2-0123456789abcdef"
assert document["bundle_id"] == (
"sdk-authority-v2-0123456789abcdef-pre-cutover"
)
+30 -4
View File
@@ -2,6 +2,7 @@
import asyncio
import json
import os
import sqlite3
from contextlib import closing
from io import StringIO
@@ -20,6 +21,7 @@ from one_memory.memory_server.durable_queue import (
DurableWriteQueue,
execute_job,
)
from one_memory.sdk_write_processor import SdkWriteReceipt
def _grant(tmp_path, source_id: str, *, sync: bool) -> EvidenceStore:
@@ -233,14 +235,14 @@ def test_hook_job_completes_after_file_identity_adoption(tmp_path) -> None:
).fetchone()
class IdentityCheckingSupervisor:
async def process_records(
async def process_sdk_job(
self,
records,
*,
request_id,
before_hy_invocation,
source_admitted,
):
assert source_admitted is True
assert request_id
with closing(sqlite3.connect(database)) as connection:
connection.execute("BEGIN")
for record in records:
@@ -249,7 +251,7 @@ def test_hook_job_completes_after_file_identity_adoption(tmp_path) -> None:
assert ingest_journal.matches_current(connection, current)
connection.rollback()
await before_hy_invocation()
return ()
return SdkWriteReceipt()
drain = DurableWriteDrain(queue, execute=lambda _job: None)
assert drain.claim(limit=1) == (job_id,)
@@ -560,6 +562,30 @@ def test_capture_failure_stays_silent(tmp_path, monkeypatch) -> None:
assert output.getvalue() == ""
def test_capture_is_a_silent_noop_during_sdk_migration(tmp_path, monkeypatch) -> None:
database = tmp_path / "one-memory.sqlite3"
marker = tmp_path / ".sdk-migration-capture-fence.json"
marker.write_text(json.dumps({"schema_version": 1, "pid": os.getpid()}))
monkeypatch.setattr("one_memory.live_capture.database_path", lambda: database)
monkeypatch.setattr(
"sys.stdin",
StringIO(
json.dumps(
{
"hook_event_name": "beforeSubmitPrompt",
"conversation_id": "session",
"prompt": "not persisted during migration",
}
)
),
)
output = StringIO()
assert cli.run(["capture", "cursor"], output=output) == 0
assert output.getvalue() == ""
assert not database.exists()
def test_codex_harness_preamble_is_not_a_user_message(tmp_path) -> None:
database = tmp_path / "one-memory.sqlite3"
_grant(tmp_path, "codex", sync=True)
@@ -2,6 +2,10 @@ from __future__ import annotations
import asyncio
import json
import sqlite3
import threading
from datetime import UTC, datetime, timedelta
from pathlib import Path
import pytest
from hy_memory.models.memory import MemoryLayer, MemoryStatus
@@ -23,6 +27,7 @@ from one_memory.sdk_data_plane_migration import (
MemoryAuthority,
SdkDataPlaneMigrationStore,
)
from one_memory.sdk_memory_runtime import SdkMemoryRuntime
from one_memory.sdk_migration_bundle import create_sdk_migration_bundle
from one_memory.sdk_migration_inventory import (
LegacyMemoryInventory,
@@ -219,7 +224,7 @@ def test_startup_cutover_freezes_pending_job_input_and_preserves_the_queue(store
database_path=store.database_path,
migration_id="m1",
bundle_id="bundle-1",
sdk_version="2.0.5.dev3",
sdk_version="2.0.5",
)
assert marker.migration_id == "m1"
assert legacy_tables_present(store.database_path)
@@ -366,6 +371,12 @@ def _legacy(
status="active",
layer="l2_fact",
):
productivity = {
"coding_experience": ("experience", "coding"),
"working_experience": ("experience", "working"),
"coding_fact": ("fact", "coding"),
"working_fact": ("fact", None),
}.get(layer)
return LegacyMemoryInventory(
legacy_memory_id=legacy_id,
canonical_memory_id=canonical_id or legacy_id,
@@ -380,16 +391,27 @@ def _legacy(
tags=(),
supersedes=(),
superseded_by=(),
productivity_kind=productivity[0] if productivity else None,
productivity_variant=productivity[1] if productivity else None,
)
def _sdk(memory_id, *, content="body", status="active"):
def _sdk(
memory_id,
*,
content="body",
status="active",
productivity_kind=None,
productivity_variant=None,
):
return SdkMemoryInventory(
memory_id=memory_id,
user_id="local",
agent_id="one-memory",
content_hash=content_hash(content),
status=status,
productivity_kind=productivity_kind,
productivity_variant=productivity_variant,
)
@@ -420,7 +442,7 @@ def test_inventory_classification_covers_every_legacy_and_sdk_state():
assert by_key["memory:delete-existing"].action == "delete"
assert by_key["memory:divergent"].action == "quarantine"
assert by_key["memory:unknown-layer"].reason == "legacy_layer_unknown"
assert by_key["sdk:sdk-only"].action == "adopt"
assert by_key["sdk:chat:sdk-only"].action == "adopt"
assert len(decisions) == 8
@@ -430,7 +452,10 @@ class _MigrationClient:
self.imported = []
self.imported_raw = []
self.deleted = []
self.deleted_productivity = []
self.deleted_raw = []
self.raw = ()
self.productivity = {}
async def list_memories(self, *, limit, offset, **_kwargs):
page = self.memories[offset : offset + limit]
@@ -449,6 +474,34 @@ class _MigrationClient:
self.deleted.append(memory_id)
return {"success": True, "deleted_count": 1}
async def read_productivity_memory(self, memory_id, *, kind, variant):
if variant is not None:
memory = self.productivity.get((memory_id, kind, variant))
return None if memory is None else (memory, variant)
for candidate in ("working", "coding"):
memory = self.productivity.get((memory_id, kind, candidate))
if memory is not None:
return memory, candidate
return None
async def list_productivity_memories(self):
return tuple(
{
"memory_id": memory.memory_id,
"content": memory.content,
"status": memory.status or "active",
"user_id": memory.user_id,
"agent_id": memory.agent_id,
"kind": kind,
"variant": variant,
}
for (_memory_id, kind, variant), memory in self.productivity.items()
)
async def delete_productivity_memory(self, memory_id, *, kind, variant, user_id):
self.deleted_productivity.append((memory_id, kind, variant, user_id))
return {"success": True, "deleted_count": 1}
async def read_raw_inventory(self, *, limit, offset, **_kwargs):
rows = self.raw[offset : offset + limit]
return HySdkRawPage(
@@ -462,6 +515,10 @@ class _MigrationClient:
self.imported_raw.append(kwargs)
return {"success": True, "raw_id": kwargs["raw_id"]}
async def delete_raw(self, raw_id):
self.deleted_raw.append(raw_id)
return {"success": True, "deleted_count": 1}
async def get_sdk_memory(self, memory_id):
return next(
(memory for memory in self.memories if memory.memory_id == memory_id),
@@ -500,6 +557,84 @@ def test_sdk_inventory_is_paged_without_dropping_lifecycle_records():
]
def test_sdk_inventory_adopts_mapped_productivity_ids_and_deletes_user_removed_fact(store):
active = _legacy("exp", layer="coding_experience")
deleted = _legacy("fact", layer="working_fact", status="deleted")
client = _MigrationClient()
client.productivity = {
("exp", "experience", "coding"): HySdkMemoryRecord(
memory_id="exp",
content="sdk experience",
layer="experience",
status="active",
is_latest=True,
user_id="local",
agent_id="scope",
),
("fact", "fact", "coding"): HySdkMemoryRecord(
memory_id="fact",
content="sdk fact",
layer="fact",
status="active",
is_latest=True,
user_id="local",
agent_id="scope",
),
("sdk-only", "experience", "working"): HySdkMemoryRecord(
memory_id="sdk-only",
content="sdk only",
layer="experience",
status="active",
is_latest=True,
user_id="local",
agent_id="another-scope",
),
}
sdk = asyncio.run(read_sdk_memory_inventory(client, legacy=(active, deleted)))
_start(store)
decisions = record_memory_migration_plan(
store,
migration_id="m1",
legacy=(active, deleted),
sdk=sdk,
)
by_key = {decision.item_key: decision for decision in decisions}
assert by_key["memory:exp"].action == "matched"
assert by_key["memory:fact"].action == "delete"
assert by_key["sdk:working:experience:sdk-only"].action == "adopt"
asyncio.run(
execute_memory_migration_plan(
client,
store,
migration_id="m1",
legacy=(active, deleted),
decisions=decisions,
)
)
assert client.imported == []
assert client.deleted_productivity == [("fact", "fact", None, "local")]
def test_productivity_id_in_both_variants_is_quarantined():
legacy = (_legacy("fact", layer="working_fact"),)
sdk = (
_sdk(
"fact",
productivity_kind="fact",
productivity_variant="working",
),
_sdk(
"fact",
productivity_kind="fact",
productivity_variant="coding",
),
)
(decision, *_sdk_only) = classify_memory_inventory(legacy, sdk)
assert decision.action == "quarantine"
assert decision.reason == "sdk_memory_id_ambiguous"
def test_recorded_plan_executes_import_delete_alias_and_adoption(store):
_start(store)
legacy = (
@@ -538,32 +673,218 @@ def test_recorded_plan_executes_import_delete_alias_and_adoption(store):
assert progress.counts == {"adopted": 1, "deleted": 2, "verified": 3}
def test_legacy_raw_inventory_preserves_exact_source_metadata(store):
def test_legacy_raw_inventory_builds_recent_user_assistant_turns_without_tools(store):
database = store.database_path
with main_database(database).write() as connection, connection:
connection.execute(
connection.executemany(
"""
INSERT INTO evidence (
evidence_id,idempotency_key,user_id,source_id,workspace,project,
conversation_id,message_id,source_ordinal,content_kind,role,
written_by,created_at,verbatim,content_hash,revision,
imported_at,updated_at,ingest_seq
) VALUES (
'e1','key','local','cursor','/repo','project','session','message',
'7','raw_conversation','user','person','2026-10-01T00:00:00+00:00',
'verbatim',?,2,'2026-10-02T00:00:00+00:00',
'2026-10-02T00:00:00+00:00',1
)
) VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)
""",
(content_hash("verbatim"),),
(
(
"u1",
"k1",
"local",
"cursor",
"/repo",
"project",
"session",
"m1",
"1",
"raw_conversation",
"user",
"person",
"2026-10-22T00:00:00+00:00",
"question",
content_hash("question"),
1,
"2026-10-22T00:00:00+00:00",
"2026-10-22T00:00:00+00:00",
1,
),
(
"t1",
"k2",
"local",
"cursor",
"/repo",
"project",
"session",
"m2",
"2",
"raw_conversation",
"tool",
"tool",
"2026-10-22T00:00:01+00:00",
"secret tool output",
content_hash("secret tool output"),
1,
"2026-10-22T00:00:01+00:00",
"2026-10-22T00:00:01+00:00",
2,
),
(
"a1",
"k3",
"local",
"cursor",
"/repo",
"project",
"session",
"m3",
"3",
"raw_conversation",
"assistant",
"agent",
"2026-10-22T00:00:02+00:00",
"answer",
content_hash("answer"),
1,
"2026-10-22T00:00:02+00:00",
"2026-10-22T00:00:02+00:00",
3,
),
(
"old-u",
"k4",
"local",
"cursor",
"/repo",
"project",
"old-session",
"m4",
"4",
"raw_conversation",
"user",
"person",
"2026-10-01T00:00:00+00:00",
"old question",
content_hash("old question"),
1,
"2026-10-01T00:00:00+00:00",
"2026-10-01T00:00:00+00:00",
4,
),
(
"old-a",
"k5",
"local",
"cursor",
"/repo",
"project",
"old-session",
"m5",
"5",
"raw_conversation",
"assistant",
"agent",
"2026-10-01T00:00:01+00:00",
"old answer",
content_hash("old answer"),
1,
"2026-10-01T00:00:01+00:00",
"2026-10-01T00:00:01+00:00",
5,
),
),
)
(raw,) = read_legacy_raw_inventory(database)
assert raw.evidence_id == "e1"
(raw,) = read_legacy_raw_inventory(
database,
cutoff=datetime.fromisoformat("2026-10-20T00:00:00+00:00"),
)
assert raw.evidence_id.startswith("turn:")
assert raw.session_id == "session"
assert raw.source == "cursor"
assert raw.metadata["role"] == "user"
assert raw.metadata["revision"] == 2
assert raw.memory_at.isoformat() == "2026-10-01T00:00:00+00:00"
assert raw.agent_id == "all-apps"
assert raw.content == "[user]: question\n[assistant]: answer"
assert raw.metadata["evidence_ids"] == ["u1", "a1"]
assert "tool" not in raw.content
assert raw.memory_at.isoformat() == "2026-10-22T00:00:00+00:00"
def test_raw_turn_cutoff_uses_completion_time_and_keeps_workspaces_separate(store):
database = store.database_path
rows = (
("u0", "k0", "shared", "w0", "user", "person", "before", "2026-10-19T23:59:00+00:00", 1),
("a0", "k1", "shared", "w0", "assistant", "agent", "after", "2026-10-20T00:01:00+00:00", 2),
("u1", "k2", "same-session", "w1", "user", "person", "one", "2026-10-21T00:00:00+00:00", 3),
(
"a1",
"k3",
"same-session",
"w1",
"assistant",
"agent",
"first",
"2026-10-21T00:01:00+00:00",
4,
),
("u2", "k4", "same-session", "w2", "user", "person", "two", "2026-10-21T00:02:00+00:00", 5),
(
"a2",
"k5",
"same-session",
"w2",
"assistant",
"agent",
"second",
"2026-10-21T00:03:00+00:00",
6,
),
)
with main_database(database).write() as connection, connection:
connection.executemany(
"""
INSERT INTO evidence (
evidence_id,idempotency_key,user_id,source_id,conversation_id,
workspace,content_kind,role,written_by,verbatim,content_hash,
revision,created_at,imported_at,updated_at,ingest_seq
) VALUES (?,?,'local','cursor',?,?,'raw_conversation',?,?,?,? ,1,?,?,?,?)
""",
tuple(
(
evidence_id,
key,
session,
workspace,
role,
written_by,
text,
content_hash(text),
timestamp,
timestamp,
timestamp,
sequence,
)
for (
evidence_id,
key,
session,
workspace,
role,
written_by,
text,
timestamp,
sequence,
) in rows
),
)
turns = read_legacy_raw_inventory(
database,
cutoff=datetime.fromisoformat("2026-10-20T00:00:00+00:00"),
)
assert {turn.content for turn in turns} == {
"[user]: before\n[assistant]: after",
"[user]: one\n[assistant]: first",
"[user]: two\n[assistant]: second",
}
assert {turn.metadata["workspace"] for turn in turns} == {"w0", "w1", "w2"}
def _legacy_raw(evidence_id, *, content="verbatim"):
@@ -591,7 +912,7 @@ def _sdk_raw(raw_id, *, content="verbatim"):
)
def test_raw_inventory_matches_exact_scope_imports_missing_and_adopts_sdk_only(store):
def test_raw_inventory_matches_turns_imports_missing_and_deletes_noncanonical_sdk_raw(store):
legacy = (_legacy_raw("e1"), _legacy_raw("e2", content="missing"))
sdk = (_sdk_raw("raw-existing"), _sdk_raw("raw-sdk-only", content="sdk-only"))
decisions = classify_raw_inventory(legacy, sdk)
@@ -600,7 +921,7 @@ def test_raw_inventory_matches_exact_scope_imports_missing_and_adopts_sdk_only(s
assert by_key["raw:e1"].raw_id == "raw-existing"
assert by_key["raw:e2"].action == "import_exact"
assert by_key["raw:e2"].raw_id == "evidence:e2"
assert by_key["sdk-raw:raw-sdk-only"].action == "adopt"
assert by_key["sdk-raw:raw-sdk-only"].action == "delete"
_start(store)
recorded = record_raw_migration_plan(
@@ -620,7 +941,8 @@ def test_raw_inventory_matches_exact_scope_imports_missing_and_adopts_sdk_only(s
)
)
assert [item["raw_id"] for item in client.imported_raw] == ["evidence:e2"]
assert store.progress("m1").counts == {"adopted": 1, "verified": 2}
assert client.deleted_raw == ["raw-sdk-only"]
assert store.progress("m1").counts == {"deleted": 1, "verified": 2}
def test_sdk_raw_inventory_pages_all_records():
@@ -675,6 +997,20 @@ def test_offline_backfill_reaches_ready_without_committing_cutover(store):
""",
(content_hash("verbatim"),),
)
connection.execute(
"""
INSERT INTO evidence (
evidence_id,idempotency_key,user_id,source_id,conversation_id,
content_kind,role,written_by,verbatim,content_hash,revision,
imported_at,updated_at,ingest_seq
) VALUES (
'a1','assistant-key','local','cursor','session','raw_conversation',
'assistant','agent','reply',?,1,
'2026-10-21T00:00:01+00:00','2026-10-21T00:00:01+00:00',2
)
""",
(content_hash("reply"),),
)
client = _MigrationClient()
result = asyncio.run(
run_sdk_data_plane_backfill(
@@ -683,7 +1019,7 @@ def test_offline_backfill_reaches_ready_without_committing_cutover(store):
migration_id="migration",
bundle_id="bundle",
source_runtime_build="v1.0.4",
target_sdk_version="2.0.5.dev3",
target_sdk_version="2.0.5",
source_schema_version=55,
namespace_fingerprint="namespace",
backend_fingerprint="backend",
@@ -694,7 +1030,9 @@ def test_offline_backfill_reaches_ready_without_committing_cutover(store):
assert result.phase == "reconciled"
assert result.counts == {"verified": 2}
assert [item["memory_id"] for item in client.imported] == ["memory-1"]
assert [item["raw_id"] for item in client.imported_raw] == ["evidence:e1"]
assert len(client.imported_raw) == 1
assert client.imported_raw[0]["content"] == "[user]: verbatim\n[assistant]: reply"
migrated_raw_id = client.imported_raw[0]["raw_id"]
assert store.authority() is MemoryAuthority.LEGACY
# Simulate a process dying after both SDK side effects but before startup
# could finish. The next inventory observes them as matched rather than
@@ -703,13 +1041,13 @@ def test_offline_backfill_reaches_ready_without_committing_cutover(store):
client.memories = (_record("memory-1", content="preference"),)
client.raw = (
HySdkRawRecord(
raw_id="evidence:e1",
content="verbatim",
raw_id=migrated_raw_id,
content="[user]: verbatim\n[assistant]: reply",
user_id="local",
agent_id="one-memory",
agent_id="all-apps",
session_id="session",
source="cursor",
custom={"role": "user", "written_by": "person"},
custom={"input_kind": "legacy_turn"},
),
)
resumed = asyncio.run(
@@ -719,7 +1057,7 @@ def test_offline_backfill_reaches_ready_without_committing_cutover(store):
migration_id="migration",
bundle_id="bundle",
source_runtime_build="v1.0.4",
target_sdk_version="2.0.5.dev3",
target_sdk_version="2.0.5",
source_schema_version=55,
namespace_fingerprint="namespace",
backend_fingerprint="backend",
@@ -752,7 +1090,7 @@ def test_offline_backfill_blocks_cutover_when_an_item_is_quarantined(store):
migration_id="migration",
bundle_id="bundle",
source_runtime_build="v1.0.4",
target_sdk_version="2.0.5.dev3",
target_sdk_version="2.0.5",
source_schema_version=55,
namespace_fingerprint="namespace",
backend_fingerprint="backend",
@@ -783,9 +1121,7 @@ class _MigrationVectorStore:
async def list_by_user(self, *, user_id, layers, **_kwargs):
return [
node
for node in self.nodes.values()
if node.user_id == user_id and node.layer in layers
node for node in self.nodes.values() if node.user_id == user_id and node.layer in layers
]
@@ -803,12 +1139,53 @@ class _MigrationGraphStore:
return node.node_id
class _MigrationFactVector:
def __init__(self):
self.deleted = []
async def delete(self, memory_id):
self.deleted.append(memory_id)
return True
class _MigrationCodingStore:
def __init__(self):
self._conn = sqlite3.connect(":memory:", check_same_thread=False)
self._conn.row_factory = sqlite3.Row
self._conn.execute(
"CREATE TABLE coding_memory_meta ("
"experience_id TEXT PRIMARY KEY,user_id TEXT,agent_id TEXT,solution TEXT)"
)
self._conn.execute(
"CREATE TABLE working_fact ("
"fact_id TEXT PRIMARY KEY,user_id TEXT,agent_id TEXT,"
"statement TEXT,status TEXT)"
)
self._conn.execute(
"INSERT INTO working_fact VALUES ('fact-1','local','scope','fact body','active')"
)
self._conn.commit()
self._lock = threading.Lock()
self._fact_vec = _MigrationFactVector()
async def get_fact(self, memory_id, *, user_id):
row = self._conn.execute(
"SELECT 1 FROM working_fact WHERE fact_id=? AND user_id=?",
(memory_id, user_id),
).fetchone()
return object() if row is not None else None
class _PrivilegedMigrationClient:
def __init__(self):
self._loop_thread = _MigrationLoop()
self._vector_store = _MigrationVectorStore()
self._embed_service = _MigrationEmbedder()
self._graph_store = _MigrationGraphStore()
self._coding = _MigrationCodingStore()
async def _coding_store_for_exact_get(self, variant):
return self._coding if variant == "coding" else None
def list_memories(self, **_kwargs):
return {"vdb": {"memories": [], "total": 0}, "request_id": "list"}
@@ -858,13 +1235,36 @@ def test_offline_migration_writes_sdk_stores_without_public_client_import_method
)
)
raw_page = asyncio.run(migration.read_raw_inventory(user_id="local"))
productivity = asyncio.run(migration.list_productivity_memories())
deleted = asyncio.run(
migration.delete_productivity_memory(
"fact-1",
kind="fact",
variant=None,
user_id="local",
)
)
assert raw_result["raw_id"] == "raw-1"
assert memory_result["memory_id"] == "memory-1"
assert [item.raw_id for item in raw_page.raw] == ["raw-1"]
assert productivity == (
{
"memory_id": "fact-1",
"content": "fact body",
"status": "active",
"user_id": "local",
"agent_id": "scope",
"kind": "fact",
"variant": "coding",
},
)
assert client._vector_store.nodes["raw-1"].layer is MemoryLayer.L1_RAW
assert client._vector_store.nodes["memory-1"].status is MemoryStatus.ACTIVE
assert client._graph_store.nodes["memory-1"].content == "prefers tea"
assert deleted["deleted_count"] == 1
assert client._coding._fact_vec.deleted == ["fact-1"]
assert client._coding._conn.execute("SELECT COUNT(*) FROM working_fact").fetchone()[0] == 0
assert not hasattr(client, "import_raw")
assert not hasattr(client, "import_memory_exact")
@@ -898,6 +1298,57 @@ def test_raw_http_dto_uses_the_unified_memory_list_shape():
assert page.raw[0].custom == {"role": "user"}
def test_sdk_raw_retention_keeps_only_the_latest_three_days():
now = datetime(2026, 10, 23, tzinfo=UTC)
class Client:
def __init__(self):
self.rows = [
{
"memory_id": "old",
"memory_at": int((now - timedelta(days=4)).timestamp()),
},
{
"memory_id": "recent",
"memory_at": int((now - timedelta(days=2)).timestamp()),
},
]
def list_memories(self, **_kwargs):
return {
"vdb": {
"memories": list(self.rows),
"total": len(self.rows),
"limit": 500,
"offset": 0,
}
}
def delete(self, memory_id):
before = len(self.rows)
self.rows = [row for row in self.rows if row["memory_id"] != memory_id]
return {
"success": True,
"deleted_count": before - len(self.rows),
}
client = Client()
class Handle:
async def client(self):
return client
runtime = SdkMemoryRuntime(
coordinator=None,
read_handle=Handle(),
write_queue=None,
extraction_settings=None,
data_dir=Path("."),
)
assert asyncio.run(runtime.prune_expired_raw(now=now)) == 1
assert [row["memory_id"] for row in client.rows] == ["recent"]
def _staging_fixture(tmp_path, store):
data_dir = tmp_path / "home"
live = data_dir / "hy-memory" / "governed-v2"
@@ -91,7 +91,7 @@ def test_bundle_contains_management_sdk_config_and_verified_manifest(tmp_path):
assert manifest["bundle_id"] == "migration-1"
assert manifest["sdk_namespace"] == "governed-v2"
assert manifest["runtime_build"] == "build-13"
assert manifest["snapshot_sdk_version"] == "2.0.5.dev3"
assert manifest["snapshot_sdk_version"] == "2.0.5"
paths = {item["path"]: item for item in manifest["files"]}
assert {
"management/one-memory.sqlite3",
+5 -5
View File
@@ -991,7 +991,7 @@ wheels = [
[[package]]
name = "hy-memory"
version = "2.0.5.dev3"
version = "2.0.5"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "chromadb" },
@@ -1008,9 +1008,9 @@ dependencies = [
{ name = "scikit-learn" },
{ name = "tiktoken" },
]
sdist = { url = "https://files.pythonhosted.org/packages/cb/83/67c261bc379eb45f1cc460cf55eb057867d485695c2799bd861827dbcedb/hy_memory-2.0.5.dev3.tar.gz", hash = "sha256:8ae8c2b2d9736b304d26654a5c2903160f430ec5065206ae724d5cb841820d75", size = 836317, upload-time = "2026-10-21T09:59:27.311Z" }
sdist = { url = "https://files.pythonhosted.org/packages/94/56/7a279122bb56b067c390b748b8a83b6a90bb2a8f237c1d3989b6317681cd/hy_memory-2.0.5.tar.gz", hash = "sha256:bfb1e9e9815377477702327fa318331d555b8256a67b5bad8d70d6a682bd9a4f", size = 839060, upload-time = "2026-10-22T12:59:30.502Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/b6/5f/30a88b0278c070c63d11f0c25a360e1cfe130d90a53a5eece9359c4ca7ec/hy_memory-2.0.5.dev3-py3-none-any.whl", hash = "sha256:621193ac3a527c0b8736d66f17c4cce4956dc4a882d9dfc724b9df4d34fe816e", size = 900779, upload-time = "2026-10-21T09:59:24.716Z" },
{ url = "https://files.pythonhosted.org/packages/c5/97/090ca18b6a13d0216d2df607cb67732ff7a8eba2d0359b9ad95283b2cbdc/hy_memory-2.0.5-py3-none-any.whl", hash = "sha256:c24649d188eaa03d3b4e4e4ec20e818da884170b5f775f3cf23ecfa6b60523d6", size = 902923, upload-time = "2026-10-22T12:59:28.018Z" },
]
[[package]]
@@ -1801,7 +1801,7 @@ wheels = [
[[package]]
name = "one-memory"
version = "1.0.4"
version = "1.0.5"
source = { editable = "." }
dependencies = [
{ name = "aiomysql" },
@@ -1837,7 +1837,7 @@ requires-dist = [
{ name = "fastapi", specifier = ">=0.141.1" },
{ name = "filelock", specifier = ">=3.32.4" },
{ name = "httpx", specifier = ">=0.28.1" },
{ name = "hy-memory", specifier = "==2.0.5.dev3" },
{ name = "hy-memory", specifier = "==2.0.5" },
{ name = "keyring", specifier = ">=25.7.0" },
{ name = "numpy", specifier = "==2.5.2" },
{ name = "onnxruntime", specifier = "==1.23.2" },
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@one-memory/desktop",
"version": "1.0.4",
"version": "1.0.5",
"private": true,
"main": "dist/main.js",
"scripts": {
@@ -1,13 +1,38 @@
import { access, mkdtemp, readFile } from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { expect, it, vi } from "vitest";
import {
requireCompletedStartupSdkMigration,
runStartupSdkMigration,
withSdkMigrationCaptureFence,
} from "./startup-sdk-migration";
const core = { command: "/core/one-memory", args: [] };
const captureFence = async <T>(
_dataDir: string,
action: () => Promise<T>,
): Promise<T> => action();
it("fences hook capture only while startup migration is running", async () => {
const dataDir = await mkdtemp(
path.join(os.tmpdir(), "memhub-sdk-migration-"),
);
const marker = path.join(dataDir, ".sdk-migration-capture-fence.json");
await withSdkMigrationCaptureFence(dataDir, async () => {
const document = JSON.parse(await readFile(marker, "utf8")) as {
pid: number;
};
expect(document.pid).toBe(process.pid);
});
await expect(access(marker)).rejects.toThrow();
});
it("runs bundle backfill and cutover inside the startup maintenance fence", async () => {
const calls: string[][] = [];
const ordering: string[] = [];
const execute = vi.fn(async (_core, args: string[]) => {
calls.push(args);
const command = args[0];
@@ -33,7 +58,19 @@ it("runs bundle backfill and cutover inside the startup maintenance fence", asyn
action: () => Promise<T>,
): Promise<T> => {
fenceCalls.push(dataDir);
return action();
ordering.push("runtime-fence:start");
const result = await action();
ordering.push("runtime-fence:end");
return result;
};
const orderedCaptureFence = async <T>(
_dataDir: string,
action: () => Promise<T>,
): Promise<T> => {
ordering.push("capture-fence:start");
const result = await action();
ordering.push("capture-fence:end");
return result;
};
const result = await runStartupSdkMigration({
@@ -42,10 +79,17 @@ it("runs bundle backfill and cutover inside the startup maintenance fence", asyn
runtimeBuild: "build-13",
execute,
fence,
captureFence: orderedCaptureFence,
});
expect(result).toEqual({ state: "cutover" });
expect(fenceCalls).toEqual(["/data"]);
expect(ordering).toEqual([
"capture-fence:start",
"runtime-fence:start",
"runtime-fence:end",
"capture-fence:end",
]);
expect(calls.map((args) => args[0])).toEqual([
"sdk-migration-status",
"migration-bundle",
@@ -73,6 +117,7 @@ it("does not stop Runtime when authority is already SDK or the home is empty", a
runtimeBuild: "build-13",
execute: vi.fn(async () => ({ exitCode: 0, document })),
fence,
captureFence,
});
expect(result).toEqual({ state: "not_required" });
expect(fenced).toBe(false);
@@ -86,6 +131,7 @@ it("reports an unreadable migration status as deferred", async () => {
runtimeBuild: "build-13",
execute: vi.fn(async () => ({ exitCode: 1, document: {} })),
fence: async (_dataDir, action) => action(),
captureFence,
});
expect(result).toEqual({
@@ -116,6 +162,7 @@ it("resumes legacy table cleanup under the maintenance fence", async () => {
: { exitCode: 0, document: { state: "clean" } };
}),
fence: async (_dataDir, action) => action(),
captureFence,
});
expect(result).toEqual({ state: "cutover" });
@@ -146,6 +193,7 @@ it("keeps legacy authority when backfill is blocked", async () => {
runtimeBuild: "build-13",
execute,
fence: async (_dataDir, action) => action(),
captureFence,
});
expect(result).toEqual({ state: "deferred", code: "quarantine" });
@@ -153,3 +201,42 @@ it("keeps legacy authority when backfill is blocked", async () => {
"sdk-cutover",
);
});
it("reuses one v2 migration identity after a blocked startup", async () => {
const calls: string[][] = [];
const migrationId = "sdk-authority-v2-0123456789abcdef";
const bundleId = `${migrationId}-pre-cutover`;
const execute = vi.fn(async (_core, args: string[]) => {
calls.push(args);
return args[0] === "sdk-migration-status"
? {
exitCode: 0,
document: {
authority: "legacy",
state: "required",
generation: "ffffffffffffffff",
migration_id: migrationId,
bundle_id: bundleId,
},
}
: args[0] === "sdk-migrate"
? { exitCode: 4, document: { state: "blocked", code: "quarantine" } }
: { exitCode: 0, document: { state: "reused" } };
});
await runStartupSdkMigration({
dataDir: "/data",
core,
runtimeBuild: "build-17",
execute,
fence: async (_dataDir, action) => action(),
captureFence,
});
expect(calls.find((args) => args[0] === "migration-bundle")).toContain(
bundleId,
);
expect(calls.find((args) => args[0] === "sdk-migrate")).toContain(
migrationId,
);
});
+101 -62
View File
@@ -1,8 +1,11 @@
import { spawn } from "node:child_process";
import { mkdir, rename, rm, writeFile } from "node:fs/promises";
import path from "node:path";
import { withRuntimeMaintenanceFence } from "@one-memory/runtime-control";
const MIGRATION_PREFIX = "sdk-authority-v1";
const MIGRATION_PREFIX = "sdk-authority-v2";
const MAX_OUTPUT_BYTES = 1024 * 1024;
const CAPTURE_FENCE = ".sdk-migration-capture-fence.json";
export interface StartupMigrationResult {
state: "not_required" | "cutover" | "deferred";
@@ -39,6 +42,7 @@ interface StartupMigrationOptions {
dataDir: string,
) => Promise<CommandResult>;
fence?: <T>(dataDir: string, action: () => Promise<T>) => Promise<T>;
captureFence?: <T>(dataDir: string, action: () => Promise<T>) => Promise<T>;
}
export async function runStartupSdkMigration(
@@ -46,6 +50,7 @@ export async function runStartupSdkMigration(
): Promise<StartupMigrationResult> {
const execute = options.execute ?? executeCoreCommand;
const fence = options.fence ?? withRuntimeMaintenanceFence;
const captureFence = options.captureFence ?? withSdkMigrationCaptureFence;
const status = await execute(
options.core,
["sdk-migration-status", "--json"],
@@ -58,16 +63,18 @@ export async function runStartupSdkMigration(
status.document.authority === "sdk" &&
status.document.state === "cleanup_required"
) {
return fence(options.dataDir, async () => {
const cleanup = await execute(
options.core,
["sdk-cleanup", "--yes", "--json"],
options.dataDir,
);
return cleanup.exitCode === 0 && cleanup.document.state === "clean"
? { state: "cutover" }
: deferred(cleanup);
});
return captureFence(options.dataDir, () =>
fence(options.dataDir, async () => {
const cleanup = await execute(
options.core,
["sdk-cleanup", "--yes", "--json"],
options.dataDir,
);
return cleanup.exitCode === 0 && cleanup.document.state === "clean"
? { state: "cutover" }
: deferred(cleanup);
}),
);
}
if (
status.document.authority === "sdk" ||
@@ -79,57 +86,89 @@ export async function runStartupSdkMigration(
if (typeof generation !== "string" || !/^[a-f0-9]{16}$/.test(generation)) {
return { state: "deferred", code: "sdk_migration_generation_invalid" };
}
const migrationId = `${MIGRATION_PREFIX}-${generation}`;
const bundleId = `${migrationId}-pre-cutover`;
return fence(options.dataDir, async () => {
const bundle = await execute(
options.core,
[
"migration-bundle",
"--bundle-id",
bundleId,
"--runtime-build",
options.runtimeBuild,
"--yes",
"--json",
],
options.dataDir,
);
if (bundle.exitCode !== 0) return deferred(bundle);
const backfill = await execute(
options.core,
[
"sdk-migrate",
"--migration-id",
migrationId,
"--bundle-id",
bundleId,
"--yes",
"--json",
],
options.dataDir,
);
if (backfill.exitCode !== 0 || backfill.document.state !== "ready") {
return deferred(backfill);
}
const cutover = await execute(
options.core,
[
"sdk-cutover",
"--migration-id",
migrationId,
"--bundle-id",
bundleId,
"--yes",
"--json",
],
options.dataDir,
);
if (cutover.exitCode !== 0 || cutover.document.state !== "cutover") {
return deferred(cutover);
}
return { state: "cutover" };
});
const previousMigrationId = status.document.migration_id;
const previousBundleId = status.document.bundle_id;
const canResume =
typeof previousMigrationId === "string" &&
previousMigrationId.startsWith(`${MIGRATION_PREFIX}-`) &&
typeof previousBundleId === "string" &&
previousBundleId === `${previousMigrationId}-pre-cutover`;
const migrationId = canResume
? previousMigrationId
: `${MIGRATION_PREFIX}-${generation}`;
const bundleId = canResume ? previousBundleId : `${migrationId}-pre-cutover`;
return captureFence(options.dataDir, () =>
fence(options.dataDir, async () => {
const bundle = await execute(
options.core,
[
"migration-bundle",
"--bundle-id",
bundleId,
"--runtime-build",
options.runtimeBuild,
"--yes",
"--json",
],
options.dataDir,
);
if (bundle.exitCode !== 0) return deferred(bundle);
const backfill = await execute(
options.core,
[
"sdk-migrate",
"--migration-id",
migrationId,
"--bundle-id",
bundleId,
"--yes",
"--json",
],
options.dataDir,
);
if (backfill.exitCode !== 0 || backfill.document.state !== "ready") {
return deferred(backfill);
}
const cutover = await execute(
options.core,
[
"sdk-cutover",
"--migration-id",
migrationId,
"--bundle-id",
bundleId,
"--yes",
"--json",
],
options.dataDir,
);
if (cutover.exitCode !== 0 || cutover.document.state !== "cutover") {
return deferred(cutover);
}
return { state: "cutover" };
}),
);
}
export async function withSdkMigrationCaptureFence<T>(
dataDir: string,
action: () => Promise<T>,
): Promise<T> {
await mkdir(dataDir, { recursive: true, mode: 0o700 });
const marker = path.join(dataDir, CAPTURE_FENCE);
const temporary = `${marker}.${process.pid}.tmp`;
await writeFile(
temporary,
`${JSON.stringify({ schema_version: 1, pid: process.pid })}\n`,
{ encoding: "utf8", mode: 0o600 },
);
await rename(temporary, marker);
try {
return await action();
} finally {
await rm(marker, { force: true });
await rm(temporary, { force: true });
}
}
function deferred(result: CommandResult): StartupMigrationResult {
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@one-memory/site",
"version": "1.0.4",
"version": "1.0.5",
"private": true,
"type": "module",
"scripts": {
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@one-memory/web",
"version": "1.0.4",
"version": "1.0.5",
"private": true,
"type": "module",
"scripts": {
+4 -1
View File
@@ -130,6 +130,7 @@ This exception does not extend to VDB, Coding, prompt, reconcile, or lifecycle.
- Storage backends:仅 SDK config/env(Chroma/FAISS/Qdrant、Kuzu/Neo4j、cache/history sqlite|mysql、coding sqlite 等)。
- Queue:pending `memory_write_jobs` 只表示尚未调用 SDK 的待处理请求,因此没有 SDK raw/memory ID。worker 单次调用 SDK `add()`;SDK 内部完成 raw 写入与 extraction/reconciliation,成功后 MemHub 仅将返回的 raw ID / memory IDs 记作 job outcome。旧用户启动迁移在 Runtime writers 停止期间直接操作 SDK 存储,不向公开 `HyMemoryClient` 增加迁移专用方法,也不拆分正常写路径。
- Raw retention:SDK L1 raw 只保留最近三天;每个完整 turn 只保存真人 user 与 assistant,tool 不进入 raw。高层 Chat/Coding/Working memory 不受该窗口影响。
- Recall:Host Bridge / Core 先做宿主安装凭据与 tool grant,再调 **SDK search/get**。当前产品没有逐 memory 的 A-harness→B-harness 可见性开关,默认允许跨 harness 搜索;未来若增加来源限制,直接透传 SDK `source` / `include_sources` / `exclude_sources`,不在 MemHub 建 memory ACL。
- Context wire:cutover marker 提交前只使用 legacy Context v1;提交后 Host Bridge 只使用 SDK-native Context v2。两种 schema 原样校验、原样返回,不把 SDK item 补造成 v1 的 `revision`、`evidence_ids`、`paths` 或 `adoption`,已选择的 v2 路径失败时也不回退 legacy 权威。
- Ask wire:marker 前保留 Governance Ask v1;marker 后 Web / Host Bridge 只调用 SDK Ask v2。v2 的引用只含 SDK 原生 `memory_id/content/layer/status/source/gmt_created`,生成过程中重复 `get` 校验同一 SDK 快照,不补造 revision、Evidence 或 scopes,SDK 失败不回退 v1。
@@ -138,6 +139,8 @@ This exception does not extend to VDB, Coding, prompt, reconcile, or lifecycle.
- SDK backfill 只写同文件系统的私有 staging 副本;验证后先原子切换 SDK
root,再提交 authority marker。切换或 marker 提交失败必须恢复原 live root;
App 不得在 migration `deferred` 状态继续启动 Runtime。
- 迁移期间宿主 capture hook 由 Desktop 活跃 PID marker 静默挡住,不再追加
Evidence;cutover 后先从宿主 History checkpoint 补齐遗漏 turn,再恢复 live sync。
- Matter 若未来重新开发,需以新的 SDK-native 合同重新设计,不继承当前占位 schema。
Pinned SDK 版本见发布锁文件 / `MEMORY.md`(随发布更新)。
@@ -282,7 +285,7 @@ Support is declared per immutable GitHub Release manifest:
- Linux x64/arm64 Runtime on native GHA runners.
- macOS arm64/x64 App/Runtime: Developer ID, notarization, native acceptance.
- Windows is not supported.
- `hy-memory==2.0.5.dev3` is the current development-preview SDK dependency; the release
- `hy-memory==2.0.5` is the current stable SDK dependency; the release
lock and [`../MEMORY.md`](../MEMORY.md) are authoritative when the pin changes.
See [`INSTALLATION.md`](INSTALLATION.md).
+1 -1
View File
@@ -1,6 +1,6 @@
# MemHub 安装与发行
MemHub `v1.0.4`(build 13)的发行目标如下:
MemHub `v1.0.5`(build 14)的发行目标如下:
| 平台 | 发行物 | 构建与验收 |
| --- | --- | --- |
@@ -19,6 +19,9 @@ Before the cutover marker is committed:
- SDK stores are inspected and backfilled, but are not merged into live search;
- new imports, writes, queue drains, corrections, deletes and session delivery
are paused.
- Host capture commands detect the live Desktop migration fence and return
silently without writing Evidence. After cutover, History catch-up runs from
the existing source cursor before ordinary live sync.
After the cutover marker is committed:
@@ -49,12 +52,9 @@ The migration preflight records and validates:
- embedding model identity and dimensions;
- source grants, model profiles, config, queue leases, and pending operations.
The migration fails closed when the installed SDK lacks:
- complete lifecycle/raw inventory;
- exact raw and exact memory import;
- durable request outcome lookup;
- consistent snapshot/export and restore.
The migration fails closed when it cannot inspect every SDK memory namespace
(Chat, Coding Experience, Working Fact), write a missing ordinary memory, or
create and restore a consistent data-plane snapshot.
MemHub tags `v1.0.0` through `v1.0.4` all use the `governed-v2` namespace and
the same SDK SQL table families (`memory_operations_v2`, `pipeline_logs_v2`,
@@ -71,10 +71,10 @@ because an earlier App wrote it.
While Core and Runtime writers are stopped, build four inventories:
- `E`: Evidence rows and revisions;
- `E`: completed user/assistant turns from the rolling three-day legacy window;
- `G`: current Governance rows, statuses, supersession links, and latest content;
- `M`: task-output mappings from legacy `memory_id` to SDK `hy_memory_id`;
- `S`: complete SDK inventory, including raw and non-active lifecycle records.
- `S`: complete SDK inventory, including Coding/Working and non-active records.
Every item must end in exactly one class:
@@ -85,7 +85,7 @@ Every item must end in exactly one class:
| Governance-only | current legacy memory has no SDK entity | exact import without extraction |
| SDK-only | SDK entity has no legacy mirror | retain and adopt |
| divergent | both exist but content, owner or lifecycle differs | reconcile explicit user intent or quarantine |
| raw-only | Evidence has no SDK raw receipt | exact raw import |
| raw turn missing | canonical recent turn has no matching SDK raw | write one bounded L1 raw turn |
| unknown outcome | a task may have reached SDK without a confirmed result | SDK outcome inquiry; never blind retry |
An unclassified item blocks cutover.
@@ -102,26 +102,30 @@ The current product has no MemHub revision history or Undo.
- SDK supersession remains an SDK lifecycle relationship. Existing SDK chains
are retained. Governance links are used only to repair a missing SDK chain
during exact import.
- Mapped Coding Experience and Working Fact IDs are resolved in their native
SDK stores and adopted without conversion to Chat layers. A mapped ID missing
from its native store blocks cutover rather than fabricating another type.
- Pending correction/delete records are settled before ordinary backfill.
`update(memory_id, content)` modifies the same SDK ID. There is no product CAS,
revision number, or restore operation.
## 5. Evidence to SDK raw
## 5. Legacy Evidence to bounded SDK raw
Evidence must not be replayed through ordinary `add()`: doing so would extract
and reconcile again, changing semantic memories.
Evidence rows are message-level legacy storage, not SDK raw chunks. Migration
uses them only to rebuild completed turns from the three days preceding the
stable migration start time:
The SDK exact-raw import preserves:
- one human `user` opener plus its following `assistant` records becomes one
deterministic L1 raw node;
- tool rows and agent-authored user envelopes are excluded;
- a turn is bounded to 60,000 characters and records whether it was truncated;
- existing SDK raw outside this canonical set is removed in staging;
- high-level Chat/Coding/Working memories are retained independently of raw.
- source, conversation/session, workspace, role and author;
- original event time and stable source ordering;
- verbatim content and source identity;
- a deterministic migration key.
The operation is idempotent and returns an SDK raw ID. Evidence revisions and
repairs are preserved in the migration bundle for audit; only the current
accepted raw record becomes the normal product read.
No extraction or reconciliation is rerun while rebuilding raw. After cutover,
the normal Cursor History catch-up scans forward from its source checkpoint and
recovers turns skipped while capture was fenced.
## 6. Full migration bundle
+2 -2
View File
@@ -1,8 +1,8 @@
{
"name": "one-memory",
"version": "1.0.4",
"version": "1.0.5",
"release": {
"build_number": 13,
"build_number": 14,
"macos_min_version": "13.0",
"app_update": "manual",
"runtime_manifest_schema": 1,
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@one-memory/contracts",
"version": "1.0.4",
"version": "1.0.5",
"private": true,
"type": "module",
"types": "./src/index.ts",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@one-memory/host-bridge",
"version": "1.0.4",
"version": "1.0.5",
"private": true,
"type": "module",
"exports": {
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@one-memory/i18n",
"version": "1.0.4",
"version": "1.0.5",
"private": true,
"type": "module",
"types": "./src/index.ts",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@one-memory/operations",
"version": "1.0.4",
"version": "1.0.5",
"private": true,
"type": "module",
"types": "./src/index.ts",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@one-memory/runtime-control",
"version": "1.0.4",
"version": "1.0.5",
"private": true,
"type": "module",
"bin": {
+1 -1
View File
@@ -36,7 +36,7 @@ MEMMY_LICENSE = (
MEMMY_PROVENANCE = MEMMY_LICENSE.with_name("PROVENANCE.md")
MEMMY_PIN = "25acd5f29fc7df73b943713abe61f1b2f9839887"
HY_MEMORY_NAME = "hy-memory"
HY_MEMORY_VERSION = "2.0.5.dev3"
HY_MEMORY_VERSION = "2.0.5"
class SbomError(RuntimeError):
+2 -2
View File
@@ -12,8 +12,8 @@ test("installation docs disclose publication and lifecycle boundaries", async ()
"utf8",
);
assert.match(readme, /`hy-memory==2\.0\.5\.dev3`/);
assert.match(readme, /`v1\.0\.4` 的发行目标是 Linux x64\/arm64 Runtime/);
assert.match(readme, /`hy-memory==2\.0\.5`/);
assert.match(readme, /`v1\.0\.5` 的发行目标是 Linux x64\/arm64 Runtime/);
assert.match(readme, /Linux 制品由 GitHub Actions/);
assert.match(readme, /macOS.*Developer ID 签名、公证/s);
assert.match(readme, /release-manifest\.json/);
+3 -3
View File
@@ -13,7 +13,7 @@ spec.loader.exec_module(module)
def test_hy_memory_notice_comes_from_the_installed_dist_info(tmp_path: Path) -> None:
dist_info = tmp_path / "hy_memory-2.0.5.dev3.dist-info"
dist_info = tmp_path / "hy_memory-2.0.5.dist-info"
licenses = dist_info / "licenses"
licenses.mkdir(parents=True)
(dist_info / "METADATA").write_text(
@@ -21,7 +21,7 @@ def test_hy_memory_notice_comes_from_the_installed_dist_info(tmp_path: Path) ->
(
"Metadata-Version: 2.4",
"Name: hy-memory",
"Version: 2.0.5.dev3",
"Version: 2.0.5",
"License-Expression: Apache-2.0",
"",
)
@@ -38,7 +38,7 @@ def test_hy_memory_notice_comes_from_the_installed_dist_info(tmp_path: Path) ->
notice = module.hy_memory_notice(dist_info)
assert notice["name"] == "hy-memory"
assert notice["version"] == "2.0.5.dev3"
assert notice["version"] == "2.0.5"
assert notice["license"] == "Apache-2.0"
assert notice["copyright"] == "Copyright (C) 2026 Tencent. All rights reserved."
assert "Licensed under the Apache License" in notice["text"]
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "hy-memory"
version = "2.0.5.dev3"
version = "2.0.5"
description = "HY Memory - Industrial-grade dual-system cognitive memory framework"
readme = "README.md"
requires-python = ">=3.9"