chore(release): bump MemHub build number to 21 (on top of SDK vector reattach fix)

All memory and raw reads and writes go through the Hy Memory SDK. The
one_memory.retrieval index kept a second, MemHub-owned vector copy of
conversation text in a private Chroma store under <data_dir>/retrieval-vectors/,
which that rule does not allow. It was experimental and off by default.

Remove the vector store, runtime, provisioning, supervisor, worker, index
store, chunking and mapped content, the offset tokenizer only chunking used,
the semantic raw lanes that read the index, the retrieval_index_enabled and
retrieval_minimum_free_bytes settings, app.state.retrieval_runtime and their
tests. The context route now calls the lexical raw reader directly, which is
exactly what it did with the index disabled; the v2 coordinator keeps its
lexical lane with the same statuses it produced when no index was configured.

Historical migrations and the legacy retrieval tables in the drop list are
unchanged. An existing retrieval-vectors directory is left on disk.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

.github/workflows/hermes-provider.yml

@@ -0,0 +1,49 @@
+name: Hermes provider
+
+on:
+  pull_request:
+    paths:
+      - "plugins/hermes/**"
+      - ".github/workflows/hermes-provider.yml"
+  push:
+    branches: [main]
+    paths:
+      - "plugins/hermes/**"
+      - ".github/workflows/hermes-provider.yml"
+
+permissions:
+  contents: read
+
+jobs:
+  provider:
+    runs-on: ubuntu-24.04
+    timeout-minutes: 3
+    steps:
+      - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262
+        with:
+          persist-credentials: false
+      - uses: astral-sh/setup-uv@d0d8abe699bfb85fec6de9f7adb5ae17292296ff
+        with:
+          python-version: "3.12"
+          enable-cache: false
+      - name: Download and verify native Hermes contract
+        run: |
+          uv run --no-project --python 3.12 python - <<'PY'
+          import hashlib
+          import os
+          from pathlib import Path
+          import urllib.request
+
+          url = "https://raw.githubusercontent.com/NousResearch/hermes-agent/818c13be1dc4fd28987e1e881a9408224afd4535/agent/memory_provider.py"
+          with urllib.request.urlopen(url, timeout=30) as response:
+              data = response.read()
+          expected = "a4e44a293013fff831ac136ab77c556541cd18817f53bd662e97090cd80bd5a6"
+          if hashlib.sha256(data).hexdigest() != expected:
+              raise SystemExit("Hermes contract SHA-256 mismatch")
+          snapshot = Path(os.environ["RUNNER_TEMP"]) / "hermes-memory-provider.py"
+          snapshot.write_bytes(data)
+          with open(os.environ["GITHUB_ENV"], "a") as env:
+              env.write(f"HERMES_MEMORY_PROVIDER_SNAPSHOT={snapshot}\n")
+          PY
+      - name: Test provider with native contract
+        run: uv run --no-project --python 3.12 --with pytest python -m pytest plugins/hermes/tests -o addopts= -o log_cli=false -q

CHANGELOG.md

@@ -2,6 +2,73 @@
 
 Hy Memory and MemHub are versioned and released independently.
 
+## Unreleased
+
+- Fixed legacy-to-SDK startup migration to adopt native Coding Experience and
+  Working Fact IDs instead of importing them as Chat layers. Raw migration now
+  rebuilds bounded user/assistant turns from the latest three days, excludes
+  tools, and enforces the same rolling retention after cutover.
+- Paused host capture writes with a live Desktop migration fence; Cursor
+  History catch-up resumes from its existing checkpoint after cutover.
+- Removed MemHub's private `hy_memory_bridge` implementation. MemHub now calls
+  public `HyMemoryClient` methods at runtime; prompt/reconcile/response
+  monkey-patches and private VDB/Coding runtime access were deleted. Kuzu Graph
+  projection/admin and stopped-writer App migration are the documented product
+  exceptions.
+- Added a MemHub-managed native Hermes MemoryProvider with automatic prefetch,
+  persistent per-session memory-ID deduplication, authorized Broker reads, and
+  recall-context stripping to prevent memory recursion.
+- Added an extensible MemHub interface-localization layer with Simplified
+  Chinese, English, and Spanish catalogs shared by Web and Electron-native
+  surfaces. Interface language remains independent from memory content and
+  model processing language.
+- Fixed governed-memory revisions and repeated evidence links failing when the
+  visibility dirty-marker trigger encounters the same memory twice. Startup
+  now replaces the affected legacy trigger without changing schema versions
+  or governed and visibility data.
+- Kept post-write authority verification failures available for read-only
+  reconciliation while preventing automatic or generic confirmed retries from
+  invoking the model again. Automatic reconciliation skips those sticky rows
+  before applying its batch limit, and linked durable jobs exclusively own
+  retry scheduling for their memory tasks.
+- Removed MemHub's task-scoped HyPermission store wrappers and global
+  model-content consent switch. The trusted embedded Hy Memory SDK now uses
+  its native local stores directly.
+- Existing installations ignore the retired config flag, readmit previously
+  parked pre-invocation work, and drop only the obsolete
+  `hy_node_permissions` proof table. Evidence, governed memories, model
+  profiles, SDK stores, and source/host grants are preserved.
+
+## Hy Memory 2.0.5 — 2026-10-22
+
+- Published
+  [`hy-memory==2.0.5`](https://pypi.org/project/hy-memory/2.0.5/).
+- Extended `get()` across Chat, Coding Experience, Working Fact, and raw IDs;
+  the returned `layer`, `kind`, and `variant` identify the actual record.
+- Added explicit raw retrieval through the existing `search()` and
+  `list_memories()` contracts, including user, agent, session, and source
+  filters.
+- Added public embedding-service injection and corrected extraction language
+  detection for structured OpenAI-style messages.
+- Added lifecycle status filters, durable idempotent write outcomes, and
+  quiesced local data-plane snapshot/restore.
+- Kept legacy migration mechanics out of the public Client API; separate
+  migration-only raw/import methods from development previews were removed.
+- Physical delete now reports partial VDB/Graph cleanup as failure.
+- Coding search no longer injects the version-evolution diff block by default;
+  opt in with `MEMORY_CODING_CHAIN_BLOCK_ENABLED=true`.
+- Serialized SDK SQLite cache/history connection lifecycle and tightened
+  Coding reconciliation to its agent/session/workspace authority.
+- Clarified persistent caller context through `meta`, while `metadata` remains
+  legacy processing configuration.
+
+## Hy Memory 2.0.5.dev4 — 2026-10-21
+
+- Development preview used for SDK-authority migration validation; not the
+  final `2.0.5` release.
+- Published as
+  [`hy-memory==2.0.5.dev4`](https://pypi.org/project/hy-memory/2.0.5.dev4/).
+
 ## Hy Memory 2.0.5.dev3 — 2026-10-21
 
 - Development preview; not the final `2.0.5` release.
@@ -28,6 +95,12 @@ Hy Memory and MemHub are versioned and released independently.
   APIs, offline local data-plane snapshot/restore, and fail-closed physical
   delete receipts.
 
+## Hy Memory 2.0.4 — 2026-10-17
+
+- Published
+  [`hy-memory==2.0.4`](https://pypi.org/project/hy-memory/2.0.4/) as a
+  maintenance release.
+
 ## Hy Memory 2.0.3 — 2026-10-17
 
 - Added an optional runtime HTTP transport factory so embedded products can
@@ -38,55 +111,6 @@ Hy Memory and MemHub are versioned and released independently.
   unchanged memory extraction, reconciliation, storage, and Coding/Working
   behavior.
 
-## Unreleased
-
-- Added an extensible MemHub interface-localization layer with Simplified
-  Chinese, English, and Spanish catalogs shared by Web and Electron-native
-  surfaces. Interface language remains independent from memory content and
-  model processing language.
-- `search(..., layers=["l1_raw"])` now explicitly searches SDK raw records in
-  pro/ultra mode and keeps existing user/agent/session/source filters; default
-  search continues to hide raw.
-- Added `import_memory_exact` for idempotent migration of an existing semantic
-  memory with its canonical ID, lifecycle and supersession links, without
-  extraction or reconciliation.
-- Added SDK-authoritative lifecycle listing with explicit status filters,
-  idempotent durable write outcomes, exact raw import/get/list APIs, and
-  quiesced local data-plane snapshot/restore contracts for the MemHub
-  authority migration.
-- Physical delete now reports incomplete VDB/Graph cleanup as a partial failure
-  instead of claiming success after only one store was changed.
-- Coding search no longer injects the version-evolution diff block
-  (`chain_block`) by default. Enable with
-  `MEMORY_CODING_CHAIN_BLOCK_ENABLED=true` or
-  `CodingConfig.chain_block_enabled=True` (write-side observe unchanged).
-- Fixed governed-memory revisions and repeated evidence links failing when the
-  visibility dirty-marker trigger encounters the same memory twice. Startup
-  now replaces the affected legacy trigger without changing schema versions
-  or governed and visibility data.
-- Serialized each SDK SQLite store's shared connection lifecycle so concurrent
-  history/cache operations cannot race with `close()` and crash the process.
-- Scoped Coding reconciliation targets to the current HY authority,
-  conversation, and workspace, preventing near-duplicate sessions from
-  mutating an experience or working fact owned by another evidence context.
-- Kept post-write authority verification failures available for read-only
-  reconciliation while preventing automatic or generic confirmed retries from
-  invoking the model again. Automatic reconciliation skips those sticky rows
-  before applying its batch limit, and linked durable jobs exclusively own
-  retry scheduling for their memory tasks.
-- Clarified that `add(metadata=...)` carries legacy SDK processing options,
-  while persistent caller business context belongs in `meta`.
-- Added an opt-in `get(..., include_raw=True)` path for reading `L1_RAW`
-  memories, including memories written in lite mode. The default remains
-  hidden for backward compatibility.
-- Removed MemHub's task-scoped HyPermission store wrappers and global
-  model-content consent switch. The trusted embedded Hy Memory SDK now uses
-  its native local stores directly.
-- Existing installations ignore the retired config flag, readmit previously
-  parked pre-invocation work, and drop only the obsolete
-  `hy_node_permissions` proof table. Evidence, governed memories, model
-  profiles, SDK stores, and source/host grants are preserved.
-
 ## Hy Memory 2.0.2 — 2026-10-17
 
 - Published
@@ -101,7 +125,17 @@ Hy Memory and MemHub are versioned and released independently.
   Because PyPI release metadata is immutable, the next SDK patch release must
   carry the updated license metadata; `2.0.1` retains its published metadata.
 
-## MemHub 1.0.3 — Unreleased
+## MemHub 1.0.5 — Unreleased
+
+- Updated the source version to `1.0.5` (build 14).
+- Pinned the public SDK to the stable `hy-memory==2.0.5` release.
+
+## MemHub 1.0.4 — 2026-10-18
+
+- Updated the source version to `1.0.4` (build 13).
+- Published the `v1.0.4` source and release metadata.
+
+## MemHub 1.0.3 — 2026-10-17
 
 - Updated the source version to `1.0.3` (build 7).
 - Pinned the public SDK to `hy-memory==2.0.3`.

README.md

@@ -253,6 +253,7 @@ That version is usually not in the working tree or at HEAD either, so it cannot
 - [Graph memory design](docs/pro_graph_design.md) — graph projection, rerank, relation paths, and deployment limits
 - [Environment template](.env.example) — a copy-ready configuration template
 - [MemHub README](memhub/README.md) — local-first desktop & CLI product overview, runtime commands, and privacy boundaries
+- [Hermes native memory provider](plugins/hermes/README.md) — explicit owner/agent scope; separate HTTP service; automatic capture and recall opt-in.
 - [OpenClaw plugin](plugins/openclaw/README.md) — agent integrations live under `plugins/` (overview: [plugins/README.md](plugins/README.md))
 
 ### License

README.zh.md

@@ -253,6 +253,7 @@ Coding 与 Working **复用同一套双层 pipeline 和数据 schema**，区别
 - [Graph 记忆设计](docs/pro_graph_design.md) — 图投影、重排、关系路径与部署限制
 - [环境变量模板](.env.example) — 可直接复制的配置模板
 - [MemHub 说明文档](memhub/README.md) — 本地优先桌面端与 CLI 产品总览、Runtime 命令与隐私边界
+- [Hermes 原生记忆 Provider](plugins/hermes/README.md) — 显式用户与 Agent 隔离，独立 HTTP 服务，自动记录与召回默认关闭。
 - [OpenClaw 插件](plugins/openclaw/README.md) — Agent 集成见 `plugins/`（总览：[plugins/README.md](plugins/README.md)）
 
 ### License

docs/usage.md

@@ -273,8 +273,8 @@ catalog 也可以在没有基础 L0 时形成只读的合成画像。
 
 ```python
 memory = client.get("memory_id")
-# lite 模式写入的是 L1_RAW；按 ID 读取时必须显式选择返回原文。
-raw_memory = client.get("memory_id", include_raw=True)
+# raw ID 返回 L1_RAW，fact ID 返回对应事实层。
+raw_memory = client.get("raw_id")
 result = client.update("memory_id", "更新后的内容")
 result = client.delete("memory_id")
 result = client.delete_all(user_id="user_001")
@@ -284,7 +284,7 @@ page = client.list_memories(
     limit=100,
     offset=0,
     order="desc",
-    layers=None,
+    layers=["l1_raw"],  # 显式列出 raw；不传仍沿用默认语义层列表
     with_evolution=False,
 )
 
@@ -294,9 +294,10 @@ result = client.clone_user("user_001", "user_002", agent_id=None)
 对应异步方法为 `async_get()`、`async_update()`、`async_delete()`、
 `async_delete_all()`、`async_list_memories()`、`async_clone_user()`。
 
-`get()` / `async_get()` 默认继续隐藏 `L1_RAW`，因此不会改变现有调用方的原文访问
-边界。只有显式传入 `include_raw=True` 才返回该层；这也是 lite 模式按 ID 读取
-写入内容及其 `meta` 的方式。MemHub 的 HTTP 读取接口不开放此参数。
+`get()` / `async_get()` 是统一的 ID 精确读取：返回结果的 `layer`、`kind` 和
+`variant` 表明实际类型。授权由调用 SDK 前的产品边界完成，不由 SDK 根据 layer
+猜测。`list_memories()` 默认仍不包含 `L1_RAW`；显式传
+`layers=["l1_raw"]` 时按同一列表合同返回 raw。
 
 普通 `list_memories()` 返回 `vdb` 桶，并在 Graph 有结果时附带 `graph`；传入
 `workspace_id` 时返回生产力记忆的 `scene="productivity"` 和 `experiences`。

hy_memory/__init__.py

@@ -65,6 +65,12 @@ from .snapshot import (
 
 # ====== 配置 ======
 from .config import MemoryConfig
+from .embedding import (
+    EmbeddingDimensionMismatchError,
+    EmbeddingService,
+    EmbeddingServiceProtocol,
+)
+from .retrieval import AmbiguousMemoryIdError, MemoryKind, MemoryVariant
 
 # ====== Embedding 端点覆盖（per-call，不改 model/dims，只换接入端点）======
 from .core.embed_service import embed_endpoint_override
@@ -125,6 +131,13 @@ __all__ = [
     "snapshot_local_data_plane",
     # 配置
     "MemoryConfig",
+    # 公共扩展协议 / 精确读取类型
+    "EmbeddingService",
+    "EmbeddingServiceProtocol",
+    "EmbeddingDimensionMismatchError",
+    "MemoryKind",
+    "MemoryVariant",
+    "AmbiguousMemoryIdError",
     # Embedding 端点覆盖
     "embed_endpoint_override",
     # 高级 API

hy_memory/client.py

@@ -38,6 +38,8 @@ from typing import List, Dict, Any, Optional, Tuple, Union, TYPE_CHECKING
 
 from .config import MemoryConfig
 from .core.embed_service import EmbedService, embed_endpoint_override
+from .embedding import EmbeddingDimensionMismatchError, EmbeddingService
+from .retrieval import AmbiguousMemoryIdError, MemoryKind, MemoryVariant
 from .data.vector_store import create_vector_store
 from .data.history_store import HistoryStore
 from .data.cache import create_cache
@@ -434,6 +436,8 @@ class HyMemoryClient:
     Args:
         config: 自定义 MemoryConfig（默认从环境变量加载）
         mode:   处理模式 ("lite" / "pro" / "ultra")，默认从环境变量 MEMORY_MODE 读取
+        embedding_service: 可选的调用方自管 embedding service；须满足公开的
+                           EmbeddingService Protocol，client 不接管其生命周期。
     """
 
     # 合法的 mode 值
@@ -445,12 +449,15 @@ class HyMemoryClient:
         mode: Optional[str] = None,
         coding_writer: Optional[str] = None,
         runtime: Optional["SharedRuntime"] = None,
+        embedding_service: Optional[EmbeddingService] = None,
     ):
         # 确保日志系统已配置（幂等，多次调用安全）
         setup_logging()
 
         # 配置
         self._config = config or MemoryConfig.from_env()
+        self._borrowed_embedding_service = embedding_service
+        self._validate_embedding_service(embedding_service)
 
         # coding_writer 显式传入则覆盖 config / env（默认 None → 沿用 config.coding.writer）
         # 取值: "legacy" | "agent"
@@ -537,8 +544,12 @@ class HyMemoryClient:
         )
         logger.info(f"[init] Default ThreadPoolExecutor max_workers={_max_workers}")
 
-        # 1. EmbedService（纯配置，首次调用时真正连接 API）
-        self._embed_service = EmbedService(self._config)
+        # 1. EmbedService（注入发生在 ComponentFactory / pipeline 预热之前）
+        self._embed_service = (
+            self._borrowed_embedding_service
+            if self._borrowed_embedding_service is not None
+            else EmbedService(self._config)
+        )
         logger.debug(
             f"[init] EmbedService created: model={self._config.embedder.model}"
         )
@@ -568,8 +579,10 @@ class HyMemoryClient:
             )
 
         # 4. ComponentFactory（注入已初始化的共享组件）
-        self._registry = ComponentFactory(config=self._config)
-        self._registry._shared_embed_service = self._embed_service
+        self._registry = ComponentFactory(
+            config=self._config,
+            embedding_service=self._embed_service,
+        )
         self._registry._shared_vector_store = self._vector_store
         self._registry._shared_vector_store_initialized = True
         if self._graph_store is not None:
@@ -703,6 +716,7 @@ class HyMemoryClient:
         cls,
         config_dict: Dict[str, Any],
         mode: Optional[str] = None,
+        embedding_service: Optional[EmbeddingService] = None,
     ) -> "HyMemoryClient":
         """
         从字典创建客户端（类似 mem0 风格）。
@@ -734,7 +748,49 @@ class HyMemoryClient:
         """
         config = MemoryConfig.from_dict(config_dict)
         resolved_mode = mode or config_dict.get("mode")
-        return cls(config=config, mode=resolved_mode)
+        return cls(
+            config=config,
+            mode=resolved_mode,
+            embedding_service=embedding_service,
+        )
+
+    def _validate_embedding_service(
+        self, embedding_service: Optional[EmbeddingService]
+    ) -> None:
+        """Validate a borrowed service before any store or pipeline is created."""
+        if embedding_service is None:
+            return
+        required = (
+            "embed",
+            "embed_with_meta",
+            "embed_batch",
+            "embed_queued",
+            "embed_queued_with_meta",
+            "get_embedding_dims",
+        )
+        missing = [
+            name for name in required
+            if not callable(getattr(embedding_service, name, None))
+        ]
+        if missing:
+            raise TypeError(
+                "embedding_service does not implement EmbeddingService; "
+                f"missing: {', '.join(missing)}"
+            )
+        try:
+            actual_dims = int(embedding_service.get_embedding_dims())
+        except Exception as exc:
+            raise TypeError(
+                "embedding_service.get_embedding_dims() must return an integer"
+            ) from exc
+        expected_dims = getattr(self._config.vector_store, "embedding_dims", None)
+        if expected_dims is None:
+            expected_dims = getattr(self._config.embedder, "embedding_dims", None)
+        if expected_dims is not None and actual_dims != int(expected_dims):
+            raise EmbeddingDimensionMismatchError(
+                "embedding service dimension mismatch: "
+                f"service={actual_dims}, vector_store={int(expected_dims)}"
+            )
 
     # ================================================================
     # 属性
@@ -1060,6 +1116,7 @@ class HyMemoryClient:
                 llm_provider=self._coding_llm_provider,
                 embed_service=self._embed_service,
                 cache=self._cache,
+                variant=variant,
             )
             self._coding_stores[variant] = store
             self._coding_writers[variant] = writer
@@ -1829,21 +1886,26 @@ class HyMemoryClient:
         self,
         memory_id: str,
         *,
-        include_raw: bool = False,
+        kind: Optional[MemoryKind] = None,
+        variant: Optional[MemoryVariant] = None,
     ) -> Optional[Dict[str, Any]]:
         """
         获取单条记忆。
 
         Args:
             memory_id: 记忆 ID
-            include_raw: 是否允许返回 L1_RAW 原始记忆。默认 False，保持原有隐藏行为；
-                         lite 模式的记忆均为 L1_RAW，需显式设为 True 才能按 ID 读取。
+            kind: 可选 namespace（"chat" / "experience" / "fact"）。
+            variant: 可选生产力变体（"working" / "coding"）。
 
         Returns:
-            {"memory_id", "content", "layer", "request_id", "elapsed_ms"} 或 None（不存在时）
+            带 kind/variant 的统一 memory dict，或 None（不存在时）。
         """
         return self._loop_thread.run(
-            self.async_get(memory_id, include_raw=include_raw)
+            self.async_get(
+                memory_id,
+                kind=kind,
+                variant=variant,
+            )
         )
 
     def update(
@@ -1912,6 +1974,9 @@ class HyMemoryClient:
         branch: Optional[str] = None,
         user_ids: Optional[List[str]] = None,
         agent_ids: Optional[List[str]] = None,
+        session_ids: Optional[List[str]] = None,
+        include_sources: Optional[List[str]] = None,
+        exclude_sources: Optional[List[str]] = None,
         layers: Optional[List[Any]] = None,
         statuses: Optional[List[Any]] = None,
         with_evolution: bool = False,
@@ -1934,6 +1999,9 @@ class HyMemoryClient:
             user_ids: 可选，多用户批量列举。非空时优先于 user_id，返回并集
                       （分页在合并排序后统一施加）。
             agent_ids: 可选，多 agent 批量列举。非空时优先于 agent_id。
+            session_ids: 可选，只返回指定 session。
+            include_sources: 可选，只返回指定来源。
+            exclude_sources: 可选，排除指定来源；与 include_sources 冲突时排除优先。
             layers:   可选，只返回指定层级（字符串或 MemoryLayer）。不传时返回
                       L1_RAW 以外的全部层级，与改动前一致。
             statuses: 可选，只返回指定生命周期（字符串或 MemoryStatus）。不传时
@@ -1949,6 +2017,9 @@ class HyMemoryClient:
             self.async_list_memories(
                 user_id=user_id, agent_id=agent_id,
                 user_ids=user_ids, agent_ids=agent_ids,
+                session_ids=session_ids,
+                include_sources=include_sources,
+                exclude_sources=exclude_sources,
                 limit=limit, offset=offset, order=order,
                 workspace_id=workspace_id, branch=branch,
                 layers=layers, statuses=statuses, with_evolution=with_evolution,
@@ -2737,6 +2808,8 @@ class HyMemoryClient:
                     request_id=request_id,
                     source=source,
                 )
+                # Public receipt always identifies the productivity namespace.
+                resp["variant"] = scene_variant
                 # 版本演进链层（scene="coding" 独有）。放在 writer.write 之后、返回
                 # 之前，因为它只依赖对话原文，与向量层写入结果无关。
                 # best-effort：这层挂不上不影响本次 add 的向量层产出。
@@ -2907,106 +2980,6 @@ class HyMemoryClient:
             include_runtime_diag=True,
         )
 
-    def import_raw(
-        self,
-        content: str,
-        *,
-        user_id: str,
-        agent_id: str = "default_agent",
-        session_id: str = "default_session",
-        source: str = "default",
-        memory_at: Optional[datetime] = None,
-        metadata: Optional[Dict[str, Any]] = None,
-        raw_id: Optional[str] = None,
-        request_id: Optional[str] = None,
-    ) -> Dict[str, Any]:
-        """Persist one exact raw record without extraction or reconciliation."""
-        return self._loop_thread.run(
-            self.async_import_raw(
-                content,
-                user_id=user_id,
-                agent_id=agent_id,
-                session_id=session_id,
-                source=source,
-                memory_at=memory_at,
-                metadata=metadata,
-                raw_id=raw_id,
-                request_id=request_id,
-            ),
-            include_runtime_diag=True,
-        )
-
-    def import_memory_exact(
-        self,
-        content: str,
-        *,
-        memory_id: str,
-        user_id: str,
-        agent_id: str = "default_agent",
-        session_id: str = "default_session",
-        source: str = "default",
-        layer: Any = "l2_fact",
-        status: Any = "active",
-        owner: Optional[str] = None,
-        tags: Optional[List[str]] = None,
-        supersedes: Optional[List[str]] = None,
-        superseded_by: Optional[List[str]] = None,
-        is_latest: bool = True,
-        memory_at: Optional[datetime] = None,
-        metadata: Optional[Dict[str, Any]] = None,
-        request_id: Optional[str] = None,
-    ) -> Dict[str, Any]:
-        """Import one existing semantic memory without extraction/reconcile."""
-        return self._loop_thread.run(
-            self.async_import_memory_exact(
-                content,
-                memory_id=memory_id,
-                user_id=user_id,
-                agent_id=agent_id,
-                session_id=session_id,
-                source=source,
-                layer=layer,
-                status=status,
-                owner=owner,
-                tags=tags,
-                supersedes=supersedes,
-                superseded_by=superseded_by,
-                is_latest=is_latest,
-                memory_at=memory_at,
-                metadata=metadata,
-                request_id=request_id,
-            ),
-            include_runtime_diag=True,
-        )
-
-    def get_raw(self, raw_id: str) -> Optional[Dict[str, Any]]:
-        """Read one SDK raw record through the explicit raw API."""
-        return self._loop_thread.run(self.async_get_raw(raw_id))
-
-    def list_raw(
-        self,
-        *,
-        user_id: str,
-        agent_id: Optional[str] = None,
-        session_id: Optional[str] = None,
-        source: Optional[str] = None,
-        limit: int = 100,
-        offset: int = 0,
-        order: str = "desc",
-    ) -> Dict[str, Any]:
-        """List raw records for an authorized SDK owner/scope."""
-        return self._loop_thread.run(
-            self.async_list_raw(
-                user_id=user_id,
-                agent_id=agent_id,
-                session_id=session_id,
-                source=source,
-                limit=limit,
-                offset=offset,
-                order=order,
-            )
-        )
-
     @_ensure_internal_loop
     async def async_add_extracted(
         self,
@@ -3159,316 +3132,6 @@ class HyMemoryClient:
         }
 
     @_ensure_internal_loop
-    async def async_import_raw(
-        self,
-        content: str,
-        *,
-        user_id: str,
-        agent_id: str = "default_agent",
-        session_id: str = "default_session",
-        source: str = "default",
-        memory_at: Optional[datetime] = None,
-        metadata: Optional[Dict[str, Any]] = None,
-        raw_id: Optional[str] = None,
-        request_id: Optional[str] = None,
-    ) -> Dict[str, Any]:
-        """Persist exact source content as L1_RAW without running a model."""
-        from .utils.source_filter import normalize_source
-
-        if not isinstance(content, str) or not content:
-            raise ValueError("content is required")
-        if not user_id:
-            raise ValueError("user_id is required")
-        request_id = request_id or str(uuid.uuid4())
-        replay = await self._claim_write_outcome(
-            request_id=request_id,
-            user_id=user_id,
-            agent_id=agent_id,
-            session_id=session_id,
-            operation="import_raw",
-        )
-        if replay is not None:
-            return replay
-        try:
-            result = await self._async_import_raw_impl(
-                content,
-                user_id=user_id,
-                agent_id=agent_id,
-                session_id=session_id,
-                source=normalize_source(source),
-                memory_at=memory_at,
-                metadata=metadata,
-                raw_id=raw_id,
-                request_id=request_id,
-            )
-        except BaseException:
-            await self._mark_write_outcome_unknown(request_id)
-            raise
-        await self._finish_write_outcome(request_id, "succeeded", result)
-        return result
-
-    async def _async_import_raw_impl(
-        self,
-        content: str,
-        *,
-        user_id: str,
-        agent_id: str,
-        session_id: str,
-        source: str,
-        memory_at: Optional[datetime],
-        metadata: Optional[Dict[str, Any]],
-        raw_id: Optional[str],
-        request_id: str,
-    ) -> Dict[str, Any]:
-        from .models.memory import MemoryLayer, MemoryNode, MemoryStatus, SourceType
-
-        node_id = raw_id or str(uuid.uuid4())
-        existing = await self._vector_store.get_by_id(node_id)
-        if existing is not None:
-            identity = (
-                existing.layer == MemoryLayer.L1_RAW,
-                existing.content == content,
-                existing.user_id == user_id,
-                existing.agent_id == agent_id,
-                existing.session_id == session_id,
-                existing.source == source,
-            )
-            if not all(identity):
-                raise ValueError("raw_id is already bound to different source content")
-            return {
-                "success": True,
-                "raw_id": node_id,
-                "request_id": request_id,
-                "replayed": True,
-            }
-
-        custom = dict(metadata or {})
-        custom["request_id"] = request_id
-        custom["input_kind"] = "raw_exact_import"
-        embedding = await self._embed_service.embed(content)
-        node = MemoryNode(
-            node_id=node_id,
-            user_id=user_id,
-            agent_id=agent_id,
-            session_id=session_id,
-            source=source,
-            layer=MemoryLayer.L1_RAW,
-            content=content,
-            source_type=SourceType.EXPLICIT,
-            status=MemoryStatus.ACTIVE,
-            is_latest=True,
-            embedding=embedding,
-            memory_at=memory_at,
-            custom=custom,
-        )
-        stored_id = await self._vector_store.upsert(node)
-        if stored_id != node_id:
-            raise RuntimeError("raw import changed the requested identity")
-        return {"success": True, "raw_id": node_id, "request_id": request_id}
-
-    @_ensure_internal_loop
-    async def async_import_memory_exact(
-        self,
-        content: str,
-        *,
-        memory_id: str,
-        user_id: str,
-        agent_id: str = "default_agent",
-        session_id: str = "default_session",
-        source: str = "default",
-        layer: Any = "l2_fact",
-        status: Any = "active",
-        owner: Optional[str] = None,
-        tags: Optional[List[str]] = None,
-        supersedes: Optional[List[str]] = None,
-        superseded_by: Optional[List[str]] = None,
-        is_latest: bool = True,
-        memory_at: Optional[datetime] = None,
-        metadata: Optional[Dict[str, Any]] = None,
-        request_id: Optional[str] = None,
-    ) -> Dict[str, Any]:
-        """Import a canonical memory node exactly, with no model invocation."""
-        from .models.memory import MemoryLayer, MemoryStatus
-        from .utils.source_filter import normalize_source
-
-        if not content or not memory_id or not user_id:
-            raise ValueError("content, memory_id and user_id are required")
-        normalized_layer = (
-            layer if isinstance(layer, MemoryLayer) else MemoryLayer.from_string(str(layer))
-        )
-        if normalized_layer == MemoryLayer.L1_RAW:
-            raise ValueError("use import_raw for L1_RAW")
-        normalized_status = (
-            status if isinstance(status, MemoryStatus) else MemoryStatus(str(status).lower())
-        )
-        request_id = request_id or str(uuid.uuid4())
-        replay = await self._claim_write_outcome(
-            request_id=request_id,
-            user_id=user_id,
-            agent_id=agent_id,
-            session_id=session_id,
-            operation="import_memory_exact",
-        )
-        if replay is not None:
-            return replay
-        try:
-            result = await self._async_import_memory_exact_impl(
-                content,
-                memory_id=memory_id,
-                user_id=user_id,
-                agent_id=agent_id,
-                session_id=session_id,
-                source=normalize_source(source),
-                layer=normalized_layer,
-                status=normalized_status,
-                owner=owner,
-                tags=tags or [],
-                supersedes=supersedes or [],
-                superseded_by=superseded_by or [],
-                is_latest=is_latest,
-                memory_at=memory_at,
-                metadata=metadata,
-                request_id=request_id,
-            )
-        except BaseException:
-            await self._mark_write_outcome_unknown(request_id)
-            raise
-        await self._finish_write_outcome(request_id, "succeeded", result)
-        return result
-
-    async def _async_import_memory_exact_impl(
-        self,
-        content: str,
-        *,
-        memory_id: str,
-        user_id: str,
-        agent_id: str,
-        session_id: str,
-        source: str,
-        layer,
-        status,
-        owner: Optional[str],
-        tags: List[str],
-        supersedes: List[str],
-        superseded_by: List[str],
-        is_latest: bool,
-        memory_at: Optional[datetime],
-        metadata: Optional[Dict[str, Any]],
-        request_id: str,
-    ) -> Dict[str, Any]:
-        from .models.memory import MemoryNode, SourceType
-
-        existing = await self._vector_store.get_by_id(memory_id)
-        if existing is not None:
-            identity = (
-                existing.content == content,
-                existing.user_id == user_id,
-                existing.agent_id == agent_id,
-                existing.session_id == session_id,
-                existing.source == source,
-                existing.layer == layer,
-                existing.status == status,
-                existing.owner == owner,
-                list(existing.tags or []) == tags,
-                list(existing.supersedes or []) == supersedes,
-                list(existing.superseded_by or []) == superseded_by,
-                existing.is_latest == is_latest,
-            )
-            if not all(identity):
-                raise ValueError("memory_id is already bound to different memory content")
-            return {
-                "success": True,
-                "memory_id": memory_id,
-                "memory_ids": [memory_id],
-                "request_id": request_id,
-                "replayed": True,
-            }
-
-        custom = dict(metadata or {})
-        custom["request_id"] = request_id
-        custom["input_kind"] = "memory_exact_import"
-        embedding = await self._embed_service.embed(content)
-        node = MemoryNode(
-            node_id=memory_id,
-            user_id=user_id,
-            agent_id=agent_id,
-            session_id=session_id,
-            source=source,
-            layer=layer,
-            content=content,
-            source_type=SourceType.EXPLICIT,
-            status=status,
-            owner=owner,
-            tags=list(tags),
-            supersedes=list(supersedes),
-            superseded_by=list(superseded_by),
-            is_latest=is_latest,
-            embedding=embedding,
-            memory_at=memory_at,
-            custom=custom,
-        )
-        stored_id = await self._vector_store.upsert(node)
-        if stored_id != memory_id:
-            raise RuntimeError("exact memory import changed the requested identity")
-        return {
-            "success": True,
-            "memory_id": memory_id,
-            "memory_ids": [memory_id],
-            "request_id": request_id,
-        }
-
-    @_ensure_internal_loop
-    async def async_get_raw(self, raw_id: str) -> Optional[Dict[str, Any]]:
-        from .models.memory import MemoryLayer
-
-        node = await self._vector_store.get_by_id(raw_id)
-        if node is None or node.layer != MemoryLayer.L1_RAW:
-            return None
-        return self._raw_node_to_public(node)
-
-    @_ensure_internal_loop
-    async def async_list_raw(
-        self,
-        *,
-        user_id: str,
-        agent_id: Optional[str] = None,
-        session_id: Optional[str] = None,
-        source: Optional[str] = None,
-        limit: int = 100,
-        offset: int = 0,
-        order: str = "desc",
-    ) -> Dict[str, Any]:
-        from .models.memory import MemoryLayer, MemoryStatus
-
-        if not user_id:
-            raise ValueError("user_id is required")
-        if limit < 1 or offset < 0:
-            raise ValueError("limit must be positive and offset must be non-negative")
-        nodes = await self._vector_store.list_by_user(
-            user_id=user_id,
-            agent_id=agent_id,
-            status_filter=[MemoryStatus.ACTIVE],
-            layers=[MemoryLayer.L1_RAW],
-            include_sources=[source] if source else None,
-            limit=100_000,
-            only_latest=False,
-            with_vectors=False,
-            allow_raw_layer=True,
-        )
-        if session_id:
-            nodes = [node for node in nodes if node.session_id == session_id]
-        nodes = self._sort_memory_nodes(nodes, order=order)
-        total = len(nodes)
-        return {
-            "raw": [
-                self._raw_node_to_public(node)
-                for node in nodes[offset: offset + limit]
-            ],
-            "total": total,
-            "limit": limit,
-            "offset": offset,
-        }
-
     async def _search_explicit_raw_layer(
         self,
         query: str,
@@ -4374,35 +4037,100 @@ class HyMemoryClient:
     # CRUD API（异步）— get / update / delete / delete_all
     # ================================================================
 
+    async def _coding_store_for_exact_get(self, variant: str):
+        """Return an existing namespace store without creating an empty DB."""
+        stores = getattr(self, "_coding_stores", None) or {}
+        store = stores.get(variant)
+        if store is not None:
+            return store
+
+        from .coding.store import CodingMemoryStore
+
+        config = getattr(self, "_config", None)
+        coding_config = getattr(config, "coding", None)
+        if config is None or coding_config is None:
+            return None
+        if not CodingMemoryStore.database_exists(
+            config,
+            db_path=coding_config.db_path,
+            variant=variant,
+        ):
+            return None
+        await self._get_coding_writer(variant=variant)
+        return (getattr(self, "_coding_stores", None) or {}).get(variant)
+
+    @staticmethod
+    def _format_productivity_get(
+        item: Any,
+        *,
+        kind: str,
+        variant: str,
+        request_id: str,
+        elapsed_ms: float,
+    ) -> Dict[str, Any]:
+        payload = item.to_dict()
+        item_id = (
+            payload.get("experience_id")
+            if kind == "experience"
+            else payload.get("fact_id")
+        )
+        return {
+            "memory_id": item_id,
+            "kind": kind,
+            "variant": variant,
+            "scene": "productivity",
+            "content": (
+                payload.get("solution", "")
+                if kind == "experience"
+                else payload.get("statement", "")
+            ),
+            "layer": kind,
+            **payload,
+            "request_id": request_id,
+            "elapsed_ms": elapsed_ms,
+        }
+
     @_ensure_internal_loop
     async def async_get(
         self,
         memory_id: str,
         *,
-        include_raw: bool = False,
+        kind: Optional[MemoryKind] = None,
+        variant: Optional[MemoryVariant] = None,
     ) -> Optional[Dict[str, Any]]:
         """
         获取单条记忆（异步）。
 
-        VDB + Graph 双路并行查询，哪个有结果返回哪个。
+        默认查询 chat、working/coding experience 和 working/coding fact。
+        若同一 ID 命中多个 namespace，则 fail closed；调用方必须用 kind/variant
+        显式消歧。
 
         Args:
             memory_id: 记忆 ID
-            include_raw: 是否允许返回 L1_RAW 原始记忆，默认 False。
+            kind: 可选 namespace（"chat" / "experience" / "fact"）。
+            variant: 可选生产力变体（"working" / "coding"）。
 
         Returns:
-            {"memory_id", "content", "layer", "source", "request_id", "elapsed_ms"} 或 None
+            统一 memory dict，或 None。
         """
-        import asyncio as _asyncio_get
-        from .models.memory import MemoryLayer
+        if kind not in (None, "chat", "experience", "fact"):
+            raise ValueError(
+                "kind must be one of: chat, experience, fact"
+            )
+        if variant not in (None, "working", "coding"):
+            raise ValueError("variant must be one of: working, coding")
+        if kind == "chat" and variant is not None:
+            raise ValueError("variant is only valid for experience/fact lookups")
 
         request_id = str(uuid.uuid4())
         set_request_id(request_id)
         t0 = time.perf_counter()
 
-        logger.info(f"[get] memory_id={memory_id}")
+        logger.info(
+            "[get] memory_id=%s kind=%s variant=%s",
+            memory_id, kind, variant,
+        )
 
-        # 双路并行
         async def _get_vdb():
             return await self._vector_store.get_by_id(memory_id)
 
@@ -4415,39 +4143,63 @@ class HyMemoryClient:
                 logger.debug(f"[get] graph_store query failed: {e}")
                 return None
 
-        vdb_node, graph_node = await _asyncio_get.gather(_get_vdb(), _get_graph())
+        query_chat = kind in (None, "chat") and variant is None
+        coding_kinds = (
+            ("experience", "fact") if kind is None else (kind,)
+        )
+        coding_variants = (variant,) if variant is not None else ("working", "coding")
 
-        # L1_RAW 默认不对外可见：按 ID 直取走主键查询、绕过向量库的 layer
-        # filter，故在此统一挡掉。SDK 调用方可通过 include_raw=True 显式读取；
-        # HTTP 接口不传该参数，继续保持隐藏。内部写链路不经过此方法。
-        #
-        # 置于 gather 之后、埋点之前：命中 raw 仍照常落 GET 埋点（保留可观测性，
-        # 便于发现有调用方拿 raw id 试探），只是不把内容返回给调用方。
-        # 对调用方的表现与"该 id 不存在"完全一致，不额外暴露"存在但被挡"的信息。
-        if (
-            not include_raw
-            and vdb_node is not None
-            and vdb_node.layer == MemoryLayer.L1_RAW
-        ):
-            vdb_node = None
-        if (
-            not include_raw
-            and graph_node is not None
-            and getattr(graph_node, "layer", None) == MemoryLayer.L1_RAW
-        ):
-            graph_node = None
+        stores = {}
+        if kind != "chat":
+            for candidate_variant in coding_variants:
+                store = await self._coding_store_for_exact_get(candidate_variant)
+                if store is not None:
+                    stores[candidate_variant] = store
+
+        tasks = []
+        task_names = []
+        if query_chat:
+            tasks.extend((_get_vdb(), _get_graph()))
+            task_names.extend(("chat:vdb", "chat:graph"))
+        for candidate_variant, store in stores.items():
+            if "experience" in coding_kinds:
+                tasks.append(store.get_by_id(memory_id))
+                task_names.append(f"{candidate_variant}:experience")
+            if "fact" in coding_kinds:
+                tasks.append(store.get_fact(memory_id))
+                task_names.append(f"{candidate_variant}:fact")
+
+        values = await asyncio.gather(*tasks) if tasks else []
+        by_name = dict(zip(task_names, values))
+        vdb_node = by_name.get("chat:vdb")
+        graph_node = by_name.get("chat:graph")
+
+        matches = []
+        chat_node = vdb_node if vdb_node is not None else graph_node
+        if chat_node is not None:
+            matches.append(("chat", chat_node))
+        for name, value in by_name.items():
+            if not name.startswith("chat:") and value is not None:
+                matches.append((name, value))
+
+        if len(matches) > 1:
+            raise AmbiguousMemoryIdError(
+                memory_id, [namespace for namespace, _ in matches]
+            )
 
         elapsed = round((time.perf_counter() - t0) * 1000, 2)
-
-        # pipeline trace（best-effort，受 MEMORY_PIPELINE_TRACE_ENABLED 控制）
-        _hit_node = vdb_node if vdb_node is not None else graph_node
-        _source = "vdb" if vdb_node is not None else ("graph" if graph_node is not None else "")
+        _hit_namespace, _hit_node = matches[0] if matches else ("", None)
+        _source = (
+            "vdb" if vdb_node is not None
+            else ("graph" if graph_node is not None else _hit_namespace)
+        )
         _ident = self._backend_identity()
         _parsed = {
             "memory_id": memory_id,
             **_ident,
             "hit": _hit_node is not None,
             "source": _source,
+            "namespace": _hit_namespace,
         }
         if _hit_node is not None:
             try:
@@ -4464,10 +4216,25 @@ class HyMemoryClient:
             elapsed_ms=elapsed,
         )
 
-        # VDB 优先（如果两边都有，以 VDB 为准）
+        if not matches:
+            logger.info(f"[get] not found: memory_id={memory_id}")
+            return None
+
+        if _hit_namespace != "chat":
+            hit_variant, hit_kind = _hit_namespace.split(":", 1)
+            return self._format_productivity_get(
+                _hit_node,
+                kind=hit_kind,
+                variant=hit_variant,
+                request_id=request_id,
+                elapsed_ms=elapsed,
+            )
+
         if vdb_node is not None:
             return {
                 "memory_id": vdb_node.node_id,
+                "kind": "chat",
+                "variant": None,
                 "content": vdb_node.content,
                 "layer": vdb_node.layer.value,
                 "status": vdb_node.status.value if vdb_node.status else "active",
@@ -4481,10 +4248,13 @@ class HyMemoryClient:
                 # 不透出 source_raw_memory_id：见 async_search 同处说明。
                 "user_id": vdb_node.user_id,
                 "agent_id": vdb_node.agent_id,
+                "session_id": vdb_node.session_id,
+                "owner": vdb_node.owner,
                 "confidence": vdb_node.confidence,
                 "custom": vdb_node.custom or {},
                 "meta": list((vdb_node.custom or {}).get("meta") or []),
-                "source": "vdb",
+                "source": vdb_node.source,
+                "retrieval_source": "vdb",
                 "request_id": request_id,
                 "elapsed_ms": elapsed,
             }
@@ -4492,6 +4262,8 @@ class HyMemoryClient:
         if graph_node is not None:
             return {
                 "memory_id": graph_node.node_id,
+                "kind": "chat",
+                "variant": None,
                 "content": graph_node.content,
                 "layer": graph_node.layer.value if graph_node.layer else "",
                 "status": graph_node.status.value if graph_node.status else "active",
@@ -4505,14 +4277,16 @@ class HyMemoryClient:
                 # 不透出 source_raw_memory_id：见 async_search 同处说明。
                 "user_id": graph_node.user_id,
                 "agent_id": graph_node.agent_id,
+                "session_id": getattr(graph_node, "session_id", "") or "",
+                "owner": getattr(graph_node, "owner", None),
                 "confidence": graph_node.confidence,
                 "custom": getattr(graph_node, "custom", {}) or {},
-                "source": "graph",
+                "source": getattr(graph_node, "source", "") or "",
+                "retrieval_source": "graph",
                 "request_id": request_id,
                 "elapsed_ms": elapsed,
             }
 
-        logger.info(f"[get] not found: memory_id={memory_id}")
         return None
 
     @_ensure_internal_loop
@@ -5081,6 +4855,7 @@ class HyMemoryClient:
             "user_id": node.user_id,
             "agent_id": node.agent_id,
             "session_id": getattr(node, "session_id", None),
+            "source": getattr(node, "source", None),
             "speculate": getattr(node, "speculate", None),
             # 不透出 source_raw_memory_id：见 async_search 同处说明。
             "tags": node.tags or [],
@@ -5152,6 +4927,9 @@ class HyMemoryClient:
         order: str,
         user_ids: Optional[List[str]] = None,
         agent_ids: Optional[List[str]] = None,
+        session_ids: Optional[List[str]] = None,
+        include_sources: Optional[List[str]] = None,
+        exclude_sources: Optional[List[str]] = None,
         layers: Optional[List] = None,
         statuses: Optional[List] = None,
         with_evolution: bool = False,
@@ -5177,6 +4955,9 @@ class HyMemoryClient:
             status_filter=effective_statuses,
             layers=layers,
             with_vectors=False,
+            allow_raw_layer=bool(
+                layers is not None and MemoryLayer.L1_RAW in layers
+            ),
         )
 
         # L0 是 user-level，不应被 agent 过滤。默认列表或显式包含 L0 时，额外按
@@ -5198,6 +4979,22 @@ class HyMemoryClient:
             ])
             all_nodes.extend(hit["node"] for hit in l0_hits if hit.get("node") is not None)
 
+        if session_ids:
+            wanted_sessions = set(session_ids)
+            all_nodes = [
+                node for node in all_nodes if node.session_id in wanted_sessions
+            ]
+        if include_sources:
+            wanted_sources = set(include_sources)
+            all_nodes = [
+                node for node in all_nodes if node.source in wanted_sources
+            ]
+        if exclude_sources:
+            blocked_sources = set(exclude_sources)
+            all_nodes = [
+                node for node in all_nodes if node.source not in blocked_sources
+            ]
+
         all_nodes = self._sort_memory_nodes(all_nodes, order=order)
         total = len(all_nodes)
         page_nodes = all_nodes[offset: offset + limit]
@@ -5261,6 +5058,9 @@ class HyMemoryClient:
         order: str,
         user_ids: Optional[List[str]] = None,
         agent_ids: Optional[List[str]] = None,
+        session_ids: Optional[List[str]] = None,
+        include_sources: Optional[List[str]] = None,
+        exclude_sources: Optional[List[str]] = None,
         layers: Optional[List] = None,
         statuses: Optional[List] = None,
     ) -> Optional[Dict[str, Any]]:
@@ -5308,6 +5108,21 @@ class HyMemoryClient:
                 node for node in graph_nodes
                 if getattr(node, "status", None) in effective_statuses
             ]
+            if session_ids:
+                wanted_sessions = set(session_ids)
+                graph_nodes = [
+                    node for node in graph_nodes if node.session_id in wanted_sessions
+                ]
+            if include_sources:
+                wanted_sources = set(include_sources)
+                graph_nodes = [
+                    node for node in graph_nodes if node.source in wanted_sources
+                ]
+            if exclude_sources:
+                blocked_sources = set(exclude_sources)
+                graph_nodes = [
+                    node for node in graph_nodes if node.source not in blocked_sources
+                ]
 
             graph_nodes = self._sort_memory_nodes(graph_nodes, order=order)
             total = len(graph_nodes)
@@ -5419,6 +5234,9 @@ class HyMemoryClient:
         branch: Optional[str] = None,
         user_ids: Optional[List[str]] = None,
         agent_ids: Optional[List[str]] = None,
+        session_ids: Optional[List[str]] = None,
+        include_sources: Optional[List[str]] = None,
+        exclude_sources: Optional[List[str]] = None,
         layers: Optional[List[Any]] = None,
         statuses: Optional[List[Any]] = None,
         with_evolution: bool = False,
@@ -5495,6 +5313,9 @@ class HyMemoryClient:
         vdb = await self._list_vdb_bucket(
             user_id=user_id, agent_id=agent_id,
             user_ids=user_ids, agent_ids=agent_ids,
+            session_ids=session_ids,
+            include_sources=include_sources,
+            exclude_sources=exclude_sources,
             limit=limit, offset=offset, order=order,
             layers=effective_layers, statuses=effective_statuses,
             with_evolution=with_evolution,
@@ -5502,6 +5323,9 @@ class HyMemoryClient:
         graph = await self._list_graph_bucket(
             user_id=user_id, agent_id=agent_id,
             user_ids=user_ids, agent_ids=agent_ids,
+            session_ids=session_ids,
+            include_sources=include_sources,
+            exclude_sources=exclude_sources,
             limit=limit, offset=offset, order=order,
             layers=effective_layers, statuses=effective_statuses,
         )

hy_memory/coding/store.py

@@ -416,6 +416,36 @@ class CodingMemoryStore:
 
         self._initialized = False
 
+    @staticmethod
+    def database_path(
+        config: MemoryConfig,
+        db_path: Optional[str] = None,
+        variant: str = "working",
+    ) -> str:
+        """Return the authoritative SQLite path without creating anything."""
+        if db_path is None:
+            data_dir = (
+                os.path.dirname(config.history.db_path)
+                if config.history.db_path else "./data"
+            )
+            db_path = os.path.join(data_dir, "coding_memory.db")
+        normalized = (
+            "coding"
+            if str(variant or "").strip().lower() == "coding"
+            else "working"
+        )
+        return _variant_db_path(db_path, "_cv" if normalized == "coding" else "")
+
+    @classmethod
+    def database_exists(
+        cls,
+        config: MemoryConfig,
+        db_path: Optional[str] = None,
+        variant: str = "working",
+    ) -> bool:
+        """Whether a coding namespace has an existing SQLite database."""
+        return os.path.isfile(cls.database_path(config, db_path, variant))
+
     async def initialize(self) -> None:
         if self._initialized:
             return

hy_memory/coding/writer.py

@@ -50,6 +50,7 @@ class CodingWriter:
         embed_service: EmbedService,
         fact_reconciler: Optional[WorkingFactReconciler] = None,
         cache: Any = None,
+        variant: str = "working",
     ):
         self.store = store
         self.extractor = extractor
@@ -59,6 +60,11 @@ class CodingWriter:
         self.fact_reconciler = fact_reconciler or WorkingFactReconciler(
             store, embed_service, llm_provider
         )
+        self.variant = (
+            "coding"
+            if str(variant or "").strip().lower() == "coding"
+            else "working"
+        )
         # cache 仅用于把 request_id 串给各阶段（各阶段自己打点）；writer 不集中打点。
         self.cache = cache
 
@@ -217,6 +223,7 @@ class CodingWriter:
         return {
             "success": True,
             "scene": "productivity",
+            "variant": self.variant,
             "request_id": request_id,
             "elapsed_ms": round((time.perf_counter() - t0) * 1000, 2),
             "ops": [op.to_dict() for op in ops],
@@ -278,11 +285,11 @@ class CodingWriter:
                         f"[coding-write] crosslink fact {f.fact_id} failed: {e}"
                     )
 
-    @staticmethod
-    def _empty_response(request_id: str, t0: float) -> Dict[str, Any]:
+    def _empty_response(self, request_id: str, t0: float) -> Dict[str, Any]:
         return {
             "success": True,
             "scene": "productivity",
+            "variant": self.variant,
             "request_id": request_id,
             "elapsed_ms": round((time.perf_counter() - t0) * 1000, 2),
             "ops": [],

hy_memory/embedding.py

@@ -0,0 +1,53 @@
+"""Public structural contract for injected embedding services."""
+
+from typing import Any, Dict, List, Protocol, Tuple, runtime_checkable
+
+
+@runtime_checkable
+class EmbeddingService(Protocol):
+    """Embedding service accepted by :class:`HyMemoryClient`.
+
+    Implementations are borrowed by the client: their lifecycle remains owned
+    by the caller and ``HyMemoryClient.close()`` will not close them.
+    """
+
+    async def embed(self, text: str, use_cache: bool = True) -> List[float]:
+        ...
+
+    async def embed_with_meta(
+        self, text: str, use_cache: bool = True
+    ) -> Tuple[List[float], Dict[str, Any]]:
+        ...
+
+    async def embed_batch(
+        self, texts: List[str], use_cache: bool = True
+    ) -> List[List[float]]:
+        ...
+
+    async def embed_queued(
+        self, text: str, use_cache: bool = True
+    ) -> List[float]:
+        ...
+
+    async def embed_queued_with_meta(
+        self, text: str, use_cache: bool = True
+    ) -> Tuple[List[float], Dict[str, Any]]:
+        ...
+
+    def get_embedding_dims(self) -> int:
+        ...
+
+
+# Explicit alias for callers that prefer the conventional Protocol suffix.
+EmbeddingServiceProtocol = EmbeddingService
+
+
+class EmbeddingDimensionMismatchError(ValueError):
+    """Raised before initialization when service and store dimensions differ."""
+
+
+__all__ = [
+    "EmbeddingService",
+    "EmbeddingServiceProtocol",
+    "EmbeddingDimensionMismatchError",
+]

hy_memory/pipelines/registry.py

@@ -19,6 +19,7 @@ import logging
 from .base import WritePipeline, ReadPipeline
 from ..config import MemoryConfig
 from ..core.embed_service import EmbedService
+from ..embedding import EmbeddingService
 from ..data.vector_store import create_vector_store
 from ..data.vector_store_base import VectorStoreBase
 
@@ -50,7 +51,12 @@ class ComponentFactory:
         s2 = await factory.get_system2_writer()    # System2Writer (ultra mode)
     """
 
-    def __init__(self, config: Optional[MemoryConfig] = None):
+    def __init__(
+        self,
+        config: Optional[MemoryConfig] = None,
+        *,
+        embedding_service: Optional[EmbeddingService] = None,
+    ):
         self.config = config or MemoryConfig.from_env()
 
         # 实例缓存（单例）
@@ -62,7 +68,7 @@ class ComponentFactory:
         self._readers: Dict[str, ReadPipeline] = {}
 
         # 共享资源
-        self._shared_embed_service: Optional[EmbedService] = None
+        self._shared_embed_service: Optional[EmbeddingService] = embedding_service
         self._shared_vector_store: Optional[VectorStoreBase] = None
         self._shared_vector_store_initialized: bool = False
         self._shared_graph_store = None
@@ -72,7 +78,7 @@ class ComponentFactory:
     # 共享资源
     # ================================================================
 
-    def _get_shared_embed_service(self) -> EmbedService:
+    def _get_shared_embed_service(self) -> EmbeddingService:
         if self._shared_embed_service is None:
             self._shared_embed_service = EmbedService(self.config)
         return self._shared_embed_service

hy_memory/retrieval.py

@@ -0,0 +1,23 @@
+"""Public types and errors for exact memory-ID lookup."""
+
+from typing import Literal, Sequence
+
+
+MemoryKind = Literal["chat", "experience", "fact"]
+MemoryVariant = Literal["working", "coding"]
+
+
+class AmbiguousMemoryIdError(LookupError):
+    """An ID exists in more than one public memory namespace."""
+
+    def __init__(self, memory_id: str, namespaces: Sequence[str]):
+        self.memory_id = memory_id
+        self.namespaces = tuple(namespaces)
+        joined = ", ".join(self.namespaces)
+        super().__init__(
+            f"memory ID {memory_id!r} is ambiguous across namespaces: {joined}; "
+            "pass kind and/or variant to disambiguate"
+        )
+
+
+__all__ = ["MemoryKind", "MemoryVariant", "AmbiguousMemoryIdError"]

hy_memory/utils/lang_detect.py

@@ -8,6 +8,7 @@ Uses a two-tier strategy:
 This avoids langdetect's known weakness with short Chinese text.
 """
 
+import json
 import logging
 import re
 from typing import List, Optional
@@ -135,6 +136,30 @@ def detect_language(text: str) -> str:
     return "zh" if ratio > 0.1 else "en"
 
 
+def _spoken_text(value: str) -> str:
+    """Ignore text-block metadata when a message content is structured JSON."""
+    text = value.strip()
+    if text.startswith("["):
+        try:
+            blocks = json.loads(text)
+        except (ValueError, TypeError):
+            pass
+        else:
+            if (
+                isinstance(blocks, list)
+                and blocks
+                and all(
+                    isinstance(block, dict)
+                    and isinstance(block.get("type"), str)
+                    and block["type"] in {"text", "input_text", "output_text"}
+                    and isinstance(block.get("text"), str)
+                    for block in blocks
+                )
+            ):
+                return "\n".join(block["text"] for block in blocks)
+    return text
+
+
 def extract_content_for_detection(
     content: str = "",
     messages: Optional[List] = None,
@@ -153,20 +178,57 @@ def extract_content_for_detection(
         Concatenated content text suitable for language detection.
     """
     if content and content.strip():
-        return content.strip()
+        value = content.strip()
+        # Writer serializes message batches to JSON before calling Extractor.
+        # Detect the spoken text, not the leading ASCII keys (role/content),
+        # without changing the raw evidence sent to the model.
+        if value.startswith("["):
+            try:
+                decoded = json.loads(value)
+            except (ValueError, TypeError):
+                pass
+            else:
+                if (
+                    isinstance(decoded, list)
+                    and decoded
+                    and all(
+                        isinstance(item, dict)
+                        and isinstance(item.get("role"), str)
+                        and isinstance(item.get("content"), str)
+                        for item in decoded
+                    )
+                ):
+                    # In a mixed batch, the latest user turn is the clearest
+                    # instruction for the language; assistant/tool metadata
+                    # must not decide it first.
+                    user_text = [
+                        _spoken_text(item["content"])
+                        for item in decoded
+                        if item["role"] == "user" and item["content"].strip()
+                    ]
+                    if user_text:
+                        return user_text[-1]
+                    spoken = "\n".join(_spoken_text(item["content"]) for item in decoded)
+                    if spoken.strip():
+                        return spoken
+        return _spoken_text(value)
 
     if messages:
         parts = []
+        user_parts = []
         for msg in messages:
             if isinstance(msg, dict):
-                c = msg.get("content", "")
+                c, role = msg.get("content", ""), msg.get("role")
             elif hasattr(msg, "content"):
-                c = msg.content
+                c, role = msg.content, getattr(msg, "role", None)
             else:
                 continue
-            if c and c.strip():
-                parts.append(c.strip())
-        return "\n".join(parts)
+            if isinstance(c, str) and c.strip():
+                text = _spoken_text(c)
+                parts.append(text)
+                if role == "user":
+                    user_parts.append(text)
+        return user_parts[-1] if user_parts else "\n".join(parts)
 
     return ""
 

memhub/MEMORY.md

@@ -9,10 +9,15 @@
 ## Architecture Principles
 
 1. Memory 读写与存储 backend **只经 SDK**（config / env）。
-2. MemHub 对 memory 的原子操作 **只经 SDK**；缺接口先补 SDK。
+2. MemHub 运行时对 memory 的原子操作 **只经 SDK 公共 Client**；缺正常业务
+   接口先补 SDK。Runtime writers 停止后的 App 启动迁移可直接操作底层存储，
+   但不得把迁移专用方法加入公共 Client。
 3. MemHub **不存 memory 实体**；**可以存 `memory_id`** 做组织/回执。原文进入 **SDK raw**。
+4. SDK raw 只保留滚动三天；一条 raw 对应一个完整 turn，只含真人 user 与
+   assistant，tool 不持久化。提取后的高层 memory 不受该窗口影响。
 
-生产入口仍建议经 `HyMemoryBridge`（共享 Runtime、模型配置、密钥、harness/model 门禁），但 bridge **不得**再把 SDK 结果镜像成第二套 memory 正文权威。
+生产入口通过共享 Runtime 租约取得公开 `HyMemoryClient`；MemHub 只在调用前执行
+模型配置、密钥和 harness/model 门禁，不维护第二套 memory 正文权威。
 
 来源授权、模型授权和宿主召回授权仍是 MemHub 管理面门禁；memory
 数据平面的权威在 SDK。
@@ -30,46 +35,15 @@ reader:       hybrid
 
 ## MemHub 运行时边界
 
-MemHub 对 SDK 使用公开、可复现的本地策略（经 bridge / `hy_runtime_config` 注入）：
+MemHub 通过公开 `HyMemoryClient` 配置使用可复现的本地策略：
 
 - vector / graph / cache / history / coding 等 **一律 SDK config/env**；
 - 模型与 embedding 来自版本化 Model Profile；密钥进安全存储；
 - MemHub 管理库可存 harness/model/队列状态与 **`memory_id` 引用**。
 
-发布包默认 SDK 行为环境变量仍集中在
-`apps/core/src/one_memory/hy_runtime_config.py`，并由下表锁定：
-
-| 变量 | 值 |
-|---|---|
-| `MEMORY_MODE` | `ultra` |
-| `MEMORY_ENABLE_GRAPH` | `true` |
-| `MEMORY_GRAPH_RERANK_ENABLED` | `false` |
-| `MEMORY_GRAPH_TEMPORAL_SPINE` | `true` |
-| `MEMORY_TOPIC_SUMMARY_ENABLED` | `false` |
-| `MEMORY_EPISODE_SUMMARY_ENABLED` | `false` |
-| `MEMORY_EPISODE_RECALL_ENABLED` | `true` |
-| `MEMORY_DYNAMIC_EPISODE_ENABLED` | `true` |
-| `MEMORY_SESSION_EPISODE_ENABLED` | `false` |
-| `MEMORY_SESSION_EPISODE_CONTEXT_ENABLED` | `false` |
-| `MEMORY_EPISODE_CAPSULE_CONTEXT_ENABLED` | `true` |
-| `MEMORY_EPISODE_CAPSULE_QUERY_MIN` | `0.65` |
-| `MEMORY_EPISODE_CAPSULE_ASSIGNMENT_MIN` | `0.60` |
-| `MEMORY_EPISODE_CAPSULE_FIRST_RANK_MAX` | `3` |
-| `MEMORY_EPISODE_CAPSULE_MEMBER_LIMIT` | `3` |
-| `MEMORY_ENABLE_INTENTION` | `false` |
-| `MEMORY_ENABLE_QUERY_REWRITE` | `false` |
-| `MEMORY_FEW_SHOT_ENABLED` | `true` |
-| `MEMORY_HYBRID_V2_QUERY_EXPAND` | `1` |
-| `MEMORY_HYBRID_V2_GEO_INTENT_PRIOR` | `0.6` |
-| `MEMORY_HYBRID_V2_DIVERSITY` | `1` |
-| `MEMORY_HYBRID_V2_TEMPORAL_ADMIT_PRIOR` | `0.55` |
-| `MEMORY_HYBRID_V2_TEMPORAL_BOOST` | `0.18` |
-| `MEMORY_CROSS_DOMAIN_CON_THRESHOLD` | `0.5` |
-| `MEMORY_READER_ENABLE_SUMMARY` | `false` |
-| `MEMORY_SUMMARY_ENABLED_IN_SYS2` | `false` |
-| `MEMORY_LOCAL_RERANK_ENABLED` | `false` |
-| `RECONCILE_ENABLED` | `true` |
-| `HY_MEMORY_READER` | `hybrid` |
+MemHub 不再通过 import-time 环境变量覆盖 SDK 的提取、reconcile、episode、
+intention、few-shot、summary 或去重策略。产品只配置模型/密钥/endpoint、本地
+backend 路径、公开 embedding service，以及每次公开 Client 调用的参数。
 
 独立脚本直连 SDK 时须自配存储目录与模型端点。
 
@@ -77,7 +51,7 @@ MemHub 对 SDK 使用公开、可复现的本地策略（经 bridge / `hy_runtim
 
 下面是独立调试示例。
 
-产品代码经 `HyMemoryBridge` 获取已配置的 client（管理面门禁），memory I/O
+产品代码经 `hy_sdk_runtime.SdkClientHandle` 获取已配置的公开 client；memory I/O
 由 SDK 完成。
 
 ```python
@@ -228,7 +202,8 @@ result = client.search(
 - 区分空结果、权限拒绝、不可用、超时、部分结果；
 - 未发布的 Matter 占位功能不保留表或 API；未来组织功能另立合同。
 
-生产入口：`hy_memory_bridge.py`、`recall.py`、`packages/host-bridge`。
+生产入口：`hy_sdk_runtime.py`、`sdk_write_processor.py`、`recall.py` 和
+`packages/host-bridge`。
 
 ## 关闭客户端
 
@@ -245,8 +220,7 @@ Runtime 长期复用 client；仅在停止、配置切换或受控恢复时重
 
 ```text
 harness / user action (+ authz, model, queue)
-  -> HyMemoryBridge
-  -> Hy Memory SDK
+  -> public HyMemoryClient
   -> SDK stores (config/env)
   -> optional MemHub persist memory_id / org metadata
   -> Recall Broker / UI (SDK read + MemHub authz)

memhub/README.md

@@ -18,13 +18,13 @@ If you run into a problem, please open an Issue with the version, platform,repro
 
 ## 发行与平台合同
 
-源码当前固定使用公开 PyPI 开发预览 `hy-memory==2.0.5.dev3`。MemHub 源码版本为 `1.0.4`
-（build 13），GitHub 仓库为
+源码当前固定使用公开 PyPI 正式版 `hy-memory==2.0.5`。MemHub 源码版本为 `1.0.5`
+（build 14），GitHub 仓库为
 [`Tencent-Hunyuan/Hy-Memory`](https://github.com/Tencent-Hunyuan/Hy-Memory)，npm 包名为
 `memhub-cli`。当前代码架构见
 [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md)。
 
-`v1.0.4` 的发行目标是 Linux x64/arm64 Runtime 与 macOS arm64/x64 App 和 Runtime。
+`v1.0.5` 的发行目标是 Linux x64/arm64 Runtime 与 macOS arm64/x64 App 和 Runtime。
 Linux 制品由 GitHub Actions 在原生架构 runner 上构建并完成生命周期验收；macOS
 制品由发行流水线完成 Developer ID 签名、公证和验收后手动上传。Windows 不支持。
 只有同时出现在 GitHub Release 与 `release-manifest.json` 中、且通过公开校验的制品才属于
@@ -45,7 +45,7 @@ Linux 制品由 GitHub Actions 在原生架构 runner 上构建并完成生命
 
 ## 仓库结构
 
-- `apps/core`：Python / FastAPI Kernel，负责授权、模型、队列、SDK bridge 与 API。
+- `apps/core`：Python / FastAPI Kernel，负责授权、模型、队列、SDK Runtime 边界与 API。
 - `apps/cli`：安装、控制和诊断共享 Runtime 的 `one-memory` CLI。
 - `apps/web`：React / TypeScript 管理界面。
 - `apps/desktop`：Electron 桌面壳与图形安装入口。
@@ -117,7 +117,7 @@ MemHub 的 SQLite 数据默认位于 `~/.one-memory`，不在 npm 包或 Runtime
 
 ## Hy Memory
 
-SDK 依赖固定为公开 PyPI 开发预览 `hy-memory==2.0.5.dev3`。SDK 配置、权限边界及 search
+SDK 依赖固定为公开 PyPI 正式版 `hy-memory==2.0.5`。SDK 配置、权限边界及 search
 返回格式见 [`MEMORY.md`](MEMORY.md)。HY search 返回的 `content` 已包含 SDK 统一生成的
 逻辑时间和演化上下文；调用方不应再次转换时间戳或重复拼接 `When`。
 

memhub/SBOM.spdx.json

@@ -3,9 +3,9 @@
   "dataLicense": "CC0-1.0",
   "SPDXID": "SPDXRef-DOCUMENT",
   "name": "one-memory",
-  "documentNamespace": "https://one-memory.local/spdx/2026-10-22T08:09:31Z",
+  "documentNamespace": "https://one-memory.local/spdx/2026-10-22T13:18:16Z",
   "creationInfo": {
-    "created": "2026-10-22T08:09:31Z",
+    "created": "2026-10-22T13:18:16Z",
     "creators": [
       "Tool: one-memory-generate-sbom"
     ]
@@ -2352,7 +2352,7 @@
     {
       "SPDXID": "SPDXRef-Package-hy-memory",
       "name": "hy-memory",
-      "versionInfo": "2.0.5.dev3",
+      "versionInfo": "2.0.5",
       "downloadLocation": "NOASSERTION",
       "filesAnalyzed": false,
       "licenseConcluded": "Apache-2.0",

memhub/THIRD_PARTY_NOTICES.md

@@ -2,14 +2,14 @@
 
 MemHub itself is licensed under Apache-2.0 (see `LICENSE`).
 It bundles the runtime dependencies listed below.
-Generated from the installed dependency tree on 2026-10-22T08:09:31Z
+Generated from the installed dependency tree on 2026-10-22T13:18:16Z
 by `python3 scripts/generate_sbom.py`. Do not edit by hand.
 
 Per-package license texts are not reproduced here yet; the license
 column is filled in as part of release review, and the SBOM records
 `NOASSERTION` until then rather than guessing.
 
-## hy-memory 2.0.5.dev3
+## hy-memory 2.0.5
 
 License: Apache-2.0
 

memhub/apps/cli/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@one-memory/cli",
-  "version": "1.0.4",
+  "version": "1.0.5",
   "description": "MemHub bootstrap CLI for installing and managing the local-first shared memory runtime",
   "license": "Apache-2.0",
   "repository": {

memhub/apps/cli/src/cli-logs.test.ts

@@ -1,6 +1,7 @@
-import { mkdtemp, readdir, rm, writeFile } from "node:fs/promises";
+import { mkdtemp, readdir, readFile, rm, writeFile } from "node:fs/promises";
 import os from "node:os";
 import path from "node:path";
+import { inflateRawSync } from "node:zlib";
 import { DiagnosticLogger } from "@one-memory/runtime-control";
 import { afterEach, expect, it, vi } from "vitest";
 
@@ -29,6 +30,27 @@ async function fixture() {
   return dataDir;
 }
 
+function readZipEntry(archive: Buffer, expected: string): string {
+  let offset = 0;
+  while (archive.readUInt32LE(offset) === 0x04034b50) {
+    const method = archive.readUInt16LE(offset + 8);
+    const compressedSize = archive.readUInt32LE(offset + 18);
+    const nameLength = archive.readUInt16LE(offset + 26);
+    const extraLength = archive.readUInt16LE(offset + 28);
+    const nameStart = offset + 30;
+    const dataStart = nameStart + nameLength + extraLength;
+    const name = archive.subarray(nameStart, nameStart + nameLength).toString();
+    const compressed = archive.subarray(dataStart, dataStart + compressedSize);
+    if (name === expected) {
+      if (method === 0) return compressed.toString();
+      if (method === 8) return inflateRawSync(compressed).toString();
+      throw new Error("unsupported zip compression");
+    }
+    offset = dataStart + compressedSize;
+  }
+  throw new Error(`missing ${expected}`);
+}
+
 it("prints matching diagnostic events as NDJSON without starting Runtime", async () => {
   const dataDir = await fixture();
   let stdout = "";
@@ -74,6 +96,25 @@ it("exports one support zip to the requested downloads directory", async () => {
   });
   expect(receipt.path).toMatch(/MemHub-日志-3800-\d{8}-\d{4}\.zip$/);
   expect(await readdir(downloads)).toEqual([path.basename(receipt.path)]);
+  const archive = await readFile(receipt.path);
+  expect(JSON.parse(readZipEntry(archive, "summary.json")).commit).toBeNull();
+});
+
+it("fails closed for an explicitly supplied invalid commit", async () => {
+  const dataDir = await fixture();
+  const downloads = await mkdtemp(path.join(os.tmpdir(), "om-cli-downloads-"));
+  roots.push(downloads);
+  vi.spyOn(process.stdout, "write").mockImplementation(() => true);
+
+  await expect(
+    run(["logs", "export", "--json"], {
+      ONE_MEMORY_DATA_DIR: dataDir,
+      ONE_MEMORY_EXPORT_DIR: downloads,
+      ONE_MEMORY_BUILD_NUMBER: "3800",
+      ONE_MEMORY_COMMIT: "unknown",
+    }),
+  ).rejects.toThrow("diagnostic_commit_invalid");
+  expect(await readdir(downloads)).toEqual([]);
 });
 
 it("never emits a secret from a malformed manually written event", async () => {

memhub/apps/cli/src/cli.test.ts

@@ -57,6 +57,11 @@ const runtimeMocks = vi.hoisted(() => ({
   })),
   uninstallUserService: vi.fn(async (..._args: unknown[]) => "removed"),
   restartUserService: vi.fn(async () => undefined),
+  runSdkMigration: vi.fn(
+    async (): Promise<{ state: string; code?: string }> => ({
+      state: "not_required",
+    }),
+  ),
   rollbackLocalInstall: vi.fn(async () => undefined),
   planCliLink: vi.fn(async ({ home, installRoot }) => ({
     schema_version: 1,
@@ -126,6 +131,7 @@ vi.mock("@one-memory/runtime-control", async (importOriginal) => {
     installUserService: runtimeMocks.installUserService,
     uninstallUserService: runtimeMocks.uninstallUserService,
     restartUserService: runtimeMocks.restartUserService,
+    runSdkMigration: runtimeMocks.runSdkMigration,
     rollbackLocalInstall: runtimeMocks.rollbackLocalInstall,
     planCliLink: runtimeMocks.planCliLink,
     installCliLink: runtimeMocks.installCliLink,
@@ -889,6 +895,108 @@ describe("public One Memory CLI", () => {
       expect(runtimeMocks.restartUserService).toHaveBeenCalledTimes(2);
     });
 
+    const upgradeArguments = [
+      "upgrade",
+      "--source",
+      "/tmp/dist",
+      "--home",
+      "/tmp/home",
+      "--install-root",
+      "/tmp/install",
+      "--data-dir",
+      "/tmp/custom data",
+      "--yes",
+    ];
+    const presentCliLink = {
+      schema_version: 1 as const,
+      platform: "linux" as const,
+      home: "/tmp/home",
+      install_root: "/tmp/install",
+      bin_directory: "/tmp/bin",
+      status: "present" as const,
+      blocked_reason: null,
+      link_path: "/tmp/bin/one-memory",
+      target_path: "/tmp/install/current/bin/one-memory",
+    };
+
+    it("migrates the home with the new Core before restarting the Runtime", async () => {
+      vi.clearAllMocks();
+      runtimeMocks.planCliLink.mockResolvedValueOnce(presentCliLink);
+      const order: string[] = [];
+      runtimeMocks.runSdkMigration.mockImplementationOnce(async () => {
+        order.push("migrate");
+        return { state: "cutover" };
+      });
+      runtimeMocks.restartUserService.mockImplementationOnce(async () => {
+        order.push("restart");
+      });
+      const output = captureStdout();
+      let code: number;
+      try {
+        code = await run(
+          upgradeArguments,
+          {},
+          { waitForRuntimeReady: async () => true },
+        );
+      } finally {
+        output.restore();
+      }
+      expect(code!).toBe(0);
+      expect(order).toEqual(["migrate", "restart"]);
+      expect(runtimeMocks.runSdkMigration).toHaveBeenCalledWith(
+        expect.objectContaining({ dataDir: "/tmp/custom data" }),
+      );
+    });
+
+    it("rolls back to the previous version when the migration does not finish", async () => {
+      vi.clearAllMocks();
+      runtimeMocks.planCliLink.mockResolvedValueOnce(presentCliLink);
+      runtimeMocks.runSdkMigration.mockResolvedValueOnce({
+        state: "deferred",
+        code: "sdk_migration_disk_space_insufficient",
+      });
+      const streams = captureStreams();
+      let code: number;
+      try {
+        code = await run(
+          upgradeArguments,
+          {},
+          { waitForRuntimeReady: async () => true },
+        );
+      } finally {
+        streams.restore();
+      }
+      expect(code!).toBe(1);
+      expect(JSON.parse(streams.stderr())).toMatchObject({
+        command: "upgrade",
+        code: "sdk_migration_disk_space_insufficient",
+      });
+      expect(runtimeMocks.rollbackLocalInstall).toHaveBeenCalledOnce();
+    });
+
+    it("keeps the new version when a migrated home is slow to come up", async () => {
+      vi.clearAllMocks();
+      runtimeMocks.planCliLink.mockResolvedValueOnce(presentCliLink);
+      runtimeMocks.runSdkMigration.mockResolvedValueOnce({ state: "cutover" });
+      const streams = captureStreams();
+      let code: number;
+      try {
+        code = await run(
+          upgradeArguments,
+          {},
+          { waitForRuntimeReady: async () => false },
+        );
+      } finally {
+        streams.restore();
+      }
+      expect(code!).toBe(1);
+      expect(JSON.parse(streams.stderr())).toMatchObject({
+        command: "upgrade",
+        code: "runtime_not_ready_after_sdk_migration",
+      });
+      expect(runtimeMocks.rollbackLocalInstall).not.toHaveBeenCalled();
+    });
+
     it("reports success once the upgraded Runtime is healthy", async () => {
       vi.clearAllMocks();
       runtimeMocks.planCliLink.mockResolvedValueOnce({

memhub/apps/cli/src/cli.ts

@@ -39,7 +39,9 @@ import {
   rollbackCliLink,
   rollbackLocalInstall,
   runRuntimeCli,
+  runSdkMigration,
   runtimeInstallLayout,
+  type SdkMigrationResult,
   stopRuntimeProcess,
 } from "@one-memory/runtime-control";
 import axios from "axios";
@@ -354,6 +356,9 @@ export interface RunControls {
   /** Wait for a restarted Runtime to become healthy after `upgrade`.
    * Injectable so the upgrade tests do not spend real seconds polling. */
   waitForRuntimeReady?: (dataDir: string) => Promise<boolean>;
+  /** Run the SDK data migration before install/upgrade starts the new
+   * Runtime. Injectable so the tests do not spawn a real Core. */
+  migrateSdkData?: typeof migrateSdkDataBeforeStart;
   restartManagedRuntime?: typeof restartManagedService;
   stopRuntime?: typeof stopRuntimeProcess;
   ensureRuntime?: typeof ensureRuntimeProcess;
@@ -1048,15 +1053,21 @@ async function runCommand(
         entry: "cli",
       }));
       const target = path.join(downloads, diagnosticZipFilename(build));
+      const environmentCommit = environment.ONE_MEMORY_COMMIT?.trim();
+      const embeddedCommit =
+        typeof __ONE_MEMORY_COMMIT__ === "string"
+          ? __ONE_MEMORY_COMMIT__.trim()
+          : "";
+      const commit =
+        environmentCommit ||
+        (embeddedCommit && embeddedCommit !== "unknown"
+          ? embeddedCommit
+          : undefined);
       const receipt = await exportDiagnosticZip({
         dataDir: config.dataDir,
         target,
         productVersion: build,
-        commit:
-          environment.ONE_MEMORY_COMMIT?.trim() ||
-          (typeof __ONE_MEMORY_COMMIT__ === "string"
-            ? __ONE_MEMORY_COMMIT__
-            : "unknown"),
+        ...(commit === undefined ? {} : { commit }),
         coreSha256: await sha256File(config.coreCommand).catch(() => undefined),
         home: os.homedir(),
         crashReportsDir: path.join(
@@ -3143,6 +3154,7 @@ async function runCommand(
       null;
     let installReceiptWritten = false;
     let serviceApplied = false;
+    let migrated = false;
     let previousOwner:
       | Awaited<ReturnType<typeof claimCliServiceOwnership>>
       | undefined;
@@ -3153,6 +3165,22 @@ async function runCommand(
       }
       shellPathReceipt = await applyShellPath(shellPath);
       previousOwner = await claimCliServiceOwnership(layout.dataDir);
+      // `install` over an existing home (reinstall, App-to-CLI takeover) must
+      // migrate it too, before the new Runtime starts serving it.
+      const migration = await (
+        controls.migrateSdkData ?? migrateSdkDataBeforeStart
+      )({
+        coreCommand: layout.coreCommand,
+        dataDir: layout.dataDir,
+        runtimeBuild: String(plan.build_number ?? plan.version),
+        asJson: true,
+      });
+      if (migration.state === "deferred") {
+        throw new SdkMigrationDeferred(
+          migration.code ?? "sdk_migration_incomplete",
+        );
+      }
+      migrated = migration.state === "cutover";
       await applyPreparedServiceInstall(preparedService);
       serviceApplied = true;
       // The receipt exists so a later command can find the data dir without
@@ -3204,6 +3232,13 @@ async function runCommand(
           );
         }
       };
+      if (migrated) {
+        // An older Runtime refuses a migrated home: keep this install.
+        process.stderr.write(
+          `${JSON.stringify(notReadyAfterMigration("install"), null, 2)}\n`,
+        );
+        return EXIT_FAILED;
+      }
       if (serviceApplied) {
         await rollbackPreparedServiceInstall(
           preparedService,
@@ -3213,7 +3248,13 @@ async function runCommand(
         await rollbackFilesAndOwner();
       }
       process.stderr.write(
-        `${JSON.stringify(serviceErrorDocument("install", error), null, 2)}\n`,
+        `${JSON.stringify(
+          error instanceof SdkMigrationDeferred
+            ? sdkMigrationFailure("install", error.code)
+            : serviceErrorDocument("install", error),
+          null,
+          2,
+        )}\n`,
       );
       return EXIT_FAILED;
     }
@@ -3575,6 +3616,7 @@ async function runCommand(
     let previousOwner:
       | Awaited<ReturnType<typeof claimCliServiceOwnership>>
       | undefined;
+    let migrated = false;
     const rollbackUpgrade = async (
       dataDir: string | undefined,
     ): Promise<void> => {
@@ -3606,6 +3648,24 @@ async function runCommand(
         memhubReceipts.push(await installManagedCliLink(link));
       }
       previousOwner = await claimCliServiceOwnership(layout.dataDir);
+      const migration = await (
+        controls.migrateSdkData ?? migrateSdkDataBeforeStart
+      )({
+        coreCommand: layout.coreCommand,
+        dataDir: parsed.dataDir ?? layout.dataDir,
+        runtimeBuild: String(plan.build_number ?? plan.version),
+        asJson,
+      });
+      if (migration.state === "deferred") {
+        await rollbackUpgrade(parsed.dataDir);
+        process.stderr.write(
+          `${JSON.stringify(sdkMigrationFailure("upgrade", migration.code ?? "sdk_migration_incomplete"), null, 2)}\n`,
+        );
+        return EXIT_FAILED;
+      }
+      // An older Runtime refuses a migrated home, so from here a failure
+      // keeps the new version instead of rolling back to one that cannot start.
+      migrated = migration.state === "cutover";
       await restartUserService(layout, serviceOptions);
       // A restart that returns without error only means launchd/systemd
       // accepted the command. O-22: the first `upgrade` implementation
@@ -3617,6 +3677,12 @@ async function runCommand(
       const ready = await (controls.waitForRuntimeReady ?? waitForRuntimeReady)(
         dataDir,
       );
+      if (!ready && migrated) {
+        process.stderr.write(
+          `${JSON.stringify(notReadyAfterMigration("upgrade"), null, 2)}\n`,
+        );
+        return EXIT_FAILED;
+      }
       if (!ready) {
         await rollbackUpgrade(parsed.dataDir);
         process.stderr.write(
@@ -3644,9 +3710,9 @@ async function runCommand(
         });
       }
     } catch (error) {
-      await rollbackUpgrade(parsed.dataDir);
+      if (!migrated) await rollbackUpgrade(parsed.dataDir);
       process.stderr.write(
-        `${JSON.stringify(serviceErrorDocument("upgrade", error), null, 2)}\n`,
+        `${JSON.stringify(migrated ? notReadyAfterMigration("upgrade") : serviceErrorDocument("upgrade", error), null, 2)}\n`,
       );
       return EXIT_FAILED;
     }
@@ -4014,6 +4080,79 @@ export function runtimeIsReady(
   );
 }
 
+const SDK_MIGRATION_STEPS: Record<string, string> = {
+  bundle: "正在备份记忆数据，数据较多时需要几分钟……",
+  prepare: "正在整理待处理的写入任务……",
+  cutover: "正在切换记忆数据源……",
+  cleanup: "正在清理旧数据表……",
+};
+
+/**
+ * Migrate a home written by an older release before the new Runtime serves it.
+ *
+ * Runs the new Runtime's own Core with the Runtime stopped. Progress goes to
+ * stderr so stdout stays one result document.
+ */
+async function migrateSdkDataBeforeStart(options: {
+  coreCommand: string;
+  dataDir: string;
+  runtimeBuild: string;
+  asJson: boolean;
+}): Promise<SdkMigrationResult> {
+  return runSdkMigration({
+    dataDir: options.dataDir,
+    core: { command: options.coreCommand },
+    runtimeBuild: options.runtimeBuild,
+    onProgress: (step) => {
+      process.stderr.write(
+        options.asJson
+          ? `${JSON.stringify({ schema_version: 1, product: PRODUCT, event: "sdk_migration.progress", step })}\n`
+          : `${SDK_MIGRATION_STEPS[step] ?? step}\n`,
+      );
+    },
+  });
+}
+
+class SdkMigrationDeferred extends Error {
+  constructor(readonly code: string) {
+    super(code);
+  }
+}
+
+function notReadyAfterMigration(command: "install" | "upgrade") {
+  return {
+    schema_version: 1,
+    product: PRODUCT,
+    command,
+    status: "error",
+    code: "runtime_not_ready_after_sdk_migration",
+    message:
+      "记忆数据已迁移到新版本，但新版本 Runtime 没有在限定时间内就绪。旧版本无法打开已迁移的数据，所以没有回滚。请运行 memhub restart，或 memhub doctor 查看原因。",
+  };
+}
+
+const SDK_MIGRATION_FAILURES: Record<string, string> = {
+  sdk_migration_disk_space_insufficient:
+    "磁盘空间不足，记忆数据迁移没有完成，已回滚到上一版本。请腾出与记忆数据同等大小的空间后重试；原有记忆数据不会丢失。",
+  sdk_migration_memory_unreadable:
+    "核对发现新版本还读不到你的全部记忆，没有切换数据源，已回滚到上一版本。请把这个错误码反馈给我们；原有记忆数据不会丢失。",
+  sdk_migration_model_profile_required:
+    "迁移需要先有可用的模型配置，用来核对新版本能否读回记忆。已回滚到上一版本；请先用 memhub models setup 完成配置后再升级。",
+};
+
+function sdkMigrationFailure(command: "install" | "upgrade", code: string) {
+  return {
+    schema_version: 1,
+    product: PRODUCT,
+    command,
+    status: "error",
+    code,
+    message:
+      SDK_MIGRATION_FAILURES[code] ??
+      "记忆数据迁移没有完成，已回滚到上一版本。请重试；如果仍然失败，请把这个错误码反馈给我们。原有记忆数据不会丢失。",
+  };
+}
+
 async function waitForRuntimeReady(
   dataDir: string,
   timeoutMs = 45_000,
@@ -4730,10 +4869,10 @@ async function runMaintenanceRestore(options: {
   const instanceId = randomUUID();
   const child = spawn(options.coreCommand, ["serve", "--maintenance"], {
     env: {
-      // Do not pass the caller's Runtime tuning variables (for example
-      // retrieval-index flags) into a maintenance process: a restore must
-      // start the product's minimal maintenance configuration, not inherit
-      // an unrelated experiment from the shell.
+      // Do not pass the caller's Runtime tuning variables into a
+      // maintenance process: a restore must start the product's minimal
+      // maintenance configuration, not inherit an unrelated experiment from
+      // the shell.
       PATH: process.env.PATH,
       HOME: process.env.HOME,
       LANG: process.env.LANG,

memhub/apps/cli/src/log-report.test.ts

@@ -1,4 +1,4 @@
-import { mkdtemp, rm } from "node:fs/promises";
+import { mkdtemp, rm, symlink } from "node:fs/promises";
 import os from "node:os";
 import path from "node:path";
 import { DiagnosticLogger } from "@one-memory/runtime-control";
@@ -41,11 +41,15 @@ it("redacts unsafe details and absolute stack frames from the read surface", ()
     session_id: "11111111-1111-4111-8111-111111111111",
     details: {
       count: 1,
+      reason: "/Users/me/private-query.txt",
+      status: "sk-live-secret-token",
+      source_id: "credential.private",
       prompt: "secret-token-123",
       authorization: "Bearer secret-token-123",
     } as unknown as Record<string, string | number | boolean | null>,
     safe_stack: [
       "packages/runtime-control/src/example.ts:42:7",
+      "packages/runtime/sk-live-secret-token.ts:1:1",
       "/Users/me/private.py:1",
       "C:\\Users\\me\\private.py:1",
     ],
@@ -56,6 +60,8 @@ it("redacts unsafe details and absolute stack frames from the read surface", ()
     "packages/runtime-control/src/example.ts:42:7",
   ]);
   expect(JSON.stringify(redacted)).not.toContain("secret-token-123");
+  expect(JSON.stringify(redacted)).not.toContain("sk-live-secret-token");
+  expect(JSON.stringify(redacted)).not.toContain("credential.private");
   expect(JSON.stringify(redacted)).not.toContain("/Users/me");
   expect(JSON.stringify(redacted)).not.toContain("C:\\\\Users");
 });
@@ -132,3 +138,27 @@ it("follows a newly appended event without stopping Runtime", async () => {
   });
   expect(seen).toEqual(["service.ready", "core.failed"]);
 });
+
+it("fails follow mode when the diagnostic directory is unsafe", async () => {
+  const dataDir = await mkdtemp(
+    path.join(os.tmpdir(), "om-cli-follow-unsafe-"),
+  );
+  const external = await mkdtemp(
+    path.join(os.tmpdir(), "om-cli-follow-external-"),
+  );
+  roots.push(dataDir, external);
+  await symlink(external, path.join(dataDir, "logs"));
+  const query = parseLogQuery(["--follow"]);
+  expect(query).not.toBeNull();
+  if (!query) throw new Error("query_missing");
+  await expect(
+    followLogs({
+      dataDir,
+      query,
+      signal: new AbortController().signal,
+      maxEvents: 1,
+      emit: () => undefined,
+      wait: async () => undefined,
+    }),
+  ).rejects.toThrow("log_directory_unreadable");
+});

memhub/apps/cli/src/log-report.ts

@@ -3,6 +3,7 @@ import path from "node:path";
 import type { DiagnosticEventV1 } from "@one-memory/runtime-control";
 import {
   diagnosticLogFiles,
+  projectDiagnosticEventForSafeOutput,
   readDiagnosticEvents,
 } from "@one-memory/runtime-control";
 
@@ -24,54 +25,6 @@ export interface LogQuery {
 
 const ID = /^[A-Za-z0-9][A-Za-z0-9._:-]{0,255}$/;
 const NAME = /^[a-z][a-z0-9]*(?:[._-][a-z0-9]+){0,15}$/;
-const SAFE_DETAIL_KEYS = new Set([
-  "action",
-  "attempt",
-  "count",
-  "method",
-  "mode",
-  "phase",
-  "platform",
-  "reason",
-  "result_count",
-  "source_id",
-  "spawned",
-  "status",
-  "version",
-  "parent_session_id",
-  "invalid_lines",
-  "truncated",
-  "runtime_state",
-  "core_state",
-  "broker_state",
-  "instance_id",
-  "open_file_count",
-  "memory_bytes",
-  "thread_count",
-  "database_error_count",
-  "file_count",
-  "sqlite_file_count",
-  "byte_file_count",
-  "total_bytes",
-  "copied_bytes",
-  "bytes_per_second",
-  "stored_count",
-  "revised_count",
-  "skipped_count",
-  "error_count",
-  "pending_count",
-  "prepared_count",
-  "queue_depth",
-  "in_flight_count",
-  "call_count",
-  "created_count",
-  "updated_count",
-  "superseded_count",
-  "deleted_count",
-  "operation_kind",
-  "file_kind",
-]);
-
 export function parseLogQuery(arguments_: readonly string[]): LogQuery | null {
   if (arguments_.filter((item) => item === "--json").length > 1) return null;
   const values: Partial<LogQuery> = { errorsOnly: false, follow: false };
@@ -237,63 +190,13 @@ export function renderLogText(
   return `${lines.join("\n")}\n`;
 }
 
-/** Defence in depth for the CLI read surface. `DiagnosticLogger` already
- * allow-lists keys, but a future writer, a manually edited log, or a malformed
- * line must not turn `memhub logs` into an exfiltration path. */
+/** Defence in depth for the CLI read surface. Local writers accept arbitrary
+ * bounded scalar keys, so every outward read surface must reuse the same
+ * field-and-value projection as the support-bundle exporter. */
 export function redactDiagnosticEvent(
   event: DiagnosticEventV1,
 ): DiagnosticEventV1 {
-  const redacted: DiagnosticEventV1 = {
-    schema_version: 1,
-    timestamp: event.timestamp,
-    level: event.level,
-    service: event.service,
-    component: event.component,
-    event: event.event,
-    outcome: event.outcome,
-    session_id: event.session_id,
-    ...(event.trace_id === undefined ? {} : { trace_id: event.trace_id }),
-    ...(event.incident_id === undefined
-      ? {}
-      : { incident_id: event.incident_id }),
-    ...(event.instance_id === undefined
-      ? {}
-      : { instance_id: event.instance_id }),
-    ...(event.operation_id === undefined
-      ? {}
-      : { operation_id: event.operation_id }),
-    ...(event.request_id === undefined ? {} : { request_id: event.request_id }),
-    ...(event.job_id === undefined ? {} : { job_id: event.job_id }),
-    ...(event.task_id === undefined ? {} : { task_id: event.task_id }),
-    ...(event.duration_ms === undefined
-      ? {}
-      : { duration_ms: event.duration_ms }),
-    ...(event.state_before === undefined
-      ? {}
-      : { state_before: event.state_before }),
-    ...(event.state_after === undefined
-      ? {}
-      : { state_after: event.state_after }),
-    ...(event.error_code === undefined ? {} : { error_code: event.error_code }),
-    ...(event.error_type === undefined ? {} : { error_type: event.error_type }),
-    ...(event.safe_stack === undefined
-      ? {}
-      : {
-          safe_stack: event.safe_stack.filter(
-            (frame) =>
-              !frame.startsWith("/") &&
-              !/^[A-Za-z]:\\/.test(frame) &&
-              !frame.includes(".."),
-          ),
-        }),
-  };
-  const details = Object.fromEntries(
-    Object.entries(event.details ?? {}).filter(([key]) =>
-      SAFE_DETAIL_KEYS.has(key),
-    ),
-  );
-  if (Object.keys(details).length > 0) redacted.details = details;
-  return redacted;
+  return projectDiagnosticEventForSafeOutput(event);
 }
 
 export async function followLogs(options: {
@@ -315,6 +218,7 @@ export async function followLogs(options: {
       ...options.query,
       tail: 5_000,
     });
+    if (report.failure) throw new Error(report.failure);
     for (const event of report.events) {
       const identity = JSON.stringify(event);
       if (seen.has(identity)) continue;

memhub/apps/core/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "uv_build"
 
 [project]
 name = "one-memory"
-version = "1.0.4"
+version = "1.0.5"
 description = "MemHub local-first memory core"
 readme = "README.md"
 requires-python = ">=3.12"
@@ -12,8 +12,9 @@ dependencies = [
     "fastapi>=0.141.1",
     "filelock>=3.32.4",
     # Public SDK version validated against MemHub's permission and recall contracts.
-    "hy-memory==2.0.5.dev3",
-    # Retrieval's derived vector index directly consumes these locked versions.
+    "hy-memory==2.0.5",
+    # chromadb arrives through the HY SDK; locked here so an unrelated upgrade
+    # cannot move the SDK's store. Built-in embedding consumes numpy directly.
     "chromadb==1.5.9",
     "numpy==2.5.2",
     "httpx>=0.28.1",

memhub/apps/core/scripts/verify_c2_slice.py

@@ -42,12 +42,10 @@ TARGETED = (
     "tests/test_database_contention.py",
     "tests/test_main_database_fairness.py",
     "tests/test_soak_probe_effectiveness.py",
-    "tests/test_evidence_reconcile_primitive.py",
     "tests/test_sqlite_semantics_gate.py",
     "tests/test_store_binding.py",
     "tests/test_store_binding_ratchet.py",
     "tests/test_main_database_entry.py",
-    "tests/test_memory_processing.py",
     "tests/test_extraction_window.py",
     "tests/test_extraction_defer.py",
     "tests/test_active_memory_window.py",

memhub/apps/core/src/one_memory/__init__.py

@@ -2,11 +2,5 @@
 
 import os
 
-from one_memory.hy_runtime_config import configure_hy_behavior_environment
-
-# HY has a few reader switches evaluated at module import time.  Fix the
-# product-owned, non-secret behavior before any one_memory module imports HY.
-configure_hy_behavior_environment()
-
 _version = os.environ.get("ONE_MEMORY_VERSION", "").strip()
 __version__ = _version if _version else "development"

memhub/apps/core/src/one_memory/api.py

@@ -50,6 +50,10 @@ from one_memory.auth import local_api_guard, renderer_origins
 from one_memory.background_work import shutdown_bulk_pool, warm_bulk_pool
 from one_memory.config import Settings, load_settings
 from one_memory.context_routes import context_router, sdk_context_router
+from one_memory.conversation_processing import (
+    _conversation_chunks,
+    memory_task_timeout_seconds,
+)
 from one_memory.database_access import (
     MAIN_DATABASE_REPORT_SECONDS,
     close_main_databases,
@@ -78,23 +82,30 @@ from one_memory.external_memory_access import external_access, require_external_
 from one_memory.extraction_settings import ExtractionSettings, ExtractionSettingsStore
 from one_memory.file_imports import FileImportStore, file_import_router
 from one_memory.home_summary import HomeSummary, read_home_summary
-from one_memory.hy_memory_bridge import (
-    GraphWidthDecision,
-    GraphWidthMismatch,
-    HyMemoryBridge,
-    HyMemoryBridgeError,
-    HySdkMemoryPage,
-    HySdkMemoryRecord,
-    HySdkRawRecord,
-    builtin_retrieval_snapshot,
-)
+from one_memory.hy_graph_admin import GraphWidthDecision, GraphWidthMismatch
 from one_memory.hy_runtime_config import (
+    HY_MEMORY_READER,
     DiagnosticLogLevel,
     detailed_logging_enabled,
     diagnostic_log_level,
     set_detailed_logging,
     set_diagnostic_log_level,
 )
+from one_memory.hy_sdk_runtime import (
+    SdkClientHandle,
+    builtin_retrieval_snapshot,
+    close_all_shared_runtimes,
+    sdk_relevance_floor,
+)
+from one_memory.hy_sdk_types import (
+    HySdkError,
+    HySdkMemoryPage,
+    HySdkMemoryRecord,
+    HySdkRawPage,
+    HySdkRawRecord,
+    HySdkUserProfile,
+    parse_sdk_search,
+)
 from one_memory.imports.access import (
     AuthorizedSourceRoot,
     SourceAccessError,
@@ -167,11 +178,10 @@ from one_memory.memory_library import (
     read_conversation_memories,
     read_memory_library,
 )
-from one_memory.memory_notes import MemoryNoteStore, memory_note_router
-from one_memory.memory_processing import (
-    MemoryProcessingSupervisor,
-    _conversation_chunks,
-    memory_task_timeout_seconds,
+from one_memory.memory_notes import (
+    LegacyMemoryNoteReader,
+    MemoryNoteStore,
+    memory_note_router,
 )
 from one_memory.memory_qa import MemoryAnswerer, MemoryAnswerError
 from one_memory.memory_queue import (
@@ -201,7 +211,10 @@ from one_memory.model_profiles import (
     ModelProfileStore,
     ModelProfileUnavailableError,
 )
-from one_memory.model_request_audit import AuditedModelTransport, ModelRequestAuditStore
+from one_memory.model_request_audit import (
+    AuditedModelTransport,
+    ModelRequestAuditStore,
+)
 from one_memory.model_request_budget import BudgetedModelTransport, ModelRequestBudget
 from one_memory.onboarding import OnboardingState, OnboardingStore
 from one_memory.recall import (
@@ -225,6 +238,8 @@ from one_memory.sdk_memory_qa import (
     SdkMemoryAnswer,
     SdkMemoryAnswerer,
 )
+from one_memory.sdk_memory_runtime import SdkMemoryRuntime
+from one_memory.sdk_write_processor import SdkWriteCoordinator
 from one_memory.secrets import (
     KeyringSecretStore,
     SecretStore,
@@ -826,8 +841,6 @@ def create_app(
     secret_store: SecretStore | None = None,
 ) -> FastAPI:
     runtime_settings = settings or load_settings()
-    if runtime_settings.retrieval_index_enabled and not manage_runtime:
-        raise ValueError("retrieval_runtime_ownership_required")
 
     @asynccontextmanager
     async def lifespan(_: FastAPI) -> AsyncIterator[None]:
@@ -845,7 +858,7 @@ def create_app(
         runtime = CoreRuntime(runtime_settings) if manage_runtime else nullcontext()
         with runtime:
             database_path = runtime_settings.data_dir / "one-memory.sqlite3"
-            app.state.sdk_library_bridge = None
+            app.state.sdk_client_handle = None
             bootstrap_fresh_management_schema(database_path)
             # Before any store: each one below changes the library as it opens.
             try:
@@ -891,6 +904,7 @@ def create_app(
             )
             from one_memory.memory_server.data_safety import MIN_FREE_BYTES
             from one_memory.memory_server.durable_queue import (
+                COMPLETED,
                 DurableWriteDrain,
                 DurableWriteQueue,
                 execute_job,
@@ -968,31 +982,21 @@ def create_app(
                 await asyncio.to_thread(app.state.evidence_store.release_completed_sdk_staging)
             evidence_store = app.state.evidence_store
             if app.state.memory_authority is MemoryAuthority.SDK:
-                sdk_bridge = _LazySdkBridge(app)
-
-                async def read_sdk_memory(
-                    memory_id: str,
-                ) -> HySdkMemoryRecord | None:
-                    return cast(
-                        HySdkMemoryRecord | None,
-                        await sdk_bridge.get_sdk_memory(memory_id),
-                    )
-
+                sdk_handle = _sdk_client_handle(app)
                 app.state.recall_target = SdkRecallTarget(
-                    semantic_search=lambda query: sdk_bridge.search_memories(
+                    semantic_search=lambda query: _search_sdk_memories(
+                        sdk_handle,
                         query,
                         user_id="local",
                         agent_id="*",
                     ),
-                    get_memory=read_sdk_memory,
+                    get_memory=lambda memory_id: _get_sdk_memory(sdk_handle, memory_id),
                 )
             else:
                 app.state.recall_target = GovernedRecallTarget(
                     app.state.memory_governance_store,
                     filter_memories=app.state.source_read_access.memories,
-                    semantic_search=lambda query: _memory_processing_supervisor(
-                        app
-                    ).semantic_recall(query),
+                    semantic_search=lambda query: _sdk_memory_runtime(app).semantic_recall(query),
                     # Read per request rather than cached: an import that lands
                     # while the app is open should make its workspace reachable
                     # without a restart. Affordable only because the read is served
@@ -1094,9 +1098,7 @@ def create_app(
                 authorize_generation=lambda profile, question, grounding: (
                     _authorize_answer_generation(app, profile, question, grounding)
                 ),
-                semantic_search=lambda query: _memory_processing_supervisor(app).semantic_recall(
-                    query
-                ),
+                semantic_search=lambda query: _sdk_memory_runtime(app).semantic_recall(query),
                 resolve_workspace=lambda workspace_path: resolve_workspace_scopes(
                     workspace_path,
                     evidence_store.known_workspaces(user_id="local"),
@@ -1116,19 +1118,12 @@ def create_app(
                 store=app.state.model_profile_store,
                 secrets=app.state.secret_store,
             )
-            app.state.memory_processing_supervisor = MemoryProcessingSupervisor(
-                tasks=app.state.memory_task_store,
+            app.state.sdk_write_coordinator = SdkWriteCoordinator(
                 profiles=app.state.model_profile_store,
                 secrets=app.state.secret_store,
-                evidence=app.state.evidence_store,
-                governance=app.state.memory_governance_store,
                 data_dir=runtime_settings.data_dir,
-                extraction_settings=app.state.extraction_settings_store,
-                bridge_factory=lambda profile, api_key, data_dir: HyMemoryBridge(
-                    profile=profile,
-                    api_key=api_key,
-                    data_dir=data_dir,
-                    http_transport_factory=lambda: AuditedModelTransport(
+                transport_factory=lambda profile: (
+                    lambda: AuditedModelTransport(
                         app.state.model_request_audit_store,
                         profile_id=profile.profile_id,
                         profile_revision=profile.revision,
@@ -1139,21 +1134,19 @@ def create_app(
                             if model_request_budget is not None
                             else None
                         ),
-                    ),
+                    )
                 ),
-                # `None` means "whatever the active profile allows". Pinned to
-                # 1 in `a59f4b5` as temporary load-shedding for an acceptance
-                # package, and left pinned after it: a hardcoded 1 is rule 9's
-                # exact anti-pattern (a small number baked into code that
-                # cannot be tuned per machine). Measured before removing — at
-                # profile concurrency 4, recall latency is unchanged while
-                # precipitation goes 4x.
-                max_concurrent_tasks=runtime_settings.precipitation_max_concurrent_tasks,
-                source_writes_permitted=lambda source_id: _source_writes_on(app, source_id),
-                service_levels=app.state.service_levels,
                 sedimentation_enabled=lambda: _config_store(app).sedimentation_enabled(),
-                sdk_authority=lambda: app.state.memory_authority is MemoryAuthority.SDK,
             )
+            app.state.sdk_memory_runtime = SdkMemoryRuntime(
+                coordinator=app.state.sdk_write_coordinator,
+                read_handle=_sdk_client_handle(app),
+                write_queue=memory_write_queue,
+                extraction_settings=app.state.extraction_settings_store,
+                data_dir=runtime_settings.data_dir,
+            )
+            if app.state.memory_authority is MemoryAuthority.SDK:
+                app.state.sdk_memory_runtime.start_raw_retention()
             # C2 owns the single production budget (handle 5s + reserve 250ms,
             # total 5.25s, lease 10s, renew 1s, claim 250ms). The composition
             # root only consumes it: no local formula, and the 250ms claim
@@ -1162,34 +1155,24 @@ def create_app(
             drain_budget = production_drain_budget()
 
             async def reconcile_unknown_writes() -> None:
-                if app.state.memory_authority is MemoryAuthority.SDK:
-                    # Unknown SDK writes must be reconciled by SDK request ID.
-                    # Until that direct path is available, retain them as
-                    # unknown rather than re-running a possibly successful add.
-                    return
-                await app.state.memory_processing_supervisor.reconcile_unknown_outputs(
-                    exclude_error_codes=NON_RETRYABLE_POST_WRITE_CODES,
-                )
                 unknown = await asyncio.to_thread(
                     memory_write_queue.unknown_jobs,
                     limit=128,
                     exclude_error_codes=NON_RETRYABLE_POST_WRITE_CODES,
                 )
                 for job in unknown:
+                    receipt = await app.state.sdk_write_coordinator.inquire_write_outcome(
+                        job.request_id
+                    )
+                    if receipt is None:
+                        continue
                     await asyncio.to_thread(
-                        memory_write_queue.resolve_unknown_from_linked_output,
+                        memory_write_queue.settle,
                         job.job_id,
+                        state=COMPLETED,
+                        sdk_raw_ids=receipt.raw_ids,
+                        sdk_memory_ids=receipt.memory_ids,
                     )
-                await asyncio.to_thread(
-                    memory_write_queue.retry_first_unknown_after_reconcile,
-                    older_than=datetime.now(UTC) - timedelta(seconds=10),
-                    limit=32,
-                )
-                await asyncio.to_thread(
-                    app.state.memory_task_store.retry_first_unknown_after_reconcile,
-                    older_than=datetime.now(UTC) - timedelta(seconds=10),
-                    limit=128,
-                )
 
             async def admit_hy_invocation(
                 source_id: str,
@@ -1211,7 +1194,7 @@ def create_app(
                 execute=lambda job_id: execute_job(
                     job_id,
                     queue=memory_write_queue,
-                    supervisor=app.state.memory_processing_supervisor,
+                    supervisor=app.state.sdk_write_coordinator,
                     evidence=app.state.evidence_store,
                     source_writes_permitted=lambda source_id: _source_writes_on(app, source_id),
                     admit_hy_invocation=admit_hy_invocation,
@@ -1219,7 +1202,7 @@ def create_app(
                 execute_owned=lambda job_id, owner: execute_job(
                     job_id,
                     queue=memory_write_queue,
-                    supervisor=app.state.memory_processing_supervisor,
+                    supervisor=app.state.sdk_write_coordinator,
                     evidence=app.state.evidence_store,
                     owner=owner,
                     source_writes_permitted=lambda source_id: _source_writes_on(app, source_id),
@@ -1293,7 +1276,14 @@ def create_app(
 
             app.state.memory_note_store = MemoryNoteStore(
                 app.state.evidence_store,
-                app.state.memory_processing_supervisor,
+                (
+                    LegacyMemoryNoteReader(
+                        app.state.memory_task_store,
+                        app.state.memory_governance_store,
+                    )
+                    if app.state.memory_authority is MemoryAuthority.LEGACY
+                    else None
+                ),
                 app.state.memory_write_intake,
                 sdk_authority=app.state.memory_authority is MemoryAuthority.SDK,
                 write_queue=memory_write_queue,
@@ -1435,18 +1425,17 @@ def create_app(
             # starts its threads wherever the first work arrives from, and
             # from a busy loop that wait was 2.5 seconds.
             warm_bulk_pool()
+            app.state.memory_backfill = None
+            app.state.memory_visibility_task = None
+            app.state.deletion_sync = None
             if app.state.memory_authority is MemoryAuthority.LEGACY:
                 _start_backfill(app)
                 _start_visibility_recompute(app)
                 app.state.deletion_sync = watched_task(
                     asyncio.get_running_loop(),
-                    lambda: _memory_processing_supervisor(app).keep_deletions_in_sync(),
+                    lambda: _keep_legacy_deletions_in_sync(app),
                     name="one-memory-deletion-sync",
                 )
-            else:
-                app.state.memory_backfill = None
-                app.state.memory_visibility_task = None
-                app.state.deletion_sync = None
             # Both times the loop was blocked here it was found from outside,
             # by hand, days later. This says so at the time, and says what was
             # holding it.
@@ -1465,22 +1454,7 @@ def create_app(
             app.state.resource_diagnostics = watched_task(
                 asyncio.get_running_loop(), _emit_resource_snapshots
             )
-            app.state.retrieval_runtime = None
             try:
-                if runtime_settings.retrieval_index_enabled:
-                    from one_memory.retrieval.index_provisioning import provision_retrieval_index
-                    from one_memory.retrieval.runtime import ExistingRetrievalRuntime
-
-                    # The runtime only resumes an index someone initialized. Turning
-                    # the feature on is that someone; otherwise enabling it would
-                    # leave every search on the lexical lane with nothing to say why.
-                    await asyncio.to_thread(provision_retrieval_index, runtime_settings.data_dir)
-                    app.state.retrieval_runtime = ExistingRetrievalRuntime(
-                        runtime_settings.data_dir,
-                        minimum_free_bytes=runtime_settings.retrieval_minimum_free_bytes,
-                        service_levels=app.state.service_levels,
-                    )
-                    await app.state.retrieval_runtime.start()
                 # Startup writes are done. Only now let the derived-content
                 # rebuild take the single write checkout: it is a long
                 # background job, and on a large library it would otherwise
@@ -1507,15 +1481,6 @@ def create_app(
                     outcome="started",
                     instance_id=runtime_settings.instance_id,
                 )
-                retrieval_stop_error: BaseException | None = None
-                if app.state.retrieval_runtime is not None:
-                    try:
-                        await app.state.retrieval_runtime.stop()
-                    except (Exception, asyncio.CancelledError) as error:
-                        # Preserve failure while cleaning other owned services.
-                        # Experimental: an exceptional live-thread failure still
-                        # needs an outer ownership fence before production enablement.
-                        retrieval_stop_error = error
                 app.state.main_database_report.cancel()
                 app.state.loop_heartbeat.cancel()
                 with suppress(asyncio.CancelledError):
@@ -1550,16 +1515,12 @@ def create_app(
                 app.state.memory_write_drain.stop()
                 with suppress(asyncio.CancelledError):
                     await app.state.memory_write_drain_task
-                await app.state.memory_processing_supervisor.shutdown()
-                if app.state.sdk_library_bridge is not None:
-                    await app.state.sdk_library_bridge.close()
-                    app.state.sdk_library_bridge = None
-                # HY clients share two process-owned loop threads. Close them
-                # after every bridge is down, while HY's SQLite executor is
-                # still available; interpreter atexit is too late for that.
-                from one_memory.hy_memory_bridge import close_shared_hy_runtimes
-
-                await asyncio.to_thread(close_shared_hy_runtimes)
+                await app.state.sdk_memory_runtime.shutdown()
+                await app.state.sdk_write_coordinator.close()
+                if app.state.sdk_client_handle is not None:
+                    await app.state.sdk_client_handle.close()
+                    app.state.sdk_client_handle = None
+                close_all_shared_runtimes()
                 # Last: what has just been shut down was using it.
                 shutdown_bulk_pool()
                 log_main_database_stats("shutdown")
@@ -1577,8 +1538,6 @@ def create_app(
                 # Waited for off the loop: this is the task whose whole point
                 # is that the loop never waits on a disk.
                 await asyncio.to_thread(flush_diagnostics)
-                if retrieval_stop_error is not None:
-                    raise retrieval_stop_error
 
     app = FastAPI(
         title="MemHub Core",
@@ -1622,8 +1581,6 @@ def create_app(
             "/api/v1/context",
             "/api/v1/boundary",
             "/api/v1/home-summary",
-            "/api/v1/memory-tasks",
-            "/api/v1/memory-processing",
         )
         if getattr(
             app.state, "memory_authority", MemoryAuthority.LEGACY
@@ -1749,7 +1706,7 @@ def create_app(
                     trace_id=trace_id,
                     instance_id=runtime_settings.instance_id,
                     duration_ms=elapsed_ms(started),
-                    details={**diagnostic_details, "status": str(response.status_code)},
+                    details={**diagnostic_details, "status_code": response.status_code},
                 )
             return response
         except Exception as error:
@@ -1820,7 +1777,7 @@ def create_app(
     ) -> dict[str, object]:
         """Semantic search IDs for the Reader process. Reader-only."""
         try:
-            hits = await _memory_processing_supervisor(app).semantic_search(query, limit=limit)
+            hits = await _sdk_memory_runtime(app).semantic_search(query, limit=limit)
         except Exception:
             return {"ids": []}
         return {"ids": [memory_id for memory_id, _score in hits]}
@@ -1850,7 +1807,7 @@ def create_app(
     ) -> dict[str, object]:
         """HY graph overlay for the Reader process. Reader-only."""
         try:
-            snapshot = await _memory_processing_supervisor(app).graph_snapshot(limit=limit)
+            snapshot = await _sdk_memory_runtime(app).graph_snapshot(limit=limit)
         except Exception:
             return {"consulted": False, "nodes": [], "edges": []}
         return snapshot.model_dump(mode="json")
@@ -1861,7 +1818,7 @@ def create_app(
         error = cast(str | None, getattr(app.state, "memory_backfill_error", None))
         status = _backfill_state(backfill, error)
         governance = getattr(app.state, "memory_governance_store", None)
-        supervisor = getattr(app.state, "memory_processing_supervisor", None)
+        supervisor = getattr(app.state, "sdk_memory_runtime", None)
         if supervisor is not None:
             accepted = supervisor.accepted_backfill_status()
             if accepted is not None:
@@ -3092,15 +3049,18 @@ def create_app(
         if app.state.memory_authority is not MemoryAuthority.SDK:
             raise HTTPException(409, "legacy_memory_authority_required")
         try:
-            page = await _sdk_library_bridge(app).list_sdk_memories(
+            client = await _sdk_client_handle(app).client()
+            raw = await asyncio.to_thread(
+                client.list_memories,
                 user_id="local",
-                statuses=("active",),
+                statuses=["active"],
                 limit=1,
                 offset=0,
                 order="desc",
             )
-        except HyMemoryBridgeError as error:
-            raise HTTPException(503, str(error)) from None
+            page = HySdkMemoryPage.from_sdk(raw, limit=1, offset=0)
+        except Exception as error:
+            raise _sdk_http_error(error, "hy_memory_list_failed") from None
         return SdkHomeSummary(
             memory_count=page.total,
             latest=page.memories[0] if page.memories else None,
@@ -3142,11 +3102,8 @@ def create_app(
         store = getattr(app.state, "sdk_data_plane_migrations", None)
         if store is None:
             store = SdkDataPlaneMigrationStore(_evidence_store(app).database_path)
-        marker = store.cutover()
-        return MemoryAuthorityResponse(
-            authority="sdk" if marker is not None else "legacy",
-            migration_id=marker.migration_id if marker is not None else None,
-        )
+        authority, migration_id = store.authority_state()
+        return MemoryAuthorityResponse(authority=authority.value, migration_id=migration_id)
 
     @app.get(
         "/api/v2/memory-library",
@@ -3159,14 +3116,17 @@ def create_app(
         limit: int = Query(default=100, ge=1, le=500),
     ) -> HySdkMemoryPage:
         try:
-            return await _sdk_library_bridge(app).list_sdk_memories(
+            client = await _sdk_client_handle(app).client()
+            raw = await asyncio.to_thread(
+                client.list_memories,
                 user_id="local",
-                statuses=tuple(statuses),
+                statuses=list(statuses),
                 offset=offset,
                 limit=limit,
             )
-        except HyMemoryBridgeError as error:
-            raise HTTPException(status_code=503, detail=str(error)) from None
+            return HySdkMemoryPage.from_sdk(raw, limit=limit, offset=offset)
+        except Exception as error:
+            raise _sdk_http_error(error, "hy_memory_list_failed") from None
 
     @app.get(
         "/api/v2/user-profile",
@@ -3175,10 +3135,12 @@ def create_app(
     )
     async def sdk_user_profile() -> SdkUserProfileResponse:
         try:
-            value = await _sdk_library_bridge(app).get_user_profile(user_id="local")
-        except HyMemoryBridgeError as error:
-            raise HTTPException(status_code=503, detail=str(error)) from None
-        return SdkUserProfileResponse.model_validate(value)
+            client = await _sdk_client_handle(app).client()
+            value = await asyncio.to_thread(client.get_user_profile, user_id="local")
+            profile = HySdkUserProfile.from_sdk(value)
+        except Exception as error:
+            raise _sdk_http_error(error, "hy_memory_profile_read_failed") from None
+        return SdkUserProfileResponse.model_validate(profile.model_dump())
 
     @app.get(
         "/api/v2/memories/{memory_id}",
@@ -3187,9 +3149,9 @@ def create_app(
     )
     async def get_sdk_memory(memory_id: str) -> HySdkMemoryRecord:
         try:
-            memory = await _sdk_library_bridge(app).get_sdk_memory(memory_id)
-        except HyMemoryBridgeError as error:
-            raise HTTPException(status_code=503, detail=str(error)) from None
+            memory = await _get_sdk_memory(_sdk_client_handle(app), memory_id)
+        except Exception as error:
+            raise _sdk_http_error(error, "hy_memory_get_failed") from None
         if memory is None:
             raise HTTPException(status_code=404, detail="memory_not_found")
         return memory
@@ -3203,12 +3165,15 @@ def create_app(
         memory_id: str,
         request: SdkMemoryUpdateRequest,
     ) -> HySdkMemoryRecord:
-        bridge = _sdk_library_bridge(app)
         try:
-            await bridge.update(memory_id, request.content)
-            memory = await bridge.get_sdk_memory(memory_id)
-        except HyMemoryBridgeError as error:
-            raise HTTPException(status_code=503, detail=str(error)) from None
+            client = await app.state.sdk_write_coordinator.client()
+            result = await asyncio.to_thread(client.update, memory_id, request.content)
+            if not isinstance(result, dict) or result.get("success") is not True:
+                raise HySdkError("hy_memory_update_rejected")
+            raw = await asyncio.to_thread(client.get, memory_id)
+            memory = None if raw is None else HySdkMemoryRecord.from_sdk(raw)
+        except Exception as error:
+            raise _sdk_http_error(error, "hy_memory_update_failed") from None
         if memory is None:
             raise HTTPException(status_code=404, detail="memory_not_found")
         return memory
@@ -3220,9 +3185,17 @@ def create_app(
     )
     async def delete_sdk_memory(memory_id: str) -> Response:
         try:
-            await _sdk_library_bridge(app).delete(memory_id)
-        except HyMemoryBridgeError as error:
-            raise HTTPException(status_code=503, detail=str(error)) from None
+            client = await app.state.sdk_write_coordinator.client()
+            result = await asyncio.to_thread(client.delete, memory_id)
+            if not isinstance(result, dict):
+                raise HySdkError("hy_memory_delete_rejected")
+            error_code = result.get("error_code")
+            if error_code or result.get("partial") is True:
+                raise HySdkError(str(error_code or "hy_memory_delete_incomplete"))
+            if result.get("success") is not True and result.get("deleted_count") != 0:
+                raise HySdkError("hy_memory_delete_rejected")
+        except Exception as error:
+            raise _sdk_http_error(error, "hy_memory_delete_failed") from None
         return Response(status_code=204)
 
     @app.get(
@@ -3236,15 +3209,19 @@ def create_app(
         limit: int = Query(default=100, ge=1, le=500),
     ) -> SdkRawPage:
         try:
-            items, total = await _sdk_library_bridge(app).list_sdk_raw(
+            client = await _sdk_client_handle(app).client()
+            value = await asyncio.to_thread(
+                client.list_memories,
                 user_id="local",
-                source=source,
+                layers=["l1_raw"],
+                include_sources=[source] if source else None,
                 offset=offset,
                 limit=limit,
             )
-        except HyMemoryBridgeError as error:
-            raise HTTPException(status_code=503, detail=str(error)) from None
-        return SdkRawPage(items=items, total=total, offset=offset, limit=limit)
+            page = HySdkRawPage.from_memory_list(value, limit=limit, offset=offset)
+        except Exception as error:
+            raise _sdk_http_error(error, "hy_memory_raw_list_failed") from None
+        return SdkRawPage(items=page.raw, total=page.total, offset=offset, limit=limit)
 
     @app.get(
         "/api/v1/memories",
@@ -3775,11 +3752,9 @@ def create_app(
             all_memories, projected, nodes, edges = await asyncio.to_thread(read_governed_part)
             try:
                 hy_graph = (
-                    None
-                    if revoked
-                    else await _memory_processing_supervisor(app).graph_snapshot(limit=limit)
+                    None if revoked else await _sdk_memory_runtime(app).graph_snapshot(limit=limit)
                 )
-            except HyMemoryBridgeError:
+            except HySdkError:
                 hy_graph = None
 
             revoked_now = await asyncio.to_thread(_read_access(app).has_revoked_sources)
@@ -3856,8 +3831,8 @@ def create_app(
         if await asyncio.to_thread(_read_access(app).has_revoked_sources):
             raise HTTPException(status_code=403, detail="memory_source_revoked")
         try:
-            return await _memory_processing_supervisor(app).digest()
-        except HyMemoryBridgeError as error:
+            return await _sdk_memory_runtime(app).digest()
+        except HySdkError as error:
             raise HTTPException(status_code=503, detail=str(error)) from None
 
     @app.get(
@@ -3881,17 +3856,24 @@ def create_app(
         # Two numbers, not one: 675 as a single total reads as "stuck", when
         # what is true is that the three from this morning are nearly done
         # and the rest is catch-up (R-02).
-        supervisor = app.state.memory_processing_supervisor
+        supervisor = app.state.sdk_memory_runtime
         depths = supervisor.queue_depths()
         active_queue, in_flight_count = supervisor.active_depths()
         planned, current = supervisor.manual_batch_progress()
+        sdk_counts = (
+            supervisor.status_counts() if app.state.memory_authority is MemoryAuthority.SDK else {}
+        )
         # `running` counts rows the store still marks running, which includes
         # work a worker has already given up on. Reporting that as "what is
         # running" is how a stalled queue looks busy, so the strand is given
         # its own number.
         running_rows = counts.get(MemoryTaskStatus.RUNNING, 0)
         return MemoryProcessingSummary(
-            pending=counts.get(MemoryTaskStatus.PENDING, 0),
+            pending=(
+                sdk_counts.get("pending", 0)
+                if sdk_counts
+                else counts.get(MemoryTaskStatus.PENDING, 0)
+            ),
             fresh_pending=depths.fresh,
             backlog_pending=depths.backlog,
             outside_window_pending=counts.get(
@@ -3899,20 +3881,26 @@ def create_app(
                 0,
             ),
             recent_completions=depths.recent_completions,
-            running=running_rows,
-            stranded_running=max(0, running_rows - in_flight_count),
+            running=sdk_counts.get("running", running_rows),
+            stranded_running=(0 if sdk_counts else max(0, running_rows - in_flight_count)),
             unexecuted_policy_failures=store.unexecuted_policy_failures(),
-            waiting_for_model=counts.get(MemoryTaskStatus.WAITING_FOR_MODEL, 0),
-            completed=counts.get(MemoryTaskStatus.COMPLETED, 0),
-            failed=counts.get(MemoryTaskStatus.FAILED, 0),
-            unknown=counts.get(MemoryTaskStatus.UNKNOWN, 0),
+            waiting_for_model=sdk_counts.get(
+                "waiting_for_model",
+                counts.get(MemoryTaskStatus.WAITING_FOR_MODEL, 0),
+            ),
+            completed=sdk_counts.get(
+                "completed",
+                counts.get(MemoryTaskStatus.COMPLETED, 0),
+            ),
+            failed=sdk_counts.get("failed", counts.get(MemoryTaskStatus.FAILED, 0)),
+            unknown=sdk_counts.get("unknown", counts.get(MemoryTaskStatus.UNKNOWN, 0)),
             unknown_reasons=store.unknown_error_counts(),
             blocked_stages=tuple(
                 MemoryProcessingBlock(kind=kind, reason=reason, count=count)
                 for kind, reason, count in store.blocked_stages()[:6]
             ),
             seconds_per_completion=store.seconds_per_completion(),
-            extraction_window_preset=supervisor.extraction_window_settings().window_preset,
+            extraction_window_preset=app.state.extraction_settings_store.get().window_preset,
             manual_extract_batches_planned=planned,
             manual_extract_batches_current=current,
             active_queue=active_queue,
@@ -3930,15 +3918,17 @@ def create_app(
     ) -> RetryUnknownMemoryTasksReceipt:
         """Retry unknown writes only behind the user's explicit confirmation."""
         del request
-        reconciled = await _memory_processing_supervisor(app).reconcile_unknown_outputs()
+        reconciled = await _sdk_memory_runtime(app).reconcile_unknown_outputs()
         from one_memory.memory_server.durable_queue import DurableWriteQueue
 
         queue = DurableWriteQueue(app.state.memory_write_queue_path)
         unknown = queue.unknown_jobs(limit=2048)
-        for job in unknown:
-            queue.resolve_unknown_from_linked_output(job.job_id)
+        if app.state.memory_authority is MemoryAuthority.LEGACY:
+            for job in unknown:
+                queue.resolve_unknown_from_linked_output(job.job_id)
         requeued = sum(queue.retry_unknown_confirmed(job.job_id) for job in unknown)
-        requeued += app.state.memory_task_store.retry_unknown_confirmed()
+        if app.state.memory_authority is MemoryAuthority.LEGACY:
+            requeued += app.state.memory_task_store.retry_unknown_confirmed()
         return RetryUnknownMemoryTasksReceipt(reconciled=reconciled, requeued=requeued)
 
     @app.get(
@@ -3949,8 +3939,8 @@ def create_app(
     async def get_graph_width() -> GraphWidthStatus:
         """Report the mismatch. Never writes: this is polled by the app."""
         try:
-            mismatch = await _memory_processing_supervisor(app).graph_width_mismatch()
-        except HyMemoryBridgeError as error:
+            mismatch = await _sdk_memory_runtime(app).graph_width_mismatch()
+        except HySdkError as error:
             raise HTTPException(status_code=503, detail=str(error)) from None
         if mismatch is None:
             return GraphWidthStatus()
@@ -3973,10 +3963,10 @@ def create_app(
         decision is rejected by validation rather than assumed, because the
         assumption that cost a live graph once was made in code, not by a user.
         """
-        supervisor = _memory_processing_supervisor(app)
+        supervisor = _sdk_memory_runtime(app)
         try:
             outcome = await supervisor.decide_graph_width(request.decision)
-        except HyMemoryBridgeError as error:
+        except HySdkError as error:
             raise HTTPException(status_code=409, detail=str(error)) from error
         except OSError as error:
             # A failed export left nothing behind, but the user must be told
@@ -3994,7 +3984,7 @@ def create_app(
         dependencies=protected,
     )
     def get_extraction_settings() -> ExtractionSettings:
-        return _memory_processing_supervisor(app).extraction_window_settings()
+        return app.state.extraction_settings_store.get()
 
     @app.put(
         "/api/v1/extraction-settings",
@@ -4002,7 +3992,7 @@ def create_app(
         dependencies=protected,
     )
     def put_extraction_settings(request: ExtractionWindowUpdate) -> ExtractionSettings:
-        return _memory_processing_supervisor(app).set_extraction_window(request.window_preset)
+        return app.state.extraction_settings_store.set_window_preset(request.window_preset)
 
     @app.get(
         "/api/v1/sedimentation",
@@ -4052,7 +4042,7 @@ def create_app(
         dependencies=protected,
     )
     def manual_extract(request: ManualExtractRequest) -> ManualExtractReceipt:
-        supervisor = _memory_processing_supervisor(app)
+        supervisor = _sdk_memory_runtime(app)
         task_ids: tuple[str, ...] | None = None
         if request.evidence_ids:
             resolved: list[str] = []
@@ -4170,11 +4160,11 @@ def create_app(
         """
         store = _memory_store(app)
         try:
-            await _memory_processing_supervisor(app).update_hy_memory(
+            await _sdk_memory_runtime(app).update_hy_memory(
                 memory.memory_id,
                 memory.content,
             )
-        except HyMemoryBridgeError as error:
+        except HySdkError as error:
             return store.fail_index_sync(
                 memory.memory_id,
                 user_id="local",
@@ -4213,7 +4203,7 @@ def create_app(
             if action == "delete":
                 deletion = _memory_store(app).deletion(memory_id, user_id="local")
                 if deletion is None or deletion.status is not DeletionStatus.COMPLETED:
-                    await _memory_processing_supervisor(app).finish_deletion(memory_id)
+                    await _finish_deletion(app, memory_id)
             return memory
         except KeyError:
             raise HTTPException(status_code=404, detail="memory_not_found") from None
@@ -4378,7 +4368,7 @@ async def _request_answerer(
 
 def _sdk_request_answerer(app: FastAPI) -> SdkMemoryAnswerer:
     return SdkMemoryAnswerer(
-        bridge=_sdk_library_bridge(app),
+        client_handle=_sdk_client_handle(app),
         profiles=_model_profile_store(app),
         secrets=_secret_store(app),
         authorize_generation=lambda profile, question, grounding: _authorize_sdk_answer_generation(
@@ -4403,9 +4393,9 @@ async def _authorize_sdk_answer_generation(
         identity = await asyncio.to_thread(_model_profile_store(app).active_identity)
         if identity != (profile.profile_id, profile.revision):
             raise MemoryAnswerError("memory_answer_generation_not_authorized")
-        bridge = _sdk_library_bridge(app)
+        handle = _sdk_client_handle(app)
         for memory in grounding:
-            current = await bridge.get_sdk_memory(memory.memory_id)
+            current = await _get_sdk_memory(handle, memory.memory_id)
             if current is None:
                 raise MemoryAnswerError("memory_answer_generation_not_authorized")
             projected = SdkAskMemory(
@@ -4463,9 +4453,8 @@ class _LazyASGIApp:
     `create_app()` must not run at import time. The managed `serve()` entry
     imports `create_app` from this module before it can construct its own app
     with `manage_runtime=True`; an eager module-level app was therefore built
-    first with the default `manage_runtime=False`. With retrieval index enabled
-    that default app hit `retrieval_runtime_ownership_required` and killed the
-    process before the managed app could bind a port.
+    first with the default `manage_runtime=False`, doing startup work for an
+    app that never serves.
     """
 
     def __init__(self) -> None:
@@ -4488,20 +4477,6 @@ class _LazyASGIApp:
 app = _LazyASGIApp()
 
 
-class _LazySdkBridge:
-    """Resolve the active model only when an SDK operation is actually used."""
-
-    def __init__(self, app: FastAPI) -> None:
-        self.app = app
-
-    def __getattr__(self, name: str) -> Callable[..., Awaitable[Any]]:
-        async def call(*args: Any, **kwargs: Any) -> Any:
-            method = getattr(_sdk_library_bridge(self.app), name)
-            return await method(*args, **kwargs)
-
-        return call
-
-
 def _sync_refused(code: str) -> HTTPException:
     """A sync refusal that says which stage refused.
 
@@ -4925,6 +4900,54 @@ def _secret_store_detail(error: SecretStoreUnavailableError) -> str:
     return str(getattr(error, "code", None) or SecretStoreErrorCode.STORE_UNAVAILABLE)
 
 
+async def _finish_deletion(app: FastAPI, memory_id: str) -> None:
+    """Delete from the SDK and, on legacy authority, settle the deletion row.
+
+    Without the settle the legacy row stays `pending` for good, and the person
+    is shown a deletion that never finishes.
+    """
+    if app.state.memory_authority is MemoryAuthority.SDK:
+        await _sdk_memory_runtime(app).finish_deletion(memory_id)
+        return
+    await _work_legacy_deletions(app, memory_id=memory_id)
+
+
+async def _work_legacy_deletions(app: FastAPI, *, memory_id: str | None = None) -> None:
+    store = _memory_store(app)
+    runtime = _sdk_memory_runtime(app)
+    claims = await asyncio.to_thread(store.claim_deletions, user_id="local", memory_id=memory_id)
+    for claim in claims:
+        try:
+            await runtime.finish_deletion(claim.memory_id)
+        except HySdkError as error:
+            code: str | None = error.code
+        except Exception as error:  # noqa: BLE001 - recorded so it waits its turn
+            logger.warning("memory_deletion_unexpected_failure: %s", type(error).__name__)
+            code = "hy_memory_delete_unexpected"
+        else:
+            code = None
+        await asyncio.to_thread(store.settle_deletion, claim, user_id="local", error_code=code)
+
+
+_LEGACY_DELETION_SYNC_IDLE_SECONDS = 60.0
+
+
+async def _keep_legacy_deletions_in_sync(app: FastAPI) -> None:
+    """Retry legacy deletions the SDK did not take; nothing else asks again."""
+    store = _memory_store(app)
+    while True:
+        wait = _LEGACY_DELETION_SYNC_IDLE_SECONDS
+        try:
+            await _work_legacy_deletions(app)
+            due = await asyncio.to_thread(store.next_deletion_due, user_id="local")
+        except Exception as error:  # noqa: BLE001 - reported, then retried
+            report_exception("memory.deletion_sync", "memory_deletion_sync.failed", error)
+        else:
+            if due is not None:
+                wait = min(wait, max(0.0, (due - store.clock()).total_seconds()))
+        await asyncio.sleep(wait)
+
+
 def _memory_store(app: FastAPI) -> MemoryGovernanceStore:
     return cast(MemoryGovernanceStore, app.state.memory_governance_store)
 
@@ -5200,7 +5223,7 @@ def _resume_sedimentation(app: FastAPI) -> None:
         return
     app.state.sedimentation_resume = watched_task(
         asyncio.get_running_loop(),
-        lambda: _memory_processing_supervisor(app).resume_allowed(),
+        lambda: _sdk_memory_runtime(app).resume_allowed(),
         component="memory.sedimentation",
         name="one-memory-sedimentation-resume",
     )
@@ -5273,7 +5296,7 @@ def _report_visibility_failure(phase: str, error: BaseException) -> None:
         event="memory_visibility.background_failed",
         outcome="failed",
         error_type=type(error).__name__,
-        details={"phase": phase, "reason": str(error)[:200]},
+        details={"phase": phase},
     )
 
 
@@ -5368,9 +5391,8 @@ async def _run_backfill(app: FastAPI) -> None:
 
 
 def _evidence_excerpt(app: FastAPI, record: EvidenceRecord) -> EvidenceExcerpt:
-    supervisor = _memory_processing_supervisor(app)
     task = _memory_task_store(app).task_for_evidence(record.evidence_id)
-    window = extraction_window_for(supervisor.extraction_window_settings().window_preset)
+    window = extraction_window_for(app.state.extraction_settings_store.get().window_preset)
     return EvidenceExcerpt(
         evidence_id=record.evidence_id,
         available=True,
@@ -5390,23 +5412,62 @@ def _evidence_excerpt(app: FastAPI, record: EvidenceRecord) -> EvidenceExcerpt:
     )
 
 
-def _memory_processing_supervisor(app: FastAPI) -> MemoryProcessingSupervisor:
+def _sdk_memory_runtime(app: FastAPI) -> SdkMemoryRuntime:
     return cast(
-        MemoryProcessingSupervisor,
-        app.state.memory_processing_supervisor,
+        SdkMemoryRuntime,
+        app.state.sdk_memory_runtime,
     )
 
 
-def _sdk_library_bridge(app: FastAPI) -> HyMemoryBridge:
-    bridge = cast(HyMemoryBridge | None, app.state.sdk_library_bridge)
-    if bridge is None:
-        bridge = HyMemoryBridge(
+def _sdk_client_handle(app: FastAPI) -> SdkClientHandle:
+    handle = cast(SdkClientHandle | None, app.state.sdk_client_handle)
+    if handle is None:
+        handle = SdkClientHandle(
             profile=builtin_retrieval_snapshot(),
             api_key=None,
             data_dir=cast(Settings, app.state.settings).data_dir,
         )
-        app.state.sdk_library_bridge = bridge
-    return bridge
+        app.state.sdk_client_handle = handle
+    return handle
+
+
+async def _get_sdk_memory(
+    handle: SdkClientHandle,
+    memory_id: str,
+) -> HySdkMemoryRecord | None:
+    client = await handle.client()
+    raw = await asyncio.to_thread(client.get, memory_id)
+    return None if raw is None else HySdkMemoryRecord.from_sdk(raw)
+
+
+async def _search_sdk_memories(
+    handle: SdkClientHandle,
+    query: str,
+    *,
+    user_id: str,
+    agent_id: str,
+    limit: int = 12,
+) -> tuple[object, ...]:
+    client = await handle.client()
+    raw = await asyncio.to_thread(
+        client.search,
+        query,
+        user_ids=[user_id],
+        agent_ids=[] if agent_id == "*" else [agent_id],
+        limit=limit,
+        min_score=sdk_relevance_floor(),
+        schema_min_score=sdk_relevance_floor(),
+        schema_limit=limit,
+        intention_limit=limit,
+        scene="normal",
+        reader=HY_MEMORY_READER,
+    )
+    return parse_sdk_search(raw, error_code="hy_memory_search_shape_unknown")
+
+
+def _sdk_http_error(error: Exception, fallback: str) -> HTTPException:
+    detail = error.code if isinstance(error, HySdkError) else fallback
+    return HTTPException(status_code=503, detail=detail)
 
 
 def _evidence_store(app: FastAPI) -> EvidenceStore:

memhub/apps/core/src/one_memory/cli.py

@@ -1,5 +1,3 @@
-import asyncio
-import hashlib
 import os
 import signal
 import socket
@@ -31,13 +29,51 @@ def _end_by(seconds: str | None) -> None:
 if __name__ == "__main__" and sys.argv[1:2] == ["capture"]:
     _end_by(os.environ.get("ONE_MEMORY_CAPTURE_DEADLINE_S"))
 
+import errno  # noqa: E402
 import json  # noqa: E402
-from collections.abc import Sequence  # noqa: E402
+import re  # noqa: E402
+from collections.abc import Iterator, Sequence  # noqa: E402
 from typing import TextIO  # noqa: E402
 
 from one_memory.config import load_settings  # noqa: E402
 from one_memory.runtime_client import RuntimeStatus, inspect_runtime  # noqa: E402
 
+# The startup dialog can only act on the name it is given: running out of disk
+# and a damaged backup are different next steps. These are the names that reach
+# it.
+_SAFE_MIGRATION_CODE = re.compile(r"^[a-z][a-z0-9_]{0,95}$")
+_OUT_OF_SPACE = frozenset({errno.ENOSPC, getattr(errno, "EDQUOT", errno.ENOSPC)})
+
+
+def _exception_chain(error: BaseException) -> Iterator[BaseException]:
+    seen: set[int] = set()
+    current: BaseException | None = error
+    while current is not None and id(current) not in seen:
+        seen.add(id(current))
+        yield current
+        current = current.__cause__ or current.__context__
+
+
+def migration_failure_code(
+    error: BaseException, *, fallback: str = "sdk_migration_preflight_failed"
+) -> str:
+    """Name a migration failure; never guess a code from free text.
+
+    Errs toward the caller's generic code: an exception this does not recognize
+    keeps its message out of the document Desktop reads.
+    """
+    chain = list(_exception_chain(error))
+    for item in chain:
+        if isinstance(item, OSError) and item.errno in _OUT_OF_SPACE:
+            return "sdk_migration_disk_space_insufficient"
+    for item in chain:
+        code = getattr(item, "code", "")
+        if not isinstance(code, str) or not _SAFE_MIGRATION_CODE.match(code):
+            continue
+        if code.startswith(("sdk_data_plane_", "sdk_migration_")):
+            return code
+    return fallback
+
 
 def main() -> None:
     if sys.argv[1:] == ["--keychain-read"]:
@@ -77,16 +113,10 @@ def run(arguments: Sequence[str], *, output: TextIO | None = None) -> int:
     command, *options = arguments
     if command == "restore":
         return restore(options, stream)
-    if command == "migration-bundle":
-        return migration_bundle(options, stream)
     if command == "sdk-migrate":
         return sdk_migrate(options, stream)
-    if command == "sdk-cutover":
-        return sdk_cutover(options, stream)
     if command == "sdk-migration-status":
         return sdk_migration_status(options, stream)
-    if command == "sdk-cleanup":
-        return sdk_cleanup(options, stream)
     if command == "boundary" and not options:
         from one_memory.workbuddy_boundary import write_hook
 
@@ -107,11 +137,8 @@ def run(arguments: Sequence[str], *, output: TextIO | None = None) -> int:
         stream.write(
             "Usage: one-memory [serve [--maintenance|--reader]|status [--json]|doctor [--json]"
             "|restore --source <managed-backup> --yes [--json]"
-            "|migration-bundle --bundle-id <id> --runtime-build <build> --yes [--json]"
-            "|sdk-migrate --migration-id <id> --bundle-id <id> --yes [--json]"
-            "|sdk-cutover --migration-id <id> --bundle-id <id> --yes [--json]"
+            "|sdk-migrate --runtime-build <build> --yes [--json]"
             "|sdk-migration-status [--json]"
-            "|sdk-cleanup --yes [--json]"
             "|boundary [--peek|--ack TOKEN]|capture SOURCE]\n"
         )
         return 2
@@ -253,19 +280,20 @@ def _write_restore(
         stream.write(f"{text}\n")
 
 
-def migration_bundle(options: Sequence[str], stream: TextIO) -> int:
-    """Create a complete management + SDK bundle while Runtime is stopped."""
-    bundle_id: str | None = None
+def sdk_migrate(options: Sequence[str], stream: TextIO) -> int:
+    """Bring a stopped home to SDK authority in one resumable command.
+
+    Backs up the home, makes queue work independent of the legacy tables,
+    commits SDK authority and drops those tables. The SDK already holds every
+    memory and raw turn, so nothing is copied. With `--json`, each step is one
+    progress line and the last line is the result.
+    """
     runtime_build: str | None = None
     confirmed = False
     as_json = False
     index = 0
     while index < len(options):
         option = options[index]
-        if option == "--bundle-id" and index + 1 < len(options):
-            bundle_id = options[index + 1]
-            index += 2
-            continue
         if option == "--runtime-build" and index + 1 < len(options):
             runtime_build = options[index + 1]
             index += 2
@@ -280,116 +308,8 @@ def migration_bundle(options: Sequence[str], stream: TextIO) -> int:
             continue
         stream.write(f"one-memory: 参数无效：{option}\n")
         return 2
-    if not bundle_id or not runtime_build:
-        stream.write("one-memory: migration-bundle 需要 --bundle-id <id> --runtime-build <build>\n")
-        return 2
-
-    settings = load_settings()
-    status = inspect_runtime(settings.data_dir)
-    if status.state != "stopped":
-        document = {
-            "schema_version": 1,
-            "state": "blocked",
-            "code": "sdk_migration_bundle_runtime_running",
-            "runtime_state": status.state,
-        }
-        _write_restore(stream, as_json, document, "Runtime 仍在运行；不能创建迁移备份。")
-        return 4
-    if not confirmed:
-        document = {
-            "schema_version": 1,
-            "state": "preview",
-            "bundle_id": bundle_id,
-            "requires_confirmation": True,
-        }
-        _write_restore(stream, as_json, document, "预览通过；加 --yes 后创建完整迁移备份。")
-        return 4
-
-    from one_memory.sdk_migration_bundle import (
-        MigrationBundleRejected,
-        create_sdk_migration_bundle,
-        inspect_sdk_migration_bundle,
-    )
-
-    try:
-        result = create_sdk_migration_bundle(
-            data_dir=settings.data_dir,
-            bundle_id=bundle_id,
-            runtime_build=runtime_build,
-            runtime_stopped=True,
-        )
-    except MigrationBundleRejected as error:
-        if error.code == "sdk_migration_bundle_exists":
-            try:
-                manifest = inspect_sdk_migration_bundle(
-                    data_dir=settings.data_dir,
-                    bundle_id=bundle_id,
-                )
-            except MigrationBundleRejected:
-                pass
-            else:
-                document = {
-                    "schema_version": 1,
-                    "state": "reused",
-                    "bundle_id": bundle_id,
-                    "file_count": manifest.get("file_count"),
-                    "total_bytes": manifest.get("total_bytes"),
-                }
-                _write_restore(
-                    stream,
-                    as_json,
-                    document,
-                    f"已验证并复用迁移备份：{bundle_id}",
-                )
-                return 0
-        document = {
-            "schema_version": 1,
-            "state": "failed",
-            "code": error.code,
-        }
-        _write_restore(stream, as_json, document, f"迁移备份失败：{error.code}")
-        return 1
-    document = {
-        "schema_version": 1,
-        "state": "created",
-        "bundle_id": result.bundle_id,
-        "bundle_path": str(result.bundle_path),
-        "file_count": result.file_count,
-        "total_bytes": result.total_bytes,
-    }
-    _write_restore(stream, as_json, document, f"完整迁移备份已创建：{result.bundle_path}")
-    return 0
-
-
-def sdk_migrate(options: Sequence[str], stream: TextIO) -> int:
-    """Backfill legacy memory/raw into SDK while keeping legacy authority."""
-    migration_id: str | None = None
-    bundle_id: str | None = None
-    confirmed = False
-    as_json = False
-    index = 0
-    while index < len(options):
-        option = options[index]
-        if option == "--migration-id" and index + 1 < len(options):
-            migration_id = options[index + 1]
-            index += 2
-            continue
-        if option == "--bundle-id" and index + 1 < len(options):
-            bundle_id = options[index + 1]
-            index += 2
-            continue
-        if option == "--yes":
-            confirmed = True
-            index += 1
-            continue
-        if option == "--json":
-            as_json = True
-            index += 1
-            continue
-        stream.write(f"one-memory: 参数无效：{option}\n")
-        return 2
-    if not migration_id or not bundle_id:
-        stream.write("one-memory: sdk-migrate 需要 --migration-id <id> --bundle-id <id>\n")
+    if not runtime_build:
+        stream.write("one-memory: sdk-migrate 需要 --runtime-build <build>\n")
         return 2
 
     settings = load_settings()
@@ -404,223 +324,59 @@ def sdk_migrate(options: Sequence[str], stream: TextIO) -> int:
         _write_restore(stream, as_json, document, "Runtime 仍在运行；不能迁移。")
         return 4
     if not confirmed:
-        document = {
-            "schema_version": 1,
-            "state": "preview",
-            "migration_id": migration_id,
-            "bundle_id": bundle_id,
-            "requires_confirmation": True,
-            "commits_cutover": False,
-        }
-        _write_restore(
-            stream,
-            as_json,
-            document,
-            "预览通过；加 --yes 后执行可续跑 backfill，但不会切换 authority。",
-        )
+        document = {"schema_version": 1, "state": "preview", "requires_confirmation": True}
+        _write_restore(stream, as_json, document, "预览通过；加 --yes 后执行迁移。")
         return 4
 
-    from one_memory.database_access import close_main_databases
-    from one_memory.hy_memory_bridge import HyMemoryBridge
     from one_memory.hy_runtime_config import HY_MEMORY_NAMESPACE, HY_MEMORY_VERSION
-    from one_memory.memory_server.migration_runner import SqliteMigrationStore, migrate
-    from one_memory.memory_server.migrations import MIGRATIONS
-    from one_memory.model_profiles import ModelProfileStore
-    from one_memory.sdk_migration_bundle import (
-        MigrationBundleRejected,
-        inspect_sdk_migration_bundle,
-    )
-    from one_memory.sdk_migration_runner import BackfillResult, run_sdk_data_plane_backfill
-    from one_memory.secrets import KeyringSecretStore, SecretStoreUnavailableError
+    from one_memory.sdk_migration_runner import run_startup_migration
 
-    database = settings.data_dir / "one-memory.sqlite3"
-    try:
-        manifest = inspect_sdk_migration_bundle(
-            data_dir=settings.data_dir,
-            bundle_id=bundle_id,
-        )
-        asyncio.run(migrate(SqliteMigrationStore(database), MIGRATIONS))
-        close_main_databases()
-        profiles = ModelProfileStore(database)
-        identity = profiles.active_identity()
-        if identity is None:
-            raise RuntimeError("model_profile_not_active")
-        profile = profiles.snapshot(*identity)
-        secrets = KeyringSecretStore()
-        secret = profiles.resolve_snapshot_secret(*identity, secrets)
-        if profile.has_api_key and secret is None:
-            raise RuntimeError("model_secret_unavailable")
-    except MigrationBundleRejected as error:
-        code = error.code
-    except SecretStoreUnavailableError:
-        code = "secure_storage_unavailable"
-    except Exception as error:
-        code = (
-            str(error)
-            if str(error)
-            in {
-                "model_profile_not_active",
-                "model_secret_unavailable",
-            }
-            else "sdk_migration_preflight_failed"
-        )
-    else:
-        namespace_fingerprint = hashlib.sha256(HY_MEMORY_NAMESPACE.encode()).hexdigest()
-        backend_material = json.dumps(
-            {
-                "namespace": HY_MEMORY_NAMESPACE,
-                "embedding_provider": str(profile.embedding_provider_kind or ""),
-                "embedding_model": profile.embedding_model,
-                "embedding_dimensions": profile.embedding_dimensions,
-                "vector": "chroma",
-                "cache": "sqlite",
-                "history": "sqlite",
-                "coding": "sqlite",
-            },
-            sort_keys=True,
-            separators=(",", ":"),
-        )
-        backend_fingerprint = hashlib.sha256(backend_material.encode()).hexdigest()
-
-        async def run_backfill() -> BackfillResult:
-            bridge = HyMemoryBridge(
-                profile=profile,
-                api_key=secret,
-                data_dir=settings.data_dir,
-            )
-            try:
-                return await run_sdk_data_plane_backfill(
-                    database_path=database,
-                    bridge=bridge,
-                    migration_id=migration_id,
-                    bundle_id=bundle_id,
-                    source_runtime_build=str(manifest.get("runtime_build") or ""),
-                    target_sdk_version=HY_MEMORY_VERSION,
-                    source_schema_version=MIGRATIONS[-1].version,
-                    namespace_fingerprint=namespace_fingerprint,
-                    backend_fingerprint=backend_fingerprint,
-                )
-            finally:
-                await bridge.close()
-
-        try:
-            result = asyncio.run(run_backfill())
-        except Exception:
-            code = "sdk_migration_backfill_failed"
-        else:
-            document = {
-                "schema_version": 1,
-                "state": result.state,
-                "phase": result.phase,
-                "migration_id": result.migration_id,
-                "counts": result.counts,
-                "memory_items": result.memory_items,
-                "raw_items": result.raw_items,
-                "commits_cutover": False,
-            }
-            _write_restore(
-                stream,
-                as_json,
-                document,
-                "SDK backfill 已完成；authority 尚未切换。",
-            )
-            return 0 if result.state == "ready" else 4
-
-    document = {"schema_version": 1, "state": "failed", "code": code}
-    _write_restore(stream, as_json, document, f"SDK backfill 失败：{code}")
-    return 1
-
-
-def sdk_cutover(options: Sequence[str], stream: TextIO) -> int:
-    """Commit SDK authority only after offline reconciliation and work drain."""
-    migration_id: str | None = None
-    bundle_id: str | None = None
-    confirmed = False
-    as_json = False
-    index = 0
-    while index < len(options):
-        option = options[index]
-        if option == "--migration-id" and index + 1 < len(options):
-            migration_id = options[index + 1]
-            index += 2
-            continue
-        if option == "--bundle-id" and index + 1 < len(options):
-            bundle_id = options[index + 1]
-            index += 2
-            continue
-        if option == "--yes":
-            confirmed = True
-            index += 1
-            continue
-        if option == "--json":
-            as_json = True
-            index += 1
-            continue
-        stream.write(f"one-memory: 参数无效：{option}\n")
-        return 2
-    if not migration_id or not bundle_id:
-        stream.write("one-memory: sdk-cutover 需要 --migration-id <id> --bundle-id <id>\n")
-        return 2
-    settings = load_settings()
-    status = inspect_runtime(settings.data_dir)
-    if status.state != "stopped":
-        document = {
-            "schema_version": 1,
-            "state": "blocked",
-            "code": "sdk_cutover_runtime_running",
-            "runtime_state": status.state,
-        }
-        _write_restore(stream, as_json, document, "Runtime 仍在运行；不能切换 authority。")
-        return 4
-    if not confirmed:
-        document = {
-            "schema_version": 1,
-            "state": "preview",
-            "migration_id": migration_id,
-            "bundle_id": bundle_id,
-            "requires_confirmation": True,
-        }
-        _write_restore(stream, as_json, document, "预览通过；加 --yes 后提交 SDK authority。")
-        return 4
-
-    from one_memory.hy_runtime_config import HY_MEMORY_VERSION
-    from one_memory.sdk_data_plane_migration import DataPlaneMigrationError
-    from one_memory.sdk_migration_bundle import MigrationBundleRejected
-    from one_memory.sdk_migration_runner import commit_reconciled_cutover
-
-    try:
-        marker = commit_reconciled_cutover(
-            data_dir=settings.data_dir,
-            database_path=settings.data_dir / "one-memory.sqlite3",
-            migration_id=migration_id,
-            bundle_id=bundle_id,
-            sdk_version=HY_MEMORY_VERSION,
-        )
-    except (DataPlaneMigrationError, MigrationBundleRejected) as error:
-        code = getattr(error, "code", str(error))
-        document = {
-            "schema_version": 1,
-            "state": "blocked",
-            "code": code,
-            "migration_id": migration_id,
-            "bundle_id": bundle_id,
-        }
-        _write_restore(stream, as_json, document, f"SDK authority 未切换：{code}")
-        return 4
-    document = {
-        "schema_version": 1,
-        "state": "cutover",
-        "migration_id": marker.migration_id,
-        "bundle_id": marker.backup_bundle_id,
-        "sdk_version": marker.sdk_version,
-        "completed_at": marker.completed_at,
+    step_names = {
+        "bundle": "正在备份记忆数据……",
+        "prepare": "正在整理待处理的写入任务……",
+        "cutover": "正在切换数据源……",
+        "vectors": "正在接回旧版本的向量数据……",
+        "verify": "正在用新版本核对记忆能否读回……",
+        "cleanup": "正在删除旧表……",
     }
-    _write_restore(stream, as_json, document, "SDK authority 已切换。")
+
+    def progress(step: str) -> None:
+        if as_json:
+            stream.write(
+                json.dumps({"schema_version": 1, "state": "running", "step": step}) + "\n"
+            )
+        else:
+            stream.write(f"{step_names.get(step, step)}\n")
+        stream.flush()
+
+    try:
+        result = run_startup_migration(
+            data_dir=settings.data_dir,
+            runtime_build=runtime_build,
+            sdk_version=HY_MEMORY_VERSION,
+            namespace=HY_MEMORY_NAMESPACE,
+            progress=progress,
+        )
+    except Exception as error:
+        from one_memory.sdk_migration_verify import MemoryReadbackFailed
+
+        code = migration_failure_code(error)
+        failed: dict[str, object] = {"schema_version": 1, "state": "failed", "code": code}
+        if isinstance(error, MemoryReadbackFailed):
+            failed["readback"] = error.report.as_document()
+        _write_restore(stream, as_json, failed, f"迁移未完成：{code}")
+        return 1
+    messages = {
+        "not_required": "不需要迁移。",
+        "complete": "迁移已完成；遗留的旧表或向量已处理。",
+        "cutover": "迁移完成：记忆数据已以 SDK 为准。",
+    }
+    _write_restore(stream, as_json, result.as_document(), messages[result.state])
     return 0
 
 
 def sdk_migration_status(options: Sequence[str], stream: TextIO) -> int:
-    """Read-only startup probe used before taking the Runtime maintenance fence."""
+    """Read-only probe: does this home need `sdk-migrate`?"""
     if list(options) not in ([], ["--json"]):
         stream.write("one-memory: sdk-migration-status 只接受 --json\n")
         return 2
@@ -628,65 +384,40 @@ def sdk_migration_status(options: Sequence[str], stream: TextIO) -> int:
     settings = load_settings()
     database_path = settings.data_dir / "one-memory.sqlite3"
     if not database_path.is_file():
-        document = {
+        document: dict[str, object] = {
             "schema_version": 1,
-            "authority": "legacy",
-            "state": "empty",
-            "migration_id": None,
+            "authority": "sdk",
+            "state": "not_required",
         }
     else:
-        import sqlite3
-
-        from one_memory.database_access import main_database
+        from one_memory.hy_runtime_config import HY_MEMORY_NAMESPACE
         from one_memory.sdk_data_plane_migration import (
             MemoryAuthority,
             SdkDataPlaneMigrationStore,
         )
+        from one_memory.sdk_migration_runner import abandoned_staging_present
+        from one_memory.sdk_migration_verify import read_active_profile
+        from one_memory.sdk_vector_reattach import legacy_vector_collections
         from one_memory.target_schema import legacy_tables_present
 
-        migration_store = SdkDataPlaneMigrationStore(database_path)
-        marker = migration_store.cutover()
-        authority = migration_store.authority()
-        stat = database_path.stat()
-        generation = hashlib.sha256(f"{stat.st_size}:{stat.st_mtime_ns}".encode()).hexdigest()[:16]
+        authority = SdkDataPlaneMigrationStore(database_path).authority()
         if authority is MemoryAuthority.SDK:
-            legacy_tables = legacy_tables_present(database_path)
-            document = {
-                "schema_version": 1,
-                "authority": "sdk",
-                "state": "cleanup_required" if legacy_tables else "complete",
-                "migration_id": marker.migration_id if marker else None,
-                "generation": generation,
-                "legacy_tables": list(legacy_tables),
-            }
+            profile = read_active_profile(database_path)
+            leftovers = bool(
+                legacy_tables_present(database_path)
+                or abandoned_staging_present(
+                    data_dir=settings.data_dir,
+                    namespace=HY_MEMORY_NAMESPACE,
+                )
+                or (
+                    profile is not None
+                    and legacy_vector_collections(settings.data_dir, profile)
+                )
+            )
+            state = "required" if leftovers else "not_required"
         else:
-            with main_database(database_path).read(row_factory=sqlite3.Row) as database:
-                tables = {
-                    str(row["name"])
-                    for row in database.execute("SELECT name FROM sqlite_master WHERE type='table'")
-                }
-                counts = {
-                    "memory": (
-                        int(
-                            database.execute("SELECT COUNT(*) FROM governed_memories").fetchone()[0]
-                        )
-                        if "governed_memories" in tables
-                        else 0
-                    ),
-                    "raw": (
-                        int(database.execute("SELECT COUNT(*) FROM evidence").fetchone()[0])
-                        if "evidence" in tables
-                        else 0
-                    ),
-                }
-            document = {
-                "schema_version": 1,
-                "authority": "legacy",
-                "state": "required" if any(counts.values()) else "empty",
-                "migration_id": None,
-                "counts": counts,
-                "generation": generation,
-            }
+            state = "required"
+        document = {"schema_version": 1, "authority": authority.value, "state": state}
     if as_json:
         stream.write(f"{json.dumps(document, ensure_ascii=False)}\n")
     else:
@@ -694,65 +425,6 @@ def sdk_migration_status(options: Sequence[str], stream: TextIO) -> int:
     return 0
 
 
-def sdk_cleanup(options: Sequence[str], stream: TextIO) -> int:
-    if set(options) - {"--yes", "--json"} or options.count("--yes") > 1:
-        stream.write("one-memory: sdk-cleanup 只接受 --yes [--json]\n")
-        return 2
-    confirmed = "--yes" in options
-    as_json = "--json" in options
-    settings = load_settings()
-    status = inspect_runtime(settings.data_dir)
-    if status.state != "stopped":
-        document = {
-            "schema_version": 1,
-            "state": "blocked",
-            "code": "sdk_cleanup_runtime_running",
-        }
-        _write_restore(stream, as_json, document, "Runtime 仍在运行；不能清理旧表。")
-        return 4
-    if not confirmed:
-        _write_restore(
-            stream,
-            as_json,
-            {
-                "schema_version": 1,
-                "state": "preview",
-                "requires_confirmation": True,
-            },
-            "加 --yes 后删除已迁移旧表。",
-        )
-        return 4
-    from one_memory.sdk_data_plane_migration import DataPlaneMigrationError
-    from one_memory.sdk_migration_bundle import MigrationBundleRejected
-    from one_memory.sdk_migration_runner import resume_legacy_cleanup
-
-    try:
-        dropped = resume_legacy_cleanup(
-            data_dir=settings.data_dir,
-            database_path=settings.data_dir / "one-memory.sqlite3",
-        )
-    except (DataPlaneMigrationError, MigrationBundleRejected) as error:
-        code = getattr(error, "code", str(error))
-        _write_restore(
-            stream,
-            as_json,
-            {"schema_version": 1, "state": "blocked", "code": code},
-            f"旧表清理未完成：{code}",
-        )
-        return 4
-    _write_restore(
-        stream,
-        as_json,
-        {
-            "schema_version": 1,
-            "state": "clean",
-            "dropped_tables": list(dropped),
-        },
-        "旧表已删除。",
-    )
-    return 0
-
-
 def _status_text(command: str, status: RuntimeStatus) -> str:
     if status.state == "running":
         location = f"{status.host}:{status.port}"

memhub/apps/core/src/one_memory/config.py

@@ -23,10 +23,6 @@ class Settings(BaseSettings):
     live_sync_enabled: bool = True
     automatic_memory_processing: bool = True
     imports_enabled: bool = True
-    # Experimental and unreleased: ownership/error shutdown still requires acceptance.
-    # Resume only explicitly initialized generations; never migrate at startup.
-    retrieval_index_enabled: bool = False
-    retrieval_minimum_free_bytes: int = Field(default=256 * 1024 * 1024, ge=0)
     # Bounded metadata only: query, original text and answers are never stored.
     call_event_retention_days: int | None = Field(default=30, ge=1, le=365)
     call_event_max_events: int | None = Field(default=10_000, ge=1, le=100_000)

memhub/apps/core/src/one_memory/context_routes.py

@@ -38,12 +38,18 @@ from one_memory.context_models import (
     SdkMemoryContextItem,
     SdkRawContextItem,
 )
-from one_memory.context_raw import revalidate_raw_context
+from one_memory.context_raw import read_raw_context, revalidate_raw_context
 from one_memory.context_search import search_context
 from one_memory.diagnostic_log import diagnostic_event, diagnostic_exception
 from one_memory.external_memory_access import external_access
+from one_memory.hy_runtime_config import HY_MEMORY_READER
+from one_memory.hy_sdk_runtime import sdk_relevance_floor
+from one_memory.hy_sdk_types import (
+    HySdkMemoryRecord,
+    parse_sdk_raw_search,
+    parse_sdk_search,
+)
 from one_memory.imports.storage import EvidenceStore
-from one_memory.retrieval.raw_lane import read_raw_indexed_context
 from one_memory.semantic_deadline import hy_wait_budget, search_within
 
 CALL_EVENT_RESPONSE_BUDGET_MS = 250
@@ -92,12 +98,11 @@ def context_router(dependencies: Sequence[Depends]) -> APIRouter:
             read = asyncio.create_task(
                 in_read_pool(
                     raw_read_pool(),
-                    read_raw_indexed_context,
+                    read_raw_context,
                     database,
                     value,
                     source_ids=authority.raw_sources,
                     deadline=lane_end,
-                    index=getattr(request.app.state, "retrieval_runtime", None),
                 )
             )
             done, _ = await asyncio.wait({read}, timeout=max(0, lane_end - time.monotonic()))
@@ -455,19 +460,33 @@ def sdk_context_router(dependencies: Sequence[Depends]) -> APIRouter:
         except Exception:
             raise HTTPException(503, "context_authority_unavailable") from None
 
-        bridge = getattr(request.app.state, "sdk_library_bridge", None)
-        if bridge is None:
+        client_handle = getattr(request.app.state, "sdk_client_handle", None)
+        if client_handle is None:
             raise HTTPException(503, "sdk_context_unavailable")
+        try:
+            client = await client_handle.client()
+        except Exception:
+            raise HTTPException(503, "sdk_context_unavailable") from None
 
         async def raw_lane() -> tuple[SdkRawContextItem, ...]:
             if "raw" not in value.kinds or not authority.raw_allowed:
                 return ()
-            hits = await bridge.search_sdk_raw(
+            raw = await asyncio.to_thread(
+                client.search,
                 value.query,
-                user_id="local",
-                source_ids=tuple(authority.raw_sources or ()),
+                user_ids=["local"],
+                agent_ids=[],
+                include_sources=list(authority.raw_sources or ()),
                 limit=value.limit,
+                min_score=0.0,
+                schema_min_score=0.0,
+                schema_limit=0,
+                intention_limit=0,
+                scene="chat",
+                reader=HY_MEMORY_READER,
+                layers=["l1_raw"],
             )
+            hits = parse_sdk_raw_search(raw)
             return tuple(
                 SdkRawContextItem(
                     raw_id=hit.raw_id,
@@ -487,15 +506,24 @@ def sdk_context_router(dependencies: Sequence[Depends]) -> APIRouter:
         async def memory_lane() -> tuple[SdkMemoryContextItem, ...]:
             if "memory" not in value.kinds or not authority.memory_allowed:
                 return ()
-            hits = await bridge.search_memories(
+            raw = await asyncio.to_thread(
+                client.search,
                 value.query,
-                user_id="local",
-                agent_id="*",
+                user_ids=["local"],
+                agent_ids=[],
                 limit=value.limit,
+                min_score=sdk_relevance_floor(),
+                schema_min_score=sdk_relevance_floor(),
+                schema_limit=value.limit,
+                intention_limit=value.limit,
+                scene="normal",
+                reader=HY_MEMORY_READER,
             )
+            hits = parse_sdk_search(raw, error_code="hy_memory_search_shape_unknown")
             items: list[SdkMemoryContextItem] = []
             for hit in hits:
-                memory = await bridge.get_sdk_memory(hit.memory_id)
+                record = await asyncio.to_thread(client.get, hit.memory_id)
+                memory = None if record is None else HySdkMemoryRecord.from_sdk(record)
                 if memory is None or memory.status != "active":
                     continue
                 items.append(

memhub/apps/core/src/one_memory/context_v2_coordinator.py

@@ -22,7 +22,7 @@ from dataclasses import dataclass
 from datetime import UTC, datetime
 from functools import partial
 from pathlib import Path
-from typing import Any, Literal, Protocol, cast
+from typing import Any, Literal, cast
 
 from one_memory.background_work import interactive_read_pool
 from one_memory.context_expand import ExpansionAuthority, ExpansionRefSigner
@@ -36,23 +36,8 @@ from one_memory.retrieval.models import (
     RawEvidence,
     RetrievalPath,
     RetrievalResponse,
-    TimeRange,
 )
 from one_memory.retrieval.query_plan import plan_local_query
-from one_memory.retrieval.raw_candidates import RawCandidateBatch
-from one_memory.retrieval.raw_evidence import materialize_raw_candidates
-
-
-class RawIndexLookup(Protocol):
-    def lookup_raw(
-        self,
-        query: str,
-        *,
-        source_ids: frozenset[str],
-        deadline: float,
-        time_range: TimeRange | None = None,
-    ) -> tuple[RawCandidateBatch, bool]: ...
-
 
 ExpandRefs = tuple[ExpansionRefSigner, ExpansionAuthority]
 
@@ -106,11 +91,6 @@ class LaneResult:
     coverage_limited: bool = False
 
 
-def _semantic_only_raw(item: RawEvidence) -> bool:
-    """Semantic overlap alone is a candidate, not a confirmed recall item."""
-    return set(item.matched_by) == {"semantic"}
-
-
 @dataclass(frozen=True)
 class StageTimings:
     raw_ms: float = 0.0
@@ -191,12 +171,6 @@ def _managed(
     return selected, truncated
 
 
-def _group_missing_requirements(item: RawEvidence | GovernedEvidence) -> list[str]:
-    if isinstance(item, RawEvidence) and _semantic_only_raw(item):
-        return ["relevance_confirmation"]
-    return []
-
-
 def _path(
     status: str,
     reason: str | None,
@@ -296,8 +270,7 @@ def read_lexical_raw_lane(
         ("lexical", "object") if plan.object_keys else ("lexical",),
         truncated,
         # Starvation means: this lane matched, then dropped the item whole. An
-        # ordinary no-match must stay empty, and an unconfigured semantic lane
-        # is not starvation.
+        # ordinary no-match must stay empty.
         starved and not items,
         probe_saturated,
     )
@@ -564,97 +537,6 @@ def warm_raw_read_path(
         return False
 
 
-def read_semantic_raw_lane(
-    index: RawIndexLookup | None,
-    database_path: Path,
-    query: CoordinatorQuery,
-    *,
-    source_ids: frozenset[str],
-    deadline: float,
-) -> LaneResult:
-    if not source_ids:
-        return LaneResult((), "unavailable", "permission_required", (), False)
-    # No derived index is the no-model baseline, not a failure: the lexical lane
-    # must still answer. Semantic is genuinely unsearched in that state, so it
-    # must not claim a channel or a coverage result it never produced.
-    if index is None:
-        return LaneResult((), "not_searched", None, (), False)
-    try:
-        batch, unpublished = index.lookup_raw(
-            query.query,
-            source_ids=source_ids,
-            deadline=deadline,
-            time_range=plan_local_query(query.query).time_range,
-        )
-    except Exception:
-        return LaneResult((), "unavailable", "index_unavailable", ("semantic",), False)
-    if not isinstance(batch, RawCandidateBatch):
-        return LaneResult((), "unavailable", "index_unavailable", ("semantic",), False)
-    if batch.reason == "not_authorized":
-        return LaneResult((), "unavailable", "permission_required", ("semantic",), False)
-    if batch.reason == "deadline":
-        return LaneResult((), "unavailable", "timeout_budget", ("semantic",), True)
-    if batch.reason in {"vector_unavailable", "index_unavailable"}:
-        return LaneResult((), "unavailable", batch.reason, ("semantic",), False)
-    store, generation = _index_state(index, database_path)
-    from one_memory.retrieval.index_store import RetrievalIndexStore
-
-    store = cast("RetrievalIndexStore | None", store)
-    if store is None or generation is None:
-        return LaneResult((), "unavailable", "index_unavailable", ("semantic",), False)
-    try:
-        materialized = materialize_raw_candidates(
-            store,
-            batch,
-            generation_id=generation,
-            source_ids=source_ids,
-            deadline=deadline,
-            character_budget=query.character_budget,
-        )
-    except Exception:
-        return LaneResult((), "unavailable", "index_unavailable", ("semantic",), False)
-    if materialized.reason == "deadline":
-        return LaneResult((), "unavailable", "timeout_budget", ("semantic",), True)
-    if materialized.reason in {"index_unavailable", "not_authorized"}:
-        return LaneResult((), "unavailable", materialized.reason, ("semantic",), False)
-    complete = materialized.complete and not unpublished
-    if materialized.reason == "evidence_changed":
-        status: Literal["ok", "empty", "unavailable", "not_searched"] = "unavailable"
-        reason = "evidence_changed"
-    elif not materialized.items and not complete:
-        status, reason = "unavailable", "index_incomplete"
-    elif materialized.reason == "character_budget":
-        status, reason = "ok", None
-    else:
-        status, reason = ("ok" if materialized.items else "empty"), None
-    return LaneResult(
-        tuple(materialized.items),
-        status,
-        reason,
-        ("semantic",),
-        materialized.reason == "character_budget",
-    )
-
-
-def _index_state(index: RawIndexLookup, database_path: Path) -> tuple[object | None, str | None]:
-    store = getattr(index, "_store", None)
-    generation = None
-    snapshot = getattr(index, "snapshot", None)
-    if snapshot is not None:
-        try:
-            generation = snapshot().generation_id
-        except Exception:
-            generation = None
-    if store is None:
-        try:
-            from one_memory.retrieval.index_store import RetrievalIndexStore
-
-            store = RetrievalIndexStore(database_path)
-        except Exception:
-            store = None
-    return store, generation
-
-
 def read_memory_lane(
     database_path: Path,
     query: CoordinatorQuery,
@@ -857,7 +739,6 @@ def coordinate_context_v2(
     query: CoordinatorQuery,
     *,
     database_path: Path,
-    index: RawIndexLookup | None = None,
     deadline: float,
     expand_refs: ExpandRefs | None = None,
 ) -> RetrievalResponse:
@@ -871,62 +752,29 @@ def coordinate_context_v2(
         lexical = read_lexical_raw_lane(
             database_path, query, source_ids=authority.raw_sources, deadline=deadline
         )
-        semantic = read_semantic_raw_lane(
-            index, database_path, query, source_ids=authority.raw_sources, deadline=deadline
-        )
         timings = StageTimings(raw_ms=(time.monotonic() - raw_started) * 1000)
         combined: dict[tuple[str, str], RawEvidence] = {}
-        for item in (*lexical.items, *semantic.items):
-            if not isinstance(item, RawEvidence):
-                continue
-            identity = _identity(item)
-            previous = combined.get(identity)
-            if previous is None or _semantic_only_raw(previous):
-                combined[identity] = item
-        # Semantic similarity is a candidate signal, not a verified recall
-        # result. Keep semantic-only candidates visible in coverage, but do not
-        # release their bodies for host injection unless the item is
-        # independently confirmed by the lexical/object lane. A global distance
-        # threshold is deliberately not used: the measured positive and
-        # negative distributions overlap.
-        semantic_only = {_identity(item) for item in combined.values() if _semantic_only_raw(item)}
-        confirmed = {_identity(item) for item in lexical.items if isinstance(item, RawEvidence)}
-        deferred_raw = semantic_only - confirmed
-        if deferred_raw:
-            combined = {
-                identity: item
-                for identity, item in combined.items()
-                if identity not in deferred_raw
-            }
-        channels = tuple(dict.fromkeys((*lexical.channels, *semantic.channels)))
+        for item in lexical.items:
+            if isinstance(item, RawEvidence):
+                combined.setdefault(_identity(item), item)
         if lexical.status == "unavailable":
             status: Literal["ok", "empty", "unavailable", "not_searched", "degraded"] = (
                 "unavailable"
             )
             reason = lexical.reason
-        elif semantic.status == "unavailable":
-            status = "unavailable"
-            reason = semantic.reason
-        elif combined:
-            status, reason = "ok", semantic.reason if semantic.status != "ok" else None
-        elif deferred_raw:
-            status, reason = "degraded", "relevance_confirmation"
-        elif lexical.status == "empty":
-            status, reason = ("empty" if semantic.status != "not_searched" else "ok"), None
+        elif combined or lexical.status == "empty":
+            status, reason = "ok", None
         else:
             status, reason = "empty", None
-        truncated = lexical.truncated or semantic.truncated
         raw = LaneResult(
             tuple(combined.values()),
             status,
             reason,
-            channels,
-            truncated,
+            lexical.channels,
+            lexical.truncated,
             lexical.starved,
             lexical.coverage_limited,
         )
-        if deferred_raw:
-            object.__setattr__(raw, "coverage_limited", True)
     elif "raw" in query.kinds and not authority.raw_allowed:
         raw = LaneResult((), "unavailable", "permission_required", (), False)
     if "memory" in query.kinds:
@@ -995,9 +843,7 @@ def coordinate_context_v2(
             return _path("unavailable", raw.reason, raw.channels, "not_searched", False, 0)
         if dropped and not final_raw:
             return _path("unavailable", "permission_required", raw.channels, "unavailable", True, 0)
-        # Lexical-only is complete when the lexical lane actually answered. An
-        # unconfigured semantic lane is a capability absence, not proof that
-        # less than the authorized range was searched.
+        # The raw lane is complete when the lexical lane actually answered.
         complete = (
             raw.status in {"ok", "empty"}
             and not dropped
@@ -1022,9 +868,7 @@ def coordinate_context_v2(
                 "count_budget"
                 if truncated
                 else "coverage_budget"
-                if raw.coverage_limited and not deferred_raw
-                else "relevance_confirmation"
-                if deferred_raw
+                if raw.coverage_limited
                 else "character_budget"
             )
         return _path(
@@ -1112,7 +956,7 @@ def coordinate_context_v2(
                 "group_id": f"group-{index}",
                 "answer_target": query.intent,
                 "decision_status": "missing_confirmation",
-                "missing_requirements": _group_missing_requirements(item),
+                "missing_requirements": [],
                 "object_refs": _object_refs(item, plan.object_keys),
                 "items": [group_item.model_dump(mode="json") for group_item in (item,)],
             }
@@ -1185,7 +1029,6 @@ async def search_context_v2(
     query: CoordinatorQuery,
     *,
     database_path: Path,
-    index: RawIndexLookup | None = None,
     deadline: float,
     expand_refs: ExpandRefs | None = None,
 ) -> RetrievalResponse:
@@ -1198,7 +1041,6 @@ async def search_context_v2(
             authority,
             query,
             database_path=database_path,
-            index=index,
             deadline=deadline,
             expand_refs=expand_refs,
         ),

memhub/apps/core/src/one_memory/conversation_processing.py

@@ -0,0 +1,277 @@
+"""Pure conversation-window helpers shared by capture, retrieval, and writes."""
+
+from __future__ import annotations
+
+import os
+import re
+
+from one_memory.imports.models import EvidenceContentKind, EvidenceOrigin, EvidenceRecord
+from one_memory.memory_tasks import segment_evidence_record
+
+MEMORY_TASK_TIMEOUT_MARGIN_SECONDS = 60.0
+MEMORY_TASK_TIMEOUT_MIN_SECONDS = 180.0
+MEMHUB_DEFAULT_CODING_TOOL_RESULT_MAX_BYTES = 512
+
+_HARNESS_PREFIXES = (
+    "<codex_internal_context",
+    "# agents.md instructions",
+    "<environment_context>",
+    "<permissions instructions>",
+    "<system-reminder>",
+    "<system_notification>",
+    "<instructions>",
+)
+_HOST_ENVELOPE = re.compile(
+    r"(?:"
+    r"^<timestamp>.*?<system_(?:notification|reminder)>|"
+    r"^<system_(?:notification|reminder)>|"
+    r'^\{\s*"request_id"\s*:.*?"steps"\s*:'
+    r")",
+    re.IGNORECASE | re.DOTALL,
+)
+_WHOLE_TURN_CHARACTERS = 60_000
+_WINDOW_BYTE_CEILING = 2_000_000
+_HISTORY_MERGE_TURNS = 8
+_HISTORY_MERGE_CHARACTERS = 8_000
+
+
+def memory_task_timeout_seconds(model_timeout_seconds: float) -> float:
+    """Bound a whole write without treating one model request as its budget."""
+    return max(
+        MEMORY_TASK_TIMEOUT_MIN_SECONDS,
+        float(model_timeout_seconds) + MEMORY_TASK_TIMEOUT_MARGIN_SECONDS,
+    )
+
+
+def _coding_tool_result_max_bytes() -> int:
+    raw = os.environ.get("MEMORY_CODING_TOOL_RESULT_MAX_BYTES")
+    if raw is None:
+        return MEMHUB_DEFAULT_CODING_TOOL_RESULT_MAX_BYTES
+    try:
+        value = int(raw)
+    except ValueError:
+        return MEMHUB_DEFAULT_CODING_TOOL_RESULT_MAX_BYTES
+    return value if value > 0 else MEMHUB_DEFAULT_CODING_TOOL_RESULT_MAX_BYTES
+
+
+_HY_TOOL_CHARACTERS = _coding_tool_result_max_bytes()
+
+
+def _conversation_chunks(
+    records: list[EvidenceRecord],
+    *,
+    max_characters: int | None = None,
+    max_turns: int | None = None,
+    completed_only: bool = False,
+) -> list[tuple[EvidenceRecord, ...]]:
+    """Build bounded, complete user/assistant/tool windows."""
+    turns: list[list[EvidenceRecord]] = []
+    for record in records:
+        if not _is_hy_turn_record(record):
+            continue
+        pending_proposal = bool(turns) and not _opens_turn(turns[-1][0])
+        if not turns or (_opens_turn(record) and not pending_proposal):
+            turns.append([record])
+        else:
+            turns[-1].append(record)
+
+    if completed_only and turns and _trailing_user_turn_is_open(turns[-1]):
+        turns.pop()
+
+    whole = _whole_turns() and max_characters is None
+    if whole:
+        size = _hy_read_length
+        max_characters = _WHOLE_TURN_CHARACTERS
+        max_turns = max_turns or 1_000_000
+    else:
+        size = lambda record: len(record.verbatim)  # noqa: E731
+        max_characters = max_characters or 12_000
+        max_turns = max_turns or 24
+
+    chunks: list[tuple[EvidenceRecord, ...]] = []
+    for turn in turns:
+        original_question = turn[0] if _opens_turn(turn[0]) else None
+        expanded: list[EvidenceRecord] = []
+        for record in turn:
+            reserve = (
+                size(original_question)
+                if original_question is not None
+                and record is not original_question
+                and size(original_question) < max_characters
+                else 0
+            )
+            segment_limit = max(1, max_characters - reserve)
+            if size(record) <= segment_limit:
+                expanded.append(record)
+                continue
+            spans = _semantic_segment_spans(record.verbatim, segment_limit)
+            expanded.extend(
+                segment_evidence_record(record, index=index, start=start, end=end)
+                if len(spans) > 1
+                else record
+                for index, (start, end) in enumerate(spans)
+            )
+        question = expanded[0] if expanded and _opens_turn(expanded[0]) else None
+        current: list[EvidenceRecord] = []
+        for record in expanded:
+            if current and (
+                len(current) >= max_turns
+                or sum(size(item) for item in current) + size(record) > max_characters
+                or sum(_window_bytes(item) for item in current) + _window_bytes(record)
+                > _WINDOW_BYTE_CEILING
+            ):
+                chunks.append(tuple(current))
+                current = (
+                    [question]
+                    if question is not None
+                    and max_turns > 1
+                    and size(question) + size(record) <= max_characters
+                    else []
+                )
+            current.append(record)
+        chunks.append(tuple(current))
+    return _merge_tool_only_chunks(chunks)
+
+
+def _semantic_segment_spans(text: str, limit: int) -> list[tuple[int, int]]:
+    if len(text) <= limit:
+        return [(0, len(text))]
+    spans: list[tuple[int, int]] = []
+    start = 0
+    separators = ("\n\n", "\n", "。", "！", "？", ". ", "! ", "? ")
+    while start < len(text):
+        hard_end = min(len(text), start + limit)
+        if hard_end == len(text):
+            end = hard_end
+        else:
+            floor = start + max(1, limit // 2)
+            candidates = [text.rfind(separator, floor, hard_end) for separator in separators]
+            boundary = max(candidates)
+            if boundary < floor:
+                end = hard_end
+            else:
+                matched = next(
+                    separator for separator in separators if text.startswith(separator, boundary)
+                )
+                end = boundary + len(matched)
+        spans.append((start, end))
+        start = end
+    return spans
+
+
+def _merge_tool_only_chunks(
+    chunks: list[tuple[EvidenceRecord, ...]],
+) -> list[tuple[EvidenceRecord, ...]]:
+    if not any(chunk and all(record.role == "tool" for record in chunk) for chunk in chunks):
+        return chunks
+    merged: list[tuple[EvidenceRecord, ...]] = []
+    leading_tools: list[EvidenceRecord] = []
+    for chunk in chunks:
+        if not chunk:
+            continue
+        if all(record.role == "tool" for record in chunk):
+            if merged:
+                merged[-1] = (*merged[-1], *chunk)
+            else:
+                leading_tools.extend(chunk)
+            continue
+        if leading_tools:
+            chunk = (*leading_tools, *chunk)
+            leading_tools = []
+        merged.append(chunk)
+    return [] if leading_tools else merged
+
+
+def _merge_historical_chunks(
+    chunks: list[tuple[EvidenceRecord, ...]],
+    *,
+    max_turns: int = _HISTORY_MERGE_TURNS,
+    max_characters: int = _HISTORY_MERGE_CHARACTERS,
+) -> list[tuple[EvidenceRecord, ...]]:
+    if not chunks:
+        return []
+    merged: list[tuple[EvidenceRecord, ...]] = []
+    current: list[EvidenceRecord] = []
+    current_turns = 0
+    current_characters = 0
+    for chunk in chunks:
+        chunk_characters = sum(len(record.verbatim) for record in chunk)
+        if current and (
+            current_turns >= max_turns or current_characters + chunk_characters > max_characters
+        ):
+            merged.append(tuple(current))
+            current = []
+            current_turns = 0
+            current_characters = 0
+        current.extend(chunk)
+        current_turns += 1
+        current_characters += chunk_characters
+    if current:
+        merged.append(tuple(current))
+    return merged
+
+
+def _chunk_input_ref(chunk: tuple[EvidenceRecord, ...]) -> str:
+    user_records = tuple(item for item in chunk if item.role == "user")
+    return user_records[-1].evidence_id if user_records else chunk[-1].evidence_id
+
+
+def _opens_turn(record: EvidenceRecord) -> bool:
+    return record.role == "user"
+
+
+def _trailing_user_turn_is_open(turn: list[EvidenceRecord]) -> bool:
+    if len(turn) != 1:
+        return False
+    opener = turn[0]
+    return (
+        _opens_turn(opener)
+        and opener.content_kind is EvidenceContentKind.RAW_CONVERSATION
+        and not _is_memory_note(opener)
+    )
+
+
+def _is_hy_turn_record(record: EvidenceRecord) -> bool:
+    if record.role == "tool":
+        return record.content_kind is EvidenceContentKind.RAW_CONVERSATION and bool(
+            record.verbatim.strip()
+        )
+    return _is_hy_input_turn(record)
+
+
+def _is_hy_input_turn(record: EvidenceRecord) -> bool:
+    source = record.verbatim.strip()
+    if not source:
+        return False
+    if record.content_kind is EvidenceContentKind.MEMORY_CONTENT:
+        return True
+    if record.role == "assistant":
+        return True
+    if record.role != "user":
+        return False
+    if source.casefold().startswith(_HARNESS_PREFIXES):
+        return False
+    return _HOST_ENVELOPE.search(source[:2_000]) is None
+
+
+def _is_memory_note(record: EvidenceRecord) -> bool:
+    meta = record.origin_meta.get(EvidenceOrigin.MANUAL_IMPORT)
+    return (
+        record.source_id == "one-memory-manual"
+        and meta is not None
+        and meta.raw_meta.get("input_mode") == "direct_memory_note"
+    )
+
+
+def _window_bytes(record: EvidenceRecord) -> int:
+    return len(record.verbatim.encode("utf-8"))
+
+
+def _whole_turns() -> bool:
+    return os.environ.get("ONE_MEMORY_SPLIT_TURNS", "").strip() != "1"
+
+
+def _hy_read_length(record: EvidenceRecord) -> int:
+    if record.role == "tool":
+        return min(len(record.verbatim), _HY_TOOL_CHARACTERS)
+    return len(record.verbatim)

memhub/apps/core/src/one_memory/diagnostic_log.py

@@ -20,8 +20,10 @@ from __future__ import annotations
 import atexit
 import json
 import logging
+import math
 import os
 import re
+import stat
 import sys
 import threading
 import time
@@ -72,80 +74,30 @@ class _Refused(OSError):
     """The path is not one we are willing to write to, and never will be."""
 
 
-DiagnosticLevel = Literal["info", "warn", "error"]
+DiagnosticLevel = Literal["debug", "info", "warn", "error"]
 _IDENTIFIER = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._:-]{0,255}$")
 _EVENT = re.compile(r"^[a-z][a-z0-9]*(?:[._-][a-z0-9]+){0,15}$")
-_DETAIL_KEYS = frozenset(
-    {
-        "action",
-        "attempt",
-        "count",
-        "method",
-        "mode",
-        "phase",
-        "platform",
-        "reason",
-        "result_count",
-        "source_id",
-        "spawned",
-        "status",
-        "version",
-        "parent_session_id",
-        "invalid_lines",
-        "truncated",
-        "open_file_count",
-        "memory_bytes",
-        "thread_count",
-        "database_error_count",
-        "route_path",
-        "sqlite_errorcode",
-        "sqlite_errorname",
-        "file_count",
-        "sqlite_file_count",
-        "byte_file_count",
-        "total_bytes",
-        "copied_bytes",
-        "bytes_per_second",
-        "stored_count",
-        "revised_count",
-        "skipped_count",
-        "error_count",
-        "pending_count",
-        "prepared_count",
-        "queue_depth",
-        "in_flight_count",
-        "call_count",
-        "created_count",
-        "updated_count",
-        "superseded_count",
-        "deleted_count",
-        "operation_kind",
-        "index_complete",
-        "lexical_hits",
-        "semantic_hits",
-        "generation",
-        "timed_out",
-        "failure_site",
-        "error_class",
-        "file_kind",
-        "file_size",
-        "file_mtime_ns",
-        "file_inode",
-        "cursor_new",
-        "cursor_old",
-        "cursor_read_to_end",
-        "cursor_rewind_branch",
-        "summed_ms",
-        "model_ms",
-        "accounted_ms",
-        "unaccounted_ms",
-        "conversation_hash",
-        # The sedimentation switch: a bounded boolean, the same shape as
-        # `spawned` and `index_complete`. Left out of this list it did not
-        # drop a log line, it turned the switch's own request into a 500.
-        "enabled",
-    }
+_DIAGNOSTIC_LOG_NAME = re.compile(
+    r"^diagnostic-(runtime|core|host-bridge|desktop|cli)\.jsonl(?:\.\d+)?$"
 )
+_DETAIL_KEY = re.compile(r"^[A-Za-z][A-Za-z0-9_.-]{0,63}$")
+_CODE_FRAME = re.compile(
+    r"^(?:(?:apps|packages|scripts)/(?:[A-Za-z0-9_-]+/)*[A-Za-z0-9_.-]+:\d+:\d+"
+    r"|one_memory/(?:[A-Za-z0-9_-]+/)*[A-Za-z0-9_.-]+:\d+:[A-Za-z_][A-Za-z0-9_]*"
+    r"|[A-Za-z0-9_][A-Za-z0-9_.-]*\.py:\d+:[A-Za-z_][A-Za-z0-9_]*)$"
+)
+_SHORT_CREDENTIAL_TOKEN = re.compile(
+    r"(?<![A-Za-z0-9])([A-Za-z0-9]{24,47})(?:[_-])?(?![A-Za-z0-9])"
+)
+_REFUSED_DETAIL_KEYS = frozenset({"__proto__", "prototype", "constructor"})
+_MAX_DETAIL_KEYS = 32
+_MAX_SAFE_INTEGER = (1 << 53) - 1
+_EXPORT_SECRET_MARKER = re.compile(
+    r"(?:^|[._:-])(?:authorization|credential|password|prompt|query|secret|token)(?:$|[._:-])",
+    re.IGNORECASE,
+)
+_TRACE_STRING_DETAIL_KEYS = frozenset({"phase", "mode", "status", "reason", "failure_site"})
+_TRACE_NUMBER_DETAIL_KEYS = frozenset({"count"})
 _trace_id: ContextVar[str | None] = ContextVar("one_memory_trace_id", default=None)
 _active: DiagnosticLog | None = None
 _database_errors: Counter[tuple[str, str]] = Counter()
@@ -264,7 +216,12 @@ class DiagnosticLog:
         }
         value.update({key: _identifier(item) for key, item in identifiers.items() if item})
         if duration_ms is not None:
-            if not isinstance(duration_ms, int) or duration_ms < 0:
+            if (
+                not isinstance(duration_ms, int)
+                or isinstance(duration_ms, bool)
+                or duration_ms < 0
+                or duration_ms > _MAX_SAFE_INTEGER
+            ):
                 raise ValueError("diagnostic_duration_invalid")
             value["duration_ms"] = duration_ms
         for field_name, item in {
@@ -276,7 +233,9 @@ class DiagnosticLog:
             if item is not None:
                 value[field_name] = _identifier(item)
         if safe_stack:
-            value["safe_stack"] = [_safe_stack_line(line) for line in safe_stack[:256]]
+            if len(safe_stack) > 16:
+                raise ValueError("diagnostic_stack_invalid")
+            value["safe_stack"] = [_safe_stack_line(line) for line in safe_stack]
         if details:
             value["details"] = _details(details)
         self._submit(_serialize(value))
@@ -289,6 +248,8 @@ class DiagnosticLog:
         event: str,
         outcome: str,
     ) -> dict[str, object]:
+        if level not in ("debug", "info", "warn", "error"):
+            raise ValueError("diagnostic_level_invalid")
         return {
             "schema_version": 1,
             "timestamp": datetime.now(UTC).isoformat().replace("+00:00", "Z"),
@@ -651,24 +612,17 @@ def _drain_diagnostics_at_exit() -> None:
 
 
 def diagnostic_event(**values: object) -> None:
-    active = _active
-    if active is None:
-        return
-    if values.get("level") == "debug" and not logging.getLogger().isEnabledFor(logging.DEBUG):
-        return
     try:
+        active = _active
+        if active is None:
+            return
+        if values.get("level") == "debug" and not logging.getLogger().isEnabledFor(logging.DEBUG):
+            return
         active.write(**values)  # type: ignore[arg-type]
-    except OSError:
-        # Diagnostics must not become a new product outage when a disk is full
-        # or the log directory is temporarily unavailable.
-        return
-    except ValueError:
-        # Same promise, same reason, for a rejected payload instead of a failed
-        # write. An unlisted `details` key once turned every external recall
-        # into a 500 and then killed the process, and the sedimentation switch
-        # into a permanent failure. The payload itself is still wrong and is
-        # still caught in CI by tests/test_diagnostic_payload_keys.py; what
-        # must not happen is the caller paying for it.
+    except Exception:
+        # Diagnostics are best-effort: validation, serialization, logging, and
+        # filesystem failures must never become a product outage. BaseException
+        # deliberately remains visible so process-control signals still work.
         return
 
 
@@ -691,46 +645,95 @@ def read_trace_events(
     if (request_id is None) == (trace_id is None):
         raise ValueError("exactly_one_trace_identity_required")
     key, expected = ("request_id", request_id) if request_id is not None else ("trace_id", trace_id)
-    if expected is None or not _IDENTIFIER.fullmatch(expected):
+    if expected is None or not _export_identifier(expected):
         raise ValueError("invalid_trace_identity")
     selected: list[dict[str, object]] = []
-    allowed_details = {"phase", "mode", "status", "reason", "failure_site", "count"}
-    files = sorted(
-        (data_dir / "logs").glob("diagnostic-*.jsonl*"),
-        key=lambda path: path.stat().st_mtime,
-    )
-    for path in files:
-        if path.is_symlink() or not path.is_file():
+    directory = data_dir / "logs"
+    try:
+        directory_info = directory.lstat()
+    except FileNotFoundError:
+        return ()
+    if stat.S_ISLNK(directory_info.st_mode) or not stat.S_ISDIR(directory_info.st_mode):
+        raise ValueError("diagnostic_directory_unsafe")
+    files: list[tuple[Path, float]] = []
+    for path in directory.iterdir():
+        if not _DIAGNOSTIC_LOG_NAME.fullmatch(path.name):
             continue
-        for line in path.read_text(encoding="utf-8", errors="replace").splitlines():
+        try:
+            info = path.lstat()
+        except OSError:
+            continue
+        if stat.S_ISREG(info.st_mode) and not stat.S_ISLNK(info.st_mode):
+            files.append((path, info.st_mtime))
+    for path, _mtime in sorted(files, key=lambda item: item[1]):
+        for line in _read_regular_file_no_follow(path).splitlines():
             try:
                 event = json.loads(line)
             except (TypeError, json.JSONDecodeError):
                 continue
             if not isinstance(event, dict) or event.get(key) != expected:
                 continue
-            details = event.get("details")
-            projected = {
-                name: event[name]
-                for name in (
-                    "timestamp",
-                    "component",
-                    "event",
-                    "outcome",
-                    "duration_ms",
-                    "error_code",
-                    "error_type",
-                )
-                if name in event
+            timestamp = event.get("timestamp")
+            if not isinstance(timestamp, str):
+                continue
+            try:
+                datetime.fromisoformat(timestamp.replace("Z", "+00:00"))
+            except ValueError:
+                continue
+            component = event.get("component")
+            event_name = event.get("event")
+            outcome = event.get("outcome")
+            projected: dict[str, object] = {
+                "timestamp": timestamp,
+                "component": (
+                    component
+                    if isinstance(component, str) and _export_identifier(component)
+                    else "diagnostic.redacted"
+                ),
+                "event": (
+                    event_name
+                    if isinstance(event_name, str) and _export_identifier(event_name)
+                    else "diagnostic.redacted"
+                ),
+                "outcome": (
+                    outcome
+                    if isinstance(outcome, str) and _export_identifier(outcome)
+                    else "redacted"
+                ),
             }
+            duration_ms = event.get("duration_ms")
+            if (
+                isinstance(duration_ms, int)
+                and not isinstance(duration_ms, bool)
+                and 0 <= duration_ms <= _MAX_SAFE_INTEGER
+            ):
+                projected["duration_ms"] = duration_ms
+            for name in ("error_code", "error_type"):
+                value = event.get(name)
+                if isinstance(value, str) and _export_identifier(value):
+                    projected[name] = value
+            details = event.get("details")
             if isinstance(details, dict):
-                projected["details"] = {
-                    name: value for name, value in details.items() if name in allowed_details
-                }
+                safe_details = _trace_details(details)
+                if safe_details:
+                    projected["details"] = safe_details
             selected.append(projected)
     return tuple(selected[-max(1, min(limit, 500)) :])
 
 
+def _read_regular_file_no_follow(path: Path) -> str:
+    descriptor = os.open(path, os.O_RDONLY | _O_NOFOLLOW)
+    try:
+        if not stat.S_ISREG(os.fstat(descriptor).st_mode):
+            raise ValueError("diagnostic_file_unsafe")
+        with os.fdopen(descriptor, encoding="utf-8", errors="replace") as handle:
+            descriptor = -1
+            return handle.read()
+    finally:
+        if descriptor >= 0:
+            os.close(descriptor)
+
+
 def current_trace_id() -> str | None:
     return _trace_id.get()
 
@@ -837,8 +840,77 @@ def diagnostic_exception(error: BaseException) -> dict[str, object]:
     }
 
 
+def _contains_credential_material(value: str) -> bool:
+    return bool(
+        re.search(
+            r"(?:^|[^A-Za-z0-9])(?:sk-|sk_|omhg_|ghp_|github_pat_)[A-Za-z0-9._-]+",
+            value,
+            re.IGNORECASE,
+        )
+        or re.search(r"(?:^|[^A-Za-z0-9])xox[baprs]-[A-Za-z0-9-]+", value, re.IGNORECASE)
+        or re.search(r"(?:^|[^A-Za-z0-9])(?:AKIA|ASIA)[A-Z0-9]{16}(?![A-Za-z0-9])", value)
+        or re.search(r"(?:^|[^A-Za-z0-9])AKID[A-Za-z0-9]{12,}", value, re.IGNORECASE)
+        or re.search(r"(?:^|[^A-Za-z0-9])AIza[A-Za-z0-9_-]{20,}", value)
+        or re.search(r"(?:^|[^A-Za-z0-9])GOCSPX-[A-Za-z0-9_-]+", value)
+        or any(
+            _is_conservative_short_credential(match.group(1))
+            for match in _SHORT_CREDENTIAL_TOKEN.finditer(value)
+        )
+        or re.search(
+            r"\b[A-Za-z0-9_-]{10,}\.[A-Za-z0-9_-]{10,}\.[A-Za-z0-9_-]{10,}\b",
+            value,
+        )
+        or any(
+            len(candidate) >= 48
+            and any(character.isalpha() for character in candidate)
+            and any(character.isdigit() for character in candidate)
+            for candidate in re.findall(r"\b[A-Za-z0-9._-]{48,}\b", value)
+        )
+    )
+
+
+def _is_conservative_short_credential(value: str) -> bool:
+    return bool(
+        re.search(r"[A-Z]", value)
+        and re.search(r"[a-z]", value)
+        and re.search(r"\d", value)
+        and not re.fullmatch(r"[A-Fa-f0-9]+", value)
+    )
+
+
+def _export_identifier(value: str) -> bool:
+    return bool(
+        _IDENTIFIER.fullmatch(value)
+        and not _EXPORT_SECRET_MARKER.search(value)
+        and not _contains_credential_material(value)
+    )
+
+
+def _trace_details(values: Mapping[object, object]) -> dict[str, str | int | float]:
+    result: dict[str, str | int | float] = {}
+    for key, value in values.items():
+        if (
+            key in _TRACE_STRING_DETAIL_KEYS
+            and isinstance(value, str)
+            and _export_identifier(value)
+        ):
+            result[str(key)] = value
+        elif key in _TRACE_NUMBER_DETAIL_KEYS and isinstance(value, (int, float)):
+            if isinstance(value, bool):
+                continue
+            safe_integer = isinstance(value, int) and abs(value) <= _MAX_SAFE_INTEGER
+            safe_float = (
+                isinstance(value, float)
+                and math.isfinite(value)
+                and (not value.is_integer() or abs(value) <= _MAX_SAFE_INTEGER)
+            )
+            if safe_integer or safe_float:
+                result[str(key)] = value
+    return result
+
+
 def _serialize(value: Mapping[str, object]) -> str:
-    return json.dumps(value, ensure_ascii=True, separators=(",", ":")) + "\n"
+    return json.dumps(value, ensure_ascii=True, allow_nan=False, separators=(",", ":")) + "\n"
 
 
 def _uuid(value: str | None) -> str | None:
@@ -864,24 +936,47 @@ def _event_name(value: str) -> str:
 
 
 def _safe_stack_line(value: str) -> str:
-    if not value or len(value) > 256 or "\n" in value or "\r" in value:
+    if (
+        not value
+        or len(value) > 256
+        or any(character in value for character in "\0\r\n")
+        or _contains_credential_material(value)
+    ):
         raise ValueError("diagnostic_stack_invalid")
-    if Path(value.split(":", 1)[0]).is_absolute() or "file://" in value:
-        raise ValueError("diagnostic_stack_invalid")
-    return value
+    if _CODE_FRAME.fullmatch(value):
+        return value
+    if value.startswith("thread|"):
+        frames = value.removeprefix("thread|").split(" > ")
+        if frames and all(_CODE_FRAME.fullmatch(frame) for frame in frames):
+            return value
+    raise ValueError("diagnostic_stack_invalid")
 
 
 def _details(
     values: Mapping[str, str | int | float | bool | None],
 ) -> dict[str, str | int | float | bool | None]:
+    if len(values) > _MAX_DETAIL_KEYS:
+        raise ValueError("diagnostic_detail_count_invalid")
     result: dict[str, str | int | float | bool | None] = {}
     for key, value in values.items():
-        if key not in _DETAIL_KEYS:
+        if (
+            not isinstance(key, str)
+            or not _DETAIL_KEY.fullmatch(key)
+            or key in _REFUSED_DETAIL_KEYS
+        ):
             raise ValueError("diagnostic_detail_key_invalid")
         if isinstance(value, str):
-            if not value or len(value) > 256 or "\n" in value or "\r" in value:
+            if not value or len(value) > 256 or any(character in value for character in "\0\r\n"):
                 raise ValueError("diagnostic_detail_value_invalid")
-        elif value is not None and not isinstance(value, (int, float, bool)):
+        elif isinstance(value, float):
+            if not math.isfinite(value) or (value.is_integer() and abs(value) > _MAX_SAFE_INTEGER):
+                raise ValueError("diagnostic_detail_value_invalid")
+        elif isinstance(value, bool) or value is None:
+            pass
+        elif isinstance(value, int):
+            if abs(value) > _MAX_SAFE_INTEGER:
+                raise ValueError("diagnostic_detail_value_invalid")
+        else:
             raise ValueError("diagnostic_detail_value_invalid")
         result[key] = value
     return result

memhub/apps/core/src/one_memory/exception_reporting.py

@@ -3,8 +3,8 @@
 Every site that catches `Exception` broadly must call `report_exception`
 before (or instead of) continuing, so a failure never looks like success
 and a stack is always on disk. Content (conversation text, keys, SQL
-parameters) never reaches the log: only exception type, a truncated
-message, and file:line:function frames.
+parameters, and exception messages) never reaches the diagnostic log: only
+exception type, a repeat count, and file:line:function frames.
 """
 
 from __future__ import annotations
@@ -67,7 +67,7 @@ def report_exception(
         event=event,
         outcome="failed",
         **payload,
-        details={"reason": _safe_message(error), "count": _RECENT[key]},
+        details={"count": _RECENT[key]},
     )
     if full:
         _logger.error(

memhub/apps/core/src/one_memory/hy_graph_admin.py

@@ -0,0 +1,271 @@
+"""Kuzu graph-width inspection and explicit user-directed administration.
+
+This module is the second graph architecture exception.  It may open Kuzu
+directly to inspect an existing graph's schema, but owns no prompts, memory
+reconciliation, coding data, vector database, or legacy governance behavior.
+"""
+
+from __future__ import annotations
+
+import logging
+import os
+import re
+import shutil
+import threading
+from dataclasses import dataclass
+from datetime import UTC, datetime
+from enum import StrEnum
+from importlib import import_module
+from pathlib import Path
+from typing import Any
+
+from pydantic import BaseModel, ConfigDict
+
+from one_memory.hy_runtime_config import HY_MEMORY_NAMESPACE
+from one_memory.model_profiles import ModelProfileSnapshot, ProviderKind
+
+
+class GraphWidthError(RuntimeError):
+    """Base error for graph-width inspection and administration."""
+
+
+class GraphWidthDecision(StrEnum):
+    """What to do when an existing graph disagrees with the embedding width."""
+
+    KEEP_EXISTING = "keep_existing"
+    REBUILD_WITH_NEW = "rebuild_with_new"
+    EXPORT_FIRST = "export_first"
+
+
+class GraphWidthMismatch(BaseModel):
+    model_config = ConfigDict(frozen=True, extra="forbid")
+
+    stored_width: int
+    expected_width: int
+    graph_path: str
+
+
+class GraphWidthOutcome(BaseModel):
+    model_config = ConfigDict(frozen=True, extra="forbid")
+
+    archived_path: str | None = None
+    exported_path: str | None = None
+
+
+class GraphWidthMismatchError(GraphWidthError):
+    """Raised instead of silently archiving a live graph."""
+
+    def __init__(self, mismatch: GraphWidthMismatch) -> None:
+        super().__init__(
+            f"graph_width_mismatch:stored={mismatch.stored_width}"
+            f":expected={mismatch.expected_width}"
+        )
+        self.mismatch = mismatch
+
+
+def graph_store_enabled() -> bool:
+    """Whether HY should open the Kuzu graph store."""
+    raw = os.environ.get("ONE_MEMORY_GRAPH_STORE", "").strip().lower()
+    if not raw:
+        return False
+    return raw != "none"
+
+
+def inspect_graph_width_mismatch(config: Any) -> GraphWidthMismatch | None:
+    """Return a mismatch without renaming or deleting anything."""
+    db_path = Path(config.graph_store.db_path)
+    expected = getattr(getattr(config, "embedder", None), "embedding_dims", None)
+    return _inspect_graph_width(db_path, expected)
+
+
+def inspect_profile_graph_width(
+    profile: ModelProfileSnapshot,
+    *,
+    api_key: str | None,
+    data_dir: Path,
+) -> GraphWidthMismatch | None:
+    """Inspect the graph path and width selected by the active profile."""
+    del api_key
+    return _inspect_graph_width(
+        _profile_graph_path(data_dir),
+        _profile_embedding_width(profile),
+    )
+
+
+def decide_profile_graph_width(
+    profile: ModelProfileSnapshot,
+    *,
+    api_key: str | None,
+    data_dir: Path,
+    decision: GraphWidthDecision,
+) -> GraphWidthOutcome:
+    """Apply the user's decision to the active profile's graph."""
+    del api_key
+    config = _GraphAdminConfig(
+        graph_store=_GraphStoreConfig(str(_profile_graph_path(data_dir))),
+        embedder=_EmbeddingWidthConfig(_profile_embedding_width(profile)),
+    )
+    return apply_graph_width_decision(
+        config,
+        decision,
+        export_root=data_dir.parent,
+    )
+
+
+def apply_graph_width_decision(
+    config: Any,
+    decision: GraphWidthDecision,
+    *,
+    export_root: Path | None = None,
+) -> GraphWidthOutcome:
+    """Act only on a current mismatch and only as explicitly requested."""
+    mismatch = inspect_graph_width_mismatch(config)
+    if mismatch is None:
+        return GraphWidthOutcome()
+    if decision is GraphWidthDecision.REBUILD_WITH_NEW:
+        archive = retire_mismatched_graph_store(config)
+        return GraphWidthOutcome(archived_path=str(archive) if archive is not None else None)
+    if decision is GraphWidthDecision.EXPORT_FIRST:
+        if export_root is None:
+            raise GraphWidthError("export_root_required")
+        return GraphWidthOutcome(
+            exported_path=str(export_graph_store(config, export_root=export_root))
+        )
+    return GraphWidthOutcome()
+
+
+def export_graph_store(config: Any, *, export_root: Path) -> Path:
+    """Copy the graph outside its data directory, atomically."""
+    db_path = Path(config.graph_store.db_path)
+    stamp = datetime.now(UTC).strftime("%Y%m%dT%H%M%S%fZ")
+    destination = export_root / f"one-memory-graph-{stamp}"
+    staging = destination.with_name(f"{destination.name}.partial")
+    sources = [db_path] + [
+        sibling
+        for suffix in (".wal", ".shadow", ".tmp")
+        if (sibling := db_path.with_name(f"{db_path.name}{suffix}")).exists()
+    ]
+    try:
+        staging.mkdir(parents=True)
+        for source in sources:
+            shutil.copy2(source, staging / source.name)
+        staging.rename(destination)
+    except BaseException:
+        shutil.rmtree(staging, ignore_errors=True)
+        raise
+    logging.getLogger(__name__).info(
+        "exported hy graph store to %s (%d files)",
+        destination,
+        len(sources),
+    )
+    return destination
+
+
+def retire_mismatched_graph_store(config: Any) -> Path | None:
+    """Archive a mismatched Kuzu graph only after the user chooses rebuild."""
+    db_path = Path(config.graph_store.db_path)
+    if not db_path.exists():
+        return None
+    expected = getattr(getattr(config, "embedder", None), "embedding_dims", None)
+    if not expected:
+        return None
+    stored = _graph_embedding_width(db_path)
+    if stored is None or stored == int(expected):
+        return None
+    stamp = datetime.now(UTC).strftime("%Y%m%dT%H%M%SZ")
+    archive = db_path.with_name(f"{db_path.name}.dim{stored}.{stamp}.bak")
+    db_path.rename(archive)
+    for suffix in (".wal", ".shadow", ".tmp"):
+        sibling = db_path.with_name(f"{db_path.name}{suffix}")
+        if sibling.exists():
+            sibling.rename(archive.with_name(f"{archive.name}{suffix}"))
+    logging.getLogger(__name__).warning(
+        "retired hy graph store with embedding width %s (model expects %s): %s",
+        stored,
+        expected,
+        archive.name,
+    )
+    return archive
+
+
+_GRAPH_WIDTH_CACHE: dict[str, int | None] = {}
+_GRAPH_WIDTH_LOCK = threading.Lock()
+
+
+def _graph_embedding_width(db_path: Path) -> int | None:
+    """Read the Memory embedding width directly from Kuzu's schema."""
+    try:
+        kuzu = import_module("kuzu")
+    except ModuleNotFoundError:
+        return None
+    key = str(db_path.resolve())
+    with _GRAPH_WIDTH_LOCK:
+        if key in _GRAPH_WIDTH_CACHE:
+            return _GRAPH_WIDTH_CACHE[key]
+    try:
+        database = kuzu.Database(str(db_path), read_only=True)
+    except Exception:
+        return None
+    width: int | None = None
+    try:
+        connection = kuzu.Connection(database)
+        result = connection.execute("CALL TABLE_INFO('Memory') RETURN *")
+        while result.has_next():
+            row = result.get_next()
+            if len(row) < 3 or row[1] != "embedding":
+                continue
+            match = re.search(r"\[(\d+)\]", str(row[2]))
+            width = int(match.group(1)) if match else None
+            break
+    except Exception:
+        width = None
+    with _GRAPH_WIDTH_LOCK:
+        _GRAPH_WIDTH_CACHE[key] = width
+    return width
+
+
+def _inspect_graph_width(
+    db_path: Path,
+    expected_width: object,
+) -> GraphWidthMismatch | None:
+    if not db_path.exists() or not expected_width:
+        return None
+    stored = _graph_embedding_width(db_path)
+    if stored is None or stored == int(expected_width):
+        return None
+    return GraphWidthMismatch(
+        stored_width=int(stored),
+        expected_width=int(expected_width),
+        graph_path=str(db_path),
+    )
+
+
+def _profile_graph_path(data_dir: Path) -> Path:
+    name = "graph" if graph_store_enabled() else "graph-disabled"
+    return data_dir / "hy-memory" / HY_MEMORY_NAMESPACE / name
+
+
+def _profile_embedding_width(profile: ModelProfileSnapshot) -> int | None:
+    if profile.embedding_dimensions is not None:
+        return profile.embedding_dimensions
+    if (profile.embedding_provider_kind or profile.provider_kind) is ProviderKind.LOCAL_ONNX:
+        from one_memory.embedding_assets import BUILTIN_EMBEDDING_DIMENSIONS
+
+        return BUILTIN_EMBEDDING_DIMENSIONS
+    return None
+
+
+@dataclass(frozen=True)
+class _GraphStoreConfig:
+    db_path: str
+
+
+@dataclass(frozen=True)
+class _EmbeddingWidthConfig:
+    embedding_dims: int | None
+
+
+@dataclass(frozen=True)
+class _GraphAdminConfig:
+    graph_store: _GraphStoreConfig
+    embedder: _EmbeddingWidthConfig

memhub/apps/core/src/one_memory/hy_graph_projection.py

@@ -0,0 +1,185 @@
+"""Read-only projection of HY's Kuzu graph for MemHub's existing graph UI.
+
+This module is one of the two explicit graph architecture exceptions.  The
+public SDK does not expose graph projection yet, so it may use the client's
+private graph handle and loop thread.  It must remain read-only and must not
+grow memory, vector, coding, prompt, or governance responsibilities.
+"""
+
+from __future__ import annotations
+
+from typing import Any, cast
+
+from pydantic import BaseModel, ConfigDict
+
+
+class HyGraphNode(BaseModel):
+    model_config = ConfigDict(frozen=True, extra="forbid")
+
+    node_id: str
+    content: str
+    layer: str | None = None
+    confidence: float | None = None
+
+
+class HyGraphEdge(BaseModel):
+    model_config = ConfigDict(frozen=True, extra="forbid")
+
+    source_node_id: str
+    target_node_id: str
+    relation: str
+    weight: float | None = None
+
+
+class HyGraphSnapshot(BaseModel):
+    model_config = ConfigDict(frozen=True, extra="forbid")
+
+    nodes: tuple[HyGraphNode, ...] = ()
+    edges: tuple[HyGraphEdge, ...] = ()
+    # False when HY was never consulted, which is not the same as HY having
+    # induced no relations.
+    consulted: bool = True
+
+
+def read_graph_snapshot(
+    client: object,
+    user_id: str,
+    agent_id: str,
+    limit: int,
+) -> HyGraphSnapshot:
+    raw_nodes, raw_edges = _read_graph_snapshot(client, user_id, agent_id, limit)
+    return HyGraphSnapshot(
+        nodes=tuple(
+            HyGraphNode(
+                node_id=str(row[0]),
+                content=str(row[1]),
+                layer=str(row[2]) if row[2] is not None else None,
+                confidence=_safe_float(row[3]),
+            )
+            for row in raw_nodes
+            if len(row) >= 4 and row[0] and row[1]
+        ),
+        edges=tuple(
+            HyGraphEdge(
+                source_node_id=str(row[0]),
+                target_node_id=str(row[1]),
+                relation=str(row[2]),
+                weight=_safe_float(row[3]),
+            )
+            for row in raw_edges
+            if len(row) >= 4 and row[0] and row[1] and row[2]
+        ),
+    )
+
+
+def _read_graph_snapshot(
+    client: object,
+    user_id: str,
+    agent_id: str,
+    limit: int,
+) -> tuple[list[list[object]], list[list[object]]]:
+    """Read the graph through the SDK handles until a public API exists."""
+    concrete = cast(Any, client)
+    # Keep the existing architecture gate unchanged; this module is the named
+    # graph-only exception while the rest of MemHub remains on public handles.
+    graph_store = getattr(concrete, "_graph_store")  # noqa: B009
+    if graph_store is None:
+        return ([], [])
+    loop_thread = getattr(concrete, "_loop_thread")  # noqa: B009
+    # Kuzu rejects a prepared statement that is handed a parameter its text
+    # never references, so a wildcard read must not bind agent_id at all.
+    params: dict[str, object] = {"user_id": user_id}
+    if agent_id != "*":
+        params["agent_id"] = agent_id
+    safe_limit = max(1, min(int(limit), 1_000))
+    memory_agent = "" if agent_id == "*" else " AND m.agent_id = $agent_id"
+    related_agent = (
+        "" if agent_id == "*" else " AND a.agent_id = $agent_id AND b.agent_id = $agent_id"
+    )
+    source_agent = "" if agent_id == "*" else " AND a.agent_id = $agent_id"
+    nodes = (
+        loop_thread.run(
+            graph_store._run(
+                f"""
+            MATCH (m:Memory)
+            WHERE m.user_id = $user_id{memory_agent}
+              AND m.status = 'active'
+            RETURN m.node_id, m.content, m.layer, m.confidence
+            LIMIT {safe_limit}
+            """,
+                params,
+            )
+        )
+        or []
+    )
+    related_edges = (
+        loop_thread.run(
+            graph_store._run(
+                f"""
+            MATCH (a:Memory)-[r:RELATED_TO]->(b:Memory)
+            WHERE a.user_id = $user_id{related_agent}
+              AND b.user_id = $user_id
+              AND a.status = 'active' AND b.status = 'active'
+            RETURN a.node_id, b.node_id, r.relation_type, r.weight
+            LIMIT {safe_limit * 4}
+            """,
+                params,
+            )
+        )
+        or []
+    )
+    derived_edges = (
+        loop_thread.run(
+            graph_store._run(
+                f"""
+            MATCH (a:Memory)-[:DERIVED_FROM]->(v:VdbRef)
+            WHERE a.user_id = $user_id{source_agent}
+              AND a.status = 'active'
+            RETURN a.node_id, v.node_id, 'derived_from', 1.0
+            LIMIT {safe_limit * 4}
+            """,
+                params,
+            )
+        )
+        or []
+    )
+    topic_nodes = (
+        loop_thread.run(
+            graph_store._run(
+                f"""
+            MATCH (m:Memory)-[:TAGGED_WITH]->(t:Topic)
+            WHERE m.user_id = $user_id{memory_agent}
+              AND m.status = 'active'
+            RETURN DISTINCT t.topic_id, t.name, 'TOPIC', 1.0
+            LIMIT {safe_limit}
+            """,
+                params,
+            )
+        )
+        or []
+    )
+    tagged_edges = (
+        loop_thread.run(
+            graph_store._run(
+                f"""
+            MATCH (m:Memory)-[:TAGGED_WITH]->(t:Topic)
+            WHERE m.user_id = $user_id{memory_agent}
+              AND m.status = 'active'
+            RETURN m.node_id, t.topic_id, 'tagged_with', 1.0
+            LIMIT {safe_limit * 4}
+            """,
+                params,
+            )
+        )
+        or []
+    )
+    return (
+        [*nodes, *topic_nodes],
+        [*related_edges, *derived_edges, *tagged_edges],
+    )
+
+
+def _safe_float(value: object) -> float | None:
+    if isinstance(value, (int, float)):
+        return float(value)
+    return None

memhub/apps/core/src/one_memory/hy_lifecycle.py

@@ -1,128 +0,0 @@
-"""Which memories HY no longer counts as current, read from HY itself.
-
-HY owns a memory's lifecycle: it supersedes, deduplicates, shadows and
-restores on its own schedule — during a search, while deleting the head of a
-chain — and most of that never passes through a write the ledger sees. So
-the ledger does not keep a copy of any of it; readers ask HY, once per read,
-for the nodes it has set aside.
-
-Only the vector store is read. HY writes a node's status there on every
-transition, and the graph holds no status the vector store lacks.
-"""
-
-import json
-from collections.abc import Collection, Mapping
-from dataclasses import dataclass, field
-from importlib import import_module
-from types import MappingProxyType
-from typing import Any
-
-#: HY's one status for a node that is current. Everything else it writes —
-#: superseded, shadow, archived, negated, conflicted — takes it out of recall.
-_ACTIVE = "active"
-#: HY's verbatim chunks. Stored as shadow from the start and never a memory,
-#: so never something a reader could have shown.
-_RAW_LAYER = "l1_raw"
-
-
-class HyLifecycleUnreadable(RuntimeError):
-    """HY's store could not be read; never to be taken for "nothing is hidden"."""
-
-
-@dataclass(frozen=True, eq=False)
-class HyLifecycle:
-    """HY's set-aside nodes for one user, each with what HY says replaced it."""
-
-    hidden: Mapping[str, tuple[str, ...]] = field(default_factory=dict)
-
-    def __post_init__(self) -> None:
-        object.__setattr__(self, "hidden", MappingProxyType(dict(self.hidden)))
-
-    def __eq__(self, other: object) -> bool:
-        return isinstance(other, HyLifecycle) and dict(self.hidden) == dict(other.hidden)
-
-    def __hash__(self) -> int:
-        return hash(tuple(sorted(self.hidden.items())))
-
-    def replaced_by(self, memory_id: str) -> tuple[str, ...]:
-        return self.hidden.get(memory_id, ())
-
-    def replacing(self, memory_id: str) -> tuple[str, ...]:
-        """The set-aside memories HY names this one as the replacement of."""
-        return tuple(sorted(old for old, by in self.hidden.items() if memory_id in by))
-
-
-def read_hy_lifecycle(
-    client: object, user_id: str, ids: Collection[str] | None = None
-) -> HyLifecycle:
-    """Every node of `user_id` HY holds as anything but active, in one read.
-
-    With `ids`, only those nodes are looked up — what a single-memory read
-    needs, and a lookup by key rather than a scan; `replacing` is then only
-    known among them.
-
-    Read one level below HY's own listing, which drops a failed read and
-    answers empty: a hidden memory would then read as current.
-    """
-    native: Any = client
-    try:
-        chroma = import_module("hy_memory.data.vector_store_chroma")
-        store = native._vector_store
-        collection = store._collection
-        if not isinstance(store, chroma.ChromaVectorStore) or collection is None:
-            raise HyLifecycleUnreadable("hy_lifecycle_store_unsupported")
-        found = collection.get(
-            ids=None if ids is None else list(ids),
-            where={
-                "$and": [
-                    {"user_id": {"$eq": user_id}},
-                    {"status": {"$ne": _ACTIVE}},
-                    {"layer": {"$ne": _RAW_LAYER}},
-                ]
-            },
-            include=["metadatas"],
-        )
-        metadatas = found["metadatas"]
-        point_ids = found["ids"]
-        if (
-            not isinstance(point_ids, list)
-            or not isinstance(metadatas, list)
-            or len(point_ids) != len(metadatas)
-        ):
-            raise HyLifecycleUnreadable("hy_lifecycle_store_mismatch")
-    except HyLifecycleUnreadable:
-        raise
-    except Exception as error:
-        raise HyLifecycleUnreadable("hy_lifecycle_unreadable") from error
-    hidden: dict[str, tuple[str, ...]] = {}
-    for point_id, metadata in zip(point_ids, metadatas, strict=True):
-        if not isinstance(point_id, str) or not point_id or not isinstance(metadata, Mapping):
-            raise HyLifecycleUnreadable("hy_lifecycle_store_mismatch")
-        payload = chroma.ChromaVectorStore._chroma_metadata_to_payload(dict(metadata))
-        node_id = payload.get("node_id") or point_id
-        if not isinstance(node_id, str) or not node_id or node_id != point_id:
-            raise HyLifecycleUnreadable("hy_lifecycle_store_mismatch")
-        hidden[node_id] = _replacements(payload)
-    return HyLifecycle(hidden=hidden)
-
-
-def _replacements(payload: Mapping[str, object]) -> tuple[str, ...]:
-    """What HY names as taking this node's place: its chain successor or the kept duplicate."""
-    names: list[str] = []
-    superseded_by = payload.get("superseded_by")
-    if isinstance(superseded_by, list):
-        names.extend(value for value in superseded_by if isinstance(value, str) and value)
-    kept = payload.get("dedup_kept_id")
-    custom = payload.get("custom")
-    if isinstance(custom, str):
-        try:
-            custom = json.loads(custom)
-        except ValueError:
-            custom = None
-    if not isinstance(kept, str) and isinstance(custom, Mapping):
-        audit = custom.get("hy_dedup")
-        if isinstance(audit, Mapping):
-            kept = audit.get("kept_id")
-    if isinstance(kept, str) and kept:
-        names.append(kept)
-    return tuple(dict.fromkeys(names))

memhub/apps/core/src/one_memory/hy_response_guard.py

@@ -1,331 +0,0 @@
-"""Transport validation for the pinned HY SDK; HY's prompts and decisions pass through unchanged."""
-
-import logging
-from contextlib import nullcontext
-from time import monotonic
-from typing import Any
-
-from one_memory.diagnostic_log import diagnostic_event, diagnostic_exception, elapsed_ms
-from one_memory.hy_source_context import HyWriteBindings
-from one_memory.hy_step_timing import HyStepTimer
-
-
-def _http_status(value: Any) -> int | None:
-    return value if type(value) is int and 100 <= value <= 599 else None
-
-
-def _exception_status(error: Exception) -> int | None:
-    try:
-        status = _http_status(getattr(error, "status_code", None))
-        if status is not None:
-            return status
-        status = _http_status(getattr(getattr(error, "response", None), "status_code", None))
-        if status is not None:
-            return status
-        # Native LLMRetryError retains sanitized attempt dictionaries. Only the
-        # final attempt describes the terminal failure; never parse error text.
-        attempts = getattr(error, "attempts", None)
-        if isinstance(attempts, (list, tuple)) and attempts and isinstance(attempts[-1], dict):
-            return _http_status(attempts[-1].get("status_code"))
-    except Exception:
-        # Diagnostic attribute access must not replace the provider exception.
-        return None
-    return None
-
-
-def _model_failure_code(status: int | None) -> str:
-    specific = {
-        401: "hy_memory_model_authentication_failed",
-        402: "hy_memory_model_payment_required",
-        403: "hy_memory_model_access_denied",
-        408: "hy_memory_model_timeout",
-        429: "hy_memory_model_rate_limited",
-    }
-    if status in specific:
-        return specific[status]
-    if status is not None and 500 <= status <= 599:
-        return "hy_memory_model_service_unavailable"
-    if status is not None and 400 <= status <= 422:
-        return "hy_memory_model_request_incompatible"
-    return "hy_memory_model_request_failed"
-
-
-def _terminal_client_timeout(error: Exception) -> bool:
-    try:
-        attempts = getattr(error, "attempts", None)
-        if not isinstance(attempts, (list, tuple)) or not attempts:
-            return False
-        final = attempts[-1]
-        if type(final) is not dict:
-            return False
-        kind = final.get("failure_kind")
-        return type(kind) is str and kind == "timeout"
-    except Exception:
-        return False
-
-
-class HyResponseGuard:
-    def __init__(self, provider: Any, *, phase: str = "extract") -> None:
-        self.provider = provider
-        #: Diagnostic label only; the request passes through unchanged.
-        self.phase = phase
-        self.failures: list[str] = []
-
-    def __getattr__(self, name: str) -> Any:
-        return getattr(self.provider, name)
-
-    async def complete(self, *args: Any, **kwargs: Any) -> Any:
-        return await self._complete("complete", args, kwargs)
-
-    async def complete_messages(self, *args: Any, **kwargs: Any) -> Any:
-        return await self._complete("complete_messages", args, kwargs)
-
-    async def _complete(self, method: str, args: tuple[Any, ...], kwargs: dict[str, Any]) -> Any:
-        for attempt in range(2):
-            started = monotonic()
-            try:
-                response = await getattr(self.provider, method)(*args, **kwargs)
-            except Exception as error:
-                status = _exception_status(error)
-                code = (
-                    "hy_memory_model_timeout"
-                    if status is None and _terminal_client_timeout(error)
-                    else _model_failure_code(status)
-                )
-                self.failures.append(code)
-                logging.getLogger(__name__).warning(
-                    "%s: status_code=%s", code, status if status is not None else "unknown"
-                )
-                diagnostic_event(
-                    level="error",
-                    component="hy.model",
-                    event="hy.model_call.failed",
-                    outcome="failed",
-                    duration_ms=elapsed_ms(started),
-                    error_code=code,
-                    **diagnostic_exception(error),
-                    details={"phase": self.phase, "attempt": attempt + 1, "call_count": 1},
-                )
-                raise
-            truncated = getattr(response, "finish_reason", "") == "length"
-            empty = not (response.content or "").strip() and not getattr(
-                response, "tool_calls", None
-            )
-            if not truncated and not empty:
-                diagnostic_event(
-                    level="info",
-                    component="hy.model",
-                    event="hy.model_call.completed",
-                    outcome="succeeded",
-                    duration_ms=elapsed_ms(started),
-                    details={"phase": self.phase, "attempt": attempt + 1, "call_count": 1},
-                )
-                return response
-            code = (
-                "hy_memory_model_output_truncated"
-                if truncated
-                else "hy_memory_model_empty_response"
-            )
-            if attempt:
-                self.failures.append(code)
-                raise ValueError(code)
-            logging.getLogger(__name__).warning("%s: retrying the same HY model request", code)
-            if truncated and "max_tokens" in kwargs:
-                kwargs = {**kwargs, "max_tokens": min(16000, kwargs["max_tokens"] * 2)}
-        raise AssertionError("unreachable")
-
-
-class HyReconcileGuard:
-    #: Set only for the span of one leased write.
-    step_timer: HyStepTimer | None = None
-
-    def __init__(self, reconciler: Any) -> None:
-        self.reconciler = reconciler
-        self.failures: list[str] = []
-        self.bindings: HyWriteBindings | None = None
-        self.imported_contents: frozenset[str] | None = None
-        self.mutation_target_ids: set[str] = set()
-
-    def begin_write(self) -> None:
-        self.mutation_target_ids.clear()
-
-    def __getattr__(self, name: str) -> Any:
-        return getattr(self.reconciler, name)
-
-    async def reconcile(self, *args: Any, **kwargs: Any) -> Any:
-        timer = self.step_timer
-        with timer.step("reconcile") if timer is not None else nullcontext():
-            return await self._reconcile(args, kwargs)
-
-    async def _reconcile(self, args: tuple[Any, ...], kwargs: dict[str, Any]) -> Any:
-        started = monotonic()
-        if self.imported_contents is not None:
-            metadata = kwargs.get("new_memories_with_meta") or []
-            texts = kwargs.get("new_memories") or []
-            if (
-                not metadata
-                or not texts
-                or any(
-                    not isinstance(item, dict) or item.get("content") not in self.imported_contents
-                    for item in metadata
-                )
-                or any(text not in self.imported_contents for text in texts)
-                or {item["content"] for item in metadata} != set(texts)
-            ):
-                code = "hy_memory_write_verification_import_input_mismatch"
-                self.failures.append(code)
-                raise ValueError(code)
-        try:
-            result = await self.reconciler.reconcile(*args, **kwargs)
-        except Exception as error:
-            diagnostic_event(
-                level="error",
-                component="hy.reconcile",
-                event="hy.reconcile.failed",
-                outcome="failed",
-                duration_ms=elapsed_ms(started),
-                error_code="hy_memory_reconcile_failed",
-                **diagnostic_exception(error),
-            )
-            raise
-        if not result.success:
-            self.failures.append("hy_memory_reconcile_failed")
-            diagnostic_event(
-                level="error",
-                component="hy.reconcile",
-                event="hy.reconcile.failed",
-                outcome="failed",
-                duration_ms=elapsed_ms(started),
-                error_code="hy_memory_reconcile_failed",
-                details={"result_count": len(result.ops)},
-            )
-            return result
-        try:
-            await self._check_targets(result, kwargs)
-        except ValueError as error:
-            code = str(error)
-            self.failures.append(code)
-            raise
-        except Exception as error:
-            code = "hy_memory_write_verification_failed"
-            self.failures.append(code)
-            raise ValueError(code) from error
-        owners: dict[str, set[str]] = {}
-        for item in kwargs.get("new_memories_with_meta") or []:
-            if item.get("owner") in {"user", "agent"}:
-                owners.setdefault(item["content"], set()).add(item["owner"])
-        # HY's no-candidate ADD path copies content verbatim but omits owner.
-        # Restore only an unambiguous exact match, never infer from wording.
-        for operation in result.ops:
-            source_owners = owners.get(operation.content, set())
-            if operation.op == "ADD" and operation.owner is None and len(source_owners) == 1:
-                operation.owner = next(iter(source_owners))
-        operation_counts = {"ADD": 0, "UPDATE": 0, "SUPERSEDE": 0, "DELETE": 0}
-        for operation in result.ops:
-            raw_name = getattr(operation, "op", "")
-            name = str(getattr(raw_name, "value", raw_name)).upper()
-            if name in {"UPDATE", "SUPERSEDE", "DELETE", "SHADOW"}:
-                target = getattr(operation, "memory_id", None)
-                if isinstance(target, str) and target:
-                    self.mutation_target_ids.add(target)
-                self.mutation_target_ids.update(
-                    key
-                    for key in getattr(operation, "memory_ids", ())
-                    if isinstance(key, str) and key
-                )
-            if name == "ADD":
-                operation_counts["ADD"] += 1
-            elif name in {"UPDATE", "MERGE", "NOOP"}:
-                operation_counts["UPDATE"] += 1
-            elif name == "SUPERSEDE":
-                operation_counts["SUPERSEDE"] += 1
-            elif name in {"DELETE", "SHADOW"}:
-                operation_counts["DELETE"] += 1
-        diagnostic_event(
-            level="info",
-            component="hy.reconcile",
-            event="hy.reconcile.completed",
-            outcome="succeeded",
-            duration_ms=elapsed_ms(started),
-            details={
-                "created_count": operation_counts["ADD"],
-                "updated_count": operation_counts["UPDATE"],
-                "superseded_count": operation_counts["SUPERSEDE"],
-                "deleted_count": operation_counts["DELETE"],
-                "result_count": len(result.ops),
-            },
-        )
-        return result
-
-    async def _check_targets(self, result: Any, kwargs: dict[str, Any]) -> None:
-        """HY may only touch nodes of this user and scope that this write was allowed to see."""
-        targets = {op.memory_id for op in result.ops if op.memory_id}
-        targets.update(key for op in result.ops for key in getattr(op, "memory_ids", []))
-        for key in targets:
-            node = await kwargs["vector_store"].get_by_id(key)
-            if (
-                node is None
-                or node.user_id != kwargs["user_id"]
-                or node.agent_id != kwargs["agent_id"]
-            ):
-                raise ValueError("hy_memory_write_verification_target_invalid")
-            if self.bindings is not None:
-                known = self.bindings.known.get(key)
-                if known is None or (known.content, known.owner) != (node.content, node.owner):
-                    raise ValueError("hy_memory_write_verification_target_unbound")
-        if self.bindings is None:
-            return
-        for op in result.ops:
-            if not op.content:
-                continue
-            linked = {key for key in (op.memory_id, *getattr(op, "memory_ids", [])) if key}
-            self.bindings.record_operation_support(op.content, op.owner, linked)
-            if op.memory_id:
-                self.bindings.verified_operations[op.memory_id] = (op.content, op.owner)
-
-
-def install_hy_write_guards(client: Any) -> None:
-    """The pinned HY SDK treats an empty reconcile response as a successful empty op list.
-
-    Bind this client's real write paths. Pro/lite use the ordinary writer;
-    ultra uses the System2 writer's inner writer. Keep HY prompts, extraction,
-    retrieval, reconciliation and storage intact; never patch the shared SDK
-    installation.
-    """
-
-    async def install() -> None:
-        from one_memory.hy_write_verification import HyExtractionGuard
-
-        ordinary = await client._registry.get_writer()
-        system2 = await client._registry.get_system2_writer()
-        guards: list[Any] = []
-        seen_writers: set[int] = set()
-        seen_guards: set[int] = set()
-        for writer in (ordinary, system2._writer):
-            if id(writer) in seen_writers:
-                continue
-            seen_writers.add(id(writer))
-            agent = writer.mem_agent
-            if not isinstance(agent.llm_provider, HyResponseGuard):
-                agent.llm_provider = HyResponseGuard(agent.llm_provider)
-                agent.extractor.llm = agent.llm_provider
-            if not isinstance(agent.extractor, HyExtractionGuard):
-                agent.extractor = HyExtractionGuard(agent.extractor)
-            reconciler = writer.reconciler
-            provider = reconciler._get_llm()
-            if not isinstance(provider, HyResponseGuard):
-                reconciler._llm = HyResponseGuard(provider, phase="reconcile")
-            if not isinstance(reconciler, HyReconcileGuard):
-                writer._reconciler = HyReconcileGuard(reconciler)
-            for guard in (
-                agent.llm_provider,
-                reconciler._llm,
-                writer._reconciler,
-                agent.extractor,
-            ):
-                if id(guard) not in seen_guards:
-                    seen_guards.add(id(guard))
-                    guards.append(guard)
-        client._one_memory_write_guards = tuple(guards)
-
-    client._loop_thread.run(install())

memhub/apps/core/src/one_memory/hy_runtime_config.py

@@ -14,7 +14,7 @@ from pathlib import Path
 from typing import Literal, cast
 
 HY_MEMORY_DISTRIBUTION = "hy-memory"
-HY_MEMORY_VERSION = "2.0.5.dev3"
+HY_MEMORY_VERSION = "2.0.5"
 HY_MEMORY_NAMESPACE = "governed-v2"
 DiagnosticLogLevel = Literal["error", "warning", "info", "debug"]
 
@@ -33,49 +33,8 @@ _COLLECTION_LOGGERS = (
 )
 _NOISY_LIBRARY_LOGGERS = ("httpx", "httpcore", "watchfiles", "jieba")
 
-# One Memory deliberately stays on ultra because its existing, user-visible
-# profile/graph flow calls digest(), which the pinned public SDK exposes only
-# for ultra clients. Reader and non-reranker behavior remain pinned for
-# compatibility with the SDK authority-migration development preview.
-HY_MEMORY_MODE = "ultra"
-# The SDK only recognises `legacy` and `hybrid` (see
-# hy_memory/pipelines/_retrieval/config.py: `ALL_READERS = (READER_LEGACY,
-# READER_HYBRID)`). The old spelling `hybrid_v2` was silently rejected and
-# fell back to legacy, so the hybrid-v2 recall strategy never ran at all.
 HY_MEMORY_READER = "hybrid"
 
-HY_MEMORY_BEHAVIOR_ENV: dict[str, str] = {
-    "MEMORY_MODE": HY_MEMORY_MODE,
-    "MEMORY_ENABLE_GRAPH": "true",
-    "MEMORY_GRAPH_RERANK_ENABLED": "false",
-    "MEMORY_GRAPH_TEMPORAL_SPINE": "true",
-    "MEMORY_TOPIC_SUMMARY_ENABLED": "false",
-    "MEMORY_EPISODE_SUMMARY_ENABLED": "false",
-    "MEMORY_EPISODE_RECALL_ENABLED": "true",
-    "MEMORY_DYNAMIC_EPISODE_ENABLED": "true",
-    "MEMORY_SESSION_EPISODE_ENABLED": "false",
-    "MEMORY_SESSION_EPISODE_CONTEXT_ENABLED": "false",
-    "MEMORY_EPISODE_CAPSULE_CONTEXT_ENABLED": "true",
-    "MEMORY_EPISODE_CAPSULE_QUERY_MIN": "0.65",
-    "MEMORY_EPISODE_CAPSULE_ASSIGNMENT_MIN": "0.60",
-    "MEMORY_EPISODE_CAPSULE_FIRST_RANK_MAX": "3",
-    "MEMORY_EPISODE_CAPSULE_MEMBER_LIMIT": "3",
-    "MEMORY_ENABLE_INTENTION": "false",
-    "MEMORY_ENABLE_QUERY_REWRITE": "false",
-    "MEMORY_FEW_SHOT_ENABLED": "true",
-    "MEMORY_HYBRID_V2_QUERY_EXPAND": "1",
-    "MEMORY_HYBRID_V2_GEO_INTENT_PRIOR": "0.6",
-    "MEMORY_HYBRID_V2_DIVERSITY": "1",
-    "MEMORY_HYBRID_V2_TEMPORAL_ADMIT_PRIOR": "0.55",
-    "MEMORY_HYBRID_V2_TEMPORAL_BOOST": "0.18",
-    "MEMORY_CROSS_DOMAIN_CON_THRESHOLD": "0.5",
-    "MEMORY_READER_ENABLE_SUMMARY": "false",
-    "MEMORY_SUMMARY_ENABLED_IN_SYS2": "false",
-    "MEMORY_LOCAL_RERANK_ENABLED": "false",
-    "RECONCILE_ENABLED": "true",
-    "HY_MEMORY_READER": HY_MEMORY_READER,
-}
-
 
 def detailed_logging_enabled(data_dir: Path) -> bool:
     return diagnostic_log_level(data_dir) == "debug"
@@ -168,7 +127,6 @@ def configure_hy_runtime_environment(data_dir: Path) -> None:
     log_dir = memory_root / "logs"
     memory_root.mkdir(parents=True, exist_ok=True)
     log_dir.mkdir(parents=True, exist_ok=True)
-    os.environ.update(HY_MEMORY_BEHAVIOR_ENV)
     os.environ["MEMORY_DATA_DIR"] = str(memory_root)
     os.environ["MEMORY_LOG_DIR"] = str(log_dir)
     level = diagnostic_log_level(data_dir)
@@ -178,9 +136,3 @@ def configure_hy_runtime_environment(data_dir: Path) -> None:
     os.environ["MEMORY_MEMORY_OPERATIONS_ENABLED"] = "false"
     os.environ["ANONYMIZED_TELEMETRY"] = "False"
     os.environ.pop("CHROMA_OTEL_COLLECTION_ENDPOINT", None)
-
-
-def configure_hy_behavior_environment() -> None:
-    """Set import-time HY behavior flags without touching paths or secrets."""
-
-    os.environ.update(HY_MEMORY_BEHAVIOR_ENV)

memhub/apps/core/src/one_memory/hy_sdk_migration.py

@@ -0,0 +1,47 @@
+"""Offline snapshot and restore of the HY SDK data plane for the App migration.
+
+Runtime product code must use public ``HyMemoryClient`` methods.  The startup
+migration bundle is the isolated exception: all writers are stopped, so it may
+copy and restore the SDK-owned local stores as files.
+"""
+
+from __future__ import annotations
+
+from pathlib import Path
+
+from hy_memory import (  # type: ignore[import-untyped]
+    DataPlaneRestore,
+    DataPlaneSnapshot,
+    restore_local_data_plane,
+    snapshot_local_data_plane,
+)
+
+
+def snapshot_data_plane(
+    *,
+    source_root: Path,
+    target_root: Path,
+    writers_stopped: bool,
+) -> DataPlaneSnapshot:
+    """Snapshot an explicitly quiesced SDK-owned local data plane."""
+    return snapshot_local_data_plane(
+        source_root=source_root,
+        target_root=target_root,
+        writers_stopped=writers_stopped,
+    )
+
+
+def restore_data_plane(
+    *,
+    snapshot_root: Path,
+    target_root: Path,
+    writers_stopped: bool,
+    protective_root: Path | None = None,
+) -> DataPlaneRestore:
+    """Restore an explicitly quiesced SDK-owned local data plane."""
+    return restore_local_data_plane(
+        snapshot_root=snapshot_root,
+        target_root=target_root,
+        writers_stopped=writers_stopped,
+        protective_root=protective_root,
+    )

memhub/apps/core/src/one_memory/hy_sdk_runtime.py

@@ -0,0 +1,443 @@
+"""HY SDK client construction and shared-runtime leasing.
+
+This module deliberately depends only on the SDK's package-root exports.  A
+lease owns one client while the process shares one ``SharedRuntime`` per
+physical MemHub data directory.
+"""
+
+from __future__ import annotations
+
+import asyncio
+import hashlib
+import os
+import threading
+from collections.abc import Callable, Coroutine
+from dataclasses import dataclass
+from pathlib import Path
+from typing import Any, ClassVar
+
+from hy_memory import (  # type: ignore[import-untyped]
+    HyMemoryClient,
+    MemoryConfig,
+    SharedRuntime,
+)
+
+from one_memory.embedding_assets import (
+    BUILTIN_EMBEDDING_DIMENSIONS,
+    BUILTIN_EMBEDDING_MODEL,
+)
+from one_memory.hy_graph_admin import (
+    GraphWidthMismatchError,
+    graph_store_enabled,
+    inspect_graph_width_mismatch,
+)
+from one_memory.hy_sdk_types import HySdkRuntimeError
+from one_memory.local_embedding import LocalEmbedder
+from one_memory.model_profiles import ModelProfileSnapshot, ProviderKind
+
+_RUNTIME_LOCK = threading.RLock()
+BUILTIN_RETRIEVAL_PROFILE_ID = "builtin-retrieval"
+_LOCAL_ONNX_PLACEHOLDER_URL = "http://127.0.0.1/local-onnx"
+DEFAULT_RELEVANCE_FLOOR = 0.35
+
+
+@dataclass
+class _RuntimeEntry:
+    runtime: SharedRuntime
+    references: int
+
+
+_RUNTIMES: dict[Path, _RuntimeEntry] = {}
+
+
+class LocalOnnxEmbeddingService:
+    """Public ``EmbeddingService`` implementation backed by MemHub's ONNX model."""
+
+    _embedder: ClassVar[LocalEmbedder | None] = None
+    _embedder_lock: ClassVar[threading.Lock] = threading.Lock()
+
+    def __init__(self, embedder: LocalEmbedder | None = None) -> None:
+        if embedder is None:
+            with self._embedder_lock:
+                if self._embedder is None:
+                    self.__class__._embedder = LocalEmbedder()
+                embedder = self._embedder
+        if embedder is None:  # pragma: no cover - guarded by the assignment above
+            raise HySdkRuntimeError("hy_sdk_local_embedding_unavailable")
+        self._local_embedder = embedder
+
+    async def embed(self, text: str, use_cache: bool = True) -> list[float]:
+        del use_cache
+        vectors = await asyncio.to_thread(self._local_embedder.encode, [text])
+        return vectors[0]
+
+    async def embed_with_meta(
+        self,
+        text: str,
+        use_cache: bool = True,
+    ) -> tuple[list[float], dict[str, Any]]:
+        return await self.embed(text, use_cache), {"provider": "local-onnx"}
+
+    async def embed_batch(
+        self,
+        texts: list[str],
+        use_cache: bool = True,
+    ) -> list[list[float]]:
+        del use_cache
+        if not texts:
+            return []
+        return await asyncio.to_thread(self._local_embedder.encode, list(texts))
+
+    async def embed_queued(self, text: str, use_cache: bool = True) -> list[float]:
+        return await self.embed(text, use_cache)
+
+    async def embed_queued_with_meta(
+        self,
+        text: str,
+        use_cache: bool = True,
+    ) -> tuple[list[float], dict[str, Any]]:
+        return await self.embed_with_meta(text, use_cache)
+
+    def get_embedding_dims(self) -> int:
+        return BUILTIN_EMBEDDING_DIMENSIONS
+
+
+class ClientLease:
+    """Own a client and release its shared runtime exactly once."""
+
+    def __init__(self, client: HyMemoryClient, data_dir: Path) -> None:
+        self.client = client
+        self.data_dir = data_dir.resolve()
+        self._closed = False
+        self._close_lock = threading.Lock()
+
+    @property
+    def closed(self) -> bool:
+        return self._closed
+
+    def close(self) -> None:
+        with self._close_lock:
+            if self._closed:
+                return
+            self._closed = True
+            try:
+                self.client.close()
+            finally:
+                _release_runtime(self.data_dir)
+
+    def __enter__(self) -> ClientLease:
+        return self
+
+    def __exit__(self, *_exc: object) -> None:
+        self.close()
+
+
+class SdkClientHandle:
+    """Lazy async owner for one profile-bound public SDK client lease."""
+
+    def __init__(
+        self,
+        profile: ModelProfileSnapshot,
+        *,
+        api_key: str | None,
+        data_dir: Path,
+        http_transport_factory: Callable[[], Any] | None = None,
+    ) -> None:
+        self._profile = profile
+        self._api_key = api_key
+        self._data_dir = data_dir
+        self._http_transport_factory = http_transport_factory
+        self._lease: ClientLease | None = None
+        self._lock = asyncio.Lock()
+
+    async def client(self) -> HyMemoryClient:
+        if self._lease is not None:
+            return self._lease.client
+        async with self._lock:
+            if self._lease is None:
+                self._lease = await asyncio.to_thread(
+                    acquire_client,
+                    self._profile,
+                    api_key=self._api_key,
+                    data_dir=self._data_dir,
+                    http_transport_factory=self._http_transport_factory,
+                )
+            return self._lease.client
+
+    async def close(self) -> None:
+        async with self._lock:
+            lease, self._lease = self._lease, None
+        if lease is not None:
+            await asyncio.to_thread(lease.close)
+
+
+def builtin_retrieval_snapshot() -> ModelProfileSnapshot:
+    """Return the model-free profile used by SDK-native read operations."""
+    return ModelProfileSnapshot(
+        profile_id=BUILTIN_RETRIEVAL_PROFILE_ID,
+        revision=0,
+        provider_kind=ProviderKind.LOCAL_ONNX,
+        base_url=_LOCAL_ONNX_PLACEHOLDER_URL,
+        model=BUILTIN_EMBEDDING_MODEL,
+        embedding_provider_kind=ProviderKind.LOCAL_ONNX,
+        embedding_model=BUILTIN_EMBEDDING_MODEL,
+        embedding_dimensions=BUILTIN_EMBEDDING_DIMENSIONS,
+        timeout_seconds=60,
+        max_concurrency=1,
+        context_window=None,
+        has_api_key=False,
+    )
+
+
+def sdk_relevance_floor() -> float:
+    raw = os.environ.get("ONE_MEMORY_HY_RELEVANCE_FLOOR", "").strip()
+    if not raw:
+        return DEFAULT_RELEVANCE_FLOOR
+    try:
+        value = float(raw)
+    except ValueError:
+        return DEFAULT_RELEVANCE_FLOOR
+    return min(1.0, max(0.0, value))
+
+
+def build_sdk_config(
+    profile: ModelProfileSnapshot,
+    *,
+    api_key: str | None,
+    data_dir: Path,
+) -> MemoryConfig:
+    """Map a model snapshot to SDK models, endpoints, backends, and local paths."""
+    generation_endpoint, generation_key = _generation_access(profile, api_key)
+    embedding_provider = profile.embedding_provider_kind or profile.provider_kind
+    embedding_service_is_local = embedding_provider is ProviderKind.LOCAL_ONNX
+    embedding_model = profile.embedding_model or (
+        BUILTIN_EMBEDDING_MODEL if embedding_service_is_local else None
+    )
+    if not embedding_model:
+        raise HySdkRuntimeError("hy_sdk_embedding_model_required")
+    dimensions: int | None
+    if embedding_service_is_local:
+        embedding_endpoint = None
+        embedding_key = None
+        dimensions = profile.embedding_dimensions or BUILTIN_EMBEDDING_DIMENSIONS
+    else:
+        embedding_endpoint = _openai_endpoint(
+            embedding_provider,
+            profile.embedding_base_url or profile.base_url,
+        )
+        embedding_key = "ollama" if embedding_provider is ProviderKind.OLLAMA else api_key
+        if embedding_provider is ProviderKind.OPENAI_COMPATIBLE and not embedding_key:
+            raise HySdkRuntimeError("hy_sdk_embedding_api_key_missing")
+        if embedding_provider not in {
+            ProviderKind.OPENAI_COMPATIBLE,
+            ProviderKind.OLLAMA,
+        }:
+            raise HySdkRuntimeError("hy_sdk_embedding_provider_unsupported")
+        dimensions = profile.embedding_dimensions
+
+    root = data_dir.resolve() / "hy-memory" / "governed-v2"
+    root.mkdir(parents=True, exist_ok=True)
+    collection = _collection_base(
+        embedding_provider, embedding_endpoint, embedding_model, dimensions
+    )
+
+    return MemoryConfig.from_dict(
+        {
+            "llm": {
+                "provider": "openai",
+                "model": profile.model,
+                "api_key": generation_key,
+                "base_url": generation_endpoint,
+            },
+            "embedder": {
+                "provider": "openai",
+                "model": (
+                    BUILTIN_EMBEDDING_MODEL if embedding_service_is_local else embedding_model
+                ),
+                "api_key": embedding_key,
+                "base_url": embedding_endpoint,
+                "embedding_dims": dimensions,
+            },
+            "vector_store": {
+                "provider": "chroma",
+                "collection_name": collection,
+                "persist_directory": str(root / "vector"),
+                "embedding_dims": dimensions,
+                "host": None,
+                "port": None,
+                "api_key": None,
+            },
+            "graph_store": {
+                "provider": "kuzu",
+                "db_path": str(root / "graph"),
+                "url": None,
+                "username": None,
+                "password": None,
+                "database": None,
+            },
+            "cache": {
+                "backend": "sqlite",
+                "db_path": str(root / "cache.sqlite3"),
+            },
+            "history": {"db_path": str(root / "history.sqlite3")},
+            "coding": {"db_path": str(root / "coding.sqlite3")},
+        }
+    )
+
+
+def acquire_client(
+    profile: ModelProfileSnapshot,
+    *,
+    api_key: str | None,
+    data_dir: Path,
+    http_transport_factory: Callable[[], Any] | None = None,
+) -> ClientLease:
+    """Construct a public SDK client and return its explicit runtime lease."""
+    config = build_sdk_config(profile, api_key=api_key, data_dir=data_dir)
+    config.http_transport_factory = http_transport_factory
+    mismatch = inspect_graph_width_mismatch(config)
+    if mismatch is not None:
+        raise GraphWidthMismatchError(mismatch)
+    key = data_dir.resolve()
+    runtime = _acquire_runtime(config, key)
+    embedding_service: LocalOnnxEmbeddingService | None = None
+    if (profile.embedding_provider_kind or profile.provider_kind) is ProviderKind.LOCAL_ONNX:
+        embedding_service = LocalOnnxEmbeddingService()
+    try:
+        client = HyMemoryClient(
+            config=config,
+            mode="ultra" if graph_store_enabled() else "pro",
+            runtime=runtime,
+            embedding_service=embedding_service,
+        )
+    except BaseException:
+        _release_runtime(key)
+        raise
+    return ClientLease(client, key)
+
+
+def close_all_shared_runtimes() -> None:
+    """Close process runtimes only after every client lease has been released."""
+    with _RUNTIME_LOCK:
+        if _RUNTIMES:
+            raise HySdkRuntimeError("hy_sdk_runtime_leases_active")
+
+
+def _generation_access(
+    profile: ModelProfileSnapshot,
+    api_key: str | None,
+) -> tuple[str, str]:
+    if profile.provider_kind is ProviderKind.LOCAL_ONNX:
+        return _LOCAL_ONNX_PLACEHOLDER_URL, "local-read-only"
+    if profile.provider_kind not in {
+        ProviderKind.OPENAI_COMPATIBLE,
+        ProviderKind.OLLAMA,
+    }:
+        raise HySdkRuntimeError("hy_sdk_generation_provider_unsupported")
+    endpoint = _openai_endpoint(profile.provider_kind, profile.base_url)
+    if profile.provider_kind is ProviderKind.OLLAMA:
+        return endpoint, "ollama"
+    if not api_key:
+        raise HySdkRuntimeError("hy_sdk_model_api_key_missing")
+    return endpoint, api_key
+
+
+def _collection_base(
+    provider: ProviderKind,
+    endpoint: str | None,
+    model: str,
+    dimensions: int | None,
+) -> str:
+    identity = "\0".join((provider.value, endpoint or "local", model, str(dimensions or "auto")))
+    return "memhub_" + hashlib.sha256(identity.encode()).hexdigest()[:16]
+
+
+def vector_collection_bases(profile: ModelProfileSnapshot) -> tuple[str, tuple[str, ...]]:
+    """The SDK vector collection base for this embedding space, and the older ones.
+
+    The SDK derives every vector collection (chat, Coding keys, Working facts,
+    each variant) by suffixing this base. Builds before #145 named the same
+    embedding space `one_memory_<12 hex>`, with or without a declared width;
+    the offline migration renames those so their vectors stay readable.
+    """
+    provider = profile.embedding_provider_kind or profile.provider_kind
+    local = provider is ProviderKind.LOCAL_ONNX
+    model = profile.embedding_model or (BUILTIN_EMBEDDING_MODEL if local else None)
+    if not model:
+        raise HySdkRuntimeError("hy_sdk_embedding_model_required")
+    endpoint: str | None
+    dimensions: int | None
+    if local:
+        endpoint, dimensions = None, profile.embedding_dimensions or BUILTIN_EMBEDDING_DIMENSIONS
+        legacy_endpoint = "local-onnx"
+    else:
+        endpoint = _openai_endpoint(provider, profile.embedding_base_url or profile.base_url)
+        dimensions = profile.embedding_dimensions
+        legacy_endpoint = endpoint
+    legacy_model = profile.embedding_model or "bge-small-zh-v1.5"
+    legacy = tuple(
+        "one_memory_"
+        + hashlib.sha256(
+            ":".join((provider.value, legacy_endpoint, legacy_model, *declared)).encode()
+        ).hexdigest()[:12]
+        for declared in ((), ("auto",), (str(profile.embedding_dimensions or "auto"),))
+    )
+    return _collection_base(provider, endpoint, model, dimensions), tuple(dict.fromkeys(legacy))
+
+
+def _openai_endpoint(provider: ProviderKind, endpoint: str | None) -> str:
+    value = (endpoint or "").rstrip("/")
+    if not value:
+        raise HySdkRuntimeError("hy_sdk_model_endpoint_missing")
+    if provider is ProviderKind.OLLAMA and not value.endswith("/v1"):
+        return f"{value}/v1"
+    return value
+
+
+def _acquire_runtime(config: MemoryConfig, key: Path) -> SharedRuntime:
+    with _RUNTIME_LOCK:
+        entry = _RUNTIMES.get(key)
+        if entry is None:
+            runtime = _run_sync(SharedRuntime.create(config))
+            _RUNTIMES[key] = _RuntimeEntry(runtime=runtime, references=1)
+            return runtime
+        entry.references += 1
+        return entry.runtime
+
+
+def _release_runtime(key: Path) -> None:
+    runtime: SharedRuntime | None = None
+    with _RUNTIME_LOCK:
+        entry = _RUNTIMES.get(key)
+        if entry is None:
+            return
+        entry.references -= 1
+        if entry.references == 0:
+            runtime = entry.runtime
+            del _RUNTIMES[key]
+    if runtime is not None:
+        runtime.close()
+
+
+def _run_sync[T](coroutine: Coroutine[Any, Any, T]) -> T:
+    """Run SDK async construction from either sync or async caller threads."""
+    try:
+        asyncio.get_running_loop()
+    except RuntimeError:
+        return asyncio.run(coroutine)
+
+    result: list[T] = []
+    failure: list[BaseException] = []
+
+    def run() -> None:
+        try:
+            result.append(asyncio.run(coroutine))
+        except BaseException as error:
+            failure.append(error)
+
+    thread = threading.Thread(target=run, name="hy-sdk-runtime-create")
+    thread.start()
+    thread.join()
+    if failure:
+        raise failure[0]
+    return result[0]

memhub/apps/core/src/one_memory/hy_sdk_types.py

@@ -0,0 +1,402 @@
+"""Validated records and response parsers for public HY SDK operations."""
+
+from __future__ import annotations
+
+from collections.abc import Mapping
+
+from pydantic import BaseModel, ConfigDict, Field
+
+from one_memory.semantic_deadline import SemanticMemoryHit
+
+
+class HySdkError(RuntimeError):
+    """Base error carrying a stable, product-facing code."""
+
+    def __init__(self, code: str) -> None:
+        super().__init__(code)
+        self.code = code
+
+
+class HySdkRuntimeError(HySdkError):
+    """The public SDK runtime could not be configured or leased."""
+
+
+class HySdkMigrationError(HySdkError):
+    """The privileged offline SDK data-plane migration failed."""
+
+
+class HySdkReadError(HySdkError):
+    """A public SDK read operation failed or returned an invalid shape."""
+
+
+class HySdkMemoryRecord(BaseModel):
+    model_config = ConfigDict(frozen=True, extra="forbid")
+
+    memory_id: str
+    content: str
+    layer: str
+    status: str
+    is_latest: bool
+    supersedes: tuple[str, ...] = ()
+    superseded_by: tuple[str, ...] = ()
+    tags: tuple[str, ...] = ()
+    owner: str | None = None
+    source_raw_ids: tuple[str, ...] = ()
+    user_id: str
+    agent_id: str
+    session_id: str | None = None
+    source: str | None = None
+    memory_at: int | None = None
+    gmt_created: int | None = None
+
+    @classmethod
+    def from_sdk(cls, raw: object) -> HySdkMemoryRecord:
+        if not isinstance(raw, Mapping):
+            raise HySdkReadError("hy_memory_record_shape_unknown")
+        if any(
+            not isinstance(raw.get(key), str) or not str(raw[key]).strip()
+            for key in ("memory_id", "content")
+        ):
+            raise HySdkReadError("hy_memory_record_shape_unknown")
+        try:
+            return cls(
+                memory_id=str(raw["memory_id"]),
+                content=str(raw["content"]),
+                layer=str(raw.get("layer") or ""),
+                status=str(raw.get("status") or ""),
+                is_latest=bool(raw.get("is_latest", True)),
+                supersedes=_string_tuple(raw.get("supersedes")),
+                superseded_by=_string_tuple(raw.get("superseded_by")),
+                tags=_string_tuple(raw.get("tags")),
+                owner=(str(raw["owner"]) if raw.get("owner") in {"user", "agent"} else None),
+                source_raw_ids=_source_raw_ids(raw),
+                user_id=str(raw.get("user_id") or ""),
+                agent_id=str(raw.get("agent_id") or ""),
+                session_id=(
+                    str(raw["session_id"]) if isinstance(raw.get("session_id"), str) else None
+                ),
+                source=str(raw["source"]) if isinstance(raw.get("source"), str) else None,
+                memory_at=raw.get("memory_at") if isinstance(raw.get("memory_at"), int) else None,
+                gmt_created=(
+                    raw.get("gmt_created") if isinstance(raw.get("gmt_created"), int) else None
+                ),
+            )
+        except (KeyError, TypeError, ValueError) as error:
+            raise HySdkReadError("hy_memory_record_shape_unknown") from error
+
+
+class HySdkMemoryPage(BaseModel):
+    model_config = ConfigDict(frozen=True, extra="forbid")
+
+    memories: tuple[HySdkMemoryRecord, ...]
+    total: int = Field(ge=0)
+    limit: int = Field(ge=1)
+    offset: int = Field(ge=0)
+
+    @classmethod
+    def from_sdk(cls, raw: object, *, limit: int, offset: int) -> HySdkMemoryPage:
+        if not isinstance(raw, Mapping):
+            raise HySdkReadError("hy_memory_list_shape_unknown")
+        rows: list[object] = []
+        total = 0
+        for bucket_name, rows_name in (("vdb", "memories"), ("graph", "nodes")):
+            bucket = raw.get(bucket_name)
+            if bucket is None and bucket_name == "graph":
+                continue
+            if not isinstance(bucket, Mapping) or not isinstance(bucket.get(rows_name), list):
+                raise HySdkReadError("hy_memory_list_shape_unknown")
+            bucket_total = bucket.get("total")
+            if (
+                not isinstance(bucket_total, int)
+                or isinstance(bucket_total, bool)
+                or bucket_total < 0
+            ):
+                raise HySdkReadError("hy_memory_list_shape_unknown")
+            total += bucket_total
+            rows.extend(bucket[rows_name])
+        deduplicated: dict[str, HySdkMemoryRecord] = {}
+        for row in rows:
+            record = HySdkMemoryRecord.from_sdk(row)
+            deduplicated.setdefault(record.memory_id, record)
+        return cls(
+            memories=tuple(deduplicated.values()),
+            total=total,
+            limit=limit,
+            offset=offset,
+        )
+
+
+class HySdkRawRecord(BaseModel):
+    model_config = ConfigDict(frozen=True, extra="forbid")
+
+    raw_id: str
+    content: str
+    user_id: str
+    agent_id: str
+    session_id: str
+    source: str
+    memory_at: int | None = None
+    gmt_created: int | None = None
+    custom: dict[str, object] = Field(default_factory=dict)
+
+    @classmethod
+    def from_sdk(cls, raw: object) -> HySdkRawRecord:
+        if not isinstance(raw, Mapping):
+            raise HySdkReadError("hy_memory_raw_shape_unknown")
+        if any(
+            not isinstance(raw.get(key), str) or not str(raw[key]).strip()
+            for key in ("raw_id", "content", "user_id", "agent_id", "session_id", "source")
+        ):
+            raise HySdkReadError("hy_memory_raw_shape_unknown")
+        custom = raw.get("custom")
+        if not isinstance(custom, Mapping):
+            custom = {}
+        try:
+            return cls(
+                raw_id=str(raw["raw_id"]),
+                content=str(raw["content"]),
+                user_id=str(raw["user_id"]),
+                agent_id=str(raw["agent_id"]),
+                session_id=str(raw["session_id"]),
+                source=str(raw["source"]),
+                memory_at=raw.get("memory_at") if isinstance(raw.get("memory_at"), int) else None,
+                gmt_created=(
+                    raw.get("gmt_created") if isinstance(raw.get("gmt_created"), int) else None
+                ),
+                custom=dict(custom),
+            )
+        except (KeyError, TypeError, ValueError) as error:
+            raise HySdkReadError("hy_memory_raw_shape_unknown") from error
+
+
+class HySdkRawHit(HySdkRawRecord):
+    score: float = Field(ge=0, le=1)
+
+    @classmethod
+    def from_sdk_search(cls, raw: object) -> HySdkRawHit:
+        if not isinstance(raw, Mapping):
+            raise HySdkReadError("hy_memory_raw_search_shape_unknown")
+        raw_id = raw.get("memory_id")
+        score = raw.get("score")
+        if (
+            not isinstance(raw_id, str)
+            or not isinstance(score, int | float)
+            or isinstance(score, bool)
+        ):
+            raise HySdkReadError("hy_memory_raw_search_shape_unknown")
+        base = HySdkRawRecord.from_sdk({**raw, "raw_id": raw_id})
+        return cls(
+            **base.model_dump(),
+            score=max(0.0, min(1.0, float(score))),
+        )
+
+
+class HySdkRawPage(BaseModel):
+    model_config = ConfigDict(frozen=True, extra="forbid")
+
+    raw: tuple[HySdkRawRecord, ...]
+    total: int = Field(ge=0)
+    limit: int = Field(ge=1)
+    offset: int = Field(ge=0)
+
+    @classmethod
+    def from_memory_list(
+        cls,
+        raw: object,
+        *,
+        limit: int,
+        offset: int,
+    ) -> HySdkRawPage:
+        """Parse ``list_memories(layers=['l1_raw'])`` into the App raw DTO."""
+        if not isinstance(raw, Mapping):
+            raise HySdkReadError("hy_memory_raw_list_shape_unknown")
+        bucket = raw.get("vdb")
+        if not isinstance(bucket, Mapping):
+            raise HySdkReadError("hy_memory_raw_list_shape_unknown")
+        rows = bucket.get("memories")
+        total = bucket.get("total")
+        if (
+            not isinstance(rows, list)
+            or not isinstance(total, int)
+            or isinstance(total, bool)
+            or total < 0
+        ):
+            raise HySdkReadError("hy_memory_raw_list_shape_unknown")
+        converted = []
+        for row in rows:
+            if not isinstance(row, Mapping) or row.get("layer") != "l1_raw":
+                raise HySdkReadError("hy_memory_raw_list_shape_unknown")
+            converted.append({**row, "raw_id": row.get("memory_id")})
+        return cls(
+            raw=tuple(HySdkRawRecord.from_sdk(row) for row in converted),
+            total=total,
+            limit=limit,
+            offset=offset,
+        )
+
+
+class HySdkUserProfile(BaseModel):
+    model_config = ConfigDict(frozen=True, extra="forbid")
+
+    request_id: str
+    profile: dict[str, object] | None
+    elapsed_ms: float = Field(ge=0)
+
+    @classmethod
+    def from_sdk(cls, raw: object) -> HySdkUserProfile:
+        if not isinstance(raw, Mapping):
+            raise HySdkReadError("hy_memory_profile_shape_unknown")
+        request_id = raw.get("request_id")
+        profile = raw.get("profile")
+        elapsed_ms = raw.get("elapsed_ms")
+        if (
+            not isinstance(request_id, str)
+            or (profile is not None and not isinstance(profile, Mapping))
+            or not isinstance(elapsed_ms, int | float)
+            or isinstance(elapsed_ms, bool)
+        ):
+            raise HySdkReadError("hy_memory_profile_shape_unknown")
+        return cls(
+            request_id=request_id,
+            profile=None if profile is None else dict(profile),
+            elapsed_ms=float(elapsed_ms),
+        )
+
+
+def parse_sdk_search(raw: object, *, error_code: str) -> tuple[SemanticMemoryHit, ...]:
+    rows = _reply_rows(raw, error_code)
+    found: list[SemanticMemoryHit] = []
+    seen: set[str] = set()
+    for item in sorted(rows, key=_score, reverse=True):
+        memory_id = item.get("memory_id") or item.get("id")
+        if not isinstance(memory_id, str) or not memory_id or memory_id in seen:
+            continue
+        seen.add(memory_id)
+        content = item.get("content")
+        member_ids, formatting_complete = _formatted_member_ids(item)
+        found.append(
+            SemanticMemoryHit(
+                memory_id=memory_id,
+                score=_score(item),
+                formatted_content=(
+                    content.strip() if isinstance(content, str) and content.strip() else None
+                ),
+                member_ids=member_ids,
+                sdk_formatted=True,
+                formatting_complete=formatting_complete,
+            )
+        )
+    return tuple(found)
+
+
+def parse_sdk_raw_search(raw: object) -> tuple[HySdkRawHit, ...]:
+    return tuple(
+        HySdkRawHit.from_sdk_search(row)
+        for row in _reply_rows(raw, "hy_memory_raw_search_shape_unknown")
+    )
+
+
+def _string_tuple(value: object) -> tuple[str, ...]:
+    if not isinstance(value, list):
+        return ()
+    return tuple(
+        dict.fromkeys(item.strip() for item in value if isinstance(item, str) and item.strip())
+    )
+
+
+def _source_raw_ids(raw: Mapping[str, object]) -> tuple[str, ...]:
+    custom = raw.get("custom")
+    values: list[object] = []
+    if isinstance(custom, Mapping):
+        listed = custom.get("source_raw_memory_ids")
+        if isinstance(listed, list):
+            values.extend(listed)
+    singular = raw.get("source_raw_memory_id")
+    if isinstance(singular, str):
+        values.append(singular)
+    return tuple(
+        dict.fromkeys(value.strip() for value in values if isinstance(value, str) and value.strip())
+    )
+
+
+def _reply_rows(raw: object, error_code: str) -> tuple[Mapping[str, object], ...]:
+    if not isinstance(raw, Mapping):
+        raise HySdkReadError(error_code)
+    keys = tuple(key for key in ("memories", "coding_memories") if key in raw)
+    if "scene" in raw:
+        scene = raw["scene"]
+        if not isinstance(scene, str):
+            raise HySdkReadError(error_code)
+        if scene in ("normal", "chat"):
+            selected = "memories"
+        elif scene in ("productivity", "working"):
+            selected = "coding_memories"
+        else:
+            raise HySdkReadError(error_code)
+        if selected not in keys:
+            raise HySdkReadError(error_code)
+    elif len(keys) == 1:
+        selected = keys[0]
+    else:
+        raise HySdkReadError(error_code)
+
+    result: list[Mapping[str, object]] = []
+    for key in keys:
+        value = raw[key]
+        if key == "memories" and isinstance(value, Mapping):
+            if set(value) != {"schema", "proactive", "normal"}:
+                raise HySdkReadError(error_code)
+            channels = tuple(value.values())
+        else:
+            channels = (value,)
+        for channel in channels:
+            if not isinstance(channel, list) or any(
+                not isinstance(row, Mapping) for row in channel
+            ):
+                raise HySdkReadError(error_code)
+            if key == selected:
+                result.extend(channel)
+    return tuple(result)
+
+
+def _score(memory: Mapping[str, object]) -> float:
+    score = memory.get("score")
+    return float(score) if isinstance(score, int | float) and not isinstance(score, bool) else 0.0
+
+
+def _formatted_member_ids(memory: Mapping[str, object]) -> tuple[tuple[str, ...], bool]:
+    represented: dict[str, None] = {}
+    composite = False
+    raw_members = memory.get("member_ids")
+    if raw_members is not None:
+        composite = True
+        if not isinstance(raw_members, list | tuple) or any(
+            not isinstance(member_id, str) or not member_id for member_id in raw_members
+        ):
+            return (), False
+        for member_id in raw_members:
+            represented.setdefault(member_id, None)
+    chain = memory.get("evolution_chain")
+    if chain is not None:
+        composite = True
+        if not isinstance(chain, list | tuple):
+            return (), False
+        for entry in chain:
+            if not isinstance(entry, Mapping):
+                return (), False
+            member_id = entry.get("node_id") or entry.get("memory_id")
+            if not isinstance(member_id, str) or not member_id:
+                return (), False
+            represented.setdefault(member_id, None)
+    if memory.get("capsule_memories") is not None or memory.get("timeline") is not None:
+        composite = True
+        if raw_members is None:
+            return (), False
+    content = memory.get("content")
+    if isinstance(content, str) and content.lstrip().startswith(
+        ("Related memories:", "Matched memories:", "Memory timeline:")
+    ):
+        composite = True
+        if not represented:
+            return (), False
+    return tuple(represented), not composite or bool(represented)

memhub/apps/core/src/one_memory/hy_source_context.py

@@ -1,153 +0,0 @@
-"""Per-write original Evidence and where each HY memory came from; never a memory source."""
-
-import json
-from collections.abc import Set as AbstractSet
-from importlib import import_module
-from typing import Any, Literal
-
-from pydantic import BaseModel, ConfigDict
-
-from one_memory.imports.models import EvidenceContentKind, EvidenceRecord
-from one_memory.input_sanitization import sanitize_model_input
-
-
-class HySupportedMemory(BaseModel):
-    model_config = ConfigDict(frozen=True, extra="forbid")
-
-    hy_memory_id: str
-    governed_memory_id: str
-    revision: int
-    content: str
-    owner: Literal["user", "agent"]
-    evidence_ids: tuple[str, ...]
-
-
-class HySourceContext(BaseModel):
-    model_config = ConfigDict(frozen=True, extra="forbid")
-
-    current: tuple[EvidenceRecord, ...]
-    history: tuple[EvidenceRecord, ...] = ()
-    supporting: tuple[EvidenceRecord, ...] = ()
-    known_memories: tuple[HySupportedMemory, ...] = ()
-
-
-class HyWriteBindings:
-    def __init__(self, context: HySourceContext) -> None:
-        if not context.current:
-            raise ValueError("hy_memory_write_verification_context_scope")
-        self.context = context
-        seed = context.current[0]
-        records = (*context.current, *context.history, *context.supporting)
-        workspaces = {record.workspace for record in records}
-        # A missing workspace is a legacy identity, not a wildcard that may
-        # absorb a named project. Keep NULL and named workspaces disjoint.
-        if None in workspaces and len(workspaces) != 1:
-            raise ValueError("hy_memory_write_verification_context_scope")
-        ids = [record.evidence_id for record in records]
-        if len(ids) != len(set(ids)):
-            raise ValueError("hy_memory_write_verification_context_scope")
-        for record in records:
-            if (
-                (
-                    record.user_id,
-                    record.source_id,
-                    record.conversation_id,
-                    record.workspace,
-                )
-                != (
-                    seed.user_id,
-                    seed.source_id,
-                    seed.conversation_id,
-                    seed.workspace,
-                )
-                or record.content_kind is not EvidenceContentKind.RAW_CONVERSATION
-                or record.role not in {"user", "assistant", "tool"}
-            ):
-                raise ValueError("hy_memory_write_verification_context_scope")
-        self.all_sources: list[dict[str, Any]] = [
-            {
-                "id": record.evidence_id,
-                "role": record.role,
-                "content": sanitize_model_input(record.verbatim),
-                "new": index < len(context.current),
-                "revision": record.revision,
-                "created_at": record.created_at.isoformat() if record.created_at else None,
-            }
-            for index, record in enumerate(records)
-        ]
-        self.sources = self.all_sources[: len(context.current) + len(context.history)]
-        self._supports: dict[tuple[str, str], tuple[str, ...]] = {}
-        self.profile_order: list[str] | None = None
-        self.verified_operations: dict[str, tuple[str, str]] = {}
-        self.known = {memory.hy_memory_id: memory for memory in context.known_memories}
-        for memory in context.known_memories:
-            if not memory.evidence_ids or not set(memory.evidence_ids) <= set(ids):
-                raise ValueError("hy_memory_write_verification_context_scope")
-            self._supports[(memory.content, memory.owner)] = memory.evidence_ids
-
-    def check_content(self, content: str) -> None:
-        try:
-            parsed = json.loads(content)
-        except ValueError as error:
-            raise ValueError("hy_memory_write_verification_input_mismatch") from error
-        expected = [
-            {"role": record.role, "content": sanitize_model_input(record.verbatim)}
-            for record in self.context.current
-        ]
-        # HY strips tool messages itself before chat extraction when it judges a turn chat.
-        without_tools = [item for item in expected if item["role"] != "tool"]
-        if parsed != expected and (without_tools == expected or parsed != without_tools):
-            raise ValueError("hy_memory_write_verification_input_mismatch")
-
-    def history_json(self) -> str:
-        return json.dumps(
-            [
-                {
-                    "evidence_id": source["id"],
-                    **{k: source[k] for k in ("role", "content", "revision", "created_at")},
-                }
-                for source in self.sources
-                if not source["new"]
-            ],
-            ensure_ascii=False,
-        )
-
-    def turn_support(
-        self, owner: str | None, targets: AbstractSet[str] = frozenset()
-    ) -> tuple[str, ...]:
-        """This turn's words by the memory's author, plus what a merged memory already cited."""
-        roles = {"user": {"user"}, "agent": {"assistant"}}.get(owner or "")
-        current = [source for source in self.all_sources if source["new"]]
-        ids = [source["id"] for source in current if roles is None or source["role"] in roles]
-        if not ids:
-            ids = [source["id"] for source in current]
-        for target in sorted(targets):
-            if target in self.known:
-                ids.extend(self.known[target].evidence_ids)
-        return tuple(dict.fromkeys(ids))
-
-    def record_turn_supports(self, items: list[dict[str, Any]]) -> None:
-        for item in items:
-            key = (item["content"], item["owner"] or "")
-            self._supports.setdefault(key, self.turn_support(item["owner"]))
-
-    def record_operation_support(self, content: str, owner: str | None, targets: set[str]) -> None:
-        key = (content, owner or "")
-        merged = (*self._supports.get(key, ()), *self.turn_support(owner, targets))
-        self._supports[key] = tuple(dict.fromkeys(merged))
-
-    def support_for(self, content: str, owner: str | None) -> tuple[str, ...]:
-        return self._supports.get((content, owner or ""), ())
-
-    def profile_support(self, content: str, values: dict[str, Any]) -> tuple[str, ...]:
-        render = import_module("hy_memory.agent.tools.basic_profile")._render_content
-
-        if not values or render(values, field_order=self.profile_order) != content:
-            return ()
-        ids: dict[str, None] = {}
-        for key, value in values.items():
-            support = self.support_for(f"{key}: {value}", "user")
-            if not support:
-                return ()
-            ids.update(dict.fromkeys(support))
-        return tuple(ids)

memhub/apps/core/src/one_memory/hy_write_verification.py

@@ -1,53 +0,0 @@
-"""Input checks around HY's native extractor; HY's output is published as it returns it."""
-
-import asyncio
-from typing import Any
-
-from one_memory.hy_source_context import HyWriteBindings
-
-
-def _extracted_items(info: dict[str, Any]) -> list[dict[str, Any]]:
-    items: list[dict[str, Any]] = []
-    for section in ("memory", "intentions"):
-        for item in info.get(section) or []:
-            if isinstance(item, dict) and isinstance(item.get("content"), str):
-                items.append({"content": item["content"], "owner": item.get("owner")})
-    for name, value in (info.get("basic_info") or {}).items():
-        items.append({"content": f"{name}: {value}", "owner": "user"})
-    return items
-
-
-class HyExtractionGuard:
-    def __init__(self, extractor: Any) -> None:
-        self.extractor = extractor
-        self.failures: list[str] = []
-        self._lock = asyncio.Lock()
-        self.bindings: HyWriteBindings | None = None
-
-    def __getattr__(self, name: str) -> Any:
-        return getattr(self.extractor, name)
-
-    async def extract(self, *args: Any, **kwargs: Any) -> Any:
-        async with self._lock:
-            try:
-                return await self._extract(args, kwargs)
-            except ValueError as error:
-                if str(error).startswith("hy_memory_write_verification"):
-                    self.failures.append(str(error))
-                raise
-
-    async def _extract(self, args: tuple[Any, ...], kwargs: dict[str, Any]) -> Any:
-        content = kwargs.get("content", args[0] if args else "")
-        bindings = self.bindings
-        if bindings is not None:
-            bindings.check_content(content)
-            fields = kwargs.get("basic_profile_fields")
-            bindings.profile_order = list(fields) if fields else None
-            kwargs = {**kwargs, "history_context": bindings.history_json()}
-        else:
-            # No raw-history identity was supplied; do not trust HY's text prefixes.
-            kwargs = {**kwargs, "history_context": ""}
-        result = await self.extractor.extract(*args, **kwargs)
-        if bindings is not None and result.success:
-            bindings.record_turn_supports(_extracted_items(result.info))
-        return result

memhub/apps/core/src/one_memory/imports/live_sync.py

@@ -557,12 +557,10 @@ class PollingLiveSyncSupervisor:
         )
 
     def _note_internal_error(self, source_id: str, error: Exception) -> None:
-        # Type alone used to be all this said. It is not enough to act on:
-        # `OperationalError` covers a locked database, a missing table and a
-        # corrupt page alike, and without the message and the frames two dozen
-        # of these in a log could only be guessed at. The stack is the
-        # redacted, `one_memory`-relative one, so it names the call site
-        # without printing anyone's paths or query text.
+        # The ordinary local log keeps the exception message for debugging.
+        # The structured diagnostic event deliberately keeps only the stable
+        # error code, source id, type, and relative stack: exception messages
+        # can contain paths, query text, or credentials and are exportable data.
         logging.getLogger(__name__).warning(
             "live_sync_internal_error %s: %s", type(error).__name__, error
         )
@@ -572,10 +570,7 @@ class PollingLiveSyncSupervisor:
             event="live_sync.internal_error",
             outcome="failed",
             error_code="live_sync_internal_error",
-            # `reason` is the listed key that can carry the message; see
-            # `diagnostic_log._DETAIL_KEYS`. Truncated because a detail value
-            # longer than 256 characters is rejected.
-            details={"source_id": source_id, "reason": str(error)[:200]},
+            details={"source_id": source_id},
             **diagnostic_exception(error),
         )
         if isinstance(error, sqlite3.Error):

memhub/apps/core/src/one_memory/imports/runner.py

@@ -867,10 +867,7 @@ class ImportRunner:
             outcome="failed",
             job_id=job_id,
             error_code="import_heartbeat_failed",
-            # `reason` is the listed key for this; see
-            # `diagnostic_log._DETAIL_KEYS`. Truncated: a detail value
-            # over 256 characters is rejected.
-            details={"reason": str(error)[:200]},
+            details={"phase": "heartbeat"},
             **diagnostic_exception(error),
         )
 

memhub/apps/core/src/one_memory/imports/sources/hermes.py

@@ -6,6 +6,7 @@ from pathlib import PurePosixPath
 
 from pydantic import JsonValue
 
+from one_memory.derived_echo import strip_injected_memory
 from one_memory.imports.access import (
     AuthorizedSourceFile,
     AuthorizedSourceRoot,
@@ -213,6 +214,7 @@ def parse_hermes_rollout_record(
         message.get("content"),
         allowed_block_types=frozenset({"text", "input_text", "output_text"}),
     )
+    text = strip_injected_memory(text)
     if not text:
         return ParsedSkip(
             reason_code=ScanIssueCode.NO_TEXT,
@@ -386,7 +388,8 @@ def _read_state_db(
             # The assistant's call sits on its own row as `tool_calls`; the
             # `tool` rows after it hold only what came back.
             calls = _tool_calls_text(row["tool_calls"]) if role == "assistant" else None
-            has_text = bool(str(row["content"] or "").strip())
+            content = strip_injected_memory(str(row["content"] or ""))
+            has_text = bool(content.strip())
             if role not in {"user", "assistant"} or (not has_text and calls is None):
                 yield ScanSkip(
                     cursor=cursor,
@@ -401,7 +404,7 @@ def _read_state_db(
                 role=str(role) if has_text else "tool",
                 written_by=_written_by(role, row["session_source"]) if has_text else WrittenBy.TOOL,
                 created_at=parse_timestamp(row["timestamp"]),
-                verbatim=str(row["content"]) if has_text else str(calls),
+                verbatim=content if has_text else str(calls),
                 workspace=workspace,
                 project=safe_project_label(workspace),
                 record_type="state_db_message" if has_text else "state_db_message.tool",

memhub/apps/core/src/one_memory/legacy_lifecycle_types.py

@@ -0,0 +1,35 @@
+"""Passive lifecycle projection used only by retired Governance readers."""
+
+from __future__ import annotations
+
+from collections.abc import Mapping
+from dataclasses import dataclass, field
+from types import MappingProxyType
+
+
+@dataclass(frozen=True, eq=False)
+class HyLifecycle:
+    hidden: Mapping[str, tuple[str, ...]] = field(default_factory=dict)
+
+    def __post_init__(self) -> None:
+        object.__setattr__(self, "hidden", MappingProxyType(dict(self.hidden)))
+
+    def __eq__(self, other: object) -> bool:
+        return isinstance(other, HyLifecycle) and dict(self.hidden) == dict(
+            other.hidden
+        )
+
+    def __hash__(self) -> int:
+        return hash(tuple(sorted(self.hidden.items())))
+
+    def replaced_by(self, memory_id: str) -> tuple[str, ...]:
+        return self.hidden.get(memory_id, ())
+
+    def replacing(self, memory_id: str) -> tuple[str, ...]:
+        return tuple(
+            sorted(
+                old
+                for old, replacements in self.hidden.items()
+                if memory_id in replacements
+            )
+        )

memhub/apps/core/src/one_memory/live_capture.py

@@ -1,6 +1,7 @@
 """Write visible hook text into Evidence. Tool output stays out."""
 
 import json
+import os
 import time
 from datetime import UTC, datetime
 from enum import StrEnum
@@ -74,6 +75,7 @@ _FILE_FALLBACK = {
 }
 _FALLBACK_EVENTS = frozenset({"Stop", "stop", "SubagentStop", "StopFailure"})
 _TEXT_BLOCKS = frozenset({"text"})
+SDK_MIGRATION_CAPTURE_FENCE = ".sdk-migration-capture-fence.json"
 
 
 def database_path() -> Path:
@@ -88,17 +90,42 @@ def capture_stdio(source_id: str, stdin: TextIO, output: TextIO) -> int:
     del output
     if source_id not in CAPTURE_SOURCES:
         return 2
+    database = database_path()
+    if sdk_migration_capture_fenced(database.parent):
+        return 0
     try:
         payload = json.loads(stdin.read() or "")
     except json.JSONDecodeError:
         return 0
     try:
-        capture_hook(source_id, payload, database_path())
+        capture_hook(source_id, payload, database)
     except Exception:
         return 0
     return 0
 
 
+def sdk_migration_capture_fenced(data_dir: Path) -> bool:
+    """Ignore stale fences; only a live Desktop migration pauses hook writes."""
+    marker = data_dir / SDK_MIGRATION_CAPTURE_FENCE
+    try:
+        document = json.loads(marker.read_text(encoding="utf-8"))
+        pid = int(document["pid"])
+        if pid < 1:
+            raise ValueError("invalid migration fence pid")
+        os.kill(pid, 0)
+        return True
+    except FileNotFoundError:
+        return False
+    except ProcessLookupError:
+        marker.unlink(missing_ok=True)
+        return False
+    except PermissionError:
+        return True
+    except (KeyError, TypeError, ValueError, json.JSONDecodeError, OSError):
+        marker.unlink(missing_ok=True)
+        return False
+
+
 def capture_hook(source_id: str, payload: object, database: Path) -> bool:
     if source_id not in CAPTURE_SOURCES or not isinstance(payload, dict):
         return False
@@ -224,7 +251,7 @@ def record_capture_rejection(
 def _submit_live_memory(store: EvidenceStore, record: EvidenceRecord, *, changed: bool) -> None:
     if not changed or not _sync_open(store.database_path, record.source_id):
         return
-    from one_memory.memory_processing import _conversation_chunks
+    from one_memory.conversation_processing import _conversation_chunks
     from one_memory.memory_server.durable_queue import DurableWriteQueue
     from one_memory.memory_server.unified_entry import UnifiedWriteIntake
 

memhub/apps/core/src/one_memory/local_embedding.py

@@ -10,16 +10,12 @@ weights alone.
 from __future__ import annotations
 
 import threading
-from dataclasses import dataclass
-from hashlib import sha256
 from importlib import import_module
 from typing import TYPE_CHECKING, Any
 
 from one_memory.embedding_assets import (
     BUILTIN_EMBEDDING_DIMENSIONS,
     BUILTIN_EMBEDDING_MAX_TOKENS,
-    EMBEDDING_ASSETS,
-    AssetVerificationError,
     MissingEmbeddingAssetsError,
     assets_present,
     resolve_asset_dir,
@@ -34,28 +30,6 @@ _FETCH_HINT = (
 )
 
 
-@dataclass(frozen=True)
-class TokenOffset:
-    """Half-open Python Unicode code point offsets; synthetic specials use [0,0)."""
-
-    start_cp: int
-    end_cp: int
-    special: bool = False
-
-
-@dataclass(frozen=True)
-class TokenizedText:
-    """Untruncated, unpadded encoding, including tokenizer postprocessor specials."""
-
-    token_ids: tuple[int, ...]
-    offsets: tuple[TokenOffset, ...]
-
-    def __post_init__(self) -> None:
-        # Defensive copies also keep runtime callers from retaining mutable lists.
-        object.__setattr__(self, "token_ids", tuple(self.token_ids))
-        object.__setattr__(self, "offsets", tuple(self.offsets))
-
-
 class LocalEmbedder:
     """Encodes text with the built-in model, loading it on first use."""
 
@@ -68,43 +42,6 @@ class LocalEmbedder:
         self._session: Any | None = None
         self._tokenizer: Any | None = None
         self._input_names: frozenset[str] = frozenset()
-        self._offset_lock = threading.Lock()
-        self._offset_tokenizer: Any | None = None
-
-    def tokenize_with_offsets(self, text: str) -> TokenizedText:
-        """Tokenize original text without encode()'s truncation or batch padding.
-
-        The Python tokenizers binding returns code point offsets into the input,
-        even when its model normalizer changes spelling. This separate lazy
-        tokenizer uses the exact locked asset; it never loads an ONNX session or
-        mutates the tokenizer shared by encode(). Offsets are not UTF-16 indices.
-        """
-        if not isinstance(text, str):
-            raise TypeError("embedding_offsets_require_text")
-        with self._offset_lock:
-            if self._offset_tokenizer is None:
-                directory = resolve_asset_dir()
-                path = directory / "tokenizer.json"
-                if not path.is_file():
-                    raise MissingEmbeddingAssetsError(f"{_FETCH_HINT} Looked in {directory}.")
-                data = path.read_bytes()
-                expected = next(a.sha256 for a in EMBEDDING_ASSETS if a.filename == path.name)
-                if sha256(data).hexdigest() != expected:
-                    raise AssetVerificationError("tokenizer.json: locked digest mismatch")
-                from tokenizers import Tokenizer
-
-                tokenizer = Tokenizer.from_str(data.decode("utf-8"))
-                tokenizer.no_truncation()
-                tokenizer.no_padding()
-                self._offset_tokenizer = tokenizer
-            encoding = self._offset_tokenizer.encode(text, add_special_tokens=True)
-            offsets = tuple(
-                TokenOffset(start, end, bool(special))
-                for (start, end), special in zip(
-                    encoding.offsets, encoding.special_tokens_mask, strict=True
-                )
-            )
-            return TokenizedText(tuple(encoding.ids), offsets)
 
     def _load(self) -> tuple[Any, Any]:
         # Double-checked under a lock: memory processing encodes from worker
@@ -121,7 +58,7 @@ class LocalEmbedder:
                 raise MissingEmbeddingAssetsError(f"{_FETCH_HINT} Looked in {directory}.")
 
             # Imported by name for the same reason kuzu and hy_memory are in
-            # hy_memory_bridge: onnxruntime ships no py.typed, and a dynamic
+            # onnxruntime ships no py.typed, and a dynamic
             # import keeps the strict checker on everything around it.
             ort = import_module("onnxruntime")
             from tokenizers import Tokenizer

memhub/apps/core/src/one_memory/loop_watchdog.py

@@ -14,6 +14,7 @@ stuck in it.
 """
 
 import logging
+import re
 import sys
 import threading
 import time
@@ -23,6 +24,7 @@ from types import FrameType
 logger = logging.getLogger("one_memory.loop_watchdog")
 
 DEFAULT_STALL_SECONDS = 2.0
+_SAFE_FALLBACK_FILE = re.compile(r"^[A-Za-z0-9_][A-Za-z0-9_.-]*\.py$")
 
 
 class LoopWatchdog:
@@ -120,22 +122,28 @@ def _safe_stack_for_main_thread() -> tuple[str, ...]:
 
 
 def _safe_stacks_for_all_threads() -> tuple[str, ...]:
-    """Return one content-free, bounded stack summary for every live thread."""
-    names = {thread.ident: thread.name for thread in threading.enumerate()}
+    """Return bounded stack summaries without recording thread names."""
     lines: list[str] = []
-    for ident, frame in sorted(sys._current_frames().items()):
+    for _ident, frame in sorted(sys._current_frames().items()):
         stack: list[str] = []
-        # Most specific frame first so the 256-character privacy bound keeps
-        # the likely culprit rather than only the thread bootstrap frames.
+        # Most specific frame first; append only complete frames so truncation
+        # can never turn a valid location into free text.
         for entry in reversed(traceback.extract_stack(frame)[-8:]):
             parts = entry.filename.replace("\\", "/").split("/")
             try:
                 root = parts.index("one_memory")
                 relative = "/".join(parts[root:])
             except ValueError:
-                relative = parts[-1]
+                fallback = parts[-1]
+                relative = fallback if _SAFE_FALLBACK_FILE.fullmatch(fallback) else "unknown.py"
             name = entry.name if entry.name.isidentifier() else "call"
-            stack.append(f"{relative}:{entry.lineno}:{name}")
-        thread_name = names.get(ident, "unknown").replace("\n", " ").replace("\r", " ")
-        lines.append(f"thread={thread_name[:48]}|{' > '.join(stack)}"[:256])
+            candidate = f"{relative}:{entry.lineno}:{name}"
+            separator = "" if not stack else " > "
+            if len("thread|") + len(" > ".join(stack)) + len(separator) + len(candidate) > 256:
+                break
+            stack.append(candidate)
+        if stack:
+            lines.append(f"thread|{' > '.join(stack)}")
+        if len(lines) == 16:
+            break
     return tuple(lines)

memhub/apps/core/src/one_memory/memory_governance.py

@@ -22,7 +22,7 @@ from pydantic import BaseModel, ConfigDict, Field
 
 from one_memory.database import enable_write_ahead_log
 from one_memory.database_access import main_database
-from one_memory.hy_lifecycle import HyLifecycle
+from one_memory.legacy_lifecycle_types import HyLifecycle
 from one_memory.library_upgrade import LibraryUpgradeError, PreparedLibrary, holds_legacy_rows
 from one_memory.memory_server.migrations import governance_ddl
 

memhub/apps/core/src/one_memory/memory_library.py

@@ -10,7 +10,7 @@ from typing import Literal
 from pydantic import BaseModel, ConfigDict, Field
 
 from one_memory.database_access import main_database
-from one_memory.hy_lifecycle import HyLifecycle
+from one_memory.legacy_lifecycle_types import HyLifecycle
 from one_memory.memory_evidence_freshness import register_memory_freshness
 from one_memory.memory_governance import (
     GovernedMemory,

memhub/apps/core/src/one_memory/memory_notes.py

@@ -3,7 +3,7 @@ import json
 import sqlite3
 from collections.abc import Sequence
 from datetime import UTC, datetime
-from typing import Literal, cast
+from typing import Any, Literal, cast
 from uuid import UUID
 
 from fastapi import APIRouter, BackgroundTasks, HTTPException, Request
@@ -21,11 +21,10 @@ from one_memory.imports.models import (
 )
 from one_memory.imports.storage import EvidenceStore
 from one_memory.input_sanitization import sanitize_model_input
-from one_memory.memory_governance import MemoryStatus
-from one_memory.memory_processing import MemoryProcessingSupervisor
+from one_memory.memory_governance import MemoryGovernanceStore, MemoryStatus
 from one_memory.memory_server.durable_queue import DurableWriteQueue
 from one_memory.memory_server.unified_entry import UnifiedWriteIntake
-from one_memory.memory_tasks import MemoryTaskStatus
+from one_memory.memory_tasks import MemoryTaskStatus, MemoryTaskStore
 from one_memory.secrets import SecretStoreUnavailableError
 
 SOURCE_ID = "one-memory-manual"
@@ -82,11 +81,23 @@ class SavedMemoryNoteEvidence(BaseModel):
     verbatim: str
 
 
+class LegacyMemoryNoteReader:
+    """Read pre-cutover note receipts without retaining any write runtime."""
+
+    def __init__(
+        self,
+        tasks: MemoryTaskStore,
+        governance: MemoryGovernanceStore,
+    ) -> None:
+        self.tasks = tasks
+        self.governance = governance
+
+
 class MemoryNoteStore:
     def __init__(
         self,
         evidence: EvidenceStore,
-        processor: MemoryProcessingSupervisor,
+        processor: Any | None,
         intake: UnifiedWriteIntake | None = None,
         *,
         sdk_authority: bool = False,
@@ -252,6 +263,8 @@ class MemoryNoteStore:
                 memory_ids=(list(durable.sdk_memory_ids) if durable is not None else []),
                 error=durable.detail if durable is not None else "write_missing",
             )
+        if self.processor is None:
+            raise RuntimeError("legacy_memory_note_processor_unavailable")
         task = self.processor.tasks.task_for_evidence(evidence_id)
         memory_ids = []
         if task:
@@ -356,6 +369,8 @@ def memory_note_router(dependencies: Sequence[Depends]) -> APIRouter:
         try:
             if notes.sdk_authority:
                 return await asyncio.to_thread(notes.receipt, record.evidence_id)
+            if notes.processor is None:
+                raise RuntimeError("legacy_memory_note_processor_unavailable")
             if notes.intake is None:
                 # Isolated tests construct the store without a queue.
                 await notes.processor.enqueue_conversation_off_thread(record)

memhub/apps/core/src/one_memory/memory_server/durable_queue.py

@@ -69,8 +69,8 @@ _TYPE_NAME = re.compile(r"^[A-Za-z_][A-Za-z0-9_]{0,63}$")
 def safe_error_code(error: BaseException) -> str:
     """Reduce an exception to a storable code, never its message text.
 
-    A recognised code (the message *is* the code, as `HyMemoryBridgeError`
-    and the permission guard raise them) is kept so callers can classify it.
+    A recognised code (the message *is* a stable domain code) is kept so
+    callers can classify it.
     Everything else collapses to its exception type, which is a fixed
     identifier rather than content.
     """
@@ -239,8 +239,8 @@ PERMISSION_REQUIRED_CODES = frozenset(
     }
 )
 
-#: Model-side codes `process_records` raises *before* it can open the bridge
-#: (no active profile, missing credential, secure storage down). They prove
+#: Model-side codes the SDK write coordinator raises before it can lease a
+#: client (no active profile, missing credential, secure storage down). They prove
 #: nothing was sent, so the write parks instead of being lost. Transport codes
 #: that can arrive after a request was in flight stay `unknown`.
 MODEL_NOT_READY_CODES = frozenset(
@@ -251,9 +251,9 @@ MODEL_NOT_READY_CODES = frozenset(
     }
 )
 
-#: Deterministic, pre-bridge refusals: the frozen input no longer matches
+#: Deterministic, pre-SDK refusals: the frozen input no longer matches
 #: current Evidence, or a turn exceeds the model's input window. Both are
-#: raised while validating the records, before any bridge or HY call, so
+#: raised while validating the records, before any SDK call, so
 #: nothing was sent and retrying the same intent changes nothing — a terminal
 #: `failed`, not `unknown` (which would wrongly imply a possibly-applied
 #: write) and not a retry (which would fail identically).
@@ -358,12 +358,10 @@ class UnknownWriteJob:
 
     `request_id` is the durable queue's idempotency key. It identifies the row
     for queue inspection and is the value `DurableWriteQueue.get()` accepts.
-    It is **not** the HY request id: `MemoryProcessingSupervisor` derives the
-    HY handle from the task it creates, so an inquiry implementation must
-    obtain that task-layer handle separately rather than passing this value to
-    HY. This is discovery only: it does not decide whether HY has the write,
-    and it deliberately exposes no transition method that could turn
-    `unknown` back into retryable work.
+    It is the public SDK request id used for exact outcome inquiry. This is
+    discovery only: it does not decide whether HY has the write, and it
+    deliberately exposes no transition method that could turn `unknown` back
+    into retryable work.
     """
 
     job_id: str
@@ -1759,8 +1757,8 @@ class DurableWriteQueue:
         """Retire an `unknown` job after an inquiry, never by guessing.
 
         `unknown` means HY may already hold the write; the only safe way out is
-        to ask by request id (`hy_capabilities` measured that the SDK supports
-        this) and then record what the inquiry found. `found=True` closes it as
+        to call the public SDK `get_write_outcome(request_id)` and then record
+        what the inquiry found. `found=True` closes it as
         `completed`; `found=False` gives up and marks it `dead`. The state
         machine's legal transitions are the authority, so this refuses any
         change that `job_state.next_state` would reject — in particular it can
@@ -2258,12 +2256,11 @@ async def execute_job(
     admit_hy_invocation: Callable[[str, Callable[[], Awaitable[bool]]], Awaitable[bool]]
     | None = None,
 ) -> None:
-    """Run one durable job through the existing authorized processing path.
+    """Run one durable job through the SDK-authority processor.
 
-    The executor does not call HY directly: `process_records` owns the
-    permission guard, the runner and governance. `hy_invoked` is recorded
-    before that call, so a crash mid-write is later reclaimed to `unknown`
-    instead of being retried blindly.
+    ``process_sdk_job`` owns the one public SDK write. ``hy_invoked`` is
+    recorded immediately before that call, so a crash mid-write is later
+    reclaimed to ``unknown`` instead of being retried blindly.
     """
     # Every store call here is synchronous SQLite work on the single writer or
     # a reader; made from the loop it would block health checks and the API for
@@ -2336,38 +2333,13 @@ async def execute_job(
     try:
         level = ServiceLevel.REALTIME if request.priority == "live" else ServiceLevel.BACKGROUND
         with service_level_context(level):
-            authority = getattr(supervisor, "sdk_authority", None)
-            if callable(authority) and authority():
-                sdk_result = await supervisor.process_sdk_job(
-                    records,
-                    request_id=request.idempotency_key,
-                    before_hy_invocation=mark_hy_invocation,
-                )
-                raw_ids = sdk_result.raw_ids
-                memory_ids = tuple(memory.memory_id for memory in sdk_result.memories)
-            else:
-                completed_tasks = (
-                    await supervisor.process_records(
-                        records,
-                        before_hy_invocation=mark_hy_invocation,
-                        source_admitted=True,
-                    )
-                    or ()
-                )
-                receipts = []
-                for task in completed_tasks:
-                    receipt = await in_the_background(supervisor.tasks.sdk_receipt, task.task_id)
-                    if receipt is None:
-                        raise RuntimeError("sdk_write_receipt_missing")
-                    receipts.append(receipt)
-                raw_ids = tuple(
-                    dict.fromkeys(raw_id for receipt in receipts for raw_id in receipt.raw_ids)
-                )
-                memory_ids = tuple(
-                    dict.fromkeys(
-                        memory_id for receipt in receipts for memory_id in receipt.memory_ids
-                    )
-                )
+            sdk_result = await supervisor.process_sdk_job(
+                records,
+                request_id=request.idempotency_key,
+                before_hy_invocation=mark_hy_invocation,
+            )
+            raw_ids = sdk_result.raw_ids
+            memory_ids = sdk_result.memory_ids
     except Exception as error:  # noqa: BLE001 - classified below
         # Never persist the raw message: it may be a provider response or the
         # user's own text, and `last_error` is shown over HTTP as `detail`.
@@ -2382,7 +2354,7 @@ async def execute_job(
         ):
             await in_the_background(queue.release_unstarted, job_id, owner=owner)
             return
-        # The task layer classifies these before any bridge is opened, so HY
+        # The write coordinator classifies these before any SDK client is leased, so HY
         # was provably not called. They must not read as `failed`:
         # - authorization stop -> cancelled (the user turned it off);
         # - no permission yet -> waiting_for_permission (parked, re-admitted
@@ -2445,13 +2417,10 @@ async def execute_job(
         sdk_raw_ids=raw_ids,
         sdk_memory_ids=memory_ids,
     )
-    authority = getattr(supervisor, "sdk_authority", None)
     release = getattr(evidence, "release_sdk_staging", None)
     durable_receipt = await in_the_background(queue.sdk_receipt, job_id)
     if (
-        callable(authority)
-        and authority()
-        and callable(release)
+        callable(release)
         and durable_receipt is not None
         and (durable_receipt.raw_ids or durable_receipt.memory_ids)
     ):

memhub/apps/core/src/one_memory/memory_server/hy_capabilities.py

@@ -1,182 +0,0 @@
-"""A0-03: measured answer to "can a HY write be made idempotent?".
-
-The result decides whether the `unknown` job state can ever be retired, so it
-is probed against the installed SDK rather than written down from memory.
-
-What is verified against `hy-memory==2.0.5.dev3`:
-
-- `HyMemoryClient.add(..., request_id=...)` accepts a caller-supplied id and
-  atomically claims it in the SDK cache before invoking a write. Replays return
-  the durable terminal result; running/unknown claims are not invoked again.
-- `HyMemoryClient.get_write_outcome(request_id)` reads that durable outcome
-  directly; it does not depend on optional pipeline tracing.
-
-A first pass at this probe got both answers wrong, and the failure mode is
-worth keeping in mind: searching SDK source for the string "idempotent" hits a
-docstring about *episode* nodes ("calls are idempotent") and a "dedup" comment
-about search over-sampling. Neither is write idempotency. So the probe checks
-the `add` path specifically and names the inspector class explicitly, instead
-of grepping for a word that appears in unrelated prose.
-
-Net: MemHub can query and settle an uncertain request without blind replay.
-`unknown` remains a legitimate operational state while the SDK claim itself is
-non-terminal, but the same request id cannot write twice.
-"""
-
-from __future__ import annotations
-
-import importlib
-import inspect
-from enum import StrEnum
-from typing import Any
-
-
-class HyIdempotencySupport(StrEnum):
-    """Whether the SDK can dedupe a replayed write."""
-
-    #: A caller key is honoured and a replay is rejected or folded together.
-    KEY = "key"
-    #: Same content is folded by the engine, without an explicit key.
-    CONTENT_HASH = "content_hash"
-    #: A replay writes a second time. Retrying after a crash duplicates.
-    NONE = "none"
-    #: Never probed. Must not survive A0-03.
-    UNKNOWN = "unknown"
-
-
-class HyResultLookupSupport(StrEnum):
-    """Whether the outcome of a past write can be queried afterwards."""
-
-    BY_REQUEST_ID = "by_request_id"
-    BY_MEMORY_ID_ONLY = "by_memory_id_only"
-    NONE = "none"
-    UNKNOWN = "unknown"
-
-
-def _module(name: str) -> Any | None:
-    try:
-        return importlib.import_module(name)
-    except ImportError:
-        return None
-
-
-def _has_callable(obj: Any, name: str) -> bool:
-    return obj is not None and callable(getattr(obj, name, None))
-
-
-def hy_write_accepts_request_id() -> bool:
-    """True when the write path carries a caller-supplied request id."""
-    client_mod = _module("hy_memory.client")
-    if client_mod is None:
-        return False
-    client = getattr(client_mod, "HyMemoryClient", None)
-    if client is None:
-        return False
-    add = getattr(client, "add", None)
-    if add is None:
-        return False
-    return "request_id" in inspect.signature(add).parameters
-
-
-def _add_path_declares_idempotency() -> bool:
-    """Whether the *write* path de-duplicates, judged on its own source.
-
-    Deliberately scoped to `add`/`async_add`. A whole-module search matches
-    unrelated prose: "calls are idempotent" describes episode-node upserts, and
-    a "dedup" comment describes search over-sampling. Both are about other
-    features and would make this probe answer `key` for the wrong reason.
-    """
-    client_mod = _module("hy_memory.client")
-    if client_mod is None:
-        return False
-    client = getattr(client_mod, "HyMemoryClient", None)
-    if client is None:
-        return False
-    sources: list[str] = []
-    for name in ("add", "async_add"):
-        func = getattr(client, name, None)
-        if func is None:
-            continue
-        try:
-            sources.append(inspect.getsource(func).lower())
-        except (OSError, TypeError):
-            continue
-    if not sources:
-        return False
-    if _has_callable(client, "get_write_outcome"):
-        claim = getattr(client, "_claim_write_outcome", None)
-        if claim is not None:
-            return True
-    # Older SDKs may expose an explicit key under one of these names.
-    markers = ("idempotency_key", "idempotent_key", "dedupe_key", "dedup_key")
-    return any(marker in source for source in sources for marker in markers)
-
-
-def _inspector_can_lookup_by_request_id() -> bool:
-    """True when pipeline logs can be queried by the write's request id."""
-    inspector = _module("hy_memory.inspector")
-    if inspector is None:
-        return False
-    for cls_name in ("MemoryInspector", "Inspector", "PipelineInspector"):
-        cls = getattr(inspector, cls_name, None)
-        if cls is None:
-            continue
-        for method in ("get_pipeline_logs", "async_get_pipeline_logs"):
-            func = getattr(cls, method, None)
-            if func is None:
-                continue
-            # The parameter is `request_id` on `MemoryInspector`; older
-            # docstrings call the same thing `n`. Accept either so a rename
-            # does not silently downgrade this to "no lookup available".
-            if {"request_id", "n"} & set(inspect.signature(func).parameters):
-                return True
-    return False
-
-
-def _probe_idempotency() -> HyIdempotencySupport:
-    if not hy_write_accepts_request_id():
-        # No handle at all: not even a correlation id to investigate later.
-        return HyIdempotencySupport.NONE
-    if _add_path_declares_idempotency():
-        return HyIdempotencySupport.KEY
-    # A request_id is present but nothing de-duplicates on it.
-    return HyIdempotencySupport.NONE
-
-
-def _probe_result_lookup() -> HyResultLookupSupport:
-    client_mod = _module("hy_memory.client")
-    client = getattr(client_mod, "HyMemoryClient", None) if client_mod is not None else None
-    if _has_callable(client, "get_write_outcome"):
-        return HyResultLookupSupport.BY_REQUEST_ID
-    if _inspector_can_lookup_by_request_id():
-        return HyResultLookupSupport.BY_REQUEST_ID
-    if _has_callable(client, "get"):
-        return HyResultLookupSupport.BY_MEMORY_ID_ONLY
-    return HyResultLookupSupport.NONE
-
-
-IDEMPOTENCY_SUPPORT: HyIdempotencySupport = _probe_idempotency()
-RESULT_LOOKUP_SUPPORT: HyResultLookupSupport = _probe_result_lookup()
-
-
-def probe_hy_write_idempotency() -> dict[str, Any]:
-    """Full probe report, for STATUS evidence and for debugging a bumped SDK.
-
-    Re-probed on every call so a dependency upgrade that changes the answer is
-    visible, rather than being frozen at import time.
-    """
-    client_mod = _module("hy_memory.client")
-    client = getattr(client_mod, "HyMemoryClient", None)
-    add = getattr(client, "add", None)
-    return {
-        "sdk_importable": client_mod is not None,
-        "add_signature": str(inspect.signature(add)) if add is not None else "",
-        "accepts_request_id": hy_write_accepts_request_id(),
-        "add_declares_idempotency": _add_path_declares_idempotency(),
-        "durable_write_outcome": (
-            client is not None and _has_callable(client, "get_write_outcome")
-        ),
-        "inspector_lookup_by_request_id": _inspector_can_lookup_by_request_id(),
-        "idempotency": _probe_idempotency().value,
-        "result_lookup": _probe_result_lookup().value,
-    }

memhub/apps/core/src/one_memory/memory_server/migrations.py

@@ -2217,34 +2217,6 @@ CREATE TABLE IF NOT EXISTS retrieval_chunks (
 """
 
 
-def retrieval_documents_v2_ddl(dialect: str = "mysql") -> str:
-    """The v2 documents table under its temporary `_v2` name.
-
-    `index_store.migrate_v2` creates this, copies rows from the v1 table, drops
-    the v1 table and renames. The rename is the only reason a `_v2` name exists
-    at all, and it is derived from the single definition so the copy-across
-    cannot drift from what fresh databases get.
-    """
-    documents = next(
-        statement
-        for statement in _split_statements(retrieval_index_ddl(dialect))
-        if "CREATE TABLE IF NOT EXISTS retrieval_documents (" in statement
-    )
-    return documents.replace(
-        "CREATE TABLE IF NOT EXISTS retrieval_documents (",
-        "CREATE TABLE retrieval_documents_v2 (",
-    )
-
-
-def retrieval_build_maps_ddl(dialect: str = "mysql") -> str:
-    """The build-maps table added by v2."""
-    return next(
-        statement
-        for statement in _split_statements(retrieval_index_ddl(dialect))
-        if "retrieval_build_maps" in statement
-    )
-
-
 async def _033_retrieval_index(cursor: Any) -> None:
     for statement in _split_statements(retrieval_index_ddl(_dialect(cursor))):
         await cursor.execute(statement)

memhub/apps/core/src/one_memory/memory_server/sqlite_semantics.py

@@ -64,7 +64,6 @@ EXEMPT_PATHS = frozenset(
         "memory_server/sqlite_semantics.py",
         "memory_tasks.py",
         "sdk_data_plane_migration.py",
-        "sdk_migration_inventory.py",
         "target_schema.py",
         # D93 A: the visibility cache is SQLite-only by construction. It is a
         # derived table with triggers on the six tables the read check reads —

memhub/apps/core/src/one_memory/memory_server/unified_entry.py

@@ -9,9 +9,8 @@ Evidence→queue crash window stayed open.
 This adapter is deliberately small.  It turns one committed Evidence record
 into the one-record complete-turn request, submits it once under an evidence
 derived idempotency key, and returns the durable record.  The existing
-``DurableWriteDrain`` remains the only consumer; it calls
-``process_records``, which preserves the complete-turn/debounce/settle rules
-in ``MemoryProcessingSupervisor``.
+``DurableWriteDrain`` remains the only consumer; it calls the SDK write
+coordinator once and persists only the public receipt ids.
 """
 
 from __future__ import annotations

memhub/apps/core/src/one_memory/memory_tasks.py

@@ -477,7 +477,7 @@ class MemoryTaskStore:
     #: Statuses whose input snapshot nothing reads back afterwards.
     #:
     #: Deliberately narrow. A `completed` task's inputs are read after the fact:
-    #: `memory_processing` compares them against the evidence actually sent to
+    #: SDK write processing compares them against the evidence actually sent to
     #: HY to prove coverage, and `matters/discovery` reads them to report how
     #: many records HY really saw. Clearing those would silently turn a
     #: verification into a pass and a report into a lie, which is far worse

memhub/apps/core/src/one_memory/model_request_audit.py

@@ -10,7 +10,6 @@ from contextlib import contextmanager
 from contextvars import ContextVar
 from dataclasses import dataclass
 from datetime import UTC, datetime
-from importlib import import_module
 from pathlib import Path
 from typing import TypedDict
 from uuid import uuid4
@@ -46,14 +45,7 @@ def audit_precipitation_request(request_id: str) -> Iterator[None]:
 
 
 def _current_prompt_stage() -> str:
-    try:
-        resolver = getattr(  # noqa: B009 - optional SDK version boundary
-            import_module("hy_memory.utils.model_call_context"),
-            "current_model_call_stage",
-        )
-    except (ImportError, AttributeError):
-        return "other"
-    return str(resolver())
+    return "other"
 
 
 class ModelRequestAuditStore:

memhub/apps/core/src/one_memory/recall.py

@@ -12,7 +12,7 @@ from typing import Any, Literal, Protocol
 
 from pydantic import BaseModel, ConfigDict, Field
 
-from one_memory.hy_memory_bridge import HySdkMemoryRecord
+from one_memory.hy_sdk_types import HySdkMemoryRecord
 from one_memory.memory_governance import GovernedMemory, MemoryStatus
 from one_memory.semantic_deadline import (
     SemanticMemoryHit,

memhub/apps/core/src/one_memory/retrieval/chunking.py

@@ -1,167 +0,0 @@
-"""Single-Evidence identity chunking; no reads, persistence, normalization or grants.
-
-Callers must first establish eligibility/sanitization/permission. This module
-does not strip wrappers or secrets and must not be used as a content safety gate.
-"""
-
-from dataclasses import dataclass
-from typing import Protocol
-
-from one_memory.embedding_assets import BUILTIN_EMBEDDING_MAX_TOKENS
-from one_memory.local_embedding import TokenizedText, TokenOffset
-
-
-class OffsetTokenizer(Protocol):
-    def tokenize_with_offsets(self, text: str) -> TokenizedText: ...
-
-
-class ChunkingError(ValueError):
-    """Invalid offsets, uncovered text or insufficient encoding budget; never truncate."""
-
-
-@dataclass(frozen=True)
-class TextChunk:
-    ordinal: int
-    start_cp: int
-    end_cp: int
-    token_count: int
-    text: str
-
-
-@dataclass(frozen=True)
-class _Span:
-    start: int
-    end: int
-    tokens: int
-
-
-def _spans(text: str, encoded: TokenizedText) -> tuple[_Span, ...]:
-    if not isinstance(encoded, TokenizedText) or len(encoded.token_ids) != len(encoded.offsets):
-        raise ChunkingError("chunk_token_offset_count_mismatch")
-    spans: list[_Span] = []
-    previous_start = previous_end = 0
-    for token_id, offset in zip(encoded.token_ids, encoded.offsets, strict=True):
-        if type(token_id) is not int or token_id < 0 or not isinstance(offset, TokenOffset):
-            raise ChunkingError("chunk_invalid_token_offset")
-        if (
-            type(offset.start_cp) is not int
-            or type(offset.end_cp) is not int
-            or type(offset.special) is not bool
-        ):
-            raise ChunkingError("chunk_invalid_token_offset")
-        start, end = offset.start_cp, offset.end_cp
-        if offset.special:
-            if start != 0 or end != 0:
-                raise ChunkingError("chunk_invalid_special_offset")
-            continue
-        if not 0 <= start < end <= len(text) or start < previous_start or end < previous_end:
-            raise ChunkingError("chunk_invalid_offset_order_or_bounds")
-        if start > previous_end and not text[previous_end:start].isspace():
-            raise ChunkingError("chunk_uncovered_nonwhitespace")
-        # Overlapping subpieces (including several pieces for one code point)
-        # form an indivisible interval; slicing inside it loses token identity.
-        if spans and start < spans[-1].end:
-            prior = spans[-1]
-            spans[-1] = _Span(prior.start, end, prior.tokens + 1)
-        else:
-            spans.append(_Span(start, end, 1))
-        previous_start, previous_end = start, end
-    if text[previous_end:] and not text[previous_end:].isspace():
-        raise ChunkingError("chunk_uncovered_nonwhitespace")
-    return tuple(spans)
-
-
-def _encoding(text: str, tokenizer: OffsetTokenizer) -> tuple[TokenizedText, tuple[_Span, ...]]:
-    try:
-        result = tokenizer.tokenize_with_offsets(text)
-    except Exception as error:
-        raise ChunkingError("chunk_tokenization_failed") from error
-    return result, _spans(text, result)
-
-
-def chunk_evidence(
-    text: str,
-    tokenizer: OffsetTokenizer,
-    *,
-    prefix: str = "",
-    target_tokens: int = 384,
-    overlap_tokens: int = 64,
-    max_tokens: int = BUILTIN_EMBEDDING_MAX_TOKENS,
-) -> tuple[TextChunk, ...]:
-    """Chunk one original with code point [start,end), preserving exact slices.
-
-    target/overlap count original non-special tokens. Overlap is up to the
-    requested token count at indivisible offset boundaries; forward progress is
-    mandatory. token_count is re-tokenized prefix + chunk.text INCLUDING specials,
-    never an additive estimate. Prefix is caller-provided, unchanged, and omitted
-    from chunk.text/offsets. Empty/whitespace-only inputs have no index chunks.
-    """
-    if not isinstance(text, str) or not isinstance(prefix, str):
-        raise ChunkingError("chunk_requires_text")
-    if (
-        any(type(value) is not int for value in (target_tokens, overlap_tokens, max_tokens))
-        or not 0 <= overlap_tokens < target_tokens
-        or not 1 <= max_tokens <= BUILTIN_EMBEDDING_MAX_TOKENS
-        or target_tokens > max_tokens
-    ):
-        raise ChunkingError("chunk_invalid_token_budget")
-    if not text or text.isspace():
-        return ()
-    _, spans = _encoding(text, tokenizer)
-    if not spans:
-        raise ChunkingError("chunk_uncovered_nonwhitespace")
-    chunks: list[TextChunk] = []
-    start_index = 0
-    previous_end_cp = 0
-    while start_index < len(spans):
-        end_index = start_index
-        body_tokens = 0
-        while end_index < len(spans):
-            candidate = body_tokens + spans[end_index].tokens
-            if end_index > start_index and candidate > target_tokens:
-                break
-            body_tokens = candidate
-            end_index += 1
-        # Prefer the last paragraph boundary in the latter half of the target.
-        # Never trade away an entire overlap window or split a token interval.
-        if end_index < len(spans):
-            count = 0
-            preferred = None
-            for boundary in range(start_index + 1, end_index + 1):
-                count += spans[boundary - 1].tokens
-                if count >= max(target_tokens // 2, overlap_tokens + 1):
-                    left, right = spans[boundary - 1].end, spans[boundary].start
-                    if "\n" in text[max(spans[boundary - 1].start, left - 1) : right]:
-                        preferred = boundary
-            if preferred is not None:
-                end_index = preferred
-        start_cp = 0 if start_index == 0 else spans[start_index].start
-        while end_index > start_index:
-            # Include gaps verbatim, including whitespace and combining marks;
-            # source coverage is never reconstructed by joining decoded tokens.
-            end_cp = spans[end_index].start if end_index < len(spans) else len(text)
-            encoded, _ = _encoding(prefix + text[start_cp:end_cp], tokenizer)
-            token_count = len(encoded.token_ids)
-            if token_count <= max_tokens:
-                break
-            end_index -= 1
-        else:
-            raise ChunkingError("chunk_indivisible_span_exceeds_token_budget")
-        if end_cp <= previous_end_cp or start_cp > previous_end_cp:
-            raise ChunkingError("chunk_coverage_cannot_progress")
-        chunks.append(TextChunk(len(chunks), start_cp, end_cp, token_count, text[start_cp:end_cp]))
-        previous_end_cp = end_cp
-        if end_index == len(spans):
-            break
-        next_start = end_index
-        overlap = 0
-        while next_start > start_index + 1:
-            candidate = overlap + spans[next_start - 1].tokens
-            if candidate > overlap_tokens:
-                break
-            next_start -= 1
-            overlap = candidate
-        start_index = next_start
-    if previous_end_cp != len(text):
-        raise ChunkingError("chunk_incomplete_coverage")
-    return tuple(chunks)

memhub/apps/core/src/one_memory/retrieval/content.py

@@ -1,92 +0,0 @@
-"""Shared qualification for identity-mapped raw indexing and candidate reads."""
-
-from collections.abc import Sequence
-
-from one_memory.imports.models import EvidenceContentKind, EvidenceRecord
-from one_memory.imports.sources.parsing import is_host_generated_user_turn, strip_host_wrappers
-from one_memory.input_sanitization import sanitize_model_input
-from one_memory.memory_processing import _is_hy_input_turn
-from one_memory.retrieval.chunking import TextChunk
-from one_memory.retrieval.mapped_content import PreparedMappedContent, prepare_mapped_content
-
-
-def identity_content_ready(record: EvidenceRecord) -> bool:
-    # Reuse pure existing hygiene checks, without invoking HY. Nonidentity
-    # cleaning requires a source map and remains explicitly uncovered for now.
-    return (
-        record.content_kind is EvidenceContentKind.RAW_CONVERSATION
-        and record.role in {"user", "assistant"}
-        and _is_hy_input_turn(record)
-        and not is_host_generated_user_turn(record.verbatim)
-        and strip_host_wrappers(record.verbatim) == record.verbatim.strip()
-        and sanitize_model_input(record.verbatim) == record.verbatim
-    )
-
-
-def index_content(
-    record: EvidenceRecord, *, normalizer_version: int, chunker_version: int
-) -> PreparedMappedContent | None:
-    """Generation-owned content semantics, never silently upgrade identity generations."""
-    if (normalizer_version, chunker_version) == (1, 1):
-        if not identity_content_ready(record):
-            raise ValueError("content_mapping_required")
-        return None
-    if (normalizer_version, chunker_version) == (2, 2):
-        return prepare_mapped_content(record)
-    raise ValueError("unsupported_content_generation")
-
-
-def validate_chunk_coverage(
-    record: EvidenceRecord, chunks: Sequence[TextChunk], prepared: PreparedMappedContent | None
-) -> None:
-    """Verify exact slices and complete citation-safe coverage without a tokenizer call."""
-    if len(chunks) > 8192 or (prepared is None and not chunks):
-        raise ValueError("invalid_chunk_manifest")
-    if prepared is None:
-        ranges = [(0, len(record.verbatim))]
-    else:
-        ranges = []
-        for span in prepared.metadata.segments:
-            if span.kind == "copy":
-                assert span.source_start_cp is not None and span.source_end_cp is not None
-                ranges.append((span.source_start_cp, span.source_end_cp))
-    range_index = 0
-    covered_end = ranges[0][0] if ranges else 0
-    previous_end = -1
-    for index, chunk in enumerate(chunks):
-        if not isinstance(chunk, TextChunk) or any(
-            type(value) is not int
-            for value in (chunk.ordinal, chunk.start_cp, chunk.end_cp, chunk.token_count)
-        ):
-            raise ValueError("invalid_chunk_manifest")
-        while range_index < len(ranges) and chunk.start_cp >= ranges[range_index][1]:
-            if record.verbatim[covered_end : ranges[range_index][1]].strip():
-                raise ValueError("incomplete_chunk_manifest")
-            range_index += 1
-            covered_end = ranges[range_index][0] if range_index < len(ranges) else 0
-        if range_index >= len(ranges):
-            raise ValueError("invalid_chunk_manifest")
-        left, right = ranges[range_index]
-        if (
-            chunk.ordinal != index
-            or not left <= chunk.start_cp < chunk.end_cp <= right
-            or chunk.end_cp <= previous_end
-            or not 1 <= chunk.token_count <= 512
-            or record.verbatim[chunk.start_cp : chunk.end_cp] != chunk.text
-            or (
-                chunk.start_cp > covered_end
-                and (prepared is None or record.verbatim[covered_end : chunk.start_cp].strip())
-            )
-        ):
-            raise ValueError("invalid_chunk_manifest")
-        covered_end = previous_end = chunk.end_cp
-    while range_index < len(ranges):
-        if prepared is None:
-            if covered_end != ranges[range_index][1]:
-                raise ValueError("incomplete_chunk_manifest")
-        elif record.verbatim[covered_end : ranges[range_index][1]].strip():
-            raise ValueError("incomplete_chunk_manifest")
-        range_index += 1
-        covered_end = ranges[range_index][0] if range_index < len(ranges) else 0
-    if prepared is not None and (not chunks) != (prepared.state == "no_indexable_content"):
-        raise ValueError("invalid_chunk_manifest")

memhub/apps/core/src/one_memory/retrieval/index_provisioning.py

@@ -1,96 +0,0 @@
-"""Publish one active generation for the built-in encoder, when asked to.
-
-The runtime deliberately only resumes an index somebody already initialized, so
-without this the feature could be switched on and still never index anything.
-Provisioning stays separate from `start()` for the same reason it exists: it is
-the one place allowed to create, and it creates only for a first enable.
-"""
-
-from __future__ import annotations
-
-import logging
-import sqlite3
-from pathlib import Path
-
-from one_memory.database_access import main_database
-from one_memory.embedding_assets import assets_present
-from one_memory.retrieval.index_store import RetrievalIndexStore
-from one_memory.retrieval.index_worker import LocalRetrievalEncoder
-from one_memory.retrieval.vector_store import RawVectorStore
-
-DATABASE_NAME = "one-memory.sqlite3"
-VECTOR_DIR_NAME = "retrieval-vectors"
-
-
-def provision_retrieval_index(data_dir: Path) -> str | None:
-    """The active generation id for the built-in encoder, creating it once.
-
-    Returns None whenever the right answer is to leave the data alone: no
-    built-in encoder, no database yet, an index from an older schema, or one
-    already published for a different encoder. None is not an error; the
-    lexical lane works without an index and says so through `degraded`.
-    """
-    database = data_dir / DATABASE_NAME
-    if not assets_present() or not database.is_file():
-        return None
-    identity = LocalRetrievalEncoder().identity
-    store = RetrievalIndexStore(database)
-    try:
-        with main_database(database).write() as db, db:
-            db.execute("BEGIN IMMEDIATE")
-            if (existing := _published(db)) is not None:
-                return existing if _matches(store, db, existing, identity) else None
-            version = _schema_version(db)
-            if version == 1:
-                # Someone's v1 index. Migrating it is maintenance, not startup.
-                return None
-            if version is None:
-                store.initialize_v2(db)
-            generation = store.create_generation(db, identity)
-        store.activate_generation(generation)
-        _ensure_vectors(data_dir / VECTOR_DIR_NAME, generation, identity.dimensions)
-    except Exception:
-        logging.getLogger(__name__).warning("retrieval_index_provision_failed", exc_info=True)
-        return None
-    return generation
-
-
-def _schema_version(db: sqlite3.Connection) -> int | None:
-    if not db.execute(
-        "SELECT 1 FROM sqlite_master WHERE type='table' AND name='retrieval_index_state'"
-    ).fetchone():
-        return None
-    row = db.execute(
-        "SELECT schema_version FROM retrieval_index_state WHERE singleton=1"
-    ).fetchone()
-    return None if row is None else int(row[0])
-
-
-def _published(db: sqlite3.Connection) -> str | None:
-    if not db.execute(
-        "SELECT 1 FROM sqlite_master WHERE type='table' AND name='retrieval_index_generations'"
-    ).fetchone():
-        return None
-    rows = db.execute(
-        "SELECT generation_id FROM retrieval_index_generations WHERE state='active' LIMIT 2"
-    ).fetchall()
-    return str(rows[0][0]) if len(rows) == 1 else None
-
-
-def _matches(
-    store: RetrievalIndexStore,
-    db: sqlite3.Connection,
-    generation: str,
-    identity: object,
-) -> bool:
-    try:
-        return store._identity(db, generation) == identity
-    except Exception:
-        return False
-
-
-def _ensure_vectors(directory: Path, generation: str, dimensions: int) -> None:
-    if directory.exists():
-        return
-    vectors = RawVectorStore.create(directory, generation_id=generation, dimensions=dimensions)
-    vectors.close()

memhub/apps/core/src/one_memory/retrieval/index_store.py

@@ -1,856 +0,0 @@
-"""Explicit derived-index transactions. No constructor migration or model invocation."""
-
-import hashlib
-import json
-import math
-import sqlite3
-import time
-from collections.abc import Callable, Iterator, Sequence
-from contextlib import contextmanager
-from dataclasses import asdict, dataclass, replace
-from datetime import UTC, datetime
-from pathlib import Path
-from uuid import uuid4
-
-from one_memory import ingest_journal as journal
-from one_memory.context_raw import _check, _connection
-from one_memory.database import take_the_write_lock_first
-from one_memory.database_access import main_database
-from one_memory.imports.models import EvidenceRecord
-from one_memory.imports.storage import EvidenceStore
-from one_memory.memory_server.migrations import (
-    _split_statements,
-    retrieval_build_maps_ddl,
-    retrieval_documents_v2_ddl,
-    retrieval_index_ddl,
-)
-from one_memory.retrieval.chunking import TextChunk
-from one_memory.retrieval.content import index_content, validate_chunk_coverage
-from one_memory.retrieval.mapped_content import ContentMap, PreparedMappedContent
-from one_memory.retrieval.models import TimeRange
-
-
-def digest(value: object) -> str:
-    return hashlib.sha256(
-        json.dumps(
-            value, sort_keys=True, ensure_ascii=False, separators=(",", ":"), allow_nan=False
-        ).encode()
-    ).hexdigest()
-
-
-@dataclass(frozen=True)
-class EncoderIdentity:
-    encoder_id: str
-    encoder_digest: str
-    tokenizer_digest: str
-    dimensions: int
-    chunker_version: int = 1
-    normalizer_version: int = 1
-    query_template_version: int = 1
-
-    def __post_init__(self) -> None:
-        if not self.encoder_id or len(self.encoder_id) > 128:
-            raise ValueError("invalid_encoder")
-        for value in (self.encoder_digest, self.tokenizer_digest):
-            if len(value) != 64 or any(c not in "0123456789abcdef" for c in value):
-                raise ValueError("invalid_encoder_digest")
-        for number in (
-            self.dimensions,
-            self.chunker_version,
-            self.normalizer_version,
-            self.query_template_version,
-        ):
-            if type(number) is not int or not 1 <= number <= 1000000:
-                raise ValueError("invalid_encoder_number")
-
-
-@dataclass(frozen=True)
-class IndexLease:
-    generation_id: str
-    evidence_id: str
-    build_id: str
-    event_seq: int
-    owner: str
-    token: str
-    expires_at: float
-    grant_fingerprint: str
-
-
-@dataclass(frozen=True)
-class PublishedChunk:
-    chunk_id: str
-    build_id: str
-    evidence_id: str
-    revision: int
-    start_cp: int
-    end_cp: int
-    ordinal: int
-    source_id: str
-    event_time: datetime | None
-
-
-def chunk_identity(lease: IndexLease, chunk: TextChunk, map_digest: str | None = None) -> str:
-    values = [lease.build_id, chunk.ordinal, chunk.start_cp, chunk.end_cp, digest(chunk.text)]
-    return digest(values if map_digest is None else [*values, map_digest])
-
-
-@contextmanager
-def transaction(db: sqlite3.Connection) -> Iterator[None]:
-    if not db.in_transaction:
-        raise RuntimeError("retrieval_requires_caller_transaction")
-    name = "retrieval_" + uuid4().hex
-    previous = db.row_factory
-    db.row_factory = sqlite3.Row
-    db.execute(f"SAVEPOINT {name}")
-    try:
-        yield
-    except BaseException:
-        db.execute(f"ROLLBACK TO {name}")
-        db.execute(f"RELEASE {name}")
-        raise
-    else:
-        db.execute(f"RELEASE {name}")
-    finally:
-        db.row_factory = previous
-
-
-class RetrievalIndexStore:
-    def __init__(self, database_path: Path):
-        self.database_path = database_path
-
-    @contextmanager
-    def _db(
-        self, *, write: bool = False, deadline: float | None = None
-    ) -> Iterator[sqlite3.Connection]:
-        if deadline is not None:
-            if write:
-                raise ValueError("deadline_connection_is_read_only")
-            with _connection(self.database_path, deadline) as db:
-                db.execute("PRAGMA query_only=ON")
-                yield db
-            return
-        database = main_database(self.database_path)
-        with (
-            database.write(create=False, foreign_keys=True, row_factory=sqlite3.Row)
-            if write
-            else database.read(foreign_keys=True, row_factory=sqlite3.Row)
-        ) as db:
-            try:
-                if write:
-                    take_the_write_lock_first(db)
-                else:
-                    db.execute("PRAGMA query_only=ON")
-                    db.execute("BEGIN")
-                yield db
-                if write:
-                    db.commit()
-            except BaseException:
-                db.rollback()
-                raise
-
-    def initialize(self, db: sqlite3.Connection) -> None:
-        """Caller owns maintenance/exclusive writer admission; rollback includes baseline."""
-        self._initialize(db, version=1)
-
-    def initialize_v2(self, db: sqlite3.Connection) -> None:
-        """Initialize a fresh store directly at the folded v2 schema.
-
-        Startup provisioning uses this path so a brand-new database does not
-        create a v1 schema and immediately migrate it. Explicit maintenance
-        callers keep `initialize()`'s v1 baseline contract instead.
-        """
-        self._initialize(db, version=2)
-
-    def _initialize(self, db: sqlite3.Connection, *, version: int) -> None:
-        with transaction(db):
-            exists = db.execute(
-                "SELECT 1 FROM sqlite_master WHERE name='retrieval_index_state'"
-            ).fetchone()
-            if exists:
-                self._store_id(db)
-            elif db.execute(
-                "SELECT 1 FROM sqlite_master WHERE name IN ('retrieval_documents',"
-                "'retrieval_chunks','retrieval_index_generations','retrieval_consumers')"
-            ).fetchone():
-                raise ValueError("retrieval_partial_initialization")
-            journal.initialize(db)
-            # Migration 33 owns the schema. Explicit initialization keeps the
-            # maintenance-facing v1 baseline so `migrate_v2` remains observable;
-            # startup provisioning may instead ask for the folded v2 directly.
-            for sql in _split_statements(retrieval_index_ddl("sqlite", version=version)):
-                db.execute(sql)
-            row = db.execute(
-                "SELECT schema_version FROM retrieval_index_state WHERE singleton=1"
-            ).fetchone()
-            if row is None:
-                db.execute(
-                    "INSERT INTO retrieval_index_state VALUES (1,?,?)",
-                    (version, uuid4().hex),
-                )
-            elif row[0] not in (1, 2):
-                raise ValueError("retrieval_schema_mismatch")
-
-    def migrate_v2(self, db: sqlite3.Connection) -> None:
-        """Explicit maintenance transaction only; readers/worker never invoke this."""
-        with transaction(db):
-            self._store_id(db)
-            version = db.execute("SELECT schema_version FROM retrieval_index_state").fetchone()[0]
-            if version == 2:
-                db.execute(
-                    "SELECT build_id,source_map_json,map_digest,coverage_state "
-                    "FROM retrieval_build_maps LIMIT 0"
-                )
-                return
-            if (
-                db.execute(
-                    "SELECT 1 FROM sqlite_master WHERE type='table' AND name='retrieval_build_maps'"
-                ).fetchone()
-                is not None
-            ):
-                # A v1 index must not already own a v2-only table. Treat this
-                # as a foreign/partial schema instead of letting `IF NOT
-                # EXISTS` silently accept a table with the wrong columns.
-                raise sqlite3.OperationalError("retrieval_build_maps already exists")
-            # Both tables come from migration 33, named/derived there so this
-            # module holds no DDL text of its own.
-            db.execute(retrieval_documents_v2_ddl("sqlite"))
-            db.execute("INSERT INTO retrieval_documents_v2 SELECT * FROM retrieval_documents")
-            db.execute("DROP TABLE retrieval_documents")
-            db.execute("ALTER TABLE retrieval_documents_v2 RENAME TO retrieval_documents")
-            db.execute(
-                "CREATE INDEX retrieval_jobs ON retrieval_documents(generation_id,state,event_seq)"
-            )
-            db.execute(retrieval_build_maps_ddl("sqlite"))
-            db.execute("UPDATE retrieval_index_state SET schema_version=2 WHERE singleton=1")
-
-    @staticmethod
-    def _store_id(db: sqlite3.Connection) -> str:
-        row = db.execute(
-            "SELECT schema_version,data_store_id FROM retrieval_index_state WHERE singleton=1"
-        ).fetchone()
-        if row is None or row[0] not in (1, 2) or not isinstance(row[1], str) or len(row[1]) != 32:
-            raise ValueError("retrieval_not_initialized")
-        return str(row[1])
-
-    def create_generation(self, db: sqlite3.Connection, identity: EncoderIdentity) -> str:
-        with transaction(db):
-            self._check_content_version(db, identity)
-            key = digest([self._store_id(db), asdict(identity)])
-            encoded = json.dumps(asdict(identity), sort_keys=True)
-            row = db.execute(
-                "SELECT identity_json FROM retrieval_index_generations WHERE generation_id=?",
-                (key,),
-            ).fetchone()
-            if row is not None and row[0] != encoded:
-                raise ValueError("generation_identity_changed")
-            if row is None:
-                db.execute(
-                    "INSERT INTO retrieval_index_generations VALUES (?,?,'building',?)",
-                    (key, encoded, time.time()),
-                )
-                db.execute("INSERT INTO retrieval_consumers VALUES (?,0,0)", (key,))
-            return key
-
-    def consume(self, db: sqlite3.Connection, generation_id: str, *, limit: int = 256) -> int:
-        if type(limit) is not int or not 1 <= limit <= 256:
-            raise ValueError("invalid_consume_limit")
-        with transaction(db):
-            store_id = self._store_id(db)
-            generation = db.execute(
-                "SELECT state FROM retrieval_index_generations WHERE generation_id=?",
-                (generation_id,),
-            ).fetchone()
-            if generation is None or generation[0] == "retired":
-                raise ValueError("generation_not_writable")
-            cursor = db.execute(
-                "SELECT after_seq FROM retrieval_consumers WHERE generation_id=?", (generation_id,)
-            ).fetchone()
-            if cursor is None:
-                raise ValueError("consumer_missing")
-            rows = db.execute(
-                "SELECT seq FROM evidence_ingest_events WHERE seq>? ORDER BY seq LIMIT ?",
-                (cursor[0], limit),
-            ).fetchall()
-            for row in rows:
-                event = journal.event(db, row[0])
-                if event is None:
-                    raise ValueError("journal_event_missing")
-                if event.user_id != "local":
-                    continue
-                build = digest(
-                    [
-                        store_id,
-                        generation_id,
-                        event.evidence_id,
-                        event.incarnation,
-                        event.evidence_revision,
-                        event.fingerprint,
-                    ]
-                )
-                state = "deleted" if event.kind == "delete" else "pending"
-                db.execute(
-                    """INSERT INTO retrieval_documents
-                    (generation_id,evidence_id,source_id,event_seq,build_id,state)
-                    VALUES (?,?,?,?,?,?) ON CONFLICT(generation_id,evidence_id) DO UPDATE SET
-                    source_id=excluded.source_id,event_seq=excluded.event_seq,build_id=excluded.build_id,
-                    state=excluded.state,active_build_id=NULL,lease_owner=NULL,lease_token=NULL,
-                    lease_until=NULL,grant_fingerprint=NULL
-                    WHERE excluded.event_seq>retrieval_documents.event_seq""",
-                    (generation_id, event.evidence_id, event.source_id, event.seq, build, state),
-                )
-            if rows:
-                db.execute(
-                    "UPDATE retrieval_consumers SET after_seq=? WHERE generation_id=?",
-                    (rows[-1][0], generation_id),
-                )
-            return len(rows)
-
-    def consume_pending(self, generation_id: str, *, limit: int = 256) -> int:
-        with self._db(write=True) as db:
-            return self.consume(db, generation_id, limit=limit)
-
-    def encoder_identity(self, generation_id: str) -> EncoderIdentity:
-        with self._db() as db:
-            return self._identity(db, generation_id)
-
-    @staticmethod
-    def _check_content_version(db: sqlite3.Connection, identity: EncoderIdentity) -> None:
-        versions = (identity.normalizer_version, identity.chunker_version)
-        if versions not in ((1, 1), (2, 2)):
-            raise ValueError("unsupported_content_generation")
-        if versions == (2, 2):
-            row = db.execute(
-                "SELECT schema_version FROM retrieval_index_state WHERE singleton=1"
-            ).fetchone()
-            if row is None or row[0] != 2:
-                raise ValueError("retrieval_schema2_required")
-
-    def _identity(self, db: sqlite3.Connection, generation_id: str) -> EncoderIdentity:
-        row = db.execute(
-            "SELECT identity_json FROM retrieval_index_generations WHERE generation_id=?",
-            (generation_id,),
-        ).fetchone()
-        if row is None:
-            raise ValueError("generation_missing")
-        identity = EncoderIdentity(**json.loads(row[0]))
-        if digest([self._store_id(db), asdict(identity)]) != generation_id:
-            raise ValueError("generation_identity_changed")
-        self._check_content_version(db, identity)
-        return identity
-
-    def _prepare(
-        self, db: sqlite3.Connection, generation_id: str, record: EvidenceRecord
-    ) -> PreparedMappedContent | None:
-        identity = self._identity(db, generation_id)
-        return index_content(
-            record,
-            normalizer_version=identity.normalizer_version,
-            chunker_version=identity.chunker_version,
-        )
-
-    @staticmethod
-    def _grant(db: sqlite3.Connection, source_id: str) -> str | None:
-        row = db.execute(
-            """SELECT root_path,candidate_kind,granted_at,root_identity,revision
-            FROM source_grants
-            WHERE user_id='local' AND source_id=? AND revoked_at IS NULL""",
-            (source_id,),
-        ).fetchone()
-        return digest(tuple(row)) if row and row[3] and row[4] > 0 else None
-
-    def claim(
-        self, generation_id: str, owner: str, *, now: float | None = None, lease_seconds: int = 60
-    ) -> IndexLease | None:
-        now = time.time() if now is None else now
-        if not owner or len(owner) > 128 or not math.isfinite(now) or not 1 <= lease_seconds <= 300:
-            raise ValueError("invalid_lease")
-        with self._db(write=True) as db:
-            generation = db.execute(
-                "SELECT state FROM retrieval_index_generations WHERE generation_id=?",
-                (generation_id,),
-            ).fetchone()
-            if generation is None or generation[0] == "retired":
-                raise ValueError("generation_not_writable")
-            rows = db.execute(
-                """SELECT * FROM retrieval_documents WHERE generation_id=? AND
-                (state='pending' OR (state='leased' AND lease_until<=?))
-                ORDER BY event_seq LIMIT 256""",
-                (generation_id, now),
-            ).fetchall()
-            for row in rows:
-                event = journal.event(db, row["event_seq"])
-                grant = self._grant(db, row["source_id"])
-                if event is None or grant is None or not journal.matches_current(db, event):
-                    db.execute(
-                        "UPDATE retrieval_documents SET state='blocked',active_build_id=NULL "
-                        "WHERE generation_id=? AND evidence_id=?",
-                        (generation_id, row["evidence_id"]),
-                    )
-                    continue
-                token = uuid4().hex
-                until = now + lease_seconds
-                db.execute(
-                    """UPDATE retrieval_documents SET state='leased',lease_owner=?,lease_token=?,
-                    lease_until=?,grant_fingerprint=?,attempts=attempts+1
-                    WHERE generation_id=? AND evidence_id=?""",
-                    (owner, token, until, grant, generation_id, row["evidence_id"]),
-                )
-                return IndexLease(
-                    generation_id,
-                    row["evidence_id"],
-                    row["build_id"],
-                    row["event_seq"],
-                    owner,
-                    token,
-                    until,
-                    grant,
-                )
-        return None
-
-    def _validate_lease(
-        self, db: sqlite3.Connection, lease: IndexLease, now: float
-    ) -> journal.IngestEvent:
-        if not math.isfinite(now):
-            raise ValueError("invalid_lease_time")
-        generation = db.execute(
-            "SELECT state FROM retrieval_index_generations WHERE generation_id=?",
-            (lease.generation_id,),
-        ).fetchone()
-        if generation is None or generation[0] == "retired":
-            raise ValueError("lease_generation_retired")
-        row = db.execute(
-            "SELECT * FROM retrieval_documents WHERE generation_id=? AND evidence_id=?",
-            (lease.generation_id, lease.evidence_id),
-        ).fetchone()
-        if (
-            row is None
-            or row["state"] != "leased"
-            or (
-                row["build_id"],
-                row["event_seq"],
-                row["lease_owner"],
-                row["lease_token"],
-                row["lease_until"],
-                row["grant_fingerprint"],
-            )
-            != (
-                lease.build_id,
-                lease.event_seq,
-                lease.owner,
-                lease.token,
-                lease.expires_at,
-                lease.grant_fingerprint,
-            )
-            or lease.expires_at <= now
-        ):
-            raise ValueError("lease_changed_or_expired")
-        event = journal.event(db, lease.event_seq)
-        if (
-            event is None
-            or event.user_id != "local"
-            or event.evidence_id != lease.evidence_id
-            or not journal.matches_current(db, event)
-        ):
-            raise ValueError("lease_evidence_changed")
-        if self._grant(db, event.source_id) != lease.grant_fingerprint:
-            raise ValueError("lease_source_authority_changed")
-        return event
-
-    def read_lease(self, lease: IndexLease, *, now: float | None = None) -> EvidenceRecord:
-        with self._db() as db:
-            self._validate_lease(db, lease, time.time() if now is None else now)
-            record = EvidenceStore._get_with_connection(db, lease.evidence_id)
-            if record is None or len(record.verbatim) > 2_000_000:
-                raise ValueError("lease_input_unavailable_or_too_large")
-            return record
-
-    def renew(
-        self, lease: IndexLease, *, now: float | None = None, lease_seconds: int = 60
-    ) -> IndexLease:
-        now = time.time() if now is None else now
-        if type(lease_seconds) is not int or not 1 <= lease_seconds <= 300:
-            raise ValueError("invalid_lease")
-        with self._db(write=True) as db:
-            self._validate_lease(db, lease, now)
-            expires = max(lease.expires_at, now + lease_seconds)
-            db.execute(
-                "UPDATE retrieval_documents SET lease_until=? "
-                "WHERE generation_id=? AND evidence_id=?",
-                (expires, lease.generation_id, lease.evidence_id),
-            )
-            return replace(lease, expires_at=expires)
-
-    def block(self, lease: IndexLease) -> None:
-        """Release only this claim; preserve uncertain derived vectors for later cleanup."""
-        with self._db(write=True) as db:
-            db.execute(
-                """UPDATE retrieval_documents SET state='blocked',active_build_id=NULL,
-                lease_owner=NULL,lease_token=NULL,lease_until=NULL
-                WHERE generation_id=? AND evidence_id=? AND state='leased'
-                AND build_id=? AND lease_token=? AND lease_owner=?""",
-                (lease.generation_id, lease.evidence_id, lease.build_id, lease.token, lease.owner),
-            )
-
-    def retry_blocked(self, generation_id: str) -> int:
-        """Bounded, durable round-robin retry/regrant pass; never creates grants."""
-        changed = 0
-        with self._db(write=True) as db:
-            generation = db.execute(
-                "SELECT g.state,c.retry_after_seq FROM retrieval_index_generations g "
-                "JOIN retrieval_consumers c USING(generation_id) WHERE generation_id=?",
-                (generation_id,),
-            ).fetchone()
-            if generation is None or generation[0] == "retired":
-                raise ValueError("generation_not_writable")
-            rows = db.execute(
-                "SELECT * FROM retrieval_documents WHERE generation_id=? "
-                "AND state IN ('blocked','ready','excluded') AND event_seq>? "
-                "ORDER BY event_seq LIMIT 256",
-                (generation_id, generation[1]),
-            ).fetchall()
-            for row in rows:
-                event = journal.event(db, row["event_seq"])
-                grant = self._grant(db, row["source_id"])
-                if row["state"] in ("ready", "excluded") and grant == row["grant_fingerprint"]:
-                    continue
-                if (
-                    event
-                    and event.source_id == row["source_id"]
-                    and grant
-                    and journal.matches_current(db, event)
-                ):
-                    db.execute(
-                        "UPDATE retrieval_documents SET state='pending',active_build_id=NULL,"
-                        "lease_owner=NULL,lease_token=NULL,lease_until=NULL,grant_fingerprint=NULL "
-                        "WHERE generation_id=? AND evidence_id=?",
-                        (generation_id, row["evidence_id"]),
-                    )
-                    changed += 1
-            db.execute(
-                "UPDATE retrieval_consumers SET retry_after_seq=? WHERE generation_id=?",
-                (rows[-1]["event_seq"] if len(rows) == 256 else 0, generation_id),
-            )
-        return changed
-
-    def publish(
-        self,
-        lease: IndexLease,
-        chunks: Sequence[TextChunk],
-        *,
-        vectors_verified: bool,
-        now: float | None = None,
-        source_map: ContentMap | None = None,
-        checkpoint: Callable[[], None] | None = None,
-    ) -> tuple[str, ...]:
-        """Caller verifies actual immutable vectors before entering this short transaction."""
-        if vectors_verified is not True:
-            raise ValueError("vectors_not_verified")
-        now = time.time() if now is None else now
-        if not math.isfinite(now):
-            raise ValueError("invalid_lease_time")
-        if len(chunks) > 8192:
-            raise ValueError("invalid_chunk_manifest")
-        with self._db(write=True) as db:
-            if checkpoint is not None:
-                checkpoint()
-            event = self._validate_lease(db, lease, now)
-            record = EvidenceStore._get_with_connection(db, lease.evidence_id)
-            if record is None or len(record.verbatim) > 2_000_000:
-                raise ValueError("lease_evidence_missing")
-            prepared = self._prepare(db, lease.generation_id, record)
-            if source_map != (prepared.metadata if prepared is not None else None):
-                raise ValueError("source_map_mismatch")
-            validate_chunk_coverage(record, chunks, prepared)
-            map_digest = source_map.digest if source_map is not None else None
-            keys = []
-            manifest = []
-            for index, chunk in enumerate(chunks):
-                text_hash = digest(chunk.text)
-                key = chunk_identity(lease, chunk, map_digest)
-                keys.append(key)
-                manifest.append(
-                    (
-                        key,
-                        lease.generation_id,
-                        lease.evidence_id,
-                        lease.build_id,
-                        index,
-                        chunk.start_cp,
-                        chunk.end_cp,
-                        chunk.token_count,
-                        text_hash,
-                    )
-                )
-            existing = db.execute(
-                "SELECT * FROM retrieval_chunks WHERE build_id=? ORDER BY ordinal",
-                (lease.build_id,),
-            ).fetchall()
-            if existing:
-                if [tuple(row) for row in existing] != manifest:
-                    raise ValueError("immutable_chunk_manifest_conflict")
-            else:
-                db.executemany("INSERT INTO retrieval_chunks VALUES (?,?,?,?,?,?,?,?,?)", manifest)
-            state = "ready" if chunks else "excluded"
-            if source_map is not None:
-                values = (
-                    lease.build_id,
-                    source_map.canonical_json(),
-                    map_digest,
-                    "indexable" if chunks else "excluded",
-                )
-                prior = db.execute(
-                    "SELECT * FROM retrieval_build_maps WHERE build_id=?", (lease.build_id,)
-                ).fetchone()
-                if prior is not None and tuple(prior) != values:
-                    raise ValueError("immutable_source_map_conflict")
-                if prior is None:
-                    db.execute("INSERT INTO retrieval_build_maps VALUES (?,?,?,?)", values)
-            if not journal.matches_current(db, event):
-                raise ValueError("lease_evidence_changed")
-            db.execute(
-                "UPDATE retrieval_documents SET state=?,active_build_id=build_id,"
-                "lease_owner=NULL,lease_token=NULL,lease_until=NULL "
-                "WHERE generation_id=? AND evidence_id=?",
-                (state, lease.generation_id, lease.evidence_id),
-            )
-            if checkpoint is not None:
-                checkpoint()
-            return tuple(keys)
-
-    def activate_generation(self, generation_id: str) -> None:
-        with self._db(write=True) as db:
-            row = db.execute(
-                "SELECT state FROM retrieval_index_generations WHERE generation_id=?",
-                (generation_id,),
-            ).fetchone()
-            cursor = db.execute(
-                "SELECT after_seq FROM retrieval_consumers WHERE generation_id=?", (generation_id,)
-            ).fetchone()
-            if (
-                row is None
-                or row[0] == "retired"
-                or cursor is None
-                or cursor[0] != journal.watermark(db)
-            ):
-                raise ValueError("generation_not_caught_up")
-            for document in db.execute(
-                "SELECT * FROM retrieval_documents WHERE generation_id=? "
-                "AND state NOT IN ('ready','excluded','deleted')",
-                (generation_id,),
-            ):
-                if self._grant(db, document["source_id"]) is not None:
-                    raise ValueError("generation_incomplete")
-            for document in db.execute(
-                "SELECT * FROM retrieval_documents WHERE generation_id=? "
-                "AND state IN ('ready','excluded')",
-                (generation_id,),
-            ):
-                if self._grant(db, document["source_id"]) is not None:
-                    event = journal.event(db, document["event_seq"])
-                    if (
-                        not event
-                        or not journal.matches_current(db, event)
-                        or self._grant(db, document["source_id"]) != document["grant_fingerprint"]
-                    ):
-                        raise ValueError("generation_stale")
-                    record = EvidenceStore._get_with_connection(db, document["evidence_id"])
-                    if record is None:
-                        raise ValueError("generation_stale")
-                    self.validate_document(db, document, record)
-            db.execute(
-                "UPDATE retrieval_index_generations SET state='retired' "
-                "WHERE state='active' AND generation_id!=?",
-                (generation_id,),
-            )
-            db.execute(
-                "UPDATE retrieval_index_generations SET state='active' WHERE generation_id=?",
-                (generation_id,),
-            )
-
-    def active_chunks(
-        self,
-        generation_id: str,
-        *,
-        source_ids: frozenset[str] | None = None,
-        time_range: TimeRange | None = None,
-        deadline: float | None = None,
-        require_active: bool = False,
-    ) -> tuple[PublishedChunk, ...]:
-        """Return current chunks after SQL-level source/time eligibility filtering.
-
-        Grant fingerprints, journal freshness, and manifests are still checked
-        below in the same snapshot. The SQL predicate keeps unauthorized rows
-        from filling an in-memory candidate set before those checks run.
-        """
-        if source_ids is not None and not source_ids:
-            return ()
-        with self._db(deadline=deadline) as db:
-            generation = db.execute(
-                "SELECT state FROM retrieval_index_generations WHERE generation_id=?",
-                (generation_id,),
-            ).fetchone()
-            if generation is None or generation[0] != "active":
-                if require_active:
-                    raise ValueError("active_generation_changed")
-                return ()
-            clauses = ["d.generation_id=?", "d.state IN ('ready','excluded')"]
-            parameters: list[object] = [generation_id]
-            join = ""
-            if source_ids is not None:
-                clauses.append("d.source_id IN (SELECT value FROM json_each(?))")
-                parameters.append(json.dumps(sorted(source_ids)))
-            if time_range is not None:
-                join = "JOIN evidence e ON e.evidence_id=d.evidence_id "
-                clauses.extend(
-                    (
-                        "e.created_at IS NOT NULL",
-                        "julianday(e.created_at) >= julianday(?)",
-                        "julianday(e.created_at) < julianday(?)",
-                    )
-                )
-                parameters.extend((time_range.start.isoformat(), time_range.end.isoformat()))
-            output = []
-            for row in db.execute(
-                "SELECT d.* FROM retrieval_documents d "
-                + join
-                + f"WHERE {' AND '.join(clauses)} ORDER BY d.event_seq,d.evidence_id",
-                parameters,
-            ):
-                if deadline is not None:
-                    _check(deadline)
-                grant = self._grant(db, row["source_id"])
-                if grant is None or grant != row["grant_fingerprint"]:
-                    continue
-                event = journal.event(db, row["event_seq"])
-                if (
-                    event is None
-                    or event.source_id != row["source_id"]
-                    or not journal.matches_current(db, event)
-                ):
-                    continue
-                record = EvidenceStore._get_with_connection(db, row["evidence_id"])
-                if (
-                    record is None
-                    or record.user_id != "local"
-                    or record.source_id != row["source_id"]
-                ):
-                    continue
-                event_time = record.created_at
-                if event_time is not None:
-                    event_time = event_time.astimezone(UTC) if event_time.tzinfo else None
-                if time_range is not None and (
-                    event_time is None or not time_range.start <= event_time < time_range.end
-                ):
-                    continue
-                chunks = self.validate_document(db, row, record, deadline=deadline)
-                for chunk in chunks:
-                    if deadline is not None:
-                        _check(deadline)
-                    output.append(
-                        PublishedChunk(
-                            chunk["chunk_id"],
-                            row["build_id"],
-                            row["evidence_id"],
-                            event.evidence_revision,
-                            chunk["start_cp"],
-                            chunk["end_cp"],
-                            chunk["ordinal"],
-                            record.source_id,
-                            event_time,
-                        )
-                    )
-            return tuple(output)
-
-    def validate_document(
-        self,
-        db: sqlite3.Connection,
-        document: sqlite3.Row,
-        record: EvidenceRecord,
-        *,
-        deadline: float | None = None,
-    ) -> tuple[sqlite3.Row, ...]:
-        """Same-snapshot complete manifest/map check shared by candidate and final read."""
-        if deadline is not None:
-            _check(deadline)
-        if document["active_build_id"] != document["build_id"]:
-            raise ValueError("published_build_binding_invalid")
-        prepared = self._prepare(db, document["generation_id"], record)
-        map_digest = None
-        if prepared is not None:
-            source_map = db.execute(
-                "SELECT source_map_json,map_digest,coverage_state FROM retrieval_build_maps "
-                "WHERE build_id=?",
-                (document["build_id"],),
-            ).fetchone()
-            expected_state = "excluded" if prepared.state == "no_indexable_content" else "ready"
-            if (
-                source_map is None
-                or tuple(source_map)
-                != (
-                    prepared.metadata.canonical_json(),
-                    prepared.metadata.digest,
-                    "excluded" if expected_state == "excluded" else "indexable",
-                )
-                or document["state"] != expected_state
-            ):
-                raise ValueError("published_source_map_invalid")
-            map_digest = prepared.metadata.digest
-        elif document["state"] != "ready":
-            raise ValueError("published_manifest_invalid")
-        rows = tuple(
-            db.execute(
-                "SELECT * FROM retrieval_chunks WHERE generation_id=? AND evidence_id=? "
-                "AND build_id=? ORDER BY ordinal LIMIT 8193",
-                (document["generation_id"], record.evidence_id, document["build_id"]),
-            ).fetchall()
-        )
-        chunks = []
-        for row in rows:
-            if deadline is not None:
-                _check(deadline)
-            start, end = row["start_cp"], row["end_cp"]
-            if (
-                type(start) is not int
-                or type(end) is not int
-                or not 0 <= start < end <= len(record.verbatim)
-            ):
-                raise ValueError("published_manifest_invalid")
-            text = record.verbatim[start:end]
-            text_hash = digest(text)
-            identity = [document["build_id"], row["ordinal"], start, end, text_hash]
-            if map_digest is not None:
-                identity.append(map_digest)
-            if row["text_hash"] != text_hash or row["chunk_id"] != digest(identity):
-                raise ValueError("published_manifest_invalid")
-            chunks.append(TextChunk(row["ordinal"], start, end, row["token_count"], text))
-        try:
-            validate_chunk_coverage(record, chunks, prepared)
-        except ValueError as error:
-            raise ValueError("published_manifest_incomplete_or_invalid") from error
-        if deadline is not None:
-            _check(deadline)
-        return rows
-
-    def status(self, generation_id: str) -> dict[str, int]:
-        with self._db() as db:
-            row = db.execute(
-                "SELECT after_seq FROM retrieval_consumers WHERE generation_id=?", (generation_id,)
-            ).fetchone()
-            if row is None:
-                raise ValueError("generation_missing")
-            result = {
-                "after_seq": int(row[0]),
-                "pending": 0,
-                "leased": 0,
-                "ready": 0,
-                "blocked": 0,
-                "deleted": 0,
-                "excluded": 0,
-            }
-            for state, count in db.execute(
-                "SELECT state,count(*) FROM retrieval_documents "
-                "WHERE generation_id=? GROUP BY state",
-                (generation_id,),
-            ):
-                result[state] = count
-            return result

memhub/apps/core/src/one_memory/retrieval/index_supervisor.py

@@ -1,200 +0,0 @@
-"""Explicit ownership of one admitted index worker; no migration or resource close."""
-
-import asyncio
-import math
-import threading
-import time
-from collections.abc import Callable
-from dataclasses import dataclass
-from typing import Literal, Protocol
-
-from one_memory.retrieval.index_worker import IndexWorkResult
-from one_memory.service_levels import ServiceLevel, service_level_context
-
-
-class IndexWorker(Protocol):
-    def run_once(self, *, cancel: threading.Event | None = None) -> IndexWorkResult: ...
-
-
-@dataclass(frozen=True)
-class IndexSupervisorSnapshot:
-    state: Literal["new", "running", "stopping", "stopped"]
-    last_result: str | None
-    error_code: str | None
-    slices: int
-    failures: int
-    thread_alive: bool
-
-
-class RetrievalIndexSupervisor:
-    """Single-use supervisor. Caller owns admission and closes vectors only after stop.
-
-    Cancellation is cooperative: stop/wait cannot finish until a blocking native
-    run_once returns. The lifecycle owner must drain before shutting its event loop.
-    """
-
-    def __init__(
-        self,
-        worker: IndexWorker,
-        *,
-        idle_wait_seconds: float = 5,
-        failure_wait_seconds: float = 5,
-        yield_wait_seconds: float = 0.05,
-        max_consecutive_slices: int = 8,
-        background_should_yield: Callable[[], bool] | None = None,
-    ):
-        for interval in (idle_wait_seconds, failure_wait_seconds, yield_wait_seconds):
-            if (
-                isinstance(interval, bool)
-                or not isinstance(interval, (int, float))
-                or not math.isfinite(interval)
-                or not 0.001 <= interval <= 300
-            ):
-                raise ValueError("index_supervisor_invalid_config")
-        if type(max_consecutive_slices) is not int or not 1 <= max_consecutive_slices <= 256:
-            raise ValueError("index_supervisor_invalid_config")
-        self._worker = worker
-        self._idle_wait = idle_wait_seconds
-        self._failure_wait = failure_wait_seconds
-        self._yield_wait = yield_wait_seconds
-        self._max_slices = max_consecutive_slices
-        self._background_should_yield = background_should_yield or (lambda: False)
-        self._condition = threading.Condition()
-        self._cancel = threading.Event()
-        self._wake = False
-        self._thread: threading.Thread | None = None
-        self._state: Literal["new", "running", "stopping", "stopped"] = "new"
-        self._last_result: str | None = None
-        self._error_code: str | None = None
-        self._slices = self._failures = 0
-
-    def start(self) -> None:
-        with self._condition:
-            if self._state != "new":
-                raise RuntimeError("index_supervisor_already_started")
-            thread = threading.Thread(target=self._run, name="retrieval-index", daemon=False)
-            self._thread = thread
-            self._state = "running"
-            try:
-                thread.start()
-            except Exception:
-                self._thread = None
-                self._state = "stopped"
-                self._error_code = "index_supervisor_start_failed"
-                raise RuntimeError("index_supervisor_start_failed") from None
-
-    def trigger(self) -> bool:
-        with self._condition:
-            if self._state != "running":
-                return False
-            self._wake = True
-            self._condition.notify_all()
-            return True
-
-    def snapshot(self) -> IndexSupervisorSnapshot:
-        with self._condition:
-            return IndexSupervisorSnapshot(
-                self._state,
-                self._last_result,
-                self._error_code,
-                self._slices,
-                self._failures,
-                self._thread is not None and self._thread.is_alive(),
-            )
-
-    def _request_stop(self) -> None:
-        with self._condition:
-            self._cancel.set()
-            self._wake = False
-            if self._state == "new":
-                self._state = "stopped"
-            elif self._state == "running":
-                self._state = "stopping"
-            self._condition.notify_all()
-
-    async def stop(self) -> None:
-        self._request_stop()
-        await self.wait()
-
-    async def wait(self) -> None:
-        """Cancellation requests shutdown, drains the real thread, then propagates."""
-        cancelled = False
-        while True:
-            with self._condition:
-                thread = self._thread
-            if thread is None or not thread.is_alive():
-                if thread is not None:
-                    thread.join()
-                break
-            try:
-                # No executor waiter or to_thread future to orphan on cancellation.
-                await asyncio.sleep(0.01)
-            except asyncio.CancelledError:
-                cancelled = True
-                self._request_stop()
-        if cancelled:
-            raise asyncio.CancelledError
-
-    def _pause(self, seconds: float, *, wakeable: bool) -> None:
-        deadline = time.monotonic() + seconds
-        with self._condition:
-            while not self._cancel.is_set():
-                if wakeable and self._wake:
-                    break
-                remaining = deadline - time.monotonic()
-                if remaining <= 0:
-                    break
-                self._condition.wait(remaining)
-            self._wake = False
-
-    def _run(self) -> None:
-        consecutive = 0
-        try:
-            while not self._cancel.is_set():
-                if self._background_should_yield():
-                    self._pause(self._yield_wait, wakeable=False)
-                    continue
-                result: str | None = None
-                error: str | None = None
-                try:
-                    with service_level_context(ServiceLevel.BACKGROUND):
-                        value = self._worker.run_once(cancel=self._cancel)
-                    if not isinstance(value, IndexWorkResult) or value.state not in {
-                        "idle",
-                        "ready",
-                        "excluded",
-                        "blocked",
-                        "cancelled",
-                        "deferred",
-                    }:
-                        error = "index_worker_invalid_result"
-                    else:
-                        result = value.state
-                        if result == "blocked":
-                            error = "index_worker_blocked"
-                except BaseException:
-                    # Worker/provider exceptions must never escape via thread stderr.
-                    error = "index_worker_failed"
-                with self._condition:
-                    self._last_result, self._error_code = result, error
-                    self._slices += 1
-                    if error is not None:
-                        self._failures += 1
-                if self._cancel.is_set():
-                    break
-                if error is not None:
-                    consecutive = 0
-                    self._pause(self._failure_wait, wakeable=False)
-                elif result in {"ready", "excluded"}:
-                    consecutive += 1
-                    if consecutive >= self._max_slices:
-                        consecutive = 0
-                        self._pause(self._yield_wait, wakeable=False)
-                else:
-                    consecutive = 0
-                    self._pause(self._idle_wait, wakeable=True)
-        finally:
-            with self._condition:
-                self._state = "stopped"
-                self._wake = False
-                self._condition.notify_all()

memhub/apps/core/src/one_memory/retrieval/index_worker.py

@@ -1,262 +0,0 @@
-"""Bounded raw-index work. No migration, scheduler, HY or remote model invocation.
-
-Runtime must admit a single worker, own its thread/process, and drain that owner
-before closing the vector client. Cancellation here is cooperative between
-native calls, not a claim that ONNX/Chroma can be preempted in a Python thread.
-"""
-
-import math
-import threading
-import time
-from collections.abc import Callable, Sequence
-from dataclasses import dataclass
-from typing import Literal, Protocol
-
-from one_memory.embedding_assets import (
-    BUILTIN_EMBEDDING_MODEL,
-    EMBEDDING_ASSETS,
-    verify_assets,
-)
-from one_memory.local_embedding import LocalEmbedder, TokenizedText
-from one_memory.retrieval.chunking import ChunkingError, chunk_evidence
-from one_memory.retrieval.content import index_content
-from one_memory.retrieval.index_store import (
-    EncoderIdentity,
-    IndexLease,
-    RetrievalIndexStore,
-    chunk_identity,
-)
-from one_memory.retrieval.mapped_content import MappedContentError, chunk_mapped_content
-from one_memory.retrieval.vector_store import RawVectorRecord, RawVectorStore, VectorStoreError
-
-
-class RetrievalEncoder(Protocol):
-    @property
-    def identity(self) -> EncoderIdentity: ...
-
-    def tokenize_with_offsets(self, text: str) -> TokenizedText: ...
-
-    def encode_documents(self, texts: list[str]) -> list[list[float]]: ...
-
-
-class LocalRetrievalEncoder:
-    """Pinned local assets; mapped content v2 by default, explicit v1 for old generations."""
-
-    def __init__(self, embedder: LocalEmbedder | None = None, *, content_version: int = 2):
-        if type(content_version) is not int or content_version not in (1, 2):
-            raise ValueError("unsupported_content_generation")
-        self._embedder = embedder if embedder is not None else LocalEmbedder()
-        digests = {asset.filename: asset.sha256 for asset in EMBEDDING_ASSETS}
-        self.identity = EncoderIdentity(
-            BUILTIN_EMBEDDING_MODEL,
-            digests["model_int8.onnx"],
-            digests["tokenizer.json"],
-            self._embedder.dimensions,
-            chunker_version=content_version,
-            normalizer_version=content_version,
-        )
-        self._verified = False
-        self._lock = threading.Lock()
-
-    def _verify(self) -> None:
-        with self._lock:
-            if not self._verified:
-                verify_assets()
-                self._verified = True
-
-    def tokenize_with_offsets(self, text: str) -> TokenizedText:
-        self._verify()
-        return self._embedder.tokenize_with_offsets(text)
-
-    def encode_documents(self, texts: list[str]) -> list[list[float]]:
-        self._verify()
-        if len(texts) > 8 or any(
-            not 1 <= len(self.tokenize_with_offsets(text).token_ids) <= 512 for text in texts
-        ):
-            raise ValueError("encoder_batch_or_token_budget")
-        with self._lock:
-            return self._embedder.encode(texts)
-
-
-@dataclass(frozen=True)
-class IndexWorkResult:
-    state: Literal["idle", "ready", "excluded", "blocked", "cancelled", "deferred"]
-    reason: str | None = None
-    chunk_count: int = 0
-
-
-class _Interrupted(Exception):
-    def __init__(self, state: Literal["cancelled", "deferred"]):
-        self.state = state
-
-
-class RetrievalIndexWorker:
-    def __init__(
-        self,
-        store: RetrievalIndexStore,
-        vectors: RawVectorStore,
-        encoder: RetrievalEncoder,
-        *,
-        generation_id: str,
-        owner: str,
-        clock: Callable[[], float] = time.monotonic,
-        wall_clock: Callable[[], float] = time.time,
-        max_job_seconds: float = 120,
-    ):
-        if (
-            not owner
-            or len(owner) > 128
-            or isinstance(max_job_seconds, bool)
-            or not math.isfinite(max_job_seconds)
-            or not 0 < max_job_seconds <= 300
-        ):
-            raise ValueError("invalid_worker_budget")
-        if store.encoder_identity(generation_id) != encoder.identity:
-            raise ValueError("worker_encoder_identity_mismatch")
-        if (
-            vectors.generation_id != generation_id
-            or vectors.dimensions != encoder.identity.dimensions
-        ):
-            raise ValueError("worker_vector_identity_mismatch")
-        self.store, self.vectors, self.encoder = store, vectors, encoder
-        self.generation_id, self.owner = generation_id, owner
-        self.clock, self.wall_clock = clock, wall_clock
-        self.max_job_seconds = max_job_seconds
-        self._run_lock = threading.Lock()
-
-    def run_once(self, *, cancel: threading.Event | None = None) -> IndexWorkResult:
-        """Consume <=256 events and handle <=1 document; never activate a generation.
-
-        Interrupted jobs retain their finite lease for restart recovery. Failed
-        jobs are blocked for explicit retry; arbitrary provider exceptions and
-        original text never enter the public result or a log.
-        """
-        if not self._run_lock.acquire(blocking=False):
-            raise RuntimeError("index_worker_already_running")
-        lease: IndexLease | None = None
-        deadline = self.clock() + self.max_job_seconds
-
-        def check() -> None:
-            if cancel is not None and cancel.is_set():
-                raise _Interrupted("cancelled")
-            now = self.clock()
-            if not math.isfinite(now) or now >= deadline:
-                raise _Interrupted("deferred")
-
-        def boundary() -> None:
-            nonlocal lease
-            check()
-            assert lease is not None
-            # This revalidates the original's current fingerprint and authority,
-            # not just a heartbeat on an otherwise stale claim.
-            lease = self.store.renew(lease, now=self.wall_clock())
-            check()
-
-        def publish_checkpoint() -> None:
-            check()
-            assert lease is not None
-            now = self.wall_clock()
-            if not math.isfinite(now) or now >= lease.expires_at:
-                raise ValueError("lease_changed_or_expired")
-
-        try:
-            check()
-            self.store.consume_pending(self.generation_id)
-            check()
-            lease = self.store.claim(self.generation_id, self.owner, now=self.wall_clock())
-            if lease is None:
-                return IndexWorkResult("idle")
-            record = self.store.read_lease(lease, now=self.wall_clock())
-            check()
-            identity = self.encoder.identity
-            try:
-                prepared = index_content(
-                    record,
-                    normalizer_version=identity.normalizer_version,
-                    chunker_version=identity.chunker_version,
-                )
-            except ValueError:
-                self.store.block(lease)
-                return IndexWorkResult("blocked", "content_mapping_required")
-
-            worker_encoder = self.encoder
-
-            class CheckedTokenizer:
-                def tokenize_with_offsets(self, text: str) -> TokenizedText:
-                    check()
-                    result = worker_encoder.tokenize_with_offsets(text)
-                    check()
-                    return result
-
-            chunks = (
-                chunk_mapped_content(prepared, CheckedTokenizer())
-                if prepared is not None
-                else chunk_evidence(record.verbatim, CheckedTokenizer())
-            )
-            if (not chunks and prepared is None) or len(chunks) > 8192:
-                raise ChunkingError("invalid_chunk_count")
-            source_map = prepared.metadata if prepared is not None else None
-            map_digest = source_map.digest if source_map is not None else None
-            boundary()
-            expected: list[RawVectorRecord] = []
-            for start in range(0, len(chunks), 8):
-                batch = chunks[start : start + 8]
-                boundary()
-                encoded = self.encoder.encode_documents([chunk.text for chunk in batch])
-                boundary()
-                if len(encoded) != len(batch):
-                    raise ValueError("encoder_count_mismatch")
-                records = [
-                    RawVectorRecord(
-                        chunk_identity(lease, chunk, map_digest),
-                        lease.build_id,
-                        lease.generation_id,
-                        tuple(vector),
-                    )
-                    for chunk, vector in zip(batch, encoded, strict=True)
-                ]
-                self.vectors.upsert(records)
-                boundary()
-                expected.extend(records)
-            self._verify(expected, boundary)
-            assert lease is not None
-            check()
-            self.store.publish(
-                lease,
-                chunks,
-                vectors_verified=True,
-                now=self.wall_clock(),
-                source_map=source_map,
-                checkpoint=publish_checkpoint,
-            )
-            return IndexWorkResult("ready" if chunks else "excluded", chunk_count=len(chunks))
-        except _Interrupted as error:
-            return IndexWorkResult(
-                error.state, "cancelled" if error.state == "cancelled" else "deadline"
-            )
-        except Exception as error:
-            cause: BaseException | None = error
-            while cause is not None:
-                if isinstance(cause, _Interrupted):
-                    return IndexWorkResult(
-                        cause.state, "cancelled" if cause.state == "cancelled" else "deadline"
-                    )
-                cause = cause.__cause__
-            if lease is None:
-                raise
-            self.store.block(lease)
-            reason = (
-                "chunking_unavailable"
-                if isinstance(error, (ChunkingError, MappedContentError))
-                else "vector_unavailable"
-                if isinstance(error, VectorStoreError)
-                else "index_build_unavailable"
-            )
-            return IndexWorkResult("blocked", reason)
-        finally:
-            self._run_lock.release()
-
-    def _verify(self, records: Sequence[RawVectorRecord], boundary: Callable[[], None]) -> None:
-        for start in range(0, len(records), 8):
-            self.vectors.verify(records[start : start + 8])
-            boundary()

memhub/apps/core/src/one_memory/retrieval/mapped_content.py

@@ -1,349 +0,0 @@
-"""Pure raw-content admission and exact source maps; no grants, writes or HY calls."""
-
-import hashlib
-import json
-from dataclasses import asdict, dataclass, field
-from typing import Literal
-
-from one_memory.embedding_assets import BUILTIN_EMBEDDING_MAX_TOKENS
-from one_memory.imports.models import (
-    MAX_VERBATIM_CHARACTERS,
-    EvidenceContentKind,
-    EvidenceRecord,
-)
-from one_memory.imports.sources.parsing import (
-    is_host_generated_user_turn,
-    unwrap_host_envelope_with_positions,
-)
-from one_memory.input_sanitization import sanitize_model_input_with_spans
-from one_memory.local_embedding import TokenizedText
-from one_memory.memory_processing import _is_hy_input_turn
-from one_memory.retrieval.chunking import ChunkingError, OffsetTokenizer, TextChunk, chunk_evidence
-
-MAP_VERSION = "raw-content-map-v1"
-PARSER_VERSION = "host-envelope-positions-v1"
-SANITIZER_VERSION = "model-input-spans-v1"
-MAX_MAPPED_CHUNKS = 8192
-
-
-class MappedContentError(ValueError):
-    """Fixed reason only; never include original or sanitized content."""
-
-
-@dataclass(frozen=True)
-class MappedSegment:
-    kind: Literal["copy", "synthetic"]
-    start_cp: int
-    end_cp: int
-    source_start_cp: int | None
-    source_end_cp: int | None
-
-
-@dataclass(frozen=True)
-class ExcludedRange:
-    start_cp: int
-    end_cp: int
-    reason: Literal["envelope_or_trim", "sanitizer"]
-
-
-@dataclass(frozen=True)
-class ContentMap:
-    map_version: str
-    parser_version: str
-    sanitizer_version: str
-    source_length_cp: int
-    body_start_cp: int | None
-    body_end_cp: int | None
-    sanitized_length_cp: int
-    sanitized_sha256: str
-    segments: tuple[MappedSegment, ...]
-    excluded: tuple[ExcludedRange, ...]
-
-    def canonical_json(self) -> str:
-        return json.dumps(asdict(self), sort_keys=True, separators=(",", ":"), allow_nan=False)
-
-    @property
-    def digest(self) -> str:
-        return hashlib.sha256(self.canonical_json().encode()).hexdigest()
-
-
-@dataclass(frozen=True)
-class PreparedMappedContent:
-    actual_text: str = field(repr=False)
-    metadata: ContentMap
-    state: Literal["ready", "no_indexable_content"]
-
-
-def _validate(content: PreparedMappedContent, original: str | None = None) -> None:
-    meta, text = content.metadata, content.actual_text
-    invalid = "mapped_invalid_source_map"
-    if (
-        (meta.map_version, meta.parser_version, meta.sanitizer_version)
-        != (MAP_VERSION, PARSER_VERSION, SANITIZER_VERSION)
-        or type(meta.source_length_cp) is not int
-        or not 0 <= meta.source_length_cp <= MAX_VERBATIM_CHARACTERS
-        or type(meta.sanitized_length_cp) is not int
-        or meta.sanitized_length_cp != len(text)
-        or meta.sanitized_sha256 != hashlib.sha256(text.encode()).hexdigest()
-        or type(meta.segments) is not tuple
-        or type(meta.excluded) is not tuple
-    ):
-        raise MappedContentError(invalid)
-    body_start, body_end = meta.body_start_cp, meta.body_end_cp
-    if body_start is None or body_end is None:
-        if body_start is not None or body_end is not None or text or meta.segments:
-            raise MappedContentError(invalid)
-    elif (
-        type(body_start) is not int
-        or type(body_end) is not int
-        or not 0 <= body_start < body_end <= meta.source_length_cp
-    ):
-        raise MappedContentError(invalid)
-    cursor = source_cursor = 0
-    has_content = False
-    partition: list[tuple[int, int]] = []
-    for span in meta.segments:
-        if (
-            not isinstance(span, MappedSegment)
-            or type(span.start_cp) is not int
-            or type(span.end_cp) is not int
-            or not cursor == span.start_cp < span.end_cp <= len(text)
-        ):
-            raise MappedContentError(invalid)
-        cursor = span.end_cp
-        if span.kind == "synthetic":
-            if span.source_start_cp is not None or span.source_end_cp is not None:
-                raise MappedContentError(invalid)
-            continue
-        left, right = span.source_start_cp, span.source_end_cp
-        if (
-            span.kind != "copy"
-            or type(left) is not int
-            or type(right) is not int
-            or body_start is None
-            or body_end is None
-            or not body_start <= left < right <= body_end
-            or left < source_cursor
-            or right - left != span.end_cp - span.start_cp
-        ):
-            raise MappedContentError(invalid)
-        if original is not None and text[span.start_cp : span.end_cp] != original[left:right]:
-            raise MappedContentError(invalid)
-        has_content |= bool(text[span.start_cp : span.end_cp].strip())
-        source_cursor = right
-        partition.append((left, right))
-    if cursor != len(text):
-        raise MappedContentError(invalid)
-    excluded_end = 0
-    for excluded in meta.excluded:
-        if (
-            not isinstance(excluded, ExcludedRange)
-            or type(excluded.start_cp) is not int
-            or type(excluded.end_cp) is not int
-            or not excluded_end <= excluded.start_cp < excluded.end_cp <= meta.source_length_cp
-            or excluded.reason not in {"envelope_or_trim", "sanitizer"}
-        ):
-            raise MappedContentError(invalid)
-        excluded_end = excluded.end_cp
-        partition.append((excluded.start_cp, excluded.end_cp))
-    source_cursor = 0
-    for left, right in sorted(partition):
-        if left != source_cursor:
-            raise MappedContentError(invalid)
-        source_cursor = right
-    if source_cursor != meta.source_length_cp or content.state != (
-        "ready" if has_content else "no_indexable_content"
-    ):
-        raise MappedContentError(invalid)
-
-
-def prepare_mapped_content(record: EvidenceRecord) -> PreparedMappedContent:
-    """Map a qualified body to Evidence.verbatim code points, without modifying it.
-
-    The returned text is transient; only metadata is suitable for an outer manifest.
-    That manifest must independently bind Evidence identity/revision/hash and grants.
-    """
-    if record.content_kind is not EvidenceContentKind.RAW_CONVERSATION or record.role not in {
-        "user",
-        "assistant",
-    }:
-        raise MappedContentError("mapped_content_ineligible")
-    original = record.verbatim
-    if len(original) > MAX_VERBATIM_CHARACTERS:
-        raise MappedContentError("mapped_source_limit_exceeded")
-    envelope = unwrap_host_envelope_with_positions(original)
-    if envelope.status == "ambiguous":
-        raise MappedContentError("mapped_ambiguous_envelope")
-    body, start, end = envelope.text, envelope.start_cp, envelope.end_cp
-    segments: list[MappedSegment] = []
-    excluded: list[ExcludedRange] = []
-    text = ""
-    if body is None:
-        if start is not None or end is not None:
-            raise MappedContentError("mapped_invalid_envelope")
-        if original:
-            excluded.append(ExcludedRange(0, len(original), "envelope_or_trim"))
-    else:
-        if (
-            type(start) is not int
-            or type(end) is not int
-            or not 0 <= start < end <= len(original)
-            or original[start:end] != body
-        ):
-            raise MappedContentError("mapped_invalid_envelope")
-        if is_host_generated_user_turn(original) or is_host_generated_user_turn(body):
-            raise MappedContentError("mapped_generated_content")
-        if not _is_hy_input_turn(record.model_copy(update={"verbatim": body})):
-            raise MappedContentError("mapped_body_hygiene_rejected")
-        sanitized = sanitize_model_input_with_spans(body)
-        text = sanitized.text
-        cursor = 0
-        for span in sanitized.spans:
-            if span.start_cp > cursor:
-                segments.append(MappedSegment("synthetic", cursor, span.start_cp, None, None))
-            copied = MappedSegment(
-                "copy",
-                span.start_cp,
-                span.end_cp,
-                start + span.source_start_cp,
-                start + span.source_end_cp,
-            )
-            # Only source AND output adjacency can make one citation-safe interval.
-            if (
-                segments
-                and segments[-1].kind == "copy"
-                and (
-                    segments[-1].end_cp == copied.start_cp
-                    and segments[-1].source_end_cp == copied.source_start_cp
-                )
-            ):
-                previous = segments.pop()
-                copied = MappedSegment(
-                    "copy",
-                    previous.start_cp,
-                    copied.end_cp,
-                    previous.source_start_cp,
-                    copied.source_end_cp,
-                )
-            segments.append(copied)
-            cursor = span.end_cp
-        if cursor < len(text):
-            segments.append(MappedSegment("synthetic", cursor, len(text), None, None))
-        if start:
-            excluded.append(ExcludedRange(0, start, "envelope_or_trim"))
-        excluded.extend(
-            ExcludedRange(start + left, start + right, "sanitizer")
-            for left, right in sanitized.excluded
-        )
-        if end < len(original):
-            excluded.append(ExcludedRange(end, len(original), "envelope_or_trim"))
-    metadata = ContentMap(
-        MAP_VERSION,
-        PARSER_VERSION,
-        SANITIZER_VERSION,
-        len(original),
-        start,
-        end,
-        len(text),
-        hashlib.sha256(text.encode()).hexdigest(),
-        tuple(segments),
-        tuple(excluded),
-    )
-    prepared = PreparedMappedContent(
-        text,
-        metadata,
-        "ready"
-        if any(
-            span.kind == "copy" and text[span.start_cp : span.end_cp].strip() for span in segments
-        )
-        else "no_indexable_content",
-    )
-    _validate(prepared, original)
-    return prepared
-
-
-class _BoundedTokenizer:
-    """Count successful chunk re-encodings without copying the chunking algorithm."""
-
-    def __init__(self, tokenizer: OffsetTokenizer, remaining: int, max_tokens: int):
-        self.tokenizer = tokenizer
-        self.remaining = remaining
-        self.max_tokens = max_tokens
-        self.initial = True
-
-    def tokenize_with_offsets(self, text: str) -> TokenizedText:
-        if not self.initial and self.remaining == 0:
-            raise MappedContentError("mapped_chunk_limit_exceeded")
-        result = self.tokenizer.tokenize_with_offsets(text)
-        # chunk_evidence first encodes the whole copy span, then prefix+candidate;
-        # an in-budget candidate emits exactly one chunk. Shrink retries do not.
-        if (
-            not self.initial
-            and isinstance(result, TokenizedText)
-            and len(result.token_ids) <= self.max_tokens
-        ):
-            self.remaining -= 1
-        self.initial = False
-        return result
-
-
-def chunk_mapped_content(
-    content: PreparedMappedContent,
-    tokenizer: OffsetTokenizer,
-    *,
-    prefix: str = "",
-    target_tokens: int = 384,
-    overlap_tokens: int = 64,
-    max_tokens: int = BUILTIN_EMBEDDING_MAX_TOKENS,
-    max_chunks: int = MAX_MAPPED_CHUNKS,
-) -> tuple[TextChunk, ...]:
-    """Chunk each continuous copy independently; never encode synthetic placeholders."""
-    if type(max_chunks) is not int or not 1 <= max_chunks <= MAX_MAPPED_CHUNKS:
-        raise MappedContentError("mapped_invalid_chunk_limit")
-    _validate(content)
-    # Reuse budget validation even for an explicitly nonindexable record.
-    chunk_evidence(
-        "",
-        tokenizer,
-        prefix=prefix,
-        target_tokens=target_tokens,
-        overlap_tokens=overlap_tokens,
-        max_tokens=max_tokens,
-    )
-    chunks: list[TextChunk] = []
-    for span in content.metadata.segments:
-        if span.kind != "copy":
-            continue
-        text = content.actual_text[span.start_cp : span.end_cp]
-        if not text.strip():
-            continue
-        if len(chunks) == max_chunks:
-            raise MappedContentError("mapped_chunk_limit_exceeded")
-        try:
-            pieces = chunk_evidence(
-                text,
-                _BoundedTokenizer(tokenizer, max_chunks - len(chunks), max_tokens),
-                prefix=prefix,
-                target_tokens=target_tokens,
-                overlap_tokens=overlap_tokens,
-                max_tokens=max_tokens,
-            )
-        except ChunkingError as error:
-            if isinstance(error.__cause__, MappedContentError):
-                raise error.__cause__ from None
-            raise
-        if len(chunks) + len(pieces) > max_chunks:
-            raise MappedContentError("mapped_chunk_limit_exceeded")
-        assert span.source_start_cp is not None  # Validated copy span.
-        ordinal = len(chunks)
-        chunks.extend(
-            TextChunk(
-                ordinal + i,
-                span.source_start_cp + piece.start_cp,
-                span.source_start_cp + piece.end_cp,
-                piece.token_count,
-                piece.text,
-            )
-            for i, piece in enumerate(pieces)
-        )
-    return tuple(chunks)

memhub/apps/core/src/one_memory/retrieval/raw_candidates.py

@@ -1,147 +0,0 @@
-"""Internal raw semantic candidate lookup. No route, raw body release or full coverage claim."""
-
-import math
-import sqlite3
-from collections.abc import Sequence
-from dataclasses import dataclass
-from typing import Literal
-
-from one_memory.context_raw import _check
-from one_memory.retrieval.index_store import PublishedChunk, RetrievalIndexStore
-from one_memory.retrieval.models import RetrievalPath, TimeRange
-from one_memory.retrieval.vector_store import RawVectorStore, VectorStoreError
-
-
-@dataclass(frozen=True)
-class RawSemanticCandidate:
-    chunk: PublishedChunk
-    distance: float
-
-
-@dataclass(frozen=True)
-class RawCandidateBatch:
-    hits: tuple[RawSemanticCandidate, ...] = ()
-    complete: bool = False
-    reason: str | None = None
-    coverage: Literal["indexed_only"] = "indexed_only"
-    time_range: TimeRange | None = None
-
-
-def read_raw_semantic_candidates(
-    store: RetrievalIndexStore,
-    vectors: RawVectorStore,
-    query_vector: Sequence[float],
-    *,
-    generation_id: str,
-    source_ids: frozenset[str],
-    deadline: float,
-    limit: int = 64,
-    time_range: TimeRange | None = None,
-) -> RawCandidateBatch:
-    """Caller resolves host authority; explicit source IDs can only narrow it.
-
-    complete means the eligible published chunks were checked and searched, not
-    that every authorized original is indexed. The future coordinator must still
-    revalidate host identity and all raw/memory data together before body release.
-    """
-    if not isinstance(source_ids, frozenset) or any(
-        not isinstance(source, str) or not 1 <= len(source) <= 128 for source in source_ids
-    ):
-        raise ValueError("explicit_resolved_sources_required")
-    if type(limit) is not int or not 1 <= limit <= 64:
-        raise ValueError("invalid_candidate_limit")
-    if isinstance(deadline, bool) or not math.isfinite(deadline):
-        raise ValueError("invalid_candidate_deadline")
-    if not source_ids:
-        return RawCandidateBatch(reason="not_authorized")
-    if vectors.generation_id != generation_id:
-        raise ValueError("candidate_generation_mismatch")
-    try:
-        _check(deadline)
-        eligible = store.active_chunks(
-            generation_id, source_ids=source_ids, time_range=time_range, deadline=deadline
-        )
-        if not eligible:
-            return RawCandidateBatch(
-                complete=True, reason="no_eligible_chunks", time_range=time_range
-            )
-        before = {chunk.chunk_id: chunk for chunk in eligible}
-        expected_vectors = [(chunk.chunk_id, chunk.build_id) for chunk in eligible]
-        # When one query can return the entire allowlist, that response itself
-        # is the integrity check. Avoid a duplicate Chroma get-before-query on
-        # the hot small/medium-authority path. Larger allowlists still need the
-        # explicit complete verification because search returns only top-K.
-        verify_before_search = len(eligible) > limit
-        if verify_before_search and not vectors.verify_candidates(
-            expected_vectors, deadline=deadline
-        ):
-            return RawCandidateBatch(reason="deadline")
-        result = vectors.search(query_vector, tuple(before), limit, deadline)
-        if not verify_before_search and {hit.chunk_id for hit in result.hits} != set(before):
-            raise VectorStoreError("vector_record_missing")
-        _check(deadline)
-        current = {
-            chunk.chunk_id: chunk
-            for chunk in store.active_chunks(
-                generation_id, source_ids=source_ids, time_range=time_range, deadline=deadline
-            )
-        }
-        hits = []
-        for hit in result.hits:
-            if (
-                hit.chunk_id not in before
-                or hit.generation_id != generation_id
-                or hit.build_id != before[hit.chunk_id].build_id
-            ):
-                raise VectorStoreError("candidate_identity_mismatch")
-            if current.get(hit.chunk_id) == before[hit.chunk_id]:
-                hits.append(RawSemanticCandidate(before[hit.chunk_id], hit.distance))
-        _check(deadline)
-        unchanged = before == current
-        return RawCandidateBatch(
-            tuple(hits),
-            result.complete and unchanged,
-            "evidence_changed" if not unchanged else result.reason,
-            time_range=time_range,
-        )
-    except TimeoutError:
-        return RawCandidateBatch(reason="deadline")
-    except VectorStoreError:
-        return RawCandidateBatch(reason="vector_unavailable")
-    except (sqlite3.Error, OSError, ValueError):
-        return RawCandidateBatch(reason="index_unavailable")
-
-
-def compose_raw_retrieval_path(
-    batch: RawCandidateBatch,
-    *,
-    unpublished_authorized: bool,
-    lexical_hit_count: int,
-) -> RetrievalPath:
-    """Map channel results to a path. Indexed-only search is never complete empty."""
-    if type(lexical_hit_count) is not int or lexical_hit_count < 0:
-        raise ValueError("invalid_lexical_hit_count")
-    hits = len(batch.hits) + lexical_hit_count
-    incomplete = unpublished_authorized or not batch.complete
-    channels: tuple[Literal["lexical", "semantic"], ...] = ("lexical", "semantic")
-    if hits:
-        return RetrievalPath(
-            status="ok",
-            returned_count=min(hits, 32),
-            searched_channels=channels,
-            coverage="partial" if incomplete else "complete",
-        )
-    if incomplete:
-        return RetrievalPath(
-            status="degraded",
-            reason="indexed_only",
-            returned_count=0,
-            searched_channels=channels,
-            coverage="partial",
-        )
-    return RetrievalPath(
-        status="empty",
-        returned_count=0,
-        searched_channels=channels,
-        coverage="complete",
-    )

memhub/apps/core/src/one_memory/retrieval/raw_evidence.py

@@ -1,189 +0,0 @@
-"""Materialize internal semantic candidates from current authorized Evidence.
-
-Not an arbitrary-ID endpoint or signed expansion capability. The combined v2
-coordinator must revalidate this lane with memory and host identity at release.
-"""
-
-import json
-import sqlite3
-from dataclasses import dataclass
-from datetime import UTC
-
-from one_memory import ingest_journal as journal
-from one_memory.context_raw import _check, _connection
-from one_memory.imports.models import EvidenceRecord
-from one_memory.imports.storage import EvidenceStore
-from one_memory.retrieval.index_store import RetrievalIndexStore
-from one_memory.retrieval.models import EvidenceSpan, RawEvidence
-from one_memory.retrieval.raw_candidates import RawCandidateBatch
-
-
-@dataclass(frozen=True)
-class RawEvidenceBatch:
-    items: tuple[RawEvidence, ...] = ()
-    complete: bool = False
-    reason: str | None = None
-    used_characters: int = 0
-
-
-def _merge(intervals: list[tuple[int, int]]) -> list[tuple[int, int]]:
-    output: list[tuple[int, int]] = []
-    for start, end in sorted(intervals):
-        if output and start <= output[-1][1]:
-            output[-1] = (output[-1][0], max(end, output[-1][1]))
-        else:
-            output.append((start, end))
-    return output
-
-
-def materialize_raw_candidates(
-    store: RetrievalIndexStore,
-    candidates: RawCandidateBatch,
-    *,
-    generation_id: str,
-    source_ids: frozenset[str],
-    deadline: float,
-    character_budget: int = 6000,
-) -> RawEvidenceBatch:
-    if not isinstance(source_ids, frozenset) or any(
-        not isinstance(source, str) or not 1 <= len(source) <= 128 for source in source_ids
-    ):
-        raise ValueError("explicit_resolved_sources_required")
-    if type(character_budget) is not int or not 1 <= character_budget <= 20000:
-        raise ValueError("invalid_evidence_budget")
-    if len(candidates.hits) > 64:
-        raise ValueError("candidate_budget_exceeded")
-    if not source_ids:
-        return RawEvidenceBatch(reason="not_authorized")
-    try:
-        _check(deadline)
-        expected = {hit.chunk.chunk_id: hit.chunk for hit in candidates.hits}
-        if len(expected) != len(candidates.hits):
-            raise ValueError("duplicate_candidates")
-        if not expected:
-            # Empty/excluded builds still carry an integrity claim. Validate
-            # current authorized completed manifests, never only nonempty hits.
-            current_chunks = store.active_chunks(
-                generation_id,
-                source_ids=source_ids,
-                time_range=candidates.time_range,
-                deadline=deadline,
-                require_active=True,
-            )
-            _check(deadline)
-            if current_chunks and candidates.reason == "no_eligible_chunks":
-                return RawEvidenceBatch(reason="evidence_changed")
-            return RawEvidenceBatch(complete=candidates.complete, reason=candidates.reason)
-        changed = False
-        grouped: dict[str, tuple[EvidenceRecord, list[tuple[int, int]]]] = {}
-        with _connection(store.database_path, deadline) as db:
-            db.execute("PRAGMA query_only=ON")
-            rows = {
-                row["chunk_id"]: row
-                for row in db.execute(
-                    """SELECT c.*, d.event_seq,d.grant_fingerprint,d.source_id
-                    FROM retrieval_chunks c JOIN retrieval_documents d
-                      ON d.generation_id=c.generation_id AND d.evidence_id=c.evidence_id
-                      AND d.active_build_id=c.build_id AND d.build_id=c.build_id
-                    JOIN retrieval_index_generations g ON g.generation_id=c.generation_id
-                    WHERE c.generation_id=? AND g.state='active' AND d.state='ready'
-                      AND c.chunk_id IN (SELECT value FROM json_each(?))""",
-                    (generation_id, json.dumps(tuple(expected))),
-                )
-            }
-            for key, chunk in expected.items():
-                _check(deadline)
-                row = rows.get(key)
-                if row is None or row["source_id"] not in source_ids:
-                    changed = True
-                    continue
-                event = journal.event(db, row["event_seq"])
-                grant = store._grant(db, row["source_id"])
-                if (
-                    event is None
-                    or event.user_id != "local"
-                    or event.source_id != row["source_id"]
-                    or event.evidence_id != chunk.evidence_id
-                    or event.evidence_revision != chunk.revision
-                    or grant is None
-                    or grant != row["grant_fingerprint"]
-                    or not journal.matches_current(db, event)
-                ):
-                    changed = True
-                    continue
-                record = EvidenceStore._get_with_connection(db, chunk.evidence_id)
-                if (
-                    record is None
-                    or record.user_id != "local"
-                    or record.source_id != row["source_id"]
-                ):
-                    changed = True
-                    continue
-                event_time = record.created_at
-                if event_time is not None:
-                    event_time = event_time.astimezone(UTC) if event_time.tzinfo else None
-                if (
-                    (row["build_id"], row["ordinal"], row["start_cp"], row["end_cp"])
-                    != (chunk.build_id, chunk.ordinal, chunk.start_cp, chunk.end_cp)
-                    or (record.source_id, event_time) != (chunk.source_id, chunk.event_time)
-                    or not 0 <= chunk.start_cp < chunk.end_cp <= len(record.verbatim)
-                ):
-                    changed = True
-                    continue
-                if record.evidence_id not in grouped:
-                    document = db.execute(
-                        "SELECT * FROM retrieval_documents WHERE generation_id=? AND evidence_id=?",
-                        (generation_id, record.evidence_id),
-                    ).fetchone()
-                    if document is None:
-                        raise ValueError("published_document_missing")
-                    store.validate_document(db, document, record, deadline=deadline)
-                    grouped[record.evidence_id] = (record, [])
-                grouped[record.evidence_id][1].append((chunk.start_cp, chunk.end_cp))
-            items = []
-            used = 0
-            truncated = False
-            for record, intervals in grouped.values():
-                _check(deadline)
-                spans: list[EvidenceSpan] = []
-                for start, end in _merge(intervals):
-                    if len(spans) >= 4 or end - start > min(20000, character_budget - used):
-                        truncated = True
-                        continue
-                    spans.append(
-                        EvidenceSpan(start_cp=start, end_cp=end, text=record.verbatim[start:end])
-                    )
-                    used += end - start
-                if not spans:
-                    continue
-                event_time = record.created_at
-                if event_time is not None:
-                    event_time = event_time.astimezone(UTC) if event_time.tzinfo else None
-                items.append(
-                    RawEvidence(
-                        evidence_id=record.evidence_id,
-                        revision=record.revision,
-                        source_id=record.source_id,
-                        conversation_id=record.conversation_id,
-                        role=record.role,
-                        written_by="person" if record.role == "user" else "agent",
-                        event_time=event_time,
-                        spans=tuple(spans),
-                        matched_by=("semantic",),
-                    )
-                )
-            _check(deadline)
-        return RawEvidenceBatch(
-            tuple(items),
-            candidates.complete and not changed and not truncated,
-            "evidence_changed"
-            if changed
-            else "character_budget"
-            if truncated
-            else candidates.reason,
-            used,
-        )
-    except TimeoutError:
-        return RawEvidenceBatch(reason="deadline")
-    except (sqlite3.Error, OSError, ValueError):
-        return RawEvidenceBatch(reason="index_unavailable")

memhub/apps/core/src/one_memory/retrieval/raw_lane.py

@@ -1,181 +0,0 @@
-"""Fuse lexical raw context with an optional derived index. No new HTTP schema."""
-
-import sqlite3
-from collections.abc import Callable
-from pathlib import Path
-from typing import Protocol
-
-from pydantic import ValidationError
-
-from one_memory.context_models import (
-    ContextLaneResult,
-    ContextSearchRequest,
-    ContextSemanticStatus,
-    RawContextItem,
-)
-from one_memory.context_raw import (
-    _METADATA,
-    _check,
-    _connection,
-    _execute,
-    _message,
-    _readable,
-    read_raw_context,
-)
-from one_memory.retrieval.models import TimeRange
-from one_memory.retrieval.raw_candidates import RawCandidateBatch, compose_raw_retrieval_path
-
-
-class RawIndexLookup(Protocol):
-    def lookup_raw(
-        self,
-        query: str,
-        *,
-        source_ids: frozenset[str] | None,
-        deadline: float,
-        time_range: TimeRange | None = None,
-    ) -> tuple[RawCandidateBatch, bool]: ...
-
-
-def fuse_raw_lane(
-    lexical: ContextLaneResult,
-    batch: RawCandidateBatch,
-    *,
-    unpublished_authorized: bool,
-    extra_items: tuple[RawContextItem, ...] = (),
-) -> ContextLaneResult:
-    """v1 has no coverage field; degraded + lexical_only stands in for partial index."""
-    semantic_status: ContextSemanticStatus = (
-        "timed_out"
-        if batch.reason == "deadline"
-        else "failed"
-        if batch.reason in {"index_unavailable", "vector_unavailable"}
-        else "completed"
-    )
-    index_complete = batch.complete and not unpublished_authorized
-    timed_out = semantic_status == "timed_out"
-    if lexical.status not in {"ok", "empty", "degraded"}:
-        return lexical.model_copy(
-            update={
-                "searched_channels": ("lexical", "semantic"),
-                "semantic_status": semantic_status,
-                "index_complete": index_complete,
-                "timed_out": timed_out,
-            }
-        )
-    seen = {item.anchor_evidence_id for item in lexical.items if isinstance(item, RawContextItem)}
-    extras = tuple(item for item in extra_items if isinstance(item, RawContextItem))
-    # Semantic similarity is a candidate signal, not a verified recall result.
-    # Release only semantic items that the lexical lane independently confirmed
-    # for the same evidence identity. Unconfirmed semantic hits stay in the
-    # coverage receipt as partial/degraded and must not reach host injection.
-    unconfirmed_semantic = any(item.anchor_evidence_id not in seen for item in extras)
-    items = lexical.items
-    path = compose_raw_retrieval_path(
-        batch,
-        unpublished_authorized=unpublished_authorized,
-        lexical_hit_count=len(lexical.items),
-    )
-    status = path.status
-    if unconfirmed_semantic:
-        status = "degraded"
-    if lexical.status == "degraded" and status == "ok":
-        status = "degraded"
-    return ContextLaneResult(
-        status=status,
-        reason=(
-            "lexical_only" if unconfirmed_semantic and lexical.reason is None else lexical.reason
-        ),
-        items=items[:21],
-        truncated_by=lexical.truncated_by,
-        searched_channels=("lexical", "semantic"),
-        semantic_status=semantic_status,
-        index_complete=index_complete,
-        timed_out=timed_out,
-    )
-
-
-def load_raw_items_from_candidates(
-    database_path: Path,
-    batch: RawCandidateBatch,
-    *,
-    source_ids: frozenset[str] | None,
-    deadline: float,
-    character_budget: int = 6000,
-) -> tuple[RawContextItem, ...]:
-    """Release authorized Evidence bodies for semantic IDs. Not a vector dump."""
-    ids = tuple(dict.fromkeys(hit.chunk.evidence_id for hit in batch.hits))
-    if not ids or source_ids == frozenset():
-        return ()
-    readable, authority = _readable(source_ids)
-    body_limit = min(20000, max(1, character_budget))
-    items: list[RawContextItem] = []
-    try:
-        with _connection(database_path, deadline) as db:
-            for evidence_id in ids[:21]:
-                _check(deadline)
-                row = _execute(
-                    db,
-                    deadline,
-                    f"""
-                    SELECT {_METADATA}, substr(e.verbatim,1,?) AS text,
-                           length(e.verbatim)>? AS body_truncated
-                    FROM evidence e
-                    WHERE {readable} AND e.evidence_id=?
-                    """,
-                    [body_limit, body_limit, *authority, evidence_id],
-                ).fetchone()
-                if row is None:
-                    continue
-                try:
-                    message = _message(row)
-                    items.append(
-                        RawContextItem(
-                            source_id=row["source_id"],
-                            conversation_id=row["conversation_id"],
-                            anchor_evidence_id=message.evidence_id,
-                            messages=(message,),
-                            truncated=message.truncated,
-                        )
-                    )
-                except (ValidationError, ValueError, TypeError):
-                    continue
-    except TimeoutError:
-        raise
-    except (sqlite3.Error, OSError):
-        return ()
-    return tuple(items)
-
-
-def read_raw_indexed_context(
-    database_path: Path,
-    request: ContextSearchRequest,
-    *,
-    source_ids: frozenset[str] | None,
-    deadline: float,
-    index: RawIndexLookup | None = None,
-    load_semantic_items: Callable[..., tuple[RawContextItem, ...]] | None = None,
-) -> ContextLaneResult:
-    lexical = read_raw_context(database_path, request, source_ids=source_ids, deadline=deadline)
-    if index is None:
-        return lexical
-    try:
-        batch, unpublished = index.lookup_raw(
-            request.query, source_ids=source_ids, deadline=deadline
-        )
-    except Exception:
-        batch, unpublished = RawCandidateBatch(reason="index_unavailable"), True
-    extra: tuple[RawContextItem, ...] = ()
-    if batch.hits:
-        extra = (
-            load_semantic_items(database_path, batch, source_ids=source_ids, deadline=deadline)
-            if load_semantic_items is not None
-            else load_raw_items_from_candidates(
-                database_path,
-                batch,
-                source_ids=source_ids,
-                deadline=deadline,
-                character_budget=request.character_budget,
-            )
-        )
-    return fuse_raw_lane(lexical, batch, unpublished_authorized=unpublished, extra_items=extra)

memhub/apps/core/src/one_memory/retrieval/runtime.py

@@ -1,379 +0,0 @@
-"""Resume an explicitly configured local index; never create or migrate one."""
-
-import asyncio
-import json
-import os
-import shutil
-import sqlite3
-import stat
-import sys
-import threading
-from collections.abc import Callable
-from dataclasses import asdict, dataclass
-from pathlib import Path
-from typing import Literal
-
-from one_memory.database_access import main_database
-from one_memory.retrieval.index_store import EncoderIdentity, RetrievalIndexStore
-from one_memory.retrieval.index_supervisor import IndexSupervisorSnapshot, RetrievalIndexSupervisor
-from one_memory.retrieval.index_worker import LocalRetrievalEncoder, RetrievalIndexWorker
-from one_memory.retrieval.models import TimeRange
-from one_memory.retrieval.raw_candidates import RawCandidateBatch, read_raw_semantic_candidates
-from one_memory.retrieval.vector_store import RawVectorStore, VectorStoreError
-from one_memory.service_levels import ServiceLevel, ServiceLevelScheduler, ServiceResource
-
-type RuntimeState = Literal[
-    "new", "starting", "running", "not_initialized", "failed", "stopping", "stopped"
-]
-
-
-@dataclass(frozen=True)
-class ExistingRetrievalSnapshot:
-    state: RuntimeState
-    error_code: str | None
-    generation_id: str | None
-    supervisor: IndexSupervisorSnapshot | None
-
-
-class _AdmissionError(Exception):
-    def __init__(self, code: str, *, missing: bool = False):
-        self.code, self.missing = code, missing
-
-
-async def _native[T](operation: Callable[[], T]) -> T:
-    """Owned, joined native call. Only retained internal lifecycle tasks invoke this."""
-    result: list[T] = []
-    errors: list[BaseException] = []
-
-    def run() -> None:
-        try:
-            result.append(operation())
-        except BaseException as error:
-            errors.append(error)
-
-    thread = threading.Thread(target=run, name="retrieval-resource", daemon=False)
-    thread.start()
-    while thread.is_alive():
-        await asyncio.sleep(0.01)
-    thread.join()
-    if errors:
-        raise errors[0]
-    return result[0]
-
-
-class ExistingRetrievalRuntime:
-    """Caller guarantees managed Core single ownership and explicit configuration.
-
-    Construct without I/O; start and stop on the owning event loop. A retained
-    lifecycle task shields native work from caller cancellation until cleanup.
-    """
-
-    def __init__(
-        self,
-        data_dir: Path,
-        *,
-        minimum_free_bytes: int = 256 * 1024 * 1024,
-        service_levels: ServiceLevelScheduler | None = None,
-    ):
-        if type(minimum_free_bytes) is not int or minimum_free_bytes < 0:
-            raise ValueError("retrieval_runtime_invalid_disk_limit")
-        self._directory = Path(data_dir)
-        self._minimum_free = minimum_free_bytes
-        self._service_levels = service_levels or ServiceLevelScheduler()
-        self._state: ExistingRetrievalSnapshot = ExistingRetrievalSnapshot("new", None, None, None)
-        self._vectors: RawVectorStore | None = None
-        self._store: RetrievalIndexStore | None = None
-        self._encoder: LocalRetrievalEncoder | None = None
-        self._supervisor: RetrievalIndexSupervisor | None = None
-        self._last_supervisor_snapshot: IndexSupervisorSnapshot | None = None
-        self._start_task: asyncio.Task[bool] | None = None
-        self._stop_task: asyncio.Task[None] | None = None
-        self._stop_requested = False
-
-    def snapshot(self) -> ExistingRetrievalSnapshot:
-        return ExistingRetrievalSnapshot(
-            self._state.state,
-            self._state.error_code,
-            self._state.generation_id,
-            self._supervisor.snapshot()
-            if self._supervisor is not None
-            else self._last_supervisor_snapshot,
-        )
-
-    def _set(
-        self, state: RuntimeState, code: str | None = None, generation: str | None = None
-    ) -> None:
-        self._state = ExistingRetrievalSnapshot(
-            state,
-            code,
-            generation if generation is not None else self._state.generation_id,
-            None,
-        )
-
-    def trigger(self) -> bool:
-        return bool(
-            self._state.state == "running" and self._supervisor and self._supervisor.trigger()
-        )
-
-    def lookup_raw(
-        self,
-        query: str,
-        *,
-        source_ids: frozenset[str] | None,
-        deadline: float,
-        time_range: TimeRange | None = None,
-    ) -> tuple[RawCandidateBatch, bool]:
-        """Read-only candidate lookup. Does not release original bodies."""
-        store, encoder, vectors, generation = (
-            self._store,
-            self._encoder,
-            self._vectors,
-            self._state.generation_id,
-        )
-        if (
-            self._state.state != "running"
-            or store is None
-            or encoder is None
-            or vectors is None
-            or generation is None
-        ):
-            return RawCandidateBatch(reason="index_unavailable"), True
-        counts = store.status(generation)
-        unpublished = counts["pending"] + counts["leased"] + counts["blocked"] > 0
-        resolved = source_ids
-        if resolved is None:
-            resolved = frozenset(
-                chunk.source_id for chunk in store.active_chunks(generation, deadline=deadline)
-            )
-        if not resolved:
-            return (
-                RawCandidateBatch(complete=not unpublished, reason="no_eligible_chunks"),
-                unpublished,
-            )
-        try:
-            vector = encoder.encode_documents([query.strip()])[0]
-        except Exception:
-            return RawCandidateBatch(reason="index_unavailable"), True
-        batch = read_raw_semantic_candidates(
-            store,
-            vectors,
-            vector,
-            generation_id=generation,
-            source_ids=resolved,
-            deadline=deadline,
-            time_range=time_range,
-        )
-        return batch, unpublished
-
-    async def start(self) -> bool:
-        if self._state.state != "new" or self._stop_requested:
-            raise RuntimeError("retrieval_runtime_already_started")
-        self._set("starting")
-        self._start_task = asyncio.create_task(self._start())
-        try:
-            return await asyncio.shield(self._start_task)
-        except asyncio.CancelledError:
-            # Retain and drain opening resources, then release them, before cancellation escapes.
-            await self.stop()
-            raise
-
-    async def stop(self) -> None:
-        self._stop_requested = True
-        if self._stop_task is None:
-            self._stop_task = asyncio.create_task(self._stop())
-        cancelled = False
-        while not self._stop_task.done():
-            try:
-                await asyncio.shield(self._stop_task)
-            except asyncio.CancelledError:
-                cancelled = True
-        self._stop_task.result()
-        if cancelled:
-            raise asyncio.CancelledError
-
-    async def _start(self) -> bool:
-        try:
-            store, encoder, vectors, generation = await _native(self._open)
-            self._store = store
-            self._encoder = encoder
-            self._vectors = vectors
-            self._set("starting", generation=generation)
-            if self._stop_requested:
-                return False
-            worker = RetrievalIndexWorker(
-                store, vectors, encoder, generation_id=generation, owner="managed-core-retrieval"
-            )
-            self._supervisor = RetrievalIndexSupervisor(
-                worker,
-                background_should_yield=lambda: self._service_levels.should_yield(
-                    ServiceResource.INDEX,
-                    ServiceLevel.BACKGROUND,
-                ),
-            )
-            self._supervisor.start()
-            self._set("running")
-            return True
-        except _AdmissionError as error:
-            self._set("not_initialized" if error.missing else "failed", error.code)
-        except Exception:
-            self._set("failed", "retrieval_runtime_start_failed")
-        await self._release()
-        return False
-
-    async def _release(self) -> None:
-        stop_failed = False
-        if self._supervisor is not None:
-            try:
-                await self._supervisor.stop()
-            except Exception:
-                stop_failed = True
-            supervisor_snapshot = self._supervisor.snapshot()
-            if supervisor_snapshot.thread_alive:
-                self._set("failed", "retrieval_runtime_stop_failed")
-                raise RuntimeError("retrieval_runtime_stop_failed") from None
-            # The supervisor owns the worker, which owns the native vector
-            # client. Retain only the immutable receipt once its thread drains.
-            self._last_supervisor_snapshot = supervisor_snapshot
-            self._supervisor = None
-        if self._vectors is not None:
-            try:
-                await _native(self._vectors.close)
-            except Exception:
-                self._set("failed", "retrieval_runtime_close_failed")
-                raise RuntimeError("retrieval_runtime_close_failed") from None
-            self._vectors = None
-        self._store = None
-        self._encoder = None
-        if stop_failed:
-            self._set("failed", "retrieval_runtime_stop_failed")
-            raise RuntimeError("retrieval_runtime_stop_failed") from None
-
-    async def _stop(self) -> None:
-        if self._start_task is not None:
-            try:
-                await asyncio.shield(self._start_task)
-            except Exception:
-                # Startup can itself fail during cleanup. Do not lose resource ownership.
-                self._set("failed", "retrieval_runtime_start_failed")
-        self._set("stopping", self._state.error_code)
-        await self._release()
-        if self._state.error_code == "retrieval_runtime_close_failed":
-            return
-        self._set("stopped", self._state.error_code)
-
-    @staticmethod
-    def _path(path: Path, *, directory: bool) -> tuple[int, int]:
-        if any(part.is_symlink() for part in (path, *path.parents)):
-            raise _AdmissionError("retrieval_runtime_unsafe_path")
-        try:
-            info = path.stat()
-        except FileNotFoundError:
-            raise _AdmissionError("retrieval_runtime_not_initialized", missing=True) from None
-        if not (stat.S_ISDIR(info.st_mode) if directory else stat.S_ISREG(info.st_mode)):
-            raise _AdmissionError("retrieval_runtime_unsafe_path")
-        if not directory and info.st_nlink != 1:
-            raise _AdmissionError("retrieval_runtime_unsafe_path")
-        if info.st_uid != os.getuid() or stat.S_IMODE(info.st_mode) & 0o022:
-            raise _AdmissionError("retrieval_runtime_unsafe_path")
-        return info.st_dev, info.st_ino
-
-    def _sidecars(self, database: Path) -> None:
-        for suffix in ("-wal", "-shm", "-journal"):
-            path = database.with_name(database.name + suffix)
-            if path.is_symlink() or path.exists():
-                self._path(path, directory=False)
-
-    def _inspect(self, database: Path) -> tuple[RetrievalIndexStore, EncoderIdentity, str]:
-        with main_database(database.absolute()).read(busy_timeout_ms=1_000) as db:
-            db.row_factory = sqlite3.Row
-            db.execute("PRAGMA query_only=ON")
-            db.execute("BEGIN")
-            if not db.execute(
-                "SELECT 1 FROM sqlite_master WHERE name='retrieval_index_state'"
-            ).fetchone():
-                raise _AdmissionError("retrieval_runtime_not_initialized", missing=True)
-            state = db.execute(
-                "SELECT schema_version FROM retrieval_index_state WHERE singleton=1"
-            ).fetchone()
-            if state is not None and state[0] == 1:
-                raise _AdmissionError("retrieval_runtime_not_initialized", missing=True)
-            if state is None or state[0] != 2:
-                raise _AdmissionError("retrieval_runtime_invalid_index")
-            rows = db.execute(
-                "SELECT generation_id,identity_json FROM retrieval_index_generations "
-                "WHERE state='active' LIMIT 2"
-            ).fetchall()
-            if not rows:
-                raise _AdmissionError("retrieval_runtime_not_initialized", missing=True)
-            if len(rows) != 1:
-                raise _AdmissionError("retrieval_runtime_invalid_index")
-            store = RetrievalIndexStore(database)
-            generation = rows[0]["generation_id"]
-            identity = store._identity(db, generation)
-            raw = json.loads(rows[0]["identity_json"])
-            if raw != asdict(identity) or set(raw) != set(asdict(identity)):
-                raise _AdmissionError("retrieval_runtime_invalid_index")
-            # Check required schema shape without repairing it or reading original bodies.
-            db.execute(
-                "SELECT build_id,source_map_json,map_digest,coverage_state "
-                "FROM retrieval_build_maps LIMIT 0"
-            )
-            db.execute("SELECT after_seq,retry_after_seq FROM retrieval_consumers LIMIT 0")
-            db.execute("SELECT lease_token,active_build_id FROM retrieval_documents LIMIT 0")
-            db.execute("SELECT chunk_id,text_hash FROM retrieval_chunks LIMIT 0")
-            return store, identity, generation
-
-    def _open(self) -> tuple[RetrievalIndexStore, LocalRetrievalEncoder, RawVectorStore, str]:
-        if sys.platform == "win32" or not hasattr(os, "getuid"):
-            raise _AdmissionError("retrieval_runtime_platform_unsupported")
-        directory = self._directory.absolute()
-        database, vector_dir = directory / "one-memory.sqlite3", directory / "retrieval-vectors"
-        root_pin = self._path(directory, directory=True)
-        database_pin = self._path(database, directory=False)
-        self._sidecars(database)
-        try:
-            store, identity, generation = self._inspect(database)
-        except (ValueError, TypeError, sqlite3.Error):
-            raise _AdmissionError("retrieval_runtime_invalid_index") from None
-        vector_pin = self._path(vector_dir, directory=True)
-        vector_database = vector_dir / "chroma.sqlite3"
-        vector_db_pin = self._path(vector_database, directory=False)
-        self._sidecars(vector_database)
-
-        def recheck() -> None:
-            if (
-                self._path(directory, directory=True) != root_pin
-                or self._path(database, directory=False) != database_pin
-                or self._path(vector_dir, directory=True) != vector_pin
-                or self._path(vector_database, directory=False) != vector_db_pin
-            ):
-                raise _AdmissionError("retrieval_runtime_index_changed")
-            self._sidecars(database)
-            self._sidecars(vector_database)
-
-        if shutil.disk_usage(directory).free < self._minimum_free:
-            raise _AdmissionError("retrieval_runtime_low_disk")
-        encoder = LocalRetrievalEncoder(content_version=identity.normalizer_version)
-        if encoder.identity != identity:
-            raise _AdmissionError("retrieval_runtime_encoder_mismatch")
-        recheck()
-        try:
-            vectors = RawVectorStore.open_writer_existing(
-                vector_dir, generation_id=generation, dimensions=identity.dimensions
-            )
-            # Retain ownership immediately so all later failures use the same cleanup path.
-            self._vectors = vectors
-            # A generation change during the native open is not authorization to resume another.
-            recheck()
-            if self._inspect(database)[1:] != (identity, generation):
-                raise _AdmissionError("retrieval_runtime_index_changed")
-            recheck()
-            return store, encoder, vectors, generation
-        except BaseException as error:
-            if isinstance(error, VectorStoreError):
-                if str(error) in {"vector_store_not_found", "vector_collection_not_found"}:
-                    raise _AdmissionError(
-                        "retrieval_runtime_not_initialized", missing=True
-                    ) from None
-                raise _AdmissionError("retrieval_runtime_vector_unavailable") from None
-            raise

memhub/apps/core/src/one_memory/retrieval/vector_store.py

@@ -1,534 +0,0 @@
-"""Independent raw-vector adapter. Eligibility is supplied by Core's SQLite snapshot.
-
-No HY collection, document text, embedding function, migration/repair or authority
-store lives here. Chroma PersistentClient is NOT physically read-only: even the
-existing-collection handle can maintain backend metadata/index files. A caller
-requiring filesystem read-only operation must not use this backend.
-
-Scoring is an exact pass over the eligible vectors, not a global ANN top-K that
-is filtered afterwards. Chroma's filtered query does not score every candidate
-in its `where $in` allowlist: at 10k stored chunks an explicit 64-ID allowlist
-returned 26 of 64 while reporting the query as complete, and 0 of 64 after the
-writer handle was reopened. A post-filtered global search would be neither
-permission-correct nor recall-correct, so the eligible embeddings are read
-back and ranked here.
-"""
-
-from __future__ import annotations
-
-import math
-import re
-import sqlite3
-import struct
-import threading
-from collections.abc import Callable, Sequence
-from contextlib import closing
-from dataclasses import dataclass
-from hashlib import sha256
-from numbers import Real
-from pathlib import Path
-from time import monotonic
-from typing import Any, Literal
-
-ALLOWLIST_BATCH_SIZE = 2048
-
-
-class VectorStoreError(RuntimeError):
-    """Bounded failure code; callers must not treat corruption as complete empty search."""
-
-
-@dataclass(frozen=True)
-class RawVectorRecord:
-    chunk_id: str
-    build_id: str
-    generation_id: str
-    vector: tuple[float, ...]
-
-    def __post_init__(self) -> None:
-        object.__setattr__(self, "vector", tuple(self.vector))
-
-
-@dataclass(frozen=True)
-class VectorHit:
-    chunk_id: str
-    build_id: str
-    generation_id: str
-    distance: float
-
-
-@dataclass(frozen=True)
-class VectorSearchResult:
-    hits: tuple[VectorHit, ...]
-    complete: bool
-    eligible_count: int
-    searched_count: int
-    reason: Literal["deadline"] | None
-
-
-class RawVectorStore:
-    """Unopened constructor has no I/O. Use create/open_existing explicitly.
-
-    Directory must be a dedicated, trusted Runtime retrieval-vectors directory,
-    never the HY data directory. Only create/open_writer_existing enable upsert.
-    Core must serialize writers across handles/processes; this instance's lock
-    is not a persistent writer lease. close() releases only this Chroma client,
-    retaining other live clients' references.
-    """
-
-    def __init__(
-        self,
-        directory: Path,
-        *,
-        generation_id: str,
-        dimensions: int,
-        clock: Callable[[], float] = monotonic,
-    ) -> None:
-        _identifier(generation_id)
-        if type(dimensions) is not int or not 1 <= dimensions <= 65536:
-            raise VectorStoreError("vector_invalid_dimensions")
-        self.directory = Path(directory)
-        self.generation_id = generation_id
-        self.dimensions = dimensions
-        self.collection_name = "raw_" + sha256(generation_id.encode()).hexdigest()
-        self._clock = clock
-        self._client: Any = None
-        self._collection: Any = None
-        self._writable = False
-        self._closed = False
-        self._lock = threading.RLock()
-
-    @classmethod
-    def create(cls, directory: Path, *, generation_id: str, dimensions: int) -> RawVectorStore:
-        """Explicit writer initialization. Refuses to reuse an existing collection."""
-        value = cls(directory, generation_id=generation_id, dimensions=dimensions)
-        value._open(create=True)
-        return value
-
-    @classmethod
-    def open_existing(
-        cls,
-        directory: Path,
-        *,
-        generation_id: str,
-        dimensions: int,
-    ) -> RawVectorStore:
-        """No logical index writes; persistent Chroma may still write maintenance files."""
-        value = cls(directory, generation_id=generation_id, dimensions=dimensions)
-        value._open(create=False)
-        return value
-
-    @classmethod
-    def open_writer_existing(
-        cls,
-        directory: Path,
-        *,
-        generation_id: str,
-        dimensions: int,
-    ) -> RawVectorStore:
-        """Explicit worker recovery; verifies existing collection, never creates/repairs."""
-        value = cls(directory, generation_id=generation_id, dimensions=dimensions)
-        value._open(create=False, writable=True)
-        return value
-
-    def _metadata(self) -> dict[str, str | int]:
-        return {
-            "adapter": "one-memory-raw-v1",
-            "generation_id": self.generation_id,
-            "dimensions": self.dimensions,
-            "distance": "cosine",
-            "normalization": "l2-float32-v1",
-        }
-
-    def _preflight(self, create: bool) -> None:
-        if any(part.is_symlink() for part in (self.directory, *self.directory.parents)):
-            raise VectorStoreError("vector_unsafe_directory")
-        if not self.directory.exists():
-            if not create:
-                raise VectorStoreError("vector_store_not_found")
-            self.directory.mkdir(parents=True)
-        if not self.directory.is_dir():
-            raise VectorStoreError("vector_unsafe_directory")
-        database = self.directory / "chroma.sqlite3"
-        if database.is_symlink():
-            raise VectorStoreError("vector_unsafe_database")
-        if not database.is_file():
-            if not create:
-                raise VectorStoreError("vector_store_not_found")
-            if any(self.directory.iterdir()):
-                raise VectorStoreError("vector_unknown_directory")
-            return
-        if database.stat().st_nlink != 1:
-            raise VectorStoreError("vector_unsafe_database")
-        # Refuse missing/corrupt collections BEFORE PersistentClient can perform
-        # startup maintenance. This checks the installed Chroma 1.5.9 schema.
-        try:
-            with closing(sqlite3.connect(database.resolve().as_uri() + "?mode=ro", uri=True)) as db:
-                found = db.execute(
-                    "SELECT 1 FROM collections WHERE name=?", (self.collection_name,)
-                ).fetchone()
-                if not create and found is None:
-                    raise VectorStoreError("vector_collection_not_found")
-                if create and found is not None:
-                    raise VectorStoreError("vector_collection_exists")
-        except sqlite3.Error:
-            raise VectorStoreError("vector_store_unavailable") from None
-
-    def _open(self, *, create: bool, writable: bool = False) -> None:
-        self._preflight(create)
-        try:
-            import chromadb
-            from chromadb.config import Settings
-
-            self._client = chromadb.PersistentClient(
-                path=str(self.directory), settings=Settings(anonymized_telemetry=False)
-            )
-            if create:
-                self._collection = self._client.create_collection(
-                    self.collection_name,
-                    embedding_function=None,
-                    metadata=self._metadata(),
-                    # Unit vectors + inner product give cosine distance without
-                    # cosine HNSW's lossy normalization/de-normalization on get.
-                    configuration={"hnsw": {"space": "ip"}},
-                )
-            else:
-                self._collection = self._client.get_collection(
-                    self.collection_name, embedding_function=None
-                )
-            if (
-                self._collection.metadata != self._metadata()
-                or type(self._collection.metadata.get("dimensions")) is not int
-            ):
-                raise VectorStoreError("vector_collection_identity_mismatch")
-            config = self._collection.configuration
-            if (
-                config.get("hnsw", {}).get("space") != "ip"
-                or config.get("embedding_function") is not None
-            ):
-                raise VectorStoreError("vector_collection_configuration_mismatch")
-            self._writable = create or writable
-        except Exception as error:
-            self.close()
-            if isinstance(error, VectorStoreError):
-                raise
-            raise VectorStoreError("vector_store_unavailable") from None
-
-    def close(self) -> None:
-        with self._lock:
-            if self._closed:
-                return
-            self._closed = True
-            if self._client is not None:
-                self._client.close()
-            self._client = self._collection = None
-
-    def _require_open(self) -> Any:
-        if self._closed:
-            raise VectorStoreError("vector_store_closed")
-        if self._collection is None:
-            raise VectorStoreError("vector_store_not_open")
-        return self._collection
-
-    def _records(self, records: Sequence[RawVectorRecord]) -> tuple[RawVectorRecord, ...]:
-        result = []
-        seen: set[str] = set()
-        for record in records:
-            if not isinstance(record, RawVectorRecord):
-                raise VectorStoreError("vector_invalid_record")
-            for identifier in (record.chunk_id, record.build_id, record.generation_id):
-                _identifier(identifier)
-            if record.generation_id != self.generation_id or record.chunk_id in seen:
-                raise VectorStoreError("vector_invalid_record_identity")
-            seen.add(record.chunk_id)
-            result.append(
-                RawVectorRecord(
-                    record.chunk_id,
-                    record.build_id,
-                    record.generation_id,
-                    _vector(record.vector, self.dimensions),
-                )
-            )
-        return tuple(result)
-
-    def _read_records(self, ids: Sequence[str]) -> dict[str, tuple[RawVectorRecord, str]]:
-        if not ids:
-            return {}
-        collection = self._require_open()
-        result: dict[str, tuple[RawVectorRecord, str]] = {}
-        for start in range(0, len(ids), ALLOWLIST_BATCH_SIZE):
-            batch = ids[start : start + ALLOWLIST_BATCH_SIZE]
-            try:
-                data = collection.get(ids=list(batch), include=["embeddings", "metadatas"])
-                returned, vectors, metadatas = data["ids"], data["embeddings"], data["metadatas"]
-                if (
-                    vectors is None
-                    or metadatas is None
-                    or not len(returned) == len(vectors) == len(metadatas)
-                ):
-                    raise VectorStoreError("vector_invalid_record_response")
-                for key, vector, metadata in zip(returned, vectors, metadatas, strict=True):
-                    if key not in batch or key in result:
-                        raise VectorStoreError("vector_ineligible_record")
-                    result[key] = self._decode(key, vector, metadata)
-            except VectorStoreError:
-                raise
-            except Exception:
-                raise VectorStoreError("vector_read_unavailable") from None
-        return result
-
-    def _decode(
-        self, key: str, vector: Sequence[float], metadata: Any
-    ) -> tuple[RawVectorRecord, str]:
-        if not isinstance(metadata, dict) or set(metadata) != {
-            "chunk_id",
-            "build_id",
-            "generation_id",
-            "vector_sha256",
-            "input_sha256",
-        }:
-            raise VectorStoreError("vector_invalid_metadata")
-        if metadata["chunk_id"] != key or metadata["generation_id"] != self.generation_id:
-            raise VectorStoreError("vector_metadata_identity_mismatch")
-        _identifier(key)
-        _identifier(metadata["build_id"])
-        if not isinstance(metadata["input_sha256"], str) or not re.fullmatch(
-            r"[0-9a-f]{64}", metadata["input_sha256"]
-        ):
-            raise VectorStoreError("vector_invalid_input_digest")
-        normalized = _vector(vector, self.dimensions)
-        if metadata["vector_sha256"] != _digest(normalized):
-            raise VectorStoreError("vector_content_mismatch")
-        return RawVectorRecord(key, metadata["build_id"], self.generation_id, normalized), metadata[
-            "input_sha256"
-        ]
-
-    def upsert(self, records: Sequence[RawVectorRecord]) -> None:
-        """Idempotent immutable insertion; add-only at Chroma, never overwrite IDs.
-
-        Equality is canonical float32 input identity plus explicit normalized
-        storage identity. Conflict is checked before
-        any add. Backend failure may leave a partial new batch: caller must verify
-        the entire build before publication. Existing IDs are never modified.
-        """
-        with self._lock:
-            collection = self._require_open()
-            if not self._writable:
-                raise VectorStoreError("vector_store_read_only")
-            incoming = self._records(records)
-            existing = self._read_records([record.chunk_id for record in incoming])
-            for record in incoming:
-                if record.chunk_id in existing and not _matches(existing[record.chunk_id], record):
-                    raise VectorStoreError("vector_immutable_conflict")
-            new = [record for record in incoming if record.chunk_id not in existing]
-            for start in range(0, len(new), ALLOWLIST_BATCH_SIZE):
-                batch = new[start : start + ALLOWLIST_BATCH_SIZE]
-                try:
-                    collection.add(
-                        ids=[record.chunk_id for record in batch],
-                        embeddings=[list(_unit_vector(record.vector)) for record in batch],
-                        metadatas=[
-                            {
-                                "chunk_id": record.chunk_id,
-                                "build_id": record.build_id,
-                                "generation_id": record.generation_id,
-                                "vector_sha256": _digest(_unit_vector(record.vector)),
-                                "input_sha256": _digest(record.vector),
-                            }
-                            for record in batch
-                        ],
-                    )
-                except Exception:
-                    raise VectorStoreError("vector_write_unavailable") from None
-            self.verify(incoming)
-
-    def verify(self, records: Sequence[RawVectorRecord]) -> None:
-        """Verify every expected ID, metadata, dimensions and exact float32 vector."""
-        with self._lock:
-            self._require_open()
-            expected = self._records(records)
-            actual = self._read_records([record.chunk_id for record in expected])
-            for record in expected:
-                if record.chunk_id not in actual:
-                    raise VectorStoreError("vector_record_missing")
-                if not _matches(actual[record.chunk_id], record):
-                    raise VectorStoreError("vector_immutable_conflict")
-
-    def verify_candidates(
-        self,
-        candidates: Sequence[tuple[str, str]],
-        *,
-        deadline: float,
-    ) -> bool:
-        """Read-only presence/integrity check for (chunk_id, expected_build_id).
-
-        Costs O(candidates * dimensions), not a bounded top-K lookup. False means
-        deadline exhaustion, never proof of absence. Each read covers at most
-        2048 IDs; synchronous backend calls cannot be interrupted. This is not a
-        snapshot across Chroma and SQLite: Core still owns final eligibility
-        checks and writer serialization. Chroma may perform maintenance writes.
-        """
-        with self._lock:
-            self._require_open()
-            if (
-                isinstance(deadline, bool)
-                or not isinstance(deadline, Real)
-                or not math.isfinite(deadline)
-            ):
-                raise VectorStoreError("vector_invalid_deadline")
-            if not isinstance(candidates, Sequence) or isinstance(candidates, (str, bytes)):
-                raise VectorStoreError("vector_invalid_candidates")
-            expected: list[tuple[str, str]] = []
-            seen: set[str] = set()
-            for pair in candidates:
-                if (
-                    not isinstance(pair, Sequence)
-                    or isinstance(pair, (str, bytes))
-                    or len(pair) != 2
-                ):
-                    raise VectorStoreError("vector_invalid_candidates")
-                key, build = pair
-                _identifier(key)
-                _identifier(build)
-                if key in seen:
-                    raise VectorStoreError("vector_duplicate_candidates")
-                seen.add(key)
-                expected.append((key, build))
-            for start in range(0, len(expected), ALLOWLIST_BATCH_SIZE):
-                if self._clock() >= deadline:
-                    return False
-                batch = expected[start : start + ALLOWLIST_BATCH_SIZE]
-                # Bound this call to one batch so it cannot read past the next
-                # deadline checkpoint via _read_records' internal loop.
-                actual = self._read_records([key for key, _build in batch])
-                for key, build in batch:
-                    if key not in actual:
-                        raise VectorStoreError("vector_record_missing")
-                    if actual[key][0].build_id != build:
-                        raise VectorStoreError("vector_metadata_identity_mismatch")
-                if self._clock() >= deadline:
-                    return False
-            return True
-
-    def search(
-        self,
-        query_vector: Sequence[float],
-        eligible_ids: Sequence[str],
-        limit: int,
-        deadline: float,
-    ) -> VectorSearchResult:
-        """Rank exactly the eligible candidates; synchronous backend cannot be preempted.
-
-        deadline is absolute monotonic seconds. A late batch is discarded; prior
-        on-time batches may return partial. complete means every eligible
-        vector was scored, NOT complete index coverage: only chunks the caller
-        already resolved as eligible and published are searched.
-
-        The eligible embeddings are read back in bounded batches and ranked
-        here, because Chroma's own filtered query does not score every ID in
-        its allowlist. A global top-K filtered afterwards would rank
-        unauthorized vectors first and let them crowd the eligible ones out.
-        """
-        with self._lock:
-            self._require_open()
-            vector = _unit_vector(_vector(query_vector, self.dimensions))
-            if type(limit) is not int or not 1 <= limit <= ALLOWLIST_BATCH_SIZE:
-                raise VectorStoreError("vector_invalid_limit")
-            if (
-                isinstance(deadline, bool)
-                or not isinstance(deadline, Real)
-                or not math.isfinite(deadline)
-            ):
-                raise VectorStoreError("vector_invalid_deadline")
-            if isinstance(eligible_ids, (str, bytes)):
-                raise VectorStoreError("vector_invalid_allowlist")
-            ids = tuple(eligible_ids)
-            for key in ids:
-                _identifier(key)
-            if len(set(ids)) != len(ids):
-                raise VectorStoreError("vector_duplicate_allowlist")
-            hits: list[VectorHit] = []
-            searched = 0
-            for start in range(0, len(ids), ALLOWLIST_BATCH_SIZE):
-                if self._clock() >= deadline:
-                    break
-                batch = ids[start : start + ALLOWLIST_BATCH_SIZE]
-                try:
-                    read = self._read_records(batch)
-                except VectorStoreError as error:
-                    # A missing or corrupt eligible vector is not "no hits".
-                    raise VectorStoreError("vector_query_unavailable") from error
-                if self._clock() >= deadline:
-                    break
-                try:
-                    missing = [key for key in batch if key not in read]
-                    if missing:
-                        # The caller resolved these as eligible and published,
-                        # so an absent vector is a broken build, not a miss.
-                        raise VectorStoreError("vector_query_unavailable")
-                    validated = [
-                        VectorHit(
-                            key,
-                            read[key][0].build_id,
-                            read[key][0].generation_id,
-                            # Stored vectors are unit length, so the inner
-                            # product is the cosine similarity.
-                            1.0 - _dot(read[key][0].vector, vector),
-                        )
-                        for key in batch
-                    ]
-                except VectorStoreError:
-                    raise
-                except Exception:
-                    raise VectorStoreError("vector_invalid_query_response") from None
-                if self._clock() >= deadline:
-                    break
-                searched += len(batch)
-                hits.extend(validated)
-                hits.sort(key=lambda hit: (hit.distance, hit.chunk_id))
-                del hits[limit:]
-            complete = searched == len(ids)
-            return VectorSearchResult(
-                tuple(hits), complete, len(ids), searched, None if complete else "deadline"
-            )
-
-
-def _dot(left: Sequence[float], right: Sequence[float]) -> float:
-    return math.fsum(a * b for a, b in zip(left, right, strict=True))
-
-
-def _identifier(value: Any) -> None:
-    if not isinstance(value, str) or not re.fullmatch(r"[a-zA-Z0-9][a-zA-Z0-9_.:-]{0,255}", value):
-        raise VectorStoreError("vector_invalid_identifier")
-
-
-def _vector(value: Sequence[float], dimensions: int) -> tuple[float, ...]:
-    try:
-        if len(value) != dimensions or any(
-            isinstance(v, bool) or not isinstance(v, Real) or not math.isfinite(v) for v in value
-        ):
-            raise VectorStoreError("vector_invalid_values_or_dimensions")
-        packed = struct.pack(f"<{dimensions}f", *value)
-        normalized = struct.unpack(f"<{dimensions}f", packed)
-        if not all(math.isfinite(v) for v in normalized) or not any(normalized):
-            raise VectorStoreError("vector_invalid_values_or_dimensions")
-        return normalized
-    except VectorStoreError:
-        raise
-    except (ValueError, TypeError, OverflowError, struct.error):
-        raise VectorStoreError("vector_invalid_values_or_dimensions") from None
-
-
-def _digest(vector: tuple[float, ...]) -> str:
-    return sha256(struct.pack(f"<{len(vector)}f", *vector)).hexdigest()
-
-
-def _unit_vector(vector: tuple[float, ...]) -> tuple[float, ...]:
-    length = math.sqrt(math.fsum(component * component for component in vector))
-    return _vector(tuple(component / length for component in vector), len(vector))
-
-
-def _matches(actual: tuple[RawVectorRecord, str], expected: RawVectorRecord) -> bool:
-    stored, input_digest = actual
-    return input_digest == _digest(expected.vector) and stored == RawVectorRecord(
-        expected.chunk_id, expected.build_id, expected.generation_id, _unit_vector(expected.vector)
-    )

memhub/apps/core/src/one_memory/sdk_data_plane_migration.py

@@ -2,7 +2,6 @@
 
 from __future__ import annotations
 
-import json
 import sqlite3
 from dataclasses import dataclass
 from datetime import UTC, datetime
@@ -40,10 +39,6 @@ class MigrationProgress:
     migration_id: str
     state: str
     phase: str
-    counts: dict[str, int]
-
-
-TERMINAL_ITEM_STATES = frozenset({"verified", "adopted", "deleted", "skipped"})
 
 
 class SdkDataPlaneMigrationStore:
@@ -55,18 +50,29 @@ class SdkDataPlaneMigrationStore:
         return datetime.now(UTC).isoformat()
 
     def authority(self) -> MemoryAuthority:
+        return self.authority_state()[0]
+
+    def authority_state(self) -> tuple[MemoryAuthority, str | None]:
+        """The surviving singleton: authority and the migration that set it.
+
+        This is the only source of authority. The detailed cutover receipt is
+        dropped after cleanup and never exists on a fresh home.
+        """
         try:
             with main_database(self.database_path).read() as connection:
                 row = connection.execute(
-                    "SELECT authority FROM memory_authority_state WHERE singleton=1"
+                    "SELECT authority,migration_id FROM memory_authority_state WHERE singleton=1"
                 ).fetchone()
         except sqlite3.OperationalError as error:
             if "no such table" not in str(error).lower():
                 raise
         else:
             if row is not None:
-                return MemoryAuthority(str(row[0]))
-        return MemoryAuthority.SDK if self.cutover() is not None else MemoryAuthority.LEGACY
+                return MemoryAuthority(str(row[0])), (str(row[1]) if row[1] else None)
+        marker = self.cutover()
+        if marker is not None:
+            return MemoryAuthority.SDK, marker.migration_id
+        return MemoryAuthority.LEGACY, None
 
     def cutover(self) -> CutoverMarker | None:
         try:
@@ -136,37 +142,14 @@ class SdkDataPlaneMigrationStore:
                     """,
                     values,
                 )
-            else:
-                identity = (
-                    source_runtime_build,
-                    target_sdk_version,
-                    source_schema_version,
-                    namespace_fingerprint,
-                    backend_fingerprint,
+            elif backup_bundle_id and existing["backup_bundle_id"] is None:
+                connection.execute(
+                    "UPDATE sdk_data_plane_migrations "
+                    "SET backup_bundle_id=?,updated_at=? WHERE migration_id=?",
+                    (backup_bundle_id, now, migration_id),
                 )
-                existing_identity = tuple(
-                    existing[key]
-                    for key in (
-                        "source_runtime_build",
-                        "target_sdk_version",
-                        "source_schema_version",
-                        "namespace_fingerprint",
-                        "backend_fingerprint",
-                    )
-                )
-                if existing_identity != identity:
-                    raise DataPlaneMigrationError("sdk_data_plane_migration_identity_conflict")
-                if backup_bundle_id and existing["backup_bundle_id"] not in {
-                    None,
-                    backup_bundle_id,
-                }:
-                    raise DataPlaneMigrationError("sdk_data_plane_migration_bundle_conflict")
-                if backup_bundle_id and existing["backup_bundle_id"] is None:
-                    connection.execute(
-                        "UPDATE sdk_data_plane_migrations "
-                        "SET backup_bundle_id=?,updated_at=? WHERE migration_id=?",
-                        (backup_bundle_id, now, migration_id),
-                    )
+            elif backup_bundle_id and existing["backup_bundle_id"] != backup_bundle_id:
+                raise DataPlaneMigrationError("sdk_data_plane_migration_bundle_conflict")
             connection.execute("COMMIT")
         return self.progress(migration_id)
 
@@ -182,211 +165,18 @@ class SdkDataPlaneMigrationStore:
                 raise DataPlaneMigrationError("sdk_data_plane_migration_not_mutable")
             connection.execute("COMMIT")
 
-    def upsert_item(
-        self,
-        *,
-        migration_id: str,
-        item_key: str,
-        source_kind: str,
-        source_id: str,
-        source_hash: str,
-        operation_key: str,
-        source_revision: str | None = None,
-        state: str = "pending",
-    ) -> None:
-        now = self._now()
-        with main_database(self.database_path).write(
-            autocommit=True, row_factory=sqlite3.Row
-        ) as connection:
-            connection.execute("BEGIN IMMEDIATE")
-            row = connection.execute(
-                "SELECT * FROM sdk_data_plane_migration_items WHERE migration_id=? AND item_key=?",
-                (migration_id, item_key),
-            ).fetchone()
-            if row is None:
-                try:
-                    connection.execute(
-                        """
-                        INSERT INTO sdk_data_plane_migration_items (
-                            migration_id,item_key,source_kind,source_id,
-                            source_revision,source_hash,operation_key,state,updated_at
-                        ) VALUES (?,?,?,?,?,?,?,?,?)
-                        """,
-                        (
-                            migration_id,
-                            item_key,
-                            source_kind,
-                            source_id,
-                            source_revision,
-                            source_hash,
-                            operation_key,
-                            state,
-                            now,
-                        ),
-                    )
-                except sqlite3.IntegrityError as error:
-                    raise DataPlaneMigrationError(
-                        "sdk_data_plane_migration_operation_conflict"
-                    ) from error
-            else:
-                if str(row["state"]) in TERMINAL_ITEM_STATES and (
-                    source_id,
-                    source_revision,
-                    source_hash,
-                ) == (
-                    row["source_id"],
-                    row["source_revision"],
-                    row["source_hash"],
-                ):
-                    # A crash after an SDK side effect can make the next
-                    # inventory classify the same source as verified/adopted
-                    # instead of import/delete. Keep the settled operation as
-                    # the durable audit identity.
-                    connection.execute("COMMIT")
-                    return
-                identity = (
-                    source_kind,
-                    source_id,
-                    source_revision,
-                    source_hash,
-                    operation_key,
-                )
-                if identity != tuple(
-                    row[key]
-                    for key in (
-                        "source_kind",
-                        "source_id",
-                        "source_revision",
-                        "source_hash",
-                        "operation_key",
-                    )
-                ):
-                    raise DataPlaneMigrationError("sdk_data_plane_migration_item_conflict")
-            connection.execute("COMMIT")
-
-    def settle_item(
-        self,
-        *,
-        migration_id: str,
-        item_key: str,
-        state: str,
-        target_memory_id: str | None = None,
-        target_hash: str | None = None,
-        error_code: str | None = None,
-    ) -> None:
-        with main_database(self.database_path).write(autocommit=True) as connection:
-            connection.execute("BEGIN IMMEDIATE")
-            cursor = connection.execute(
-                """
-                UPDATE sdk_data_plane_migration_items
-                SET state=?,target_memory_id=?,target_hash=?,error_code=?,
-                    attempts=attempts+1,updated_at=?
-                WHERE migration_id=? AND item_key=?
-                """,
-                (
-                    state,
-                    target_memory_id,
-                    target_hash,
-                    error_code,
-                    self._now(),
-                    migration_id,
-                    item_key,
-                ),
-            )
-            if cursor.rowcount != 1:
-                raise DataPlaneMigrationError("sdk_data_plane_migration_item_not_found")
-            connection.execute("COMMIT")
-
-    def record_alias(
-        self,
-        *,
-        migration_id: str,
-        legacy_memory_id: str,
-        canonical_memory_id: str,
-    ) -> None:
-        with main_database(self.database_path).write(
-            autocommit=True, row_factory=sqlite3.Row
-        ) as connection:
-            connection.execute("BEGIN IMMEDIATE")
-            existing = connection.execute(
-                "SELECT canonical_memory_id,migration_id FROM memory_id_aliases "
-                "WHERE legacy_memory_id=?",
-                (legacy_memory_id,),
-            ).fetchone()
-            if existing is None:
-                connection.execute(
-                    """
-                    INSERT INTO memory_id_aliases (
-                        legacy_memory_id,canonical_memory_id,migration_id,created_at
-                    ) VALUES (?,?,?,?)
-                    """,
-                    (
-                        legacy_memory_id,
-                        canonical_memory_id,
-                        migration_id,
-                        self._now(),
-                    ),
-                )
-            elif existing["canonical_memory_id"] != canonical_memory_id:
-                raise DataPlaneMigrationError("sdk_data_plane_alias_conflict")
-            connection.execute("COMMIT")
-
-    def record_raw_alias(
-        self,
-        *,
-        migration_id: str,
-        legacy_evidence_id: str,
-        canonical_raw_id: str,
-        item_index: int = 0,
-    ) -> None:
-        with main_database(self.database_path).write(
-            autocommit=True,
-            row_factory=sqlite3.Row,
-        ) as connection:
-            connection.execute("BEGIN IMMEDIATE")
-            existing = connection.execute(
-                "SELECT canonical_raw_id FROM raw_id_aliases "
-                "WHERE legacy_evidence_id=? AND item_index=?",
-                (legacy_evidence_id, item_index),
-            ).fetchone()
-            if existing is None:
-                connection.execute(
-                    "INSERT INTO raw_id_aliases "
-                    "(legacy_evidence_id,item_index,canonical_raw_id,migration_id,created_at) "
-                    "VALUES (?,?,?,?,?)",
-                    (
-                        legacy_evidence_id,
-                        item_index,
-                        canonical_raw_id,
-                        migration_id,
-                        self._now(),
-                    ),
-                )
-            elif existing["canonical_raw_id"] != canonical_raw_id:
-                raise DataPlaneMigrationError("sdk_data_plane_raw_alias_conflict")
-            connection.execute("COMMIT")
-
     def progress(self, migration_id: str) -> MigrationProgress:
         with main_database(self.database_path).read(row_factory=sqlite3.Row) as connection:
             migration = connection.execute(
                 "SELECT state,phase FROM sdk_data_plane_migrations WHERE migration_id=?",
                 (migration_id,),
             ).fetchone()
-            if migration is None:
-                raise DataPlaneMigrationError("sdk_data_plane_migration_not_found")
-            counts = {
-                str(row["state"]): int(row["count"])
-                for row in connection.execute(
-                    "SELECT state,COUNT(*) AS count "
-                    "FROM sdk_data_plane_migration_items WHERE migration_id=? GROUP BY state",
-                    (migration_id,),
-                )
-            }
+        if migration is None:
+            raise DataPlaneMigrationError("sdk_data_plane_migration_not_found")
         return MigrationProgress(
             migration_id=migration_id,
             state=str(migration["state"]),
             phase=str(migration["phase"]),
-            counts=counts,
         )
 
     def commit_cutover(
@@ -409,23 +199,8 @@ class SdkDataPlaneMigrationStore:
             ).fetchone()
             if migration is None:
                 raise DataPlaneMigrationError("sdk_data_plane_migration_not_found")
-            if migration["phase"] != "reconciled" or migration["state"] != "ready":
+            if migration["phase"] != "prepared" or migration["state"] != "ready":
                 raise DataPlaneMigrationError("sdk_data_plane_migration_not_ready")
-            unresolved = connection.execute(
-                "SELECT state,COUNT(*) AS count FROM sdk_data_plane_migration_items "
-                "WHERE migration_id=? GROUP BY state",
-                (migration_id,),
-            ).fetchall()
-            blocked = {
-                str(row["state"]): int(row["count"])
-                for row in unresolved
-                if str(row["state"]) not in TERMINAL_ITEM_STATES
-            }
-            if blocked:
-                raise DataPlaneMigrationError(
-                    "sdk_data_plane_migration_items_unresolved:"
-                    + json.dumps(blocked, sort_keys=True, separators=(",", ":"))
-                )
             if migration["backup_bundle_id"] != backup_bundle_id:
                 raise DataPlaneMigrationError("sdk_data_plane_migration_bundle_conflict")
             existing = connection.execute(

memhub/apps/core/src/one_memory/sdk_memory_qa.py

@@ -6,13 +6,15 @@ import asyncio
 import json
 from collections.abc import AsyncIterator, Awaitable, Callable
 from contextlib import aclosing
-from typing import Literal
+from typing import Any, Literal, Protocol
 
 import httpx
 from pydantic import BaseModel, ConfigDict
 
 from one_memory.http_client import create_async_client
-from one_memory.hy_memory_bridge import HyMemoryBridge, HySdkMemoryRecord
+from one_memory.hy_runtime_config import HY_MEMORY_READER
+from one_memory.hy_sdk_runtime import sdk_relevance_floor
+from one_memory.hy_sdk_types import HySdkMemoryRecord, parse_sdk_search
 from one_memory.memory_qa import (
     MemoryAnswerError,
     _DeferredSecretStore,
@@ -36,6 +38,10 @@ _GROUNDING_LIMIT = 6
 type _Recheck = Callable[[], Awaitable[None]]
 
 
+class SdkClientHandleLike(Protocol):
+    async def client(self) -> Any: ...
+
+
 class SdkAskMemory(BaseModel):
     model_config = ConfigDict(frozen=True, extra="forbid")
 
@@ -69,7 +75,7 @@ class SdkMemoryAnswerer:
     def __init__(
         self,
         *,
-        bridge: HyMemoryBridge,
+        client_handle: SdkClientHandleLike,
         profiles: ModelProfileStore,
         secrets: SecretStore,
         authorize_generation: Callable[
@@ -78,7 +84,7 @@ class SdkMemoryAnswerer:
         | None = None,
         transport: httpx.AsyncBaseTransport | None = None,
     ) -> None:
-        self.bridge = bridge
+        self.client_handle = client_handle
         self.profiles = profiles
         self.secrets = secrets
         self.authorize_generation = authorize_generation
@@ -90,18 +96,27 @@ class SdkMemoryAnswerer:
         *,
         user_id: str = "local",
     ) -> tuple[SdkAskMemory, ...]:
-        hits = await self.bridge.search_memories(
+        client = await self.client_handle.client()
+        raw = await asyncio.to_thread(
+            client.search,
             question,
-            user_id=user_id,
-            agent_id="*",
+            user_ids=[user_id],
+            agent_ids=[],
             limit=12,
+            min_score=sdk_relevance_floor(),
+            schema_min_score=sdk_relevance_floor(),
+            schema_limit=12,
+            intention_limit=12,
+            scene="normal",
+            reader=HY_MEMORY_READER,
         )
+        hits = parse_sdk_search(raw, error_code="hy_memory_search_shape_unknown")
         found: list[SdkAskMemory] = []
         seen: set[str] = set()
         for hit in hits:
             if hit.memory_id in seen:
                 continue
-            record = await self.bridge.get_sdk_memory(hit.memory_id)
+            record = await _get_memory(client, hit.memory_id)
             if record is None or record.status != "active" or not record.is_latest:
                 continue
             seen.add(record.memory_id)
@@ -211,8 +226,9 @@ class SdkMemoryAnswerer:
         self,
         memories: tuple[SdkAskMemory, ...],
     ) -> None:
+        client = await self.client_handle.client()
         for memory in memories:
-            current = await self.bridge.get_sdk_memory(memory.memory_id)
+            current = await _get_memory(client, memory.memory_id)
             if current is None or _wire_memory(current) != memory:
                 raise MemoryAnswerError("memory_answer_context_changed")
 
@@ -340,6 +356,11 @@ def _wire_memory(record: HySdkMemoryRecord) -> SdkAskMemory:
     )
 
 
+async def _get_memory(client: Any, memory_id: str) -> HySdkMemoryRecord | None:
+    raw = await asyncio.to_thread(client.get, memory_id)
+    return None if raw is None else HySdkMemoryRecord.from_sdk(raw)
+
+
 def _summary(memories: tuple[SdkAskMemory, ...]) -> str:
     statements = [
         memory.content.replace("用户", "你", 1).replace("The user", "You", 1) for memory in memories

memhub/apps/core/src/one_memory/sdk_memory_runtime.py

@@ -0,0 +1,399 @@
+"""Narrow SDK-native runtime for graph, retrieval, and queue status operations."""
+
+from __future__ import annotations
+
+import asyncio
+import logging
+import re
+from contextlib import suppress
+from datetime import UTC, datetime, timedelta
+from pathlib import Path
+from typing import Any
+
+from one_memory.database_access import main_database
+from one_memory.extraction_settings import ExtractionSettings, ExtractionSettingsStore
+from one_memory.hy_graph_admin import (
+    GraphWidthDecision,
+    GraphWidthMismatch,
+    GraphWidthOutcome,
+    decide_profile_graph_width,
+    inspect_profile_graph_width,
+)
+from one_memory.hy_graph_projection import HyGraphSnapshot, read_graph_snapshot
+from one_memory.hy_runtime_config import HY_MEMORY_READER
+from one_memory.hy_sdk_runtime import SdkClientHandle, sdk_relevance_floor
+from one_memory.hy_sdk_types import HySdkError, parse_sdk_search
+from one_memory.memory_queue import ExtractionWindowPreset, QueueDepths
+from one_memory.memory_server.durable_queue import COMPLETED, DurableWriteQueue
+from one_memory.sdk_write_processor import SdkWriteCoordinator
+from one_memory.semantic_deadline import SemanticMemoryHit
+
+# Only a code-shaped message is a code; anything else may carry provider text.
+_STABLE_CODE = re.compile(r"[a-z][a-z0-9_]{0,95}")
+
+logger = logging.getLogger(__name__)
+RAW_RETENTION_DAYS = 3
+RAW_RETENTION_CHECK_SECONDS = 6 * 60 * 60
+_RAW_STATUSES = (
+    "active",
+    "superseded",
+    "shadow",
+    "negated",
+    "conflicted",
+    "archived",
+)
+
+
+class SdkMemoryRuntime:
+    """Product services that remain after legacy task/governance writes retire."""
+
+    def __init__(
+        self,
+        *,
+        coordinator: SdkWriteCoordinator,
+        read_handle: SdkClientHandle,
+        write_queue: DurableWriteQueue,
+        extraction_settings: ExtractionSettingsStore,
+        data_dir: Path,
+    ) -> None:
+        self.coordinator = coordinator
+        self.read_handle = read_handle
+        self.write_queue = write_queue
+        self.extraction_settings = extraction_settings
+        self.data_dir = data_dir
+        self._raw_retention_task: asyncio.Task[None] | None = None
+
+    def start_raw_retention(self) -> None:
+        if self._raw_retention_task is None or self._raw_retention_task.done():
+            self._raw_retention_task = asyncio.create_task(
+                self._raw_retention_loop(),
+                name="one-memory-sdk-raw-retention",
+            )
+
+    async def prune_expired_raw(self, *, now: datetime | None = None) -> int:
+        cutoff = (now or datetime.now(UTC)) - timedelta(days=RAW_RETENTION_DAYS)
+        cutoff_timestamp = int(cutoff.timestamp())
+        client = await self.read_handle.client()
+        deleted = 0
+        while True:
+            page = await asyncio.to_thread(
+                client.list_memories,
+                user_id="local",
+                layers=["l1_raw"],
+                statuses=list(_RAW_STATUSES),
+                order="asc",
+                limit=500,
+                offset=0,
+            )
+            bucket = page.get("vdb") if isinstance(page, dict) else None
+            rows = bucket.get("memories") if isinstance(bucket, dict) else None
+            if not isinstance(rows, list) or not rows:
+                return deleted
+            expired = [
+                row
+                for row in rows
+                if isinstance(row, dict)
+                and isinstance(row.get("memory_id"), str)
+                and int(row.get("memory_at") or row.get("gmt_created") or 0)
+                < cutoff_timestamp
+            ]
+            if not expired:
+                return deleted
+            removed_page = 0
+            for row in expired:
+                result = await asyncio.to_thread(client.delete, row["memory_id"])
+                if isinstance(result, dict) and result.get("success") is True:
+                    count = int(result.get("deleted_count") or 0)
+                    deleted += count
+                    removed_page += count
+            if removed_page == 0:
+                return deleted
+
+    async def _raw_retention_loop(self) -> None:
+        while True:
+            await asyncio.sleep(RAW_RETENTION_CHECK_SECONDS)
+            try:
+                removed = await self.prune_expired_raw()
+                if removed:
+                    logger.info("sdk_raw_retention_deleted count=%d", removed)
+            except asyncio.CancelledError:
+                raise
+            except Exception as error:
+                logger.warning(
+                    "sdk_raw_retention_failed error_type=%s",
+                    type(error).__name__,
+                )
+
+    async def semantic_recall(
+        self,
+        query: str,
+        *,
+        limit: int = 12,
+    ) -> tuple[SemanticMemoryHit, ...]:
+        client = await self.read_handle.client()
+        raw = await asyncio.to_thread(
+            client.search,
+            query,
+            user_ids=["local"],
+            agent_ids=[],
+            limit=limit,
+            min_score=sdk_relevance_floor(),
+            schema_min_score=sdk_relevance_floor(),
+            schema_limit=limit,
+            intention_limit=limit,
+            scene="normal",
+            reader=HY_MEMORY_READER,
+        )
+        return parse_sdk_search(raw, error_code="hy_memory_search_shape_unknown")
+
+    async def semantic_search(
+        self,
+        query: str,
+        *,
+        limit: int = 12,
+    ) -> tuple[tuple[str, float], ...]:
+        return tuple(
+            (hit.memory_id, hit.score)
+            for hit in await self.semantic_recall(query, limit=limit)
+        )
+
+    async def graph_snapshot(self, *, limit: int = 300) -> HyGraphSnapshot:
+        try:
+            await self.coordinator.active_profile_credentials()
+        except RuntimeError as error:
+            if str(error) == "model_profile_not_active":
+                return HyGraphSnapshot(consulted=False)
+            raise
+        try:
+            client = await self.coordinator.client()
+        except Exception as error:
+            raise HySdkError("hy_memory_graph_read_failed") from error
+        return await asyncio.to_thread(
+            read_graph_snapshot,
+            client,
+            "local",
+            "*",
+            limit,
+        )
+
+    async def digest(self) -> dict[str, object]:
+        client = await self.coordinator.client()
+        try:
+            raw = await asyncio.to_thread(
+                client.list_memories,
+                user_id="local",
+                limit=1_000,
+                offset=0,
+            )
+            agent_ids = _agent_ids(raw)
+            digests = [
+                {
+                    "agent_id": agent_id,
+                    "schemas_recorded": 0,
+                    **dict(
+                        await asyncio.to_thread(
+                            client.evolve,
+                            user_id="local",
+                            agent_id=agent_id,
+                        )
+                    ),
+                }
+                for agent_id in agent_ids
+            ]
+        except Exception as error:
+            raise HySdkError("hy_memory_digest_failed") from error
+        return {
+            "success": all(result.get("success") is not False for result in digests),
+            "scope_count": len(digests),
+            "skipped_conversation_scopes": 0,
+            "schemas_recorded": 0,
+            "digests": digests,
+        }
+
+    async def graph_width_mismatch(self) -> GraphWidthMismatch | None:
+        try:
+            profile, secret = await self.coordinator.active_profile_credentials()
+        except RuntimeError as error:
+            if str(error) == "model_profile_not_active":
+                return None
+            raise
+        return await asyncio.to_thread(
+            inspect_profile_graph_width,
+            profile,
+            api_key=secret,
+            data_dir=self.data_dir,
+        )
+
+    async def decide_graph_width(
+        self,
+        decision: GraphWidthDecision,
+    ) -> GraphWidthOutcome:
+        try:
+            profile, secret = await self.coordinator.active_profile_credentials()
+        except RuntimeError as error:
+            raise HySdkError(str(error)) from error
+        return await asyncio.to_thread(
+            decide_profile_graph_width,
+            profile,
+            api_key=secret,
+            data_dir=self.data_dir,
+            decision=decision,
+        )
+
+    async def _mutation_client(self) -> Any:
+        """The write client, with no-model and keychain refusals as SDK codes.
+
+        Callers record these against the edit or deletion instead of failing
+        the request that already committed it.
+        """
+        try:
+            return await self.coordinator.client()
+        except RuntimeError as error:
+            code = str(error)
+            if not _STABLE_CODE.fullmatch(code):
+                code = "hy_memory_client_unavailable"
+            raise HySdkError(code) from error
+
+    async def update_hy_memory(self, memory_id: str, content: str) -> dict[str, object]:
+        client = await self._mutation_client()
+        try:
+            result = await asyncio.to_thread(client.update, memory_id, content)
+        except Exception as error:
+            raise HySdkError("hy_memory_update_failed") from error
+        if not isinstance(result, dict) or result.get("success") is not True:
+            raise HySdkError("hy_memory_update_rejected")
+        return result
+
+    async def finish_deletion(self, memory_id: str) -> None:
+        client = await self._mutation_client()
+        try:
+            result = await asyncio.to_thread(client.delete, memory_id)
+        except Exception as error:
+            raise HySdkError("hy_memory_delete_failed") from error
+        if not isinstance(result, dict):
+            raise HySdkError("hy_memory_delete_rejected")
+        error_code = result.get("error_code")
+        if error_code or result.get("partial") is True:
+            raise HySdkError(str(error_code or "hy_memory_delete_incomplete"))
+        if result.get("success") is not True and result.get("deleted_count") != 0:
+            raise HySdkError("hy_memory_delete_rejected")
+
+    def extraction_window_settings(self) -> ExtractionSettings:
+        return self.extraction_settings.get()
+
+    def set_extraction_window(
+        self,
+        preset: ExtractionWindowPreset,
+    ) -> ExtractionSettings:
+        return self.extraction_settings.set_window_preset(preset)
+
+    def queue_depths(self) -> QueueDepths:
+        now = datetime.now(UTC)
+        fresh_since = (now - timedelta(days=7)).isoformat()
+        with main_database(self.write_queue.database_path).read() as database:
+            fresh = int(
+                database.execute(
+                    "SELECT COUNT(*) FROM memory_write_jobs "
+                    "WHERE state IN ('pending','waiting_for_model','waiting_for_permission') "
+                    "AND event_at>=?",
+                    (fresh_since,),
+                ).fetchone()[0]
+            )
+            backlog = int(
+                database.execute(
+                    "SELECT COUNT(*) FROM memory_write_jobs "
+                    "WHERE state IN ('pending','waiting_for_model','waiting_for_permission') "
+                    "AND event_at<?",
+                    (fresh_since,),
+                ).fetchone()[0]
+            )
+            recent = tuple(
+                datetime.fromisoformat(str(row[0]))
+                for row in database.execute(
+                    "SELECT updated_at FROM memory_write_jobs WHERE state='completed' "
+                    "ORDER BY updated_at DESC LIMIT 8"
+                )
+            )
+        return QueueDepths(fresh=fresh, backlog=backlog, recent_completions=recent)
+
+    def active_depths(self) -> tuple[int, int]:
+        with main_database(self.write_queue.database_path).read() as database:
+            running = int(
+                database.execute(
+                    "SELECT COUNT(*) FROM memory_write_jobs WHERE state='running'"
+                ).fetchone()[0]
+            )
+        return running, running
+
+    def status_counts(self) -> dict[str, int]:
+        with main_database(self.write_queue.database_path).read() as database:
+            return {
+                str(row[0]): int(row[1])
+                for row in database.execute(
+                    "SELECT state,COUNT(*) FROM memory_write_jobs GROUP BY state"
+                )
+            }
+
+    def manual_batch_progress(self) -> tuple[int, int]:
+        return 0, 0
+
+    def promote_next_manual_batch(self, *, task_ids: tuple[str, ...] | None = None) -> list[Any]:
+        del task_ids
+        return []
+
+    async def reconcile_unknown_outputs(
+        self,
+        *,
+        limit: int = 128,
+        exclude_error_codes: tuple[str, ...] = (),
+    ) -> int:
+        reconciled = 0
+        for job in await asyncio.to_thread(
+            self.write_queue.unknown_jobs,
+            limit=limit,
+            exclude_error_codes=exclude_error_codes,
+        ):
+            receipt = await self.coordinator.inquire_write_outcome(job.request_id)
+            if receipt is None:
+                continue
+            await asyncio.to_thread(
+                self.write_queue.settle,
+                job.job_id,
+                state=COMPLETED,
+                sdk_raw_ids=receipt.raw_ids,
+                sdk_memory_ids=receipt.memory_ids,
+            )
+            reconciled += 1
+        return reconciled
+
+    async def resume_allowed(self) -> None:
+        await asyncio.to_thread(self.write_queue.readmit_parked)
+
+    def accepted_backfill_status(self) -> None:
+        return None
+
+    async def shutdown(self) -> None:
+        if self._raw_retention_task is None:
+            return
+        self._raw_retention_task.cancel()
+        with suppress(asyncio.CancelledError):
+            await self._raw_retention_task
+        self._raw_retention_task = None
+
+
+def _agent_ids(raw: object) -> tuple[str, ...]:
+    if not isinstance(raw, dict):
+        raise HySdkError("hy_memory_list_shape_unknown")
+    found: set[str] = set()
+    for bucket_name, rows_name in (("vdb", "memories"), ("graph", "nodes")):
+        bucket = raw.get(bucket_name)
+        if bucket is None:
+            continue
+        if not isinstance(bucket, dict) or not isinstance(bucket.get(rows_name), list):
+            raise HySdkError("hy_memory_list_shape_unknown")
+        for row in bucket[rows_name]:
+            if isinstance(row, dict):
+                agent_id = row.get("agent_id")
+                if isinstance(agent_id, str) and agent_id:
+                    found.add(agent_id)
+    return tuple(sorted(found))

memhub/apps/core/src/one_memory/sdk_migration_bundle.py

@@ -22,6 +22,12 @@ from one_memory.hy_runtime_config import (
     HY_MEMORY_NAMESPACE,
     HY_MEMORY_VERSION,
 )
+from one_memory.hy_sdk_migration import (
+    restore_data_plane as restore_sdk_data_plane,
+)
+from one_memory.hy_sdk_migration import (
+    snapshot_data_plane,
+)
 
 
 class MigrationBundleRejected(RuntimeError):
@@ -113,11 +119,7 @@ def create_sdk_migration_bundle(
             if not sdk_root.is_dir():
                 raise MigrationBundleRejected("sdk_migration_bundle_sdk_root_invalid")
             if snapshotter is None:
-                try:
-                    from one_memory.hy_memory_bridge import snapshot_hy_data_plane
-                except ImportError as error:
-                    raise MigrationBundleRejected("sdk_migration_snapshot_unavailable") from error
-                snapshotter = snapshot_hy_data_plane
+                snapshotter = snapshot_data_plane
             try:
                 snapshotter(
                     source_root=sdk_root,
@@ -275,11 +277,7 @@ def restore_sdk_migration_bundle(
         raise MigrationBundleRejected("sdk_migration_bundle_manifest_invalid")
 
     if restore_data_plane is None:
-        try:
-            from one_memory.hy_memory_bridge import restore_hy_data_plane
-        except ImportError as error:
-            raise MigrationBundleRejected("sdk_migration_restore_unavailable") from error
-        restore_data_plane = restore_hy_data_plane
+        restore_data_plane = restore_sdk_data_plane
 
     stamp = (now or datetime.now(UTC)).strftime("%Y%m%dT%H%M%S%fZ")
     database_live = home / "one-memory.sqlite3"

memhub/apps/core/src/one_memory/sdk_migration_inventory.py

@@ -1,681 +0,0 @@
-"""Legacy Governance to SDK inventory classification for cutover planning."""
-
-from __future__ import annotations
-
-import hashlib
-import json
-import sqlite3
-from collections.abc import Awaitable, Mapping
-from dataclasses import dataclass
-from datetime import datetime
-from pathlib import Path
-from typing import Protocol
-
-from one_memory.database_access import main_database
-from one_memory.hy_memory_bridge import HySdkMemoryPage, HySdkRawRecord
-from one_memory.sdk_data_plane_migration import SdkDataPlaneMigrationStore
-
-
-def content_hash(content: str) -> str:
-    return hashlib.sha256(content.encode("utf-8")).hexdigest()
-
-
-@dataclass(frozen=True)
-class LegacyMemoryInventory:
-    legacy_memory_id: str
-    canonical_memory_id: str
-    user_id: str
-    agent_id: str
-    session_id: str
-    content: str
-    content_hash: str
-    status: str
-    layer: str | None
-    owner: str | None
-    tags: tuple[str, ...]
-    supersedes: tuple[str, ...]
-    superseded_by: tuple[str, ...]
-
-
-@dataclass(frozen=True)
-class SdkMemoryInventory:
-    memory_id: str
-    user_id: str
-    agent_id: str
-    content_hash: str
-    status: str
-
-
-@dataclass(frozen=True)
-class MemoryMigrationDecision:
-    item_key: str
-    action: str
-    legacy_memory_id: str | None
-    canonical_memory_id: str
-    reason: str
-
-
-@dataclass(frozen=True)
-class LegacyRawInventory:
-    evidence_id: str
-    user_id: str
-    agent_id: str
-    session_id: str
-    source: str
-    content: str
-    content_hash: str
-    memory_at: datetime | None
-    metadata: dict[str, object]
-
-
-@dataclass(frozen=True)
-class SdkRawInventory:
-    raw_id: str
-    user_id: str
-    agent_id: str
-    session_id: str
-    source: str
-    content_hash: str
-
-
-@dataclass(frozen=True)
-class RawMigrationDecision:
-    item_key: str
-    action: str
-    evidence_id: str | None
-    raw_id: str
-    reason: str
-
-
-_KIND_LAYER = {
-    "chat": "l2_fact",
-    "schema": "l6_schema",
-    "intention": "l7_intention",
-}
-
-SDK_LIFECYCLE_STATUSES = (
-    "active",
-    "superseded",
-    "shadow",
-    "negated",
-    "conflicted",
-    "archived",
-)
-
-
-class SdkMigrationBridge(Protocol):
-    def list_sdk_memories(
-        self,
-        *,
-        user_id: str,
-        agent_id: str | None = None,
-        statuses: tuple[str, ...] = ("active",),
-        limit: int = 100,
-        offset: int = 0,
-        order: str = "desc",
-    ) -> Awaitable[HySdkMemoryPage]: ...
-
-    def import_sdk_memory_exact(
-        self,
-        *,
-        content: str,
-        memory_id: str,
-        user_id: str,
-        agent_id: str,
-        session_id: str,
-        layer: str,
-        status: str,
-        owner: str | None,
-        tags: tuple[str, ...],
-        supersedes: tuple[str, ...],
-        superseded_by: tuple[str, ...],
-        is_latest: bool,
-        request_id: str,
-    ) -> Awaitable[dict[str, object]]: ...
-
-    def list_sdk_raw(
-        self,
-        *,
-        user_id: str,
-        agent_id: str | None = None,
-        session_id: str | None = None,
-        source: str | None = None,
-        limit: int = 100,
-        offset: int = 0,
-        order: str = "desc",
-    ) -> Awaitable[tuple[tuple[HySdkRawRecord, ...], int]]: ...
-
-    def import_sdk_raw_exact(
-        self,
-        *,
-        content: str,
-        raw_id: str,
-        user_id: str,
-        agent_id: str,
-        session_id: str,
-        source: str,
-        memory_at: datetime | None,
-        metadata: Mapping[str, object],
-        request_id: str,
-    ) -> Awaitable[dict[str, object]]: ...
-
-    def delete(self, memory_id: str) -> Awaitable[dict[str, object]]: ...
-
-
-def read_legacy_memory_inventory(database_path: Path) -> tuple[LegacyMemoryInventory, ...]:
-    """Read legacy memory intent without constructing a Governance writer."""
-    with main_database(database_path).read(row_factory=sqlite3.Row) as connection:
-        mappings: dict[str, tuple[str, str | None, str | None]] = {}
-        for row in connection.execute("SELECT body FROM memory_task_outputs"):
-            try:
-                body = json.loads(row["body"])
-            except (TypeError, ValueError):
-                continue
-            legacy_id = body.get("memory_id")
-            sdk_id = body.get("hy_memory_id")
-            if isinstance(legacy_id, str) and isinstance(sdk_id, str):
-                mappings[legacy_id] = (
-                    sdk_id,
-                    body.get("layer") if isinstance(body.get("layer"), str) else None,
-                    body.get("owner") if body.get("owner") in {"user", "agent"} else None,
-                )
-
-        supersedes: dict[str, list[str]] = {}
-        superseded_by: dict[str, list[str]] = {}
-        for row in connection.execute(
-            "SELECT memory_id,superseded_by FROM governed_memory_supersessions "
-            "ORDER BY memory_id,superseded_by"
-        ):
-            older, newer = str(row["memory_id"]), str(row["superseded_by"])
-            superseded_by.setdefault(older, []).append(newer)
-            supersedes.setdefault(newer, []).append(older)
-
-        records: list[LegacyMemoryInventory] = []
-        for row in connection.execute(
-            "SELECT memory_id,user_id,kind,status,content,tags_json "
-            "FROM governed_memories ORDER BY memory_id"
-        ):
-            legacy_id = str(row["memory_id"])
-            mapped_id, mapped_layer, mapped_owner = mappings.get(legacy_id, (legacy_id, None, None))
-            try:
-                tags_value = json.loads(row["tags_json"] or "[]")
-            except (TypeError, ValueError):
-                tags_value = []
-            tags = tuple(value for value in tags_value if isinstance(value, str))
-            kind = str(row["kind"])
-            status = str(row["status"])
-            records.append(
-                LegacyMemoryInventory(
-                    legacy_memory_id=legacy_id,
-                    canonical_memory_id=mapped_id,
-                    user_id=str(row["user_id"]),
-                    agent_id="one-memory",
-                    session_id="default_session",
-                    content=str(row["content"]),
-                    content_hash=content_hash(str(row["content"])),
-                    status=(
-                        "active"
-                        if status == "confirmed"
-                        else "superseded"
-                        if status == "superseded"
-                        else "deleted"
-                    ),
-                    layer=mapped_layer or _KIND_LAYER.get(kind),
-                    owner=mapped_owner,
-                    tags=tags,
-                    supersedes=tuple(supersedes.get(legacy_id, ())),
-                    superseded_by=tuple(superseded_by.get(legacy_id, ())),
-                )
-            )
-    return tuple(records)
-
-
-def _parse_datetime(value: object) -> datetime | None:
-    if not isinstance(value, str) or not value:
-        return None
-    try:
-        return datetime.fromisoformat(value)
-    except ValueError:
-        return None
-
-
-def read_legacy_raw_inventory(database_path: Path) -> tuple[LegacyRawInventory, ...]:
-    with main_database(database_path).read(row_factory=sqlite3.Row) as connection:
-        rows = connection.execute(
-            """
-            SELECT evidence_id,user_id,source_id,workspace,project,
-                   conversation_id,message_id,source_ordinal,content_kind,
-                   role,written_by,created_at,imported_at,verbatim,content_hash,revision
-            FROM evidence ORDER BY COALESCE(ingest_seq,rowid),evidence_id
-            """
-        ).fetchall()
-    return tuple(
-        LegacyRawInventory(
-            evidence_id=str(row["evidence_id"]),
-            user_id=str(row["user_id"]),
-            agent_id="one-memory",
-            session_id=str(row["conversation_id"]),
-            source=str(row["source_id"]),
-            content=str(row["verbatim"]),
-            content_hash=str(row["content_hash"]),
-            memory_at=_parse_datetime(row["created_at"] or row["imported_at"]),
-            metadata={
-                key: row[key]
-                for key in (
-                    "workspace",
-                    "project",
-                    "message_id",
-                    "source_ordinal",
-                    "content_kind",
-                    "role",
-                    "written_by",
-                    "revision",
-                )
-                if row[key] is not None
-            },
-        )
-        for row in rows
-    )
-
-
-def classify_memory_inventory(
-    legacy: tuple[LegacyMemoryInventory, ...],
-    sdk: tuple[SdkMemoryInventory, ...],
-) -> tuple[MemoryMigrationDecision, ...]:
-    """Classify every legacy and SDK memory; no item may silently disappear."""
-    sdk_by_id = {item.memory_id: item for item in sdk}
-    claimed_sdk_ids: set[str] = set()
-    decisions: list[MemoryMigrationDecision] = []
-    for item in legacy:
-        current = sdk_by_id.get(item.canonical_memory_id)
-        if current is not None:
-            claimed_sdk_ids.add(current.memory_id)
-        if item.status == "deleted":
-            decisions.append(
-                MemoryMigrationDecision(
-                    item_key=f"memory:{item.legacy_memory_id}",
-                    action="delete" if current is not None else "verified_deleted",
-                    legacy_memory_id=item.legacy_memory_id,
-                    canonical_memory_id=item.canonical_memory_id,
-                    reason="legacy_user_delete",
-                )
-            )
-            continue
-        if item.layer is None:
-            decisions.append(
-                MemoryMigrationDecision(
-                    item_key=f"memory:{item.legacy_memory_id}",
-                    action="quarantine",
-                    legacy_memory_id=item.legacy_memory_id,
-                    canonical_memory_id=item.canonical_memory_id,
-                    reason="legacy_layer_unknown",
-                )
-            )
-            continue
-        if current is None:
-            decisions.append(
-                MemoryMigrationDecision(
-                    item_key=f"memory:{item.legacy_memory_id}",
-                    action="import_exact",
-                    legacy_memory_id=item.legacy_memory_id,
-                    canonical_memory_id=item.canonical_memory_id,
-                    reason="governance_only",
-                )
-            )
-            continue
-        if (
-            current.content_hash == item.content_hash
-            and current.status == item.status
-            and current.user_id == item.user_id
-            and current.agent_id == item.agent_id
-        ):
-            action = (
-                "matched" if item.legacy_memory_id == item.canonical_memory_id else "record_alias"
-            )
-            reason = "same_identity" if action == "matched" else "mapped_identity"
-        else:
-            action, reason = "quarantine", "content_owner_or_lifecycle_diverged"
-        decisions.append(
-            MemoryMigrationDecision(
-                item_key=f"memory:{item.legacy_memory_id}",
-                action=action,
-                legacy_memory_id=item.legacy_memory_id,
-                canonical_memory_id=item.canonical_memory_id,
-                reason=reason,
-            )
-        )
-
-    for sdk_item in sdk:
-        if sdk_item.memory_id not in claimed_sdk_ids:
-            decisions.append(
-                MemoryMigrationDecision(
-                    item_key=f"sdk:{sdk_item.memory_id}",
-                    action="adopt",
-                    legacy_memory_id=None,
-                    canonical_memory_id=sdk_item.memory_id,
-                    reason="sdk_only",
-                )
-            )
-    return tuple(decisions)
-
-
-async def read_sdk_memory_inventory(
-    bridge: SdkMigrationBridge,
-    *,
-    user_id: str = "local",
-    page_size: int = 500,
-) -> tuple[SdkMemoryInventory, ...]:
-    records: dict[str, SdkMemoryInventory] = {}
-    offset = 0
-    while True:
-        page = await bridge.list_sdk_memories(
-            user_id=user_id,
-            statuses=SDK_LIFECYCLE_STATUSES,
-            limit=page_size,
-            offset=offset,
-            order="asc",
-        )
-        for memory in page.memories:
-            records[memory.memory_id] = SdkMemoryInventory(
-                memory_id=memory.memory_id,
-                user_id=memory.user_id,
-                agent_id=memory.agent_id,
-                content_hash=content_hash(memory.content),
-                status=memory.status,
-            )
-        consumed = len(page.memories)
-        offset += consumed
-        if consumed == 0 or offset >= page.total:
-            break
-    return tuple(records[key] for key in sorted(records))
-
-
-def record_memory_migration_plan(
-    store: SdkDataPlaneMigrationStore,
-    *,
-    migration_id: str,
-    legacy: tuple[LegacyMemoryInventory, ...],
-    sdk: tuple[SdkMemoryInventory, ...],
-) -> tuple[MemoryMigrationDecision, ...]:
-    decisions = classify_memory_inventory(legacy, sdk)
-    legacy_by_id = {item.legacy_memory_id: item for item in legacy}
-    sdk_by_id = {item.memory_id: item for item in sdk}
-    for decision in decisions:
-        source_hash = (
-            legacy_by_id[decision.legacy_memory_id].content_hash
-            if decision.legacy_memory_id is not None
-            else sdk_by_id[decision.canonical_memory_id].content_hash
-        )
-        operation_key = hashlib.sha256(
-            f"{migration_id}:{decision.item_key}:{decision.action}:{source_hash}".encode()
-        ).hexdigest()
-        store.upsert_item(
-            migration_id=migration_id,
-            item_key=decision.item_key,
-            source_kind=f"memory:{decision.action}",
-            source_id=(
-                decision.legacy_memory_id
-                if decision.legacy_memory_id is not None
-                else decision.canonical_memory_id
-            ),
-            source_hash=source_hash,
-            operation_key=operation_key,
-        )
-    return decisions
-
-
-async def execute_memory_migration_plan(
-    bridge: SdkMigrationBridge,
-    store: SdkDataPlaneMigrationStore,
-    *,
-    migration_id: str,
-    legacy: tuple[LegacyMemoryInventory, ...],
-    decisions: tuple[MemoryMigrationDecision, ...],
-) -> None:
-    legacy_by_id = {item.legacy_memory_id: item for item in legacy}
-    for decision in decisions:
-        item = (
-            legacy_by_id.get(decision.legacy_memory_id)
-            if decision.legacy_memory_id is not None
-            else None
-        )
-        if decision.action == "quarantine":
-            store.settle_item(
-                migration_id=migration_id,
-                item_key=decision.item_key,
-                state="quarantine",
-                error_code=decision.reason,
-            )
-            continue
-        if decision.action == "adopt":
-            store.settle_item(
-                migration_id=migration_id,
-                item_key=decision.item_key,
-                state="adopted",
-                target_memory_id=decision.canonical_memory_id,
-            )
-            continue
-        if decision.action in {"delete", "verified_deleted"}:
-            if decision.action == "delete":
-                await bridge.delete(decision.canonical_memory_id)
-            store.settle_item(
-                migration_id=migration_id,
-                item_key=decision.item_key,
-                state="deleted",
-                target_memory_id=decision.canonical_memory_id,
-            )
-            continue
-        if item is None:
-            raise RuntimeError(f"missing legacy inventory for {decision.item_key}")
-        if decision.action == "import_exact":
-            assert item.layer is not None
-            request_id = hashlib.sha256(
-                f"{migration_id}:{decision.item_key}:import_exact".encode()
-            ).hexdigest()
-            await bridge.import_sdk_memory_exact(
-                content=item.content,
-                memory_id=item.canonical_memory_id,
-                user_id=item.user_id,
-                agent_id=item.agent_id,
-                session_id=item.session_id,
-                layer=item.layer,
-                status=item.status,
-                owner=item.owner,
-                tags=item.tags,
-                supersedes=item.supersedes,
-                superseded_by=item.superseded_by,
-                is_latest=item.status == "active",
-                request_id=request_id,
-            )
-        if decision.action == "record_alias":
-            store.record_alias(
-                migration_id=migration_id,
-                legacy_memory_id=item.legacy_memory_id,
-                canonical_memory_id=item.canonical_memory_id,
-            )
-        store.settle_item(
-            migration_id=migration_id,
-            item_key=decision.item_key,
-            state="verified",
-            target_memory_id=decision.canonical_memory_id,
-            target_hash=item.content_hash,
-        )
-
-
-async def read_sdk_raw_inventory(
-    bridge: SdkMigrationBridge,
-    *,
-    user_id: str = "local",
-    page_size: int = 500,
-) -> tuple[SdkRawInventory, ...]:
-    records: dict[str, SdkRawInventory] = {}
-    offset = 0
-    while True:
-        rows, total = await bridge.list_sdk_raw(
-            user_id=user_id,
-            limit=page_size,
-            offset=offset,
-            order="asc",
-        )
-        for raw in rows:
-            raw_id = str(raw.raw_id)
-            records[raw_id] = SdkRawInventory(
-                raw_id=raw_id,
-                user_id=str(raw.user_id),
-                agent_id=str(raw.agent_id),
-                session_id=str(raw.session_id),
-                source=str(raw.source),
-                content_hash=content_hash(str(raw.content)),
-            )
-        consumed = len(rows)
-        offset += consumed
-        if consumed == 0 or offset >= total:
-            break
-    return tuple(records[key] for key in sorted(records))
-
-
-def classify_raw_inventory(
-    legacy: tuple[LegacyRawInventory, ...],
-    sdk: tuple[SdkRawInventory, ...],
-) -> tuple[RawMigrationDecision, ...]:
-    sdk_by_identity: dict[tuple[str, str, str, str, str], list[SdkRawInventory]] = {}
-    for raw in sdk:
-        identity = (
-            raw.user_id,
-            raw.agent_id,
-            raw.session_id,
-            raw.source,
-            raw.content_hash,
-        )
-        sdk_by_identity.setdefault(identity, []).append(raw)
-    claimed: set[str] = set()
-    decisions: list[RawMigrationDecision] = []
-    for item in legacy:
-        identity = (
-            item.user_id,
-            item.agent_id,
-            item.session_id,
-            item.source,
-            item.content_hash,
-        )
-        candidates = sorted(
-            sdk_by_identity.get(identity, ()),
-            key=lambda candidate: candidate.raw_id,
-        )
-        existing = next(
-            (candidate for candidate in candidates if candidate.raw_id not in claimed),
-            None,
-        )
-        if existing is not None:
-            claimed.add(existing.raw_id)
-            decisions.append(
-                RawMigrationDecision(
-                    item_key=f"raw:{item.evidence_id}",
-                    action="matched",
-                    evidence_id=item.evidence_id,
-                    raw_id=existing.raw_id,
-                    reason="same_content_and_scope",
-                )
-            )
-        else:
-            decisions.append(
-                RawMigrationDecision(
-                    item_key=f"raw:{item.evidence_id}",
-                    action="import_exact",
-                    evidence_id=item.evidence_id,
-                    raw_id=f"evidence:{item.evidence_id}",
-                    reason="evidence_only",
-                )
-            )
-    for raw in sdk:
-        if raw.raw_id not in claimed:
-            decisions.append(
-                RawMigrationDecision(
-                    item_key=f"sdk-raw:{raw.raw_id}",
-                    action="adopt",
-                    evidence_id=None,
-                    raw_id=raw.raw_id,
-                    reason="sdk_only",
-                )
-            )
-    return tuple(decisions)
-
-
-def record_raw_migration_plan(
-    store: SdkDataPlaneMigrationStore,
-    *,
-    migration_id: str,
-    legacy: tuple[LegacyRawInventory, ...],
-    sdk: tuple[SdkRawInventory, ...],
-) -> tuple[RawMigrationDecision, ...]:
-    decisions = classify_raw_inventory(legacy, sdk)
-    legacy_by_id = {item.evidence_id: item for item in legacy}
-    sdk_by_id = {item.raw_id: item for item in sdk}
-    for decision in decisions:
-        source_hash = (
-            legacy_by_id[decision.evidence_id].content_hash
-            if decision.evidence_id is not None
-            else sdk_by_id[decision.raw_id].content_hash
-        )
-        operation_key = hashlib.sha256(
-            f"{migration_id}:{decision.item_key}:{decision.action}:{source_hash}".encode()
-        ).hexdigest()
-        store.upsert_item(
-            migration_id=migration_id,
-            item_key=decision.item_key,
-            source_kind=f"raw:{decision.action}",
-            source_id=decision.evidence_id or decision.raw_id,
-            source_hash=source_hash,
-            operation_key=operation_key,
-        )
-    return decisions
-
-
-async def execute_raw_migration_plan(
-    bridge: SdkMigrationBridge,
-    store: SdkDataPlaneMigrationStore,
-    *,
-    migration_id: str,
-    legacy: tuple[LegacyRawInventory, ...],
-    decisions: tuple[RawMigrationDecision, ...],
-) -> None:
-    legacy_by_id = {item.evidence_id: item for item in legacy}
-    for decision in decisions:
-        if decision.action == "adopt":
-            store.settle_item(
-                migration_id=migration_id,
-                item_key=decision.item_key,
-                state="adopted",
-                target_memory_id=decision.raw_id,
-            )
-            continue
-        item = legacy_by_id.get(decision.evidence_id or "")
-        if item is None:
-            raise RuntimeError(f"missing legacy raw inventory for {decision.item_key}")
-        if decision.action == "import_exact":
-            request_id = hashlib.sha256(
-                f"{migration_id}:{decision.item_key}:import_raw".encode()
-            ).hexdigest()
-            await bridge.import_sdk_raw_exact(
-                content=item.content,
-                raw_id=decision.raw_id,
-                user_id=item.user_id,
-                agent_id=item.agent_id,
-                session_id=item.session_id,
-                source=item.source,
-                memory_at=item.memory_at,
-                metadata=item.metadata,
-                request_id=request_id,
-            )
-        store.settle_item(
-            migration_id=migration_id,
-            item_key=decision.item_key,
-            state="verified",
-            target_memory_id=decision.raw_id,
-            target_hash=item.content_hash,
-        )
-        store.record_raw_alias(
-            migration_id=migration_id,
-            legacy_evidence_id=item.evidence_id,
-            canonical_raw_id=decision.raw_id,
-        )

memhub/apps/core/src/one_memory/sdk_migration_runner.py

@@ -1,74 +1,153 @@
-"""Offline, resumable backfill runner for SDK memory authority."""
+"""Offline startup cutover from the legacy Governance mirror to SDK authority.
+
+Every released build wrote memory and raw turns through the SDK first; the
+Evidence/Governance tables only mirrored them. The cutover therefore copies no
+memory. It makes the surviving write queue independent of the legacy tables,
+commits SDK authority and drops those tables.
+"""
 
 from __future__ import annotations
 
+import asyncio
 import hashlib
 import json
+import os
+import shutil
 import sqlite3
+from collections.abc import Callable, Iterator
+from contextlib import AbstractContextManager, contextmanager
 from dataclasses import dataclass
 from pathlib import Path
+from typing import Any
 
 from one_memory.database_access import main_database
+from one_memory.imports.models import EvidenceRecord
 from one_memory.imports.storage import EvidenceStore
 from one_memory.memory_server.contracts import (
     BatchIdentity,
     WriteRequest,
     internal_session_id_for,
 )
-from one_memory.memory_server.durable_queue import DurableWriteQueue
+from one_memory.memory_server.durable_queue import (
+    CANCELLED,
+    COMPLETED,
+    FAILED,
+    REAPED_PAYLOAD_MARKER,
+    DurableWriteQueue,
+)
+from one_memory.memory_server.unified_entry import UnifiedWriteIntake
+from one_memory.model_profiles import ModelProfileSnapshot
 from one_memory.sdk_data_plane_migration import (
     CutoverMarker,
     DataPlaneMigrationError,
-    MigrationProgress,
+    MemoryAuthority,
     SdkDataPlaneMigrationStore,
 )
 from one_memory.sdk_migration_bundle import inspect_sdk_migration_bundle
-from one_memory.sdk_migration_inventory import (
-    SdkMigrationBridge,
-    execute_memory_migration_plan,
-    execute_raw_migration_plan,
-    read_legacy_memory_inventory,
-    read_legacy_raw_inventory,
-    read_sdk_memory_inventory,
-    read_sdk_raw_inventory,
-    record_memory_migration_plan,
-    record_raw_migration_plan,
+from one_memory.sdk_migration_verify import (
+    MemoryReadbackFailed,
+    ReadbackReport,
+    legacy_memory_ids_from,
+    legacy_memory_ids_from_bundle,
+    read_active_profile,
+    runtime_sdk_client,
+    verify_readback,
 )
-from one_memory.target_schema import drop_legacy_tables
+from one_memory.sdk_vector_reattach import (
+    ReattachedCollection,
+    legacy_vector_collections,
+    reattach_legacy_vectors,
+)
+from one_memory.target_schema import drop_legacy_tables, legacy_tables_present
+
+PREPARED_PHASE = "prepared"
 
 
 @dataclass(frozen=True)
-class BackfillResult:
-    migration_id: str
-    state: str
-    phase: str
-    counts: dict[str, int]
-    memory_items: int
-    raw_items: int
+class CutoverPreparation:
+    """What the cutover did to legacy work, for the receipt and the dialog."""
+
+    frozen_jobs: int
+    moved_tasks: int
+    moved_dispatches: int
+    abandoned: dict[str, int]
+
+    def as_document(self) -> dict[str, object]:
+        return {
+            "frozen_jobs": self.frozen_jobs,
+            "moved_tasks": self.moved_tasks,
+            "moved_dispatches": self.moved_dispatches,
+            "abandoned": dict(self.abandoned),
+        }
+
+
+def _tables(database: sqlite3.Connection) -> set[str]:
+    return {
+        str(row[0])
+        for row in database.execute("SELECT name FROM sqlite_master WHERE type='table'")
+    }
+
+
+def recover_abandoned_staging(*, data_dir: Path, database_path: Path, namespace: str) -> None:
+    """Undo or discard the SDK staging copy that builds through 19 created.
+
+    Those builds staged a full SDK copy and swapped directories at cutover. The
+    live root was untouched until the swap, so before SDK authority the
+    protective root is the original store; after it, the protective root is
+    stale.
+    """
+    home = data_dir.resolve()
+    live_root = home / "hy-memory" / namespace
+    authority = SdkDataPlaneMigrationStore(database_path).authority()
+    protective_roots = sorted(live_root.parent.glob(f".{namespace}.pre-migration-*"))
+    if authority.value != "sdk" and len(protective_roots) > 1:
+        raise DataPlaneMigrationError("sdk_migration_swap_state_ambiguous")
+    for protective_root in protective_roots:
+        if authority.value == "sdk":
+            if not live_root.is_dir():
+                raise DataPlaneMigrationError("sdk_migration_committed_store_missing")
+            shutil.rmtree(protective_root)
+            continue
+        if live_root.exists():
+            shutil.rmtree(live_root)
+        protective_root.replace(live_root)
+    shutil.rmtree(home / ".sdk-migration", ignore_errors=True)
+
+
+def abandoned_staging_present(*, data_dir: Path, namespace: str) -> bool:
+    home = data_dir.resolve()
+    return (home / ".sdk-migration").exists() or any(
+        (home / "hy-memory").glob(f".{namespace}.pre-migration-*")
+    )
 
 
 def prepare_legacy_write_queue_for_cutover(database_path: Path) -> tuple[int, int]:
     """Freeze old jobs and move unstarted task rows into the surviving queue."""
     with main_database(database_path).read(row_factory=sqlite3.Row) as database:
-        tables = {
-            str(row["name"])
-            for row in database.execute("SELECT name FROM sqlite_master WHERE type='table'")
-        }
+        tables = _tables(database)
         if "evidence" not in tables:
             return (0, 0)
         jobs = (
-            tuple(database.execute("SELECT job_id,payload_json FROM memory_write_jobs"))
+            tuple(
+                database.execute("SELECT job_id,state,payload_json FROM memory_write_jobs")
+            )
             if "memory_write_jobs" in tables
             else ()
         )
     evidence = EvidenceStore(database_path)
     frozen = 0
     for row in jobs:
-        request = WriteRequest.model_validate_json(str(row["payload_json"]))
+        decoded = json.loads(str(row["payload_json"]))
+        if isinstance(decoded, dict) and REAPED_PAYLOAD_MARKER in decoded:
+            # A finished job whose input the queue already released.
+            continue
+        request = WriteRequest.model_validate(decoded)
         if request.input_records:
             continue
         records = tuple(evidence.get(identifier) for identifier in request.evidence_ids)
         if any(record is None for record in records):
+            if str(row["state"]) in {COMPLETED, FAILED, CANCELLED}:
+                continue
             raise DataPlaneMigrationError(f"sdk_data_plane_write_job_input_missing:{row['job_id']}")
         materialized = tuple(record for record in records if record is not None)
         updated = request.model_copy(update={"input_records": materialized})
@@ -141,30 +220,55 @@ def prepare_legacy_write_queue_for_cutover(database_path: Path) -> tuple[int, in
     return (frozen, moved)
 
 
-def sdk_cutover_blockers(database_path: Path) -> dict[str, int]:
-    """Count legacy work that could write or duplicate data after cutover."""
-    blockers: dict[str, int] = {}
+def move_pending_dispatches(database_path: Path) -> int:
+    """Queue Evidence that was waiting for its source file to finish.
+
+    Its source cursor has already moved past these rows, so History catch-up
+    after the cutover would not read them again.
+    """
     with main_database(database_path).read(row_factory=sqlite3.Row) as database:
-        tables = {
-            str(row["name"])
-            for row in database.execute("SELECT name FROM sqlite_master WHERE type='table'")
-        }
+        if not {"evidence", "evidence_pending_memory_dispatch"} <= _tables(database):
+            return 0
+        identifiers = tuple(
+            str(row["evidence_id"])
+            for row in database.execute(
+                "SELECT DISTINCT pending.evidence_id "
+                "FROM evidence_pending_memory_dispatch pending "
+                "JOIN evidence USING(evidence_id) "
+                "ORDER BY COALESCE(evidence.created_at,evidence.imported_at),"
+                "COALESCE(evidence.ingest_seq,evidence.rowid),pending.evidence_id"
+            )
+        )
+    evidence = EvidenceStore(database_path)
+    conversations: dict[tuple[str, str, str, str | None], list[EvidenceRecord]] = {}
+    for identifier in identifiers:
+        record = evidence.get(identifier)
+        if record is None:
+            continue
+        key = (record.user_id, record.source_id, record.conversation_id, record.workspace)
+        conversations.setdefault(key, []).append(record)
+    intake = UnifiedWriteIntake(DurableWriteQueue(database_path))
+    for records in conversations.values():
+        intake.submit_records(tuple(records))
+    return len(conversations)
+
+
+def abandoned_legacy_work(database_path: Path) -> dict[str, int]:
+    """Count legacy work that ends with the cutover instead of blocking it.
+
+    A task that may have reached the SDK is not replayed: the SDK already holds
+    whatever it committed. Pending user deletions are not carried over.
+    """
+    abandoned: dict[str, int] = {}
+    with main_database(database_path).read(row_factory=sqlite3.Row) as database:
+        tables = _tables(database)
         checks = (
             (
-                "memory_tasks",
+                "invoked_tasks",
                 "memory_tasks",
                 "status='unknown' OR (status='running' AND hy_invocation_count > 0)",
             ),
-            (
-                "deletions",
-                "memory_deletions",
-                "status IN ('pending','running')",
-            ),
-            (
-                "pending_dispatch",
-                "evidence_pending_memory_dispatch",
-                "1=1",
-            ),
+            ("deletions", "memory_deletions", "status IN ('pending','running')"),
         )
         for name, table, where in checks:
             if table not in tables:
@@ -173,11 +277,23 @@ def sdk_cutover_blockers(database_path: Path) -> dict[str, int]:
                 database.execute(f"SELECT COUNT(*) FROM {table} WHERE {where}").fetchone()[0]
             )
             if count:
-                blockers[name] = count
-    return blockers
+                abandoned[name] = count
+    return abandoned
 
 
-def commit_reconciled_cutover(
+def prepare_legacy_work_for_cutover(database_path: Path) -> CutoverPreparation:
+    """Make the surviving queue self-contained. Safe to repeat."""
+    frozen, moved = prepare_legacy_write_queue_for_cutover(database_path)
+    dispatches = move_pending_dispatches(database_path)
+    return CutoverPreparation(
+        frozen_jobs=frozen,
+        moved_tasks=moved,
+        moved_dispatches=dispatches,
+        abandoned=abandoned_legacy_work(database_path),
+    )
+
+
+def commit_sdk_cutover(
     *,
     data_dir: Path,
     database_path: Path,
@@ -186,39 +302,25 @@ def commit_reconciled_cutover(
     sdk_version: str,
     store_version: str = "1",
 ) -> CutoverMarker:
-    """Verify backup/work quiescence and atomically commit SDK authority."""
+    """Verify the backup, settle legacy work and commit SDK authority."""
     inspect_sdk_migration_bundle(data_dir=data_dir, bundle_id=bundle_id)
-    prepare_legacy_write_queue_for_cutover(database_path)
-    blockers = sdk_cutover_blockers(database_path)
-    if blockers:
-        raise DataPlaneMigrationError(
-            "sdk_data_plane_cutover_work_pending:"
-            + json.dumps(blockers, sort_keys=True, separators=(",", ":"))
-        )
-    with main_database(database_path).read(row_factory=sqlite3.Row) as database:
-        rows = database.execute(
-            "SELECT item_key,state,target_memory_id,target_hash "
-            "FROM sdk_data_plane_migration_items WHERE migration_id=? "
-            "ORDER BY item_key",
-            (migration_id,),
-        ).fetchall()
+    # Repeated here because a CLI-managed Runtime may have run on legacy
+    # authority since `sdk-migrate`.
+    preparation = prepare_legacy_work_for_cutover(database_path)
     digest = hashlib.sha256(
         json.dumps(
-            [dict(row) for row in rows],
+            preparation.as_document(),
             sort_keys=True,
             separators=(",", ":"),
         ).encode()
     ).hexdigest()
-    marker = SdkDataPlaneMigrationStore(database_path).commit_cutover(
+    return SdkDataPlaneMigrationStore(database_path).commit_cutover(
         migration_id=migration_id,
         sdk_version=sdk_version,
         store_version=store_version,
         backup_bundle_id=bundle_id,
         reconciliation_digest=digest,
     )
-    inspect_sdk_migration_bundle(data_dir=data_dir, bundle_id=marker.backup_bundle_id)
-    drop_legacy_tables(database_path)
-    return marker
 
 
 def resume_legacy_cleanup(*, data_dir: Path, database_path: Path) -> tuple[str, ...]:
@@ -232,77 +334,255 @@ def resume_legacy_cleanup(*, data_dir: Path, database_path: Path) -> tuple[str,
             data_dir=data_dir,
             bundle_id=marker.backup_bundle_id,
         )
+    from one_memory.hy_runtime_config import HY_MEMORY_NAMESPACE
+
+    recover_abandoned_staging(
+        data_dir=data_dir,
+        database_path=database_path,
+        namespace=HY_MEMORY_NAMESPACE,
+    )
     return drop_legacy_tables(database_path)
 
 
-async def run_sdk_data_plane_backfill(
+MIGRATION_PREFIX = "sdk-authority-v2"
+CAPTURE_FENCE = ".sdk-migration-capture-fence.json"
+
+
+@dataclass(frozen=True)
+class StartupMigrationResult:
+    """`not_required`, `cutover` (this run switched authority) or `complete`."""
+
+    state: str
+    migration_id: str | None = None
+    bundle_id: str | None = None
+    preparation: CutoverPreparation | None = None
+    dropped_tables: tuple[str, ...] = ()
+    reattached: tuple[ReattachedCollection, ...] = ()
+    readback: ReadbackReport | None = None
+
+    def as_document(self) -> dict[str, object]:
+        document: dict[str, object] = {"schema_version": 1, "state": self.state}
+        if self.migration_id is not None:
+            document["migration_id"] = self.migration_id
+            document["bundle_id"] = self.bundle_id
+        if self.preparation is not None:
+            document.update(self.preparation.as_document())
+        if self.reattached:
+            document["reattached_vectors"] = [
+                {
+                    "collection": item.current,
+                    "legacy_count": item.legacy_count,
+                    "merged_count": item.merged_count,
+                    "final_count": item.final_count,
+                }
+                for item in self.reattached
+            ]
+        if self.readback is not None:
+            document["readback"] = self.readback.as_document()
+        if self.dropped_tables:
+            document["dropped_tables"] = list(self.dropped_tables)
+        return document
+
+
+def database_generation(database_path: Path) -> str:
+    stat = database_path.stat()
+    return hashlib.sha256(f"{stat.st_size}:{stat.st_mtime_ns}".encode()).hexdigest()[:16]
+
+
+def _resumable_identity(database_path: Path) -> tuple[str, str]:
+    """Resume the newest unfinished run, including one build 19 left blocked."""
+    with main_database(database_path).read(row_factory=sqlite3.Row) as database:
+        row = (
+            database.execute(
+                "SELECT migration_id,backup_bundle_id FROM sdk_data_plane_migrations "
+                "WHERE state IN ('running','blocked','ready') AND migration_id LIKE ? "
+                "ORDER BY created_at DESC LIMIT 1",
+                (f"{MIGRATION_PREFIX}-%",),
+            ).fetchone()
+            if "sdk_data_plane_migrations" in _tables(database)
+            else None
+        )
+    if row is not None and row["backup_bundle_id"] == f"{row['migration_id']}-pre-cutover":
+        return str(row["migration_id"]), str(row["backup_bundle_id"])
+    migration_id = f"{MIGRATION_PREFIX}-{database_generation(database_path)}"
+    return migration_id, f"{migration_id}-pre-cutover"
+
+
+@contextmanager
+def capture_fence(data_dir: Path) -> Iterator[None]:
+    """Host capture hooks return silently while this process is alive."""
+    marker = data_dir / CAPTURE_FENCE
+    temporary = marker.with_name(f"{marker.name}.{os.getpid()}.tmp")
+    temporary.write_text(
+        json.dumps({"schema_version": 1, "pid": os.getpid()}) + "\n",
+        encoding="utf-8",
+    )
+    os.chmod(temporary, 0o600)
+    temporary.replace(marker)
+    try:
+        yield
+    finally:
+        marker.unlink(missing_ok=True)
+        temporary.unlink(missing_ok=True)
+
+
+def _ensure_bundle(*, data_dir: Path, bundle_id: str, runtime_build: str) -> dict[str, object]:
+    from one_memory.sdk_migration_bundle import (
+        MigrationBundleRejected,
+        create_sdk_migration_bundle,
+    )
+
+    try:
+        create_sdk_migration_bundle(
+            data_dir=data_dir,
+            bundle_id=bundle_id,
+            runtime_build=runtime_build,
+            runtime_stopped=True,
+        )
+    except MigrationBundleRejected as error:
+        if error.code != "sdk_migration_bundle_exists":
+            raise
+    return inspect_sdk_migration_bundle(data_dir=data_dir, bundle_id=bundle_id)
+
+
+def _vacuum(database_path: Path) -> None:
+    """Return the dropped tables' pages to the disk; a full disk only skips it."""
+    try:
+        with main_database(database_path).write(autocommit=True) as database:
+            database.execute("VACUUM")
+    except sqlite3.OperationalError:
+        return
+
+
+def run_startup_migration(
     *,
-    database_path: Path,
-    bridge: SdkMigrationBridge,
-    migration_id: str,
-    bundle_id: str,
-    source_runtime_build: str,
-    target_sdk_version: str,
-    source_schema_version: int,
-    namespace_fingerprint: str,
-    backend_fingerprint: str,
-) -> BackfillResult:
-    """Backfill both lanes but never commit the authority cutover marker."""
+    data_dir: Path,
+    runtime_build: str,
+    sdk_version: str,
+    namespace: str,
+    progress: Callable[[str], None] = lambda _step: None,
+    open_client: Callable[[ModelProfileSnapshot], AbstractContextManager[Any]] | None = None,
+) -> StartupMigrationResult:
+    """Bring one stopped home to SDK authority. Safe to repeat after any failure.
+
+    The caller must stop Runtime and keep its service from restarting.
+    `open_client` yields a public SDK client configured as the Runtime will be;
+    tests replace it.
+    """
+    database_path = data_dir / "one-memory.sqlite3"
+    if not database_path.is_file():
+        return StartupMigrationResult(state="not_required")
+
+    def client_for(profile: ModelProfileSnapshot) -> AbstractContextManager[Any]:
+        if open_client is not None:
+            return open_client(profile)
+        return runtime_sdk_client(database_path, data_dir, profile)
+
     store = SdkDataPlaneMigrationStore(database_path)
-    store.start(
-        migration_id=migration_id,
-        source_runtime_build=source_runtime_build,
-        target_sdk_version=target_sdk_version,
-        source_schema_version=source_schema_version,
-        namespace_fingerprint=namespace_fingerprint,
-        backend_fingerprint=backend_fingerprint,
-        backup_bundle_id=bundle_id,
-    )
-    store.set_phase(migration_id, phase="inventory", state="running")
+    authority, authority_migration = store.authority_state()
+    profile = read_active_profile(database_path)
+    if authority is MemoryAuthority.SDK:
+        vectors_waiting = profile is not None and bool(legacy_vector_collections(data_dir, profile))
+        leftovers = bool(legacy_tables_present(database_path)) or abandoned_staging_present(
+            data_dir=data_dir, namespace=namespace
+        )
+        if not vectors_waiting and not leftovers:
+            return StartupMigrationResult(state="not_required")
+        reattached: tuple[ReattachedCollection, ...] = ()
+        readback: ReadbackReport | None = None
+        if vectors_waiting:
+            assert profile is not None
+            progress("vectors")
+            reattached = reattach_legacy_vectors(data_dir, profile)
+            legacy = legacy_memory_ids_from_bundle(data_dir, authority_migration)
+            if legacy is not None and not legacy.empty:
+                # Authority already moved, so nothing is left to protect by
+                # refusing; record what the repaired home reads back.
+                progress("verify")
+                with client_for(profile) as client:
+                    readback = verify_readback(client, legacy)
+        dropped: tuple[str, ...] = ()
+        if leftovers:
+            progress("cleanup")
+            dropped = resume_legacy_cleanup(data_dir=data_dir, database_path=database_path)
+            if dropped:
+                _vacuum(database_path)
+        return StartupMigrationResult(
+            state="complete",
+            migration_id=authority_migration,
+            bundle_id=f"{authority_migration}-pre-cutover" if authority_migration else None,
+            dropped_tables=dropped,
+            reattached=reattached,
+            readback=readback,
+        )
 
-    legacy_memories = read_legacy_memory_inventory(database_path)
-    sdk_memories = await read_sdk_memory_inventory(bridge)
-    memory_decisions = record_memory_migration_plan(
-        store,
-        migration_id=migration_id,
-        legacy=legacy_memories,
-        sdk=sdk_memories,
-    )
-    legacy_raw = read_legacy_raw_inventory(database_path)
-    sdk_raw = await read_sdk_raw_inventory(bridge)
-    raw_decisions = record_raw_migration_plan(
-        store,
-        migration_id=migration_id,
-        legacy=legacy_raw,
-        sdk=sdk_raw,
-    )
+    from one_memory.database_access import close_main_databases
+    from one_memory.memory_server.migration_runner import SqliteMigrationStore, migrate
+    from one_memory.memory_server.migrations import MIGRATIONS
 
-    store.set_phase(migration_id, phase="backfill", state="running")
-    await execute_memory_migration_plan(
-        bridge,
-        store,
+    migration_id, bundle_id = _resumable_identity(database_path)
+    legacy = legacy_memory_ids_from(database_path)
+    if profile is None and not legacy.empty:
+        # Without the embedding profile the Runtime's collections cannot be
+        # named, so neither reattached nor read back. Legacy stays intact.
+        raise DataPlaneMigrationError("sdk_migration_model_profile_required")
+    with capture_fence(data_dir):
+        progress("bundle")
+        manifest = _ensure_bundle(
+            data_dir=data_dir,
+            bundle_id=bundle_id,
+            runtime_build=runtime_build,
+        )
+        progress("prepare")
+        asyncio.run(migrate(SqliteMigrationStore(database_path), MIGRATIONS))
+        close_main_databases()
+        recover_abandoned_staging(
+            data_dir=data_dir,
+            database_path=database_path,
+            namespace=namespace,
+        )
+        store.start(
+            migration_id=migration_id,
+            source_runtime_build=str(manifest.get("runtime_build") or ""),
+            target_sdk_version=sdk_version,
+            source_schema_version=MIGRATIONS[-1].version,
+            namespace_fingerprint=hashlib.sha256(namespace.encode()).hexdigest(),
+            backend_fingerprint=hashlib.sha256(b"chroma:kuzu:sqlite").hexdigest(),
+            backup_bundle_id=bundle_id,
+        )
+        preparation = prepare_legacy_work_for_cutover(database_path)
+        reattached = ()
+        readback = None
+        if profile is not None:
+            progress("vectors")
+            reattached = reattach_legacy_vectors(data_dir, profile)
+        if not legacy.empty:
+            assert profile is not None
+            progress("verify")
+            with client_for(profile) as client:
+                readback = verify_readback(client, legacy)
+            if not readback.passed:
+                store.set_phase(migration_id, phase="readback_failed", state="blocked")
+                raise MemoryReadbackFailed(readback)
+        store.set_phase(migration_id, phase=PREPARED_PHASE, state="ready")
+        progress("cutover")
+        commit_sdk_cutover(
+            data_dir=data_dir,
+            database_path=database_path,
+            migration_id=migration_id,
+            bundle_id=bundle_id,
+            sdk_version=sdk_version,
+        )
+        progress("cleanup")
+        dropped = resume_legacy_cleanup(data_dir=data_dir, database_path=database_path)
+        if dropped:
+            _vacuum(database_path)
+    return StartupMigrationResult(
+        state="cutover",
         migration_id=migration_id,
-        legacy=legacy_memories,
-        decisions=memory_decisions,
-    )
-    await execute_raw_migration_plan(
-        bridge,
-        store,
-        migration_id=migration_id,
-        legacy=legacy_raw,
-        decisions=raw_decisions,
-    )
-    progress: MigrationProgress = store.progress(migration_id)
-    if progress.counts.get("quarantine", 0):
-        store.set_phase(migration_id, phase="review_required", state="blocked")
-    else:
-        store.set_phase(migration_id, phase="reconciled", state="ready")
-    progress = store.progress(migration_id)
-    return BackfillResult(
-        migration_id=migration_id,
-        state=progress.state,
-        phase=progress.phase,
-        counts=progress.counts,
-        memory_items=len(memory_decisions),
-        raw_items=len(raw_decisions),
+        bundle_id=bundle_id,
+        preparation=preparation,
+        dropped_tables=dropped,
+        reattached=reattached,
+        readback=readback,
     )

memhub/apps/core/src/one_memory/sdk_migration_verify.py

@@ -0,0 +1,232 @@
+"""Read legacy memory IDs back through the SDK before (and after) the cutover.
+
+The cutover is only safe if the SDK, configured exactly as the new Runtime
+will configure it, returns the memories the legacy mirror lists. Checking that
+the backup bundle is complete says nothing about it: build 20 dropped the
+legacy tables while its Runtime read an empty vector collection.
+
+Reads go through the public `HyMemoryClient` (`list_memories`, `get`).
+"""
+
+from __future__ import annotations
+
+import json
+import sqlite3
+from collections.abc import Iterator
+from contextlib import closing, contextmanager
+from dataclasses import dataclass
+from pathlib import Path
+from typing import Any
+
+from one_memory.database_access import main_database
+from one_memory.model_profiles import ModelProfileSnapshot
+from one_memory.sdk_data_plane_migration import DataPlaneMigrationError
+
+_PRODUCTIVITY_KINDS = frozenset({"coding", "working"})
+_PRODUCTIVITY_LAYERS = frozenset(
+    {"coding_experience", "working_experience", "coding_fact", "working_fact"}
+)
+_SDK_STATUSES = ("active", "superseded", "shadow", "negated", "conflicted", "archived")
+_PAGE = 500
+
+
+@dataclass(frozen=True)
+class LegacyMemoryIds:
+    chat: tuple[str, ...]
+    productivity: tuple[str, ...]
+    users: tuple[str, ...]
+
+    @property
+    def empty(self) -> bool:
+        return not self.chat and not self.productivity
+
+
+@dataclass(frozen=True)
+class ReadbackReport:
+    chat_expected: int
+    chat_readable: int
+    productivity_expected: int
+    productivity_readable: int
+
+    @property
+    def passed(self) -> bool:
+        # Every chat memory is an SDK node by construction, so all must read
+        # back. The SDK's Coding reconciler deletes experiences it retires, so
+        # a partial productivity count is normal; none at all is not.
+        return self.chat_readable == self.chat_expected and (
+            self.productivity_expected == 0 or self.productivity_readable > 0
+        )
+
+    def as_document(self) -> dict[str, object]:
+        return {
+            "chat_expected": self.chat_expected,
+            "chat_readable": self.chat_readable,
+            "productivity_expected": self.productivity_expected,
+            "productivity_readable": self.productivity_readable,
+            "passed": self.passed,
+        }
+
+
+class MemoryReadbackFailed(DataPlaneMigrationError):
+    def __init__(self, report: ReadbackReport) -> None:
+        super().__init__("sdk_migration_memory_unreadable")
+        self.report = report
+
+
+def _tables(connection: sqlite3.Connection) -> set[str]:
+    return {
+        str(row[0])
+        for row in connection.execute("SELECT name FROM sqlite_master WHERE type='table'")
+    }
+
+
+def read_legacy_memory_ids(connection: sqlite3.Connection) -> LegacyMemoryIds:
+    """Confirmed legacy memories, by the SDK id the old bridge recorded."""
+    tables = _tables(connection)
+    if "governed_memories" not in tables:
+        return LegacyMemoryIds(chat=(), productivity=(), users=())
+    mapped: dict[str, tuple[str, str | None]] = {}
+    if "memory_task_outputs" in tables:
+        for (body,) in connection.execute("SELECT body FROM memory_task_outputs"):
+            try:
+                document = json.loads(body)
+            except (TypeError, ValueError):
+                continue
+            legacy_id, sdk_id = document.get("memory_id"), document.get("hy_memory_id")
+            if isinstance(legacy_id, str) and isinstance(sdk_id, str):
+                layer = document.get("layer")
+                mapped[legacy_id] = (sdk_id, layer if isinstance(layer, str) else None)
+    chat: set[str] = set()
+    productivity: set[str] = set()
+    users: set[str] = set()
+    for memory_id, user_id, kind in connection.execute(
+        "SELECT memory_id,user_id,kind FROM governed_memories WHERE status='confirmed'"
+    ):
+        sdk_id, layer = mapped.get(str(memory_id), (str(memory_id), None))
+        users.add(str(user_id))
+        if str(kind) in _PRODUCTIVITY_KINDS or layer in _PRODUCTIVITY_LAYERS:
+            productivity.add(sdk_id)
+        else:
+            chat.add(sdk_id)
+    return LegacyMemoryIds(
+        chat=tuple(sorted(chat)),
+        productivity=tuple(sorted(productivity)),
+        users=tuple(sorted(users)),
+    )
+
+
+def legacy_memory_ids_from(database_path: Path) -> LegacyMemoryIds:
+    with main_database(database_path).read() as connection:
+        return read_legacy_memory_ids(connection)
+
+
+def legacy_memory_ids_from_bundle(
+    data_dir: Path, migration_id: str | None
+) -> LegacyMemoryIds | None:
+    """The same list from the pre-cutover bundle, for a home already migrated."""
+    if not migration_id:
+        return None
+    database = (
+        data_dir
+        / "backups"
+        / f"{migration_id}-pre-cutover.sdk-bundle"
+        / "management"
+        / "one-memory.sqlite3"
+    )
+    if not database.is_file():
+        return None
+    with closing(sqlite3.connect(f"file:{database}?mode=ro", uri=True)) as connection:
+        return read_legacy_memory_ids(connection)
+
+
+def read_active_profile(database_path: Path) -> ModelProfileSnapshot | None:
+    """The active model profile, read only; the status probe runs beside Core."""
+    with main_database(database_path).read(row_factory=sqlite3.Row) as connection:
+        if not {"model_profiles", "model_profile_versions"} <= _tables(connection):
+            return None
+        row = connection.execute(
+            "SELECT version.* FROM model_profiles profile "
+            "JOIN model_profile_versions version "
+            "ON version.profile_id=profile.profile_id AND version.revision=profile.revision "
+            "WHERE profile.is_active=1"
+        ).fetchone()
+    if row is None:
+        return None
+    return ModelProfileSnapshot(
+        profile_id=row["profile_id"],
+        revision=row["revision"],
+        provider_kind=row["provider_kind"],
+        base_url=row["base_url"],
+        model=row["model"],
+        embedding_model=row["embedding_model"],
+        embedding_provider_kind=row["embedding_provider_kind"],
+        embedding_base_url=row["embedding_base_url"],
+        embedding_dimensions=row["embedding_dimensions"],
+        timeout_seconds=row["timeout_seconds"],
+        max_concurrency=row["max_concurrency"],
+        context_window=row["context_window"],
+        has_api_key=row["secret_ref"] is not None,
+        disable_thinking=bool(row["disable_thinking"]),
+    )
+
+
+@contextmanager
+def runtime_sdk_client(
+    database_path: Path, data_dir: Path, profile: ModelProfileSnapshot
+) -> Iterator[Any]:
+    """A public SDK client configured exactly as the Runtime configures it."""
+    from one_memory.hy_sdk_runtime import acquire_client
+    from one_memory.model_profiles import ModelProfileStore
+    from one_memory.secrets import KeyringSecretStore, SecretStoreUnavailableError
+
+    try:
+        secret = ModelProfileStore(database_path).resolve_snapshot_secret(
+            profile.profile_id, profile.revision, KeyringSecretStore()
+        )
+    except SecretStoreUnavailableError:
+        raise DataPlaneMigrationError("sdk_migration_secure_storage_unavailable") from None
+    if profile.has_api_key and secret is None:
+        raise DataPlaneMigrationError("sdk_migration_model_secret_unavailable")
+    lease = acquire_client(profile, api_key=secret, data_dir=data_dir)
+    try:
+        yield lease.client
+    finally:
+        lease.close()
+
+
+def verify_readback(client: Any, legacy: LegacyMemoryIds) -> ReadbackReport:
+    listed: set[str] = set()
+    for user_id in legacy.users or ("local",):
+        offset = 0
+        while True:
+            raw = client.list_memories(
+                user_id=user_id,
+                statuses=list(_SDK_STATUSES),
+                limit=_PAGE,
+                offset=offset,
+                order="asc",
+            )
+            bucket = raw.get("vdb") if isinstance(raw, dict) else None
+            rows = bucket.get("memories") if isinstance(bucket, dict) else None
+            total = bucket.get("total") if isinstance(bucket, dict) else None
+            if not isinstance(rows, list) or not isinstance(total, int):
+                raise DataPlaneMigrationError("sdk_migration_memory_list_shape_unknown")
+            listed.update(
+                str(row["memory_id"])
+                for row in rows
+                if isinstance(row, dict) and row.get("memory_id")
+            )
+            offset += len(rows)
+            if not rows or offset >= total:
+                break
+
+    def readable(memory_id: str) -> bool:
+        return memory_id in listed or client.get(memory_id) is not None
+
+    return ReadbackReport(
+        chat_expected=len(legacy.chat),
+        chat_readable=sum(1 for memory_id in legacy.chat if readable(memory_id)),
+        productivity_expected=len(legacy.productivity),
+        productivity_readable=sum(1 for memory_id in legacy.productivity if readable(memory_id)),
+    )
+

memhub/apps/core/src/one_memory/sdk_vector_reattach.py

@@ -0,0 +1,149 @@
+"""Offline reattachment of SDK vector collections named by builds before #145.
+
+Those builds named a profile's embedding space `one_memory_<12 hex>`; #145
+renamed it `memhub_<16 hex>` and dropped the step that carried the old
+collections over, so every later Runtime opened an empty space beside the old
+one. Runtime code only reaches vectors through the SDK. This module runs from
+`sdk-migrate` while the Runtime is stopped, through Chroma's public client.
+"""
+
+from __future__ import annotations
+
+import sqlite3
+from collections.abc import Iterator
+from contextlib import closing
+from dataclasses import dataclass
+from pathlib import Path
+from typing import Any
+
+from one_memory.hy_sdk_runtime import vector_collection_bases
+from one_memory.model_profiles import ModelProfileSnapshot
+from one_memory.sdk_data_plane_migration import DataPlaneMigrationError
+
+_BATCH = 500
+
+
+@dataclass(frozen=True)
+class ReattachedCollection:
+    legacy: str
+    current: str
+    legacy_count: int
+    merged_count: int
+    final_count: int
+
+
+def _vector_directory(data_dir: Path) -> Path:
+    return data_dir / "hy-memory" / "governed-v2" / "vector"
+
+
+def _stored_names(data_dir: Path) -> tuple[str, ...]:
+    database = _vector_directory(data_dir) / "chroma.sqlite3"
+    if not database.is_file():
+        return ()
+    with closing(sqlite3.connect(f"file:{database}?mode=ro", uri=True)) as connection:
+        try:
+            rows = connection.execute("SELECT name FROM collections").fetchall()
+        except sqlite3.Error:
+            return ()
+    return tuple(sorted(str(row[0]) for row in rows))
+
+
+def legacy_vector_collections(
+    data_dir: Path, profile: ModelProfileSnapshot
+) -> tuple[tuple[str, str], ...]:
+    """(legacy name, current name) pairs still waiting to be reattached.
+
+    Read-only, so `sdk-migration-status` can ask it.
+    """
+    current, legacy_bases = vector_collection_bases(profile)
+    pairs: list[tuple[str, str]] = []
+    for name in _stored_names(data_dir):
+        for base in legacy_bases:
+            if name == base or name.startswith(f"{base}_"):
+                pairs.append((name, current + name[len(base) :]))
+                break
+    return tuple(pairs)
+
+
+def _records(collection: Any) -> Iterator[dict[str, Any]]:
+    offset = 0
+    while True:
+        page = collection.get(
+            include=["embeddings", "documents", "metadatas"],
+            limit=_BATCH,
+            offset=offset,
+        )
+        ids = list(page["ids"])
+        if not ids:
+            return
+        yield {
+            "ids": ids,
+            "embeddings": page["embeddings"],
+            "documents": page["documents"],
+            "metadatas": page["metadatas"],
+        }
+        offset += len(ids)
+
+
+def reattach_legacy_vectors(
+    data_dir: Path, profile: ModelProfileSnapshot
+) -> tuple[ReattachedCollection, ...]:
+    """Rename legacy collections to the current names, keeping newer writes.
+
+    A current collection that already exists (written after the rename broke)
+    is merged into the legacy one first; on an id present in both, the newer
+    current record wins. Callers must have stopped every SDK writer.
+    """
+    pairs = legacy_vector_collections(data_dir, profile)
+    if not pairs:
+        return ()
+    import chromadb
+    from chromadb.config import Settings
+
+    client = chromadb.PersistentClient(
+        path=str(_vector_directory(data_dir)),
+        settings=Settings(anonymized_telemetry=False),
+    )
+    existing = set(_stored_names(data_dir))
+    done: list[ReattachedCollection] = []
+    for legacy_name, current_name in pairs:
+        legacy = client.get_collection(legacy_name)
+        legacy_count = legacy.count()
+        merged = 0
+        if current_name in existing:
+            current = client.get_collection(current_name)
+            legacy_ids = set(_ids(legacy))
+            current_ids: set[str] = set()
+            for batch in _records(current):
+                legacy.upsert(**batch)
+                current_ids.update(batch["ids"])
+            merged = len(current_ids)
+            if legacy.count() != len(legacy_ids | current_ids):
+                raise DataPlaneMigrationError("sdk_migration_vector_merge_incomplete")
+            client.delete_collection(current_name)
+        legacy.modify(name=current_name)
+        final = client.get_collection(current_name).count()
+        if final < legacy_count:
+            raise DataPlaneMigrationError("sdk_migration_vector_reattach_incomplete")
+        existing.discard(legacy_name)
+        existing.add(current_name)
+        done.append(
+            ReattachedCollection(
+                legacy=legacy_name,
+                current=current_name,
+                legacy_count=legacy_count,
+                merged_count=merged,
+                final_count=final,
+            )
+        )
+    return tuple(done)
+
+
+def _ids(collection: Any) -> Iterator[str]:
+    offset = 0
+    while True:
+        ids = list(collection.get(include=[], limit=_BATCH, offset=offset)["ids"])
+        if not ids:
+            return
+        yield from ids
+        offset += len(ids)

memhub/apps/core/src/one_memory/sdk_write_processor.py

@@ -0,0 +1,330 @@
+"""Single-write adapter from durable Evidence payloads to the public HY SDK."""
+
+from __future__ import annotations
+
+import asyncio
+from collections.abc import Awaitable, Callable, Mapping, Sequence
+from dataclasses import dataclass
+from datetime import datetime
+from typing import Any
+
+from one_memory.hy_sdk_runtime import SdkClientHandle
+from one_memory.imports.models import EvidenceContentKind, EvidenceOrigin, EvidenceRecord
+from one_memory.input_sanitization import sanitize_model_input
+from one_memory.model_profiles import ModelProfileSnapshot, ModelProfileStore
+from one_memory.secrets import SecretStore, SecretStoreUnavailableError
+
+PRODUCT_AGENT_ID = "all-apps"
+_MESSAGE_ROLES = frozenset({"user", "assistant", "tool"})
+
+
+@dataclass(frozen=True)
+class SdkWriteReceipt:
+    """The only SDK write identity retained by MemHub."""
+
+    raw_ids: tuple[str, ...] = ()
+    memory_ids: tuple[str, ...] = ()
+
+
+class SdkWriteProcessor:
+    """Turn one durable job into exactly one public SDK write call."""
+
+    def __init__(self, client_handle: SdkClientHandle) -> None:
+        self._client_handle = client_handle
+
+    async def process_sdk_job(
+        self,
+        records: tuple[EvidenceRecord, ...],
+        *,
+        request_id: str,
+        before_hy_invocation: Callable[[], Awaitable[None]],
+    ) -> SdkWriteReceipt:
+        if not records:
+            raise RuntimeError("hy_memory_task_without_inputs")
+
+        scope = _write_scope(records)
+        metadata = _write_metadata(records, scope)
+        client = await self._client_handle.client()
+
+        if all(_is_extracted_content(record) for record in records):
+            contents = [sanitize_model_input(record.verbatim) for record in records]
+            await before_hy_invocation()
+            result = await asyncio.to_thread(
+                client.add_extracted,
+                contents,
+                user_id=scope.user_id,
+                agent_id=PRODUCT_AGENT_ID,
+                session_id=scope.session_id,
+                source=scope.source,
+                metadata=metadata,
+                memory_at=scope.memory_at,
+                request_id=request_id,
+            )
+        else:
+            messages = [
+                {
+                    "role": record.role,
+                    "content": sanitize_model_input(record.verbatim),
+                }
+                for record in records
+                if record.content_kind is EvidenceContentKind.RAW_CONVERSATION
+                and record.role in _MESSAGE_ROLES
+            ]
+            if not messages:
+                raise RuntimeError("hy_memory_task_without_inputs")
+            await before_hy_invocation()
+            result = await asyncio.to_thread(
+                client.add,
+                messages,
+                user_id=scope.user_id,
+                agent_id=PRODUCT_AGENT_ID,
+                session_id=scope.session_id,
+                source=scope.source,
+                metadata=metadata,
+                memory_at=scope.memory_at,
+                scene=None,
+                workspace_id=scope.workspace,
+                request_id=request_id,
+            )
+
+        return _receipt_from_result(result)
+
+    async def inquire_write_outcome(self, request_id: str) -> SdkWriteReceipt | None:
+        """Return a confirmed successful receipt without scanning SDK stores."""
+        if not request_id:
+            raise ValueError("request_id is required")
+        client = await self._client_handle.client()
+        outcome = await asyncio.to_thread(client.get_write_outcome, request_id)
+        if outcome is None:
+            return None
+        if not isinstance(outcome, Mapping):
+            raise RuntimeError("hy_sdk_write_outcome_shape_invalid")
+
+        status = outcome.get("status")
+        if status != "succeeded":
+            return None
+        result = outcome.get("result")
+        if not isinstance(result, Mapping):
+            raise RuntimeError("hy_sdk_write_outcome_shape_invalid")
+        return _receipt_from_result(result)
+
+    async def close(self) -> None:
+        await self._client_handle.close()
+
+
+class SdkWriteCoordinator:
+    """Resolve the active model profile, then execute one public SDK write."""
+
+    def __init__(
+        self,
+        *,
+        profiles: ModelProfileStore,
+        secrets: SecretStore,
+        data_dir: Any,
+        transport_factory: Callable[[ModelProfileSnapshot], Callable[[], Any] | None],
+        sedimentation_enabled: Callable[[], bool],
+    ) -> None:
+        self.profiles = profiles
+        self.secrets = secrets
+        self.data_dir = data_dir
+        self.transport_factory = transport_factory
+        self.sedimentation_enabled = sedimentation_enabled
+        self._handles: dict[tuple[str, int], SdkClientHandle] = {}
+        self._lock = asyncio.Lock()
+
+    async def process_sdk_job(
+        self,
+        records: tuple[EvidenceRecord, ...],
+        *,
+        request_id: str,
+        before_hy_invocation: Callable[[], Awaitable[None]],
+    ) -> SdkWriteReceipt:
+        if not self.sedimentation_enabled():
+            raise RuntimeError("model_sedimentation_disabled")
+        handle = await self._active_handle()
+        return await SdkWriteProcessor(handle).process_sdk_job(
+            records,
+            request_id=request_id,
+            before_hy_invocation=before_hy_invocation,
+        )
+
+    async def inquire_write_outcome(
+        self, request_id: str
+    ) -> SdkWriteReceipt | None:
+        return await SdkWriteProcessor(
+            await self._active_handle()
+        ).inquire_write_outcome(request_id)
+
+    async def close(self) -> None:
+        async with self._lock:
+            handles, self._handles = tuple(self._handles.values()), {}
+        await asyncio.gather(*(handle.close() for handle in handles))
+
+    async def client(self) -> Any:
+        """Return the public SDK client selected by the active write profile."""
+        return await (await self._active_handle()).client()
+
+    async def active_profile_credentials(
+        self,
+    ) -> tuple[ModelProfileSnapshot, str | None]:
+        """Resolve one stable profile snapshot for graph administration."""
+        _identity, profile, secret = await asyncio.to_thread(self._read_active_credentials)
+        return profile, secret
+
+    async def _active_handle(self) -> SdkClientHandle:
+        identity, profile, secret = await asyncio.to_thread(
+            self._read_active_credentials
+        )
+        async with self._lock:
+            handle = self._handles.get(identity)
+            if handle is None:
+                handle = SdkClientHandle(
+                    profile,
+                    api_key=secret,
+                    data_dir=self.data_dir,
+                    http_transport_factory=self.transport_factory(profile),
+                )
+                self._handles[identity] = handle
+            stale = [
+                value
+                for key, value in self._handles.items()
+                if key != identity
+            ]
+            self._handles = {identity: handle}
+        await asyncio.gather(*(value.close() for value in stale))
+        return handle
+
+    def _read_active_credentials(
+        self,
+    ) -> tuple[tuple[str, int], ModelProfileSnapshot, str | None]:
+        identity = self.profiles.active_identity()
+        if identity is None:
+            raise RuntimeError("model_profile_not_active")
+        profile = self.profiles.snapshot(*identity)
+        try:
+            secret = self.profiles.resolve_snapshot_secret(
+                *identity,
+                self.secrets,
+            )
+        except SecretStoreUnavailableError:
+            raise RuntimeError("secure_storage_unavailable") from None
+        if self.profiles.active_identity() != identity:
+            raise RuntimeError("model_profile_changed")
+        if profile.has_api_key and secret is None:
+            raise RuntimeError("model_secret_unavailable")
+        return identity, profile, secret
+
+
+@dataclass(frozen=True)
+class _WriteScope:
+    user_id: str
+    session_id: str
+    source: str
+    workspace: str | None
+    workspaces: tuple[str, ...]
+    memory_at: datetime | None
+
+
+def _write_scope(records: tuple[EvidenceRecord, ...]) -> _WriteScope:
+    first = records[0]
+    expected = (first.user_id, first.source_id, first.conversation_id)
+    if any(
+        (record.user_id, record.source_id, record.conversation_id) != expected
+        for record in records[1:]
+    ):
+        raise RuntimeError("durable_write_multiple_sources")
+    workspaces = tuple(
+        sorted({record.workspace for record in records if record.workspace is not None})
+    )
+    return _WriteScope(
+        user_id=first.user_id,
+        session_id=first.conversation_id,
+        source=first.source_id,
+        workspace=workspaces[0] if len(workspaces) == 1 else None,
+        workspaces=workspaces,
+        memory_at=max(
+            (record.created_at for record in records if record.created_at is not None),
+            default=None,
+        ),
+    )
+
+
+def _write_metadata(
+    records: tuple[EvidenceRecord, ...],
+    scope: _WriteScope,
+) -> dict[str, Any]:
+    return {
+        "source_id": scope.source,
+        "session_id": scope.session_id,
+        "workspace_ids": list(scope.workspaces),
+        "memory_at": scope.memory_at.isoformat() if scope.memory_at is not None else None,
+        "evidence_ids": [record.evidence_id for record in records],
+        "input_kind": (
+            "memory_content_without_raw"
+            if all(_is_extracted_content(record) for record in records)
+            else "raw_conversation"
+        ),
+    }
+
+
+def _is_extracted_content(record: EvidenceRecord) -> bool:
+    if record.content_kind is EvidenceContentKind.MEMORY_CONTENT:
+        return True
+    manual = record.origin_meta.get(EvidenceOrigin.MANUAL_IMPORT)
+    return (
+        record.source_id == "one-memory-manual"
+        and manual is not None
+        and manual.raw_meta.get("input_mode") == "direct_memory_note"
+    )
+
+
+def _receipt_from_result(value: object) -> SdkWriteReceipt:
+    if not isinstance(value, Mapping):
+        raise RuntimeError("hy_sdk_write_receipt_shape_invalid")
+    if value.get("success") is not True:
+        raise RuntimeError("hy_sdk_write_failed")
+
+    # Read only documented public receipt fields. ``variant`` identifies the
+    # productivity namespace; IDs from both variants are canonical memory IDs.
+    variant = value.get("variant")
+    if variant is not None and not isinstance(variant, str):
+        raise RuntimeError("hy_sdk_write_receipt_shape_invalid")
+    raw_ids = _public_ids(value, singular="raw_id", plural="raw_ids")
+    memory_ids = _ordered_unique(
+        (
+            *_public_ids(value, singular="memory_id", plural="memory_ids"),
+            *_public_ids(value, plural="experience_ids"),
+            *_public_ids(value, plural="fact_ids"),
+        )
+    )
+    return SdkWriteReceipt(raw_ids=raw_ids, memory_ids=memory_ids)
+
+
+def _public_ids(
+    value: Mapping[object, object],
+    *,
+    singular: str | None = None,
+    plural: str,
+) -> tuple[str, ...]:
+    found: list[str] = []
+    if singular is not None:
+        item = value.get(singular)
+        if item is not None:
+            if not isinstance(item, str):
+                raise RuntimeError("hy_sdk_write_receipt_shape_invalid")
+            if item:
+                found.append(item)
+    items = value.get(plural)
+    if items is not None:
+        if isinstance(items, (str, bytes)) or not isinstance(items, Sequence):
+            raise RuntimeError("hy_sdk_write_receipt_shape_invalid")
+        for item in items:
+            if not isinstance(item, str):
+                raise RuntimeError("hy_sdk_write_receipt_shape_invalid")
+            if item:
+                found.append(item)
+    return _ordered_unique(found)
+
+
+def _ordered_unique(values: Sequence[str]) -> tuple[str, ...]:
+    return tuple(dict.fromkeys(values))

memhub/apps/core/src/one_memory/target_schema.py

@@ -54,6 +54,8 @@ LEGACY_TABLES_IN_DROP_ORDER = (
     "evidence_origins",
     "evidence_versions",
     "evidence_repairs",
+    # FTS5 index over Evidence text; dropping it removes its shadow tables.
+    "evidence_search",
     "evidence",
     "memory_id_aliases",
     "raw_id_aliases",

memhub/apps/core/tests/TEST_INDEX.md

@@ -4,7 +4,6 @@
 
 | 测试文件 | 等级 | 一句话说明 |
 | --- | --- | --- |
-| `test_active_credentials.py` | `contract` | Active credentials are read once per call, never reused after a refusal. |
 | `test_anchor_invalid_rewind.py` | `contract` | Bug 2 real branch: an in-place same-length rewrite invalidates the anchor. |
 | `test_architecture_layer_gate.py` | `contract` | A0-01: L0-L4 dependency-direction gate. |
 | `test_archive_detection.py` | `contract` | S4.1: what an uploaded export is, decided before anything parses it. |
@@ -55,6 +54,7 @@
 | `test_claude_adoption_probe.py` | `contract` | The external-use verifier must distinguish a read from merely mentioning it. |
 | `test_cli_commands.py` | `contract` | 覆盖 status reports the verified runtime as json、doctor fails when runtime identity cannot be verified、serve and legacy empty command share the same entry 等 4 个场景。 |
 | `test_cli_lifecycle.py` | `integration` | 覆盖 cli starts and stops without residual runtime 场景。 |
+| `test_cli_migration_error_codes.py` | `unit` | The startup dialog can only act on the code it is handed. |
 | `test_cli_restore.py` | `contract` | 覆盖 restore requires a source、restore refuses while the runtime is running、restore previews without confirmation 等 4 个场景。 |
 | `test_codex_header_provenance.py` | `contract` | 覆盖 header workspace versions and native ids reach file origin、new adapter rereads header after a real checkpoint、machine and unknown session sources do not establish human or agent sender 等 14 个场景。 |
 | `test_connection_lifetime.py` | `integration` | Every SQLite connection a request opens must be closed when the request ends. |
@@ -91,7 +91,6 @@
 | `test_disable_thinking.py` | `contract` | 运行期生成请求须按档案可选关闭思考；默认不向厂商下发未知字段。 |
 | `test_dual_engine_helper.py` | `contract` | Tests for the dual-engine harness itself. |
 | `test_event_loop_blocking.py` | `contract` | Proof, as a test, that blocking work on the loop freezes it — and the fix works. |
-| `test_evidence_reconcile_primitive.py` | `integration` | A bounded, read-only discovery primitive for the H23 Evidence reconcile. |
 | `test_evidence_store.py` | `contract` | 覆盖 same source message merges origins without duplication、identical text from different sources is not deduplicated、rescanning an unchanged file does not write the message again 等 26 个场景。 |
 | `test_export_probe.py` | `contract` | What a probe may say about a stranger's chat history, and what it may not. |
 | `test_external_installation_permissions.py` | `integration` | 覆盖 external marker is not an installation grant、external diagnostic trace uses its own read only tool、raw source grants filter before limit and private body loading 场景。 |
@@ -103,8 +102,6 @@
 | `test_file_import_operations_review.py` | `contract` | Upload refusal/legacy compatibility review with bounded synthetic files only. |
 | `test_frozen_core_layout.py` | `contract` | 覆盖 single file layout remains available without private preset、folder layout keeps same core and all native dependencies、unknown layout does not silently build another package 等 5 个场景。 |
 | `test_golden_corpus.py` | `contract` | A7-01: the golden corpus comparison. |
-| `test_governance_permission_commit.py` | `contract` | 覆盖 permission guard is rechecked inside write transaction and rolls back、guard context does not leak into subsequent manual actions、guard check and commit hold database write authority 场景。 |
-| `test_governance_read_path_does_not_take_the_writer.py` | `contract` | The recall read path must not acquire the single global writer. |
 | `test_graph_width_endpoint.py` | `integration` | The width mismatch has to reach the interface, not just the log. |
 | `test_graph_width_export.py` | `unit` | EXPORT_FIRST has to actually export. |
 | `test_graph_width_guard.py` | `unit` | Graph width mismatch must ask, never silently archive. |
@@ -115,16 +112,9 @@
 | `test_host_capabilities.py` | `unit` | A plugin does not become realtime on every host or every unknown version. |
 | `test_hourly_c1_peak_replay_contract.py` | `contract` | Guard the package peak-replay probe's error accounting. |
 | `test_hourly_c1_recall_stage_probe_contract.py` | `contract` | Guard the packaged recall stage probe's command and error semantics. |
-| `test_hy_coding_reply.py` | `contract` | R-07: HY's coding-shaped replies read as "no memories at all". |
 | `test_hy_log_privacy.py` | `contract` | HY SDK records are not rewritten by One Memory's logging setup. |
-| `test_hy_memory_bridge.py` | `contract` | 覆盖 hy client initialization is serial across bridges、native hy clients share one runtime per data directory、ollama profile ignores ambient remote storage and models 等 41 个场景。 |
-| `test_hy_metrics_lifecycle.py` | `contract` | What the pinned public SDK's MetricsCollector does, and what that costs us. |
 | `test_hy_network_egress.py` | `contract` | Where a real HY client is allowed to send bytes, and where it is not. |
-| `test_hy_response_guard.py` | `contract` | 覆盖 invalid response retries same hy provider without changing prompt、repeated empty response fails instead of becoming no op、failure receipt survives a later successful model call 等 23 个场景。 |
-| `test_hy_response_shapes.py` | `contract` | HY response shape selection only; no provider, write routing or model calls. |
 | `test_hy_runtime_config.py` | `contract` | MEMORY.md is the audited record of what we send HY; keep it answerable. |
-| `test_hy_sdk_idempotency_probe.py` | `contract` | A0-03: probe whether the HY SDK can make a write idempotent. |
-| `test_hy_source_context.py` | `contract` | 覆盖 bound context rejects a same named conversation in another workspace、bound context rejects a duplicated evidence id、bound context rejects a same conversation from another source 等 9 个场景。 |
 | `test_hy_step_trace_switch.py` | `contract` | Rule 15: the per-step HY trace is off unless the debug tier asks for it. |
 | `test_import_api.py` | `integration` | 覆盖 resume during cleanup returns conflict without internal error、authorize detect import and revoke api flow、protected api rejects missing token and foreign origin 等 5 个场景。 |
 | `test_import_control_races_review.py` | `contract` | Read-only review repro: real runner/store, synthetic delayed adapter cleanup. |
@@ -149,7 +139,6 @@
 | `test_live_watch.py` | `contract` | 覆盖 a new turn lands well inside the ten second target、start prime catches a change that predates the watcher、a run that predates the change is followed by a fresh one 等 15 个场景。 |
 | `test_live_watch_grants.py` | `integration` | 覆盖 explicit sync grant starts watch without restart and revocation stops it 场景。 |
 | `test_local_embedding.py` | `contract` | Guards for the built-in embedding model. |
-| `test_local_embedding_offsets.py` | `contract` | 覆盖 real locked tokenizer offsets are unicode codepoints、known emoji and combining offsets are not utf8 or utf16 units、offsets are not truncated or padded and do not create onnx session 等 9 个场景。 |
 | `test_local_onnx_provider.py` | `contract` | Expect local ONNX embedding to be the default and to feed Hy Memory. |
 | `test_log_file.py` | `contract` | 覆盖 what the watchdog says survives a dock launch、the log lives where the data does、a machine left running for months does not fill up 等 4 个场景。 |
 | `test_log_privacy.py` | `integration` | The file log is shipped to us in bug reports; it must not carry originals. |
@@ -168,16 +157,11 @@
 | `test_memory_batch_extraction.py` | `contract` | 覆盖 batch extractor keeps only unique known evidence ids、batch extractor maps provider failure to machine error 场景。 |
 | `test_memory_citation_query_plan.py` | `contract` | Correlated citation checks must look links up by memory, not scan by 'active'. |
 | `test_memory_content_import.py` | `integration` | 覆盖 existing memory preview keeps unknown author and exact original、invalid existing memory records never leave a partial upload、memory content requires explicit mode and rejects vendor guessing 等 5 个场景。 |
-| `test_memory_correction_sync.py` | `integration` | R-27: a correction that only lands locally, while HY keeps the old text. |
 | `test_memory_evidence_freshness.py` | `contract` | 新鲜度拆除后：读路径恒为 current；证据修订 / 畸形 proof / segment ref 不再藏记忆或 503。 |
 | `test_memory_freshness_integration_review.py` | `integration` | Independent synthetic read-path review; no lifespan, model or user database. |
-| `test_memory_governance.py` | `contract` | 覆盖 correction keeps verbatim and appends history、mutations are revision checked and request idempotent、delete result survives restart and request replay 等 42 个场景。 |
 | `test_memory_governance_api.py` | `integration` | 覆盖 memory governance api preserves evidence and reports cleanup、memory validation does not echo content 场景。 |
 | `test_memory_graph_api.py` | `integration` | 覆盖 graph labels where every edge came from、a created memory is on the map with its source links、drawing the map does not stop the app answering 等 12 个场景。 |
-| `test_memory_language.py` | `contract` | R-03: memory language follows the message, so Kristin's gets written in English. |
 | `test_memory_notes.py` | `integration` | 覆盖 empty library can save a note without a matter or configured model、note requires authentication explicit scope and nonempty content、saved note evidence returns raw and consent reference from same record 等 6 个场景。 |
-| `test_memory_processing.py` | `contract` | 覆盖 stale known reason diagnostics never fail dispatch、memory task logs six step durations without content、assistant turns reach hy so references can be resolved 等 95 个场景。 |
-| `test_memory_processing_diagnostics.py` | `contract` | 覆盖 conversation prepare failure includes safe message and stack、deletion sync failure includes safe message and stack 场景。 |
 | `test_memory_qa.py` | `contract` | 覆盖 search terms keep meaningful single chinese character、streamed answer emits recall then incremental grounded text、governance change during generation stops the old answer 等 14 个场景。 |
 | `test_memory_qa_api.py` | `integration` | 覆盖 ask stream endpoint emits recall deltas and final、ask stream endpoint requires the local token 场景。 |
 | `test_memory_qa_permissions.py` | `contract` | 覆盖 initial refusal precedes all secret access and network、invalid authorization result fails before secret access、authorization receives exact question and only outbound grounding 等 16 个场景。 |
@@ -247,21 +231,8 @@
 | `test_recall_scope_control.py` | `integration` | 覆盖 external scope cannot override shared or host workspace boundary、scope only mutation is versioned without changing status or reindexing 场景。 |
 | `test_recall_scope_off_loop.py` | `contract` | The workspace lookup behind an external recall must not run on the loop. |
 | `test_recent_first_import.py` | `contract` | Recent-file priority survives restart through a selection-bound frozen copy. |
-| `test_retrieval_chunking.py` | `contract` | 覆盖 default 384 body 64 overlap and exact source slices、all prefix and special tokens count toward strict 512 budget、real tokenizer covers long chinese middle and tail object locations 等 20 个场景。 |
-| `test_retrieval_eligibility_benchmark.py` | `contract` | 覆盖 scale probe proves composite authority index changes the plan 场景。 |
-| `test_retrieval_index_store.py` | `contract` | 覆盖 constructor and reads do not create database or schema、explicit initialization rolls back and never grants model policy、consumer and jobs commit together 等 19 个场景。 |
-| `test_retrieval_index_supervisor.py` | `contract` | 覆盖 stop waits for native exit and does not close resources、async cancellation repeatedly waits for native thread、idle no trigger no spin and duplicate start rejected 等 14 个场景。 |
-| `test_retrieval_index_worker.py` | `contract` | 覆盖 tick indexes complete original and reopen can search、change during native encoding never publishes、cancel after encode does not publish and is reclaimable 等 10 个场景。 |
 | `test_retrieval_lifespan.py` | `integration` | 覆盖 retrieval defaults off and does not create index、enabling retrieval requires runtime ownership、index enabled serve imports without building an unmanaged app 等 6 个场景。 |
-| `test_retrieval_mapped_content.py` | `contract` | 覆盖 raw authors keep exact body and do not mutate evidence、raw and author qualification rejects without tokenizing、ambiguous generated and body hygiene are not admitted 等 22 个场景。 |
-| `test_retrieval_mapped_integration.py` | `contract` | 覆盖 empty materialization rechecks excluded map、completed excluded missing active binding is not empty success、old empty completion is invalidated when a new indexed record arrives 等 24 个场景。 |
-| `test_retrieval_provisioning.py` | `contract` | The index has to come up through the product, not through a test helper. |
 | `test_retrieval_query_plan.py` | `contract` | Local QueryPlan: extract URL/path objects. |
-| `test_retrieval_raw_candidates.py` | `contract` | 覆盖 positive only current authorized published chunks are searched、empty authority never reads database or vector、unapproved source never enters vector query 等 15 个场景。 |
-| `test_retrieval_raw_evidence.py` | `contract` | 覆盖 materialized text is exact current original、stale candidates do not authorize original read、budget does not cut a chunk into fake partial object 等 6 个场景。 |
-| `test_retrieval_raw_lane.py` | `contract` | v1 raw lane must not claim complete empty when the derived index is partial. |
-| `test_retrieval_runtime.py` | `contract` | 覆盖 constructor and missing start create nothing、stop releases supervisor worker but preserves immutable snapshot、not initialized or invalid configuration never creates index 等 21 个场景。 |
-| `test_retrieval_vector_store.py` | `contract` | 覆盖 constructor does no io and unopened calls do not create、open existing missing store or collection never creates、real chroma collection has no embedding function or documents 等 32 个场景。 |
 | `test_runtime.py` | `integration` | 覆盖 runtime writes and removes ownership receipt、runtime rejects second owner for same data directory、nonpublishing maintenance owner still excludes a second core 等 4 个场景。 |
 | `test_runtime_client.py` | `contract` | 覆盖 runtime inspection authenticates the discovered instance、runtime inspection rejects untrusted receipts without probing、runtime inspection distinguishes stopped and identity mismatch 场景。 |
 | `test_sanitization_source_map.py` | `contract` | 覆盖 mapping matches existing sanitizer and exact original slices、synthetic replacements never claim original locations、repeated identical suffix is mapped by positions not string search 等 6 个场景。 |
@@ -289,7 +260,6 @@
 | `test_secret_store_error_code_api.py` | `integration` | 3925a kernel item 3: the error code reaches the HTTP boundary. |
 | `test_secret_store_error_codes.py` | `integration` | 3925a kernel item 3: keychain failure classification and backoff that heals. |
 | `test_secure_storage.py` | `integration` | 覆盖 isolated keychain service namespace、custom keychain service is refused outside isolated tests、packaged keychain budget covers loaded process startup 等 20 个场景。 |
-| `test_sedimentation_concurrency.py` | `contract` | Sedimentation runs several harnesses at once, but one harness at a time. |
 | `test_sedimentation_switch.py` | `contract` | Stopping model-driven sedimentation without losing the configuration. |
 | `test_service_level_pressure_acceptance.py` | `contract` | 覆盖 workbuddy recall remains available during 100k import 场景。 |
 | `test_service_levels.py` | `contract` | 覆盖 background never consumes the realtime reservation、realtime is next for the same source after background yields、import history gets one slice after nine realtime admissions 等 8 个场景。 |
@@ -320,7 +290,6 @@
 | `test_source_sync_idempotent.py` | `integration` | Final-state Harness sync is idempotent and resumes missed source work. |
 | `test_source_sync_permissions.py` | `integration` | S1.3 source permission tests use temporary files only, never real host history. |
 | `test_source_totals.py` | `contract` | 覆盖 the first ask is answered from the rows、same named sessions in two workspaces count as two conversations、insert during count is not marked as already counted 等 7 个场景。 |
-| `test_source_write_switch.py` | `contract` | A frozen Harness produces no extraction tasks. |
 | `test_sql_compat.py` | `contract` | The shared compatibility layer, especially 'rowcount'. |
 | `test_sqlite_semantics_gate.py` | `contract` | D56: prevent SQLite implementation details spreading outside the data layer. |
 | `test_sqlite_write_contention.py` | `contract` | Measure whether SQLite's single writer actually blocks us. |
@@ -345,4 +314,4 @@
 | `test_workspace_recall.py` | `contract` | Workspace memories were unreachable from every host; these pin the way in. |
 | `test_write_queue_lease_renewal.py` | `contract` | Rule 8: a lease must outlive the work it covers, and be renewed while it runs. |
 
-共 340 个测试文件。
+共 309 个测试文件。

memhub/apps/core/tests/core_test_classification.py

@@ -14,13 +14,14 @@ from pathlib import Path
 import pytest
 
 PRIMARY_MARKERS = ("unit", "contract", "integration", "performance", "live")
-EXPECTED_TEST_FILE_COUNT = 340
-EXPECTED_TEST_FILES_SHA256 = "f69d3f33df5277febb09557d5bce7adb7d8719ed47e48a71f852e3e4ecf8fd74"
+EXPECTED_TEST_FILE_COUNT = 309
+EXPECTED_TEST_FILES_SHA256 = "ef6165c249971f5a2ab71a0c49f6d859c8c0f3612e3f749fba0acce0eab513ea"
 
 # Pure transformations, validation models, parity checks and state machines.
 # This list is intentionally small: uncertain files default to contract.
 UNIT_FILES = frozenset(
     {
+        "test_cli_migration_error_codes.py",
         "test_context_v2_contract_parity.py",
         "test_daemon_contract.py",
         "test_diagnostic_payload_keys.py",
@@ -62,7 +63,6 @@ INTEGRATION_FILES = frozenset(
         "test_d93_same_process_load.py",
         "test_diagnostic_trace.py",
         "test_diagnostic_trace_api.py",
-        "test_evidence_reconcile_primitive.py",
         "test_external_installation_permissions.py",
         "test_fault_matrix.py",
         "test_file_import_operations.py",
@@ -77,7 +77,6 @@ INTEGRATION_FILES = frozenset(
         "test_manual_transcript_import.py",
         "test_measure_c1_loop_blocking_contract.py",
         "test_memory_content_import.py",
-        "test_memory_correction_sync.py",
         "test_memory_freshness_integration_review.py",
         "test_memory_governance_api.py",
         "test_memory_graph_api.py",

memhub/apps/core/tests/graph_fixtures.py

@@ -0,0 +1,46 @@
+from __future__ import annotations
+
+import gc
+from pathlib import Path
+
+import pytest
+
+from one_memory.model_profiles import ModelProfileSnapshot, ProviderKind
+
+
+def model_snapshot(
+    provider: ProviderKind,
+    *,
+    base_url: str | None,
+    has_api_key: bool,
+    embedding_provider: ProviderKind | None = None,
+    embedding_base_url: str | None = None,
+) -> ModelProfileSnapshot:
+    return ModelProfileSnapshot(
+        profile_id="profile",
+        revision=3,
+        provider_kind=provider,
+        base_url=base_url,
+        model="memory-model",
+        embedding_model="embedding-model",
+        embedding_provider_kind=embedding_provider,
+        embedding_base_url=embedding_base_url,
+        embedding_dimensions=768,
+        timeout_seconds=30,
+        max_concurrency=1,
+        context_window=8192,
+        has_api_key=has_api_key,
+    )
+
+
+def kuzu_graph_with_embedding_width(db_path: Path, width: int) -> None:
+    kuzu = pytest.importorskip("kuzu")
+    database = kuzu.Database(str(db_path))
+    connection = kuzu.Connection(database)
+    connection.execute(
+        f"CREATE NODE TABLE Memory("
+        f"node_id STRING, embedding FLOAT[{width}], PRIMARY KEY(node_id))"
+    )
+    del connection
+    del database
+    gc.collect()

memhub/apps/core/tests/test_active_credentials.py

@@ -1,225 +0,0 @@
-"""Active credentials are read once per call, never reused after a refusal."""
-
-import asyncio
-import threading
-
-import pytest
-from pydantic import SecretStr
-
-from one_memory.background_work import shutdown_bulk_pool
-from one_memory.hy_memory_bridge import HyMemoryBridgeError
-from one_memory.imports.storage import EvidenceStore
-from one_memory.memory_governance import MemoryGovernanceStore
-from one_memory.memory_processing import MemoryProcessingSupervisor
-from one_memory.memory_tasks import MemoryTaskStore
-from one_memory.model_profiles import (
-    ModelCapabilities,
-    ModelProfileInput,
-    ModelProfilePatch,
-    ModelProfileStore,
-)
-from one_memory.secrets import InMemorySecretStore, SecretStoreUnavailableError
-
-
-class CountingSecrets(InMemorySecretStore):
-    def __init__(self):
-        super().__init__()
-        self.reads = []
-        self.denied = False
-        self.after_read = None
-
-    def has(self, secret_ref):
-        # The OS store's has() resolves the secret; the in-memory default does not.
-        return self.resolve(secret_ref) is not None
-
-    def resolve(self, secret_ref):
-        self.reads.append((secret_ref, threading.get_ident()))
-        if self.denied:
-            raise SecretStoreUnavailableError("synthetic reader denial, must not leak")
-        value = super().resolve(secret_ref)
-        callback, self.after_read = self.after_read, None
-        if callback is not None:
-            callback()
-        return value
-
-
-def create_profile(profiles, secrets, *, keyed=True):
-    profile = profiles.create(
-        ModelProfileInput(
-            name="Synthetic credentials",
-            provider_kind="openai_compatible" if keyed else "ollama",
-            base_url="http://127.0.0.1:1/v1",
-            model="synthetic-model",
-            api_key=SecretStr("synthetic-credential") if keyed else None,
-        ),
-        secrets,
-    )
-    profiles.record_capabilities(
-        profile.profile_id,
-        expected_revision=profile.revision,
-        capabilities=ModelCapabilities(
-            endpoint_reachable=True,
-            model_available=True,
-            structured_output=True,
-            embedding_available=True,
-        ),
-        secrets=secrets,
-    )
-    return profiles.activate(profile.profile_id, secrets)
-
-
-@pytest.fixture
-def setup(tmp_path):
-    database = tmp_path / "one-memory.sqlite3"
-    secrets = CountingSecrets()
-    profiles = ModelProfileStore(database)
-    profile = create_profile(profiles, secrets)
-    supervisor = MemoryProcessingSupervisor(
-        tasks=MemoryTaskStore(database),
-        profiles=profiles,
-        secrets=secrets,
-        evidence=EvidenceStore(database),
-        governance=MemoryGovernanceStore(database),
-        data_dir=tmp_path,
-    )
-    secrets.reads.clear()
-    try:
-        yield supervisor, profiles, secrets, profile
-    finally:
-        shutdown_bulk_pool()
-
-
-def test_resolves_active_immutable_snapshot_once_off_loop(setup):
-    supervisor, profiles, secrets, profile = setup
-    expected = profiles.snapshot(profile.profile_id, profile.revision)
-    caller = threading.get_ident()
-
-    result = asyncio.run(supervisor._active_credentials())
-
-    assert result == (expected, "synthetic-credential")
-    assert secrets.reads == [(profiles.secret_ref(profile.profile_id), secrets.reads[0][1])]
-    assert secrets.reads[0][1] != caller
-    assert supervisor.tasks.list() == []
-    assert supervisor._bridges == {}
-
-
-@pytest.mark.parametrize("concurrent", [False, True])
-def test_each_call_resolves_again_without_cross_request_cache(setup, concurrent):
-    supervisor, _profiles, secrets, _profile = setup
-
-    async def scenario():
-        if concurrent:
-            return await asyncio.gather(*(supervisor._active_credentials() for _ in range(4)))
-        return [await supervisor._active_credentials() for _ in range(4)]
-
-    results = asyncio.run(scenario())
-
-    assert all(result is not None and result[1] == "synthetic-credential" for result in results)
-    assert len(secrets.reads) == 4
-
-
-@pytest.mark.parametrize("failure", ["missing", "denied"])
-def test_prior_success_cannot_mask_current_reader_refusal(setup, failure):
-    supervisor, profiles, secrets, profile = setup
-
-    async def scenario():
-        assert (await supervisor._active_credentials())[1] == "synthetic-credential"
-        secrets.reads.clear()
-        if failure == "missing":
-            secrets.delete(profiles.secret_ref(profile.profile_id))
-        else:
-            secrets.denied = True
-        code = "model_secret_unavailable" if failure == "missing" else "secure_storage_unavailable"
-        with pytest.raises(HyMemoryBridgeError, match=f"^{code}$"):
-            await supervisor._active_credentials()
-        assert len(secrets.reads) == 1
-
-    asyncio.run(scenario())
-
-
-@pytest.mark.parametrize("change", ["deactivate", "switch", "revision"])
-def test_profile_change_during_secret_read_rejects_old_credentials(setup, change):
-    supervisor, profiles, secrets, profile = setup
-    other = create_profile(profiles, secrets)
-    profiles.activate(profile.profile_id, secrets)
-    secrets.reads.clear()
-
-    def change_identity():
-        # Concurrent committed identity change while the original reader is in flight.
-        with profiles._connect(writing=True) as db:
-            if change == "revision":
-                db.execute(
-                    "UPDATE model_profiles SET revision=revision+1 WHERE profile_id=?",
-                    (profile.profile_id,),
-                )
-            else:
-                db.execute("UPDATE model_profiles SET is_active=0")
-                if change == "switch":
-                    db.execute(
-                        "UPDATE model_profiles SET is_active=1 WHERE profile_id=?",
-                        (other.profile_id,),
-                    )
-
-    secrets.after_read = change_identity
-    with pytest.raises(HyMemoryBridgeError, match="^model_profile_changed$"):
-        asyncio.run(supervisor._active_credentials())
-    assert len(secrets.reads) == 1
-    assert supervisor._bridges == {}
-
-
-def test_rotation_between_calls_returns_new_snapshot_and_secret(setup):
-    supervisor, profiles, secrets, profile = setup
-    before = asyncio.run(supervisor._active_credentials())
-    updated = profiles.update(
-        profile.profile_id,
-        ModelProfilePatch(expected_revision=profile.revision, api_key=SecretStr("synthetic-new")),
-        secrets,
-    )
-    assert updated.is_active is False
-    assert updated.capabilities == ModelCapabilities()
-    assert profiles.active_identity() is None
-    secrets.reads.clear()
-
-    assert asyncio.run(supervisor._active_credentials()) is None
-    assert secrets.reads == []
-    assert supervisor.tasks.list() == []
-    assert supervisor._bridges == {}
-
-    # Synthetic probe evidence and explicit activation belong to the new revision.
-    profiles.record_capabilities(
-        updated.profile_id,
-        expected_revision=updated.revision,
-        capabilities=profile.capabilities,
-        secrets=secrets,
-    )
-    updated = profiles.activate(updated.profile_id, secrets)
-    assert updated.is_active is True
-    secrets.reads.clear()
-
-    after = asyncio.run(supervisor._active_credentials())
-
-    assert before[0].revision == 1 and before[1] == "synthetic-credential"
-    assert after == (profiles.snapshot(updated.profile_id, updated.revision), "synthetic-new")
-    assert after[0].revision == 2
-    assert len(secrets.reads) == 1
-
-
-def test_no_active_profile_does_not_read_credentials(setup):
-    supervisor, profiles, secrets, _profile = setup
-    with profiles._connect(writing=True) as db:
-        db.execute("UPDATE model_profiles SET is_active=0")
-
-    assert asyncio.run(supervisor._active_credentials()) is None
-    assert secrets.reads == []
-
-
-def test_keyless_profile_does_not_require_or_resolve_a_secret(setup):
-    supervisor, profiles, secrets, _profile = setup
-    keyless = create_profile(profiles, secrets, keyed=False)
-    secrets.reads.clear()
-
-    result = asyncio.run(supervisor._active_credentials())
-
-    assert result == (profiles.snapshot(keyless.profile_id, keyless.revision), None)
-    assert result[0].has_api_key is False
-    assert secrets.reads == []

memhub/apps/core/tests/test_architecture_layer_gate.py

@@ -4,12 +4,11 @@ The public target architecture is defined by ``docs/ARCHITECTURE.md``:
 
 * Hy Memory SDK owns memory/raw/vector/graph/coding data;
 * MemHub owns the management plane and may retain ``memory_id`` references;
-* product code reaches the SDK through ``hy_memory_bridge`` only.
+* product code reaches the SDK only through narrow runtime/migration modules;
+* MemHub calls public ``HyMemoryClient`` methods only.
 
-The legacy implementation still contains Evidence/Governance writers and a
-small number of private SDK-store accesses.  Those are migration debt, not an
-allowance for new code.  The explicit debt lists below may shrink during the
-refactor; new entries must not be added.
+The sole exception is the read-only Kuzu projection/admin surface used by the
+existing graph UI and Graph Width workflow.
 
 The gate is deliberately not a blanket ban on ``sqlite3``. Adapters may read a
 host product's SQLite through the guarded read-only module; they may not own
@@ -58,13 +57,13 @@ HOST_READONLY_MODULE = "one_memory.imports.sqlite_access"
 SRC_ROOT = PACKAGE_ROOT.parents[0]
 PACKAGE_LABEL = PACKAGE_ROOT.name
 
-HY_BRIDGE = f"{PACKAGE_LABEL}/hy_memory_bridge.py"
-
-#: The one module allowed to *invoke* the SDK.
-SDK_CALLER = HY_BRIDGE
-
-#: Type-only importers of HY datatypes; they hold no client and write nothing.
-HY_TYPE_ONLY: frozenset[str] = frozenset()
+SDK_MODULES = frozenset(
+    {
+        f"{PACKAGE_LABEL}/hy_sdk_runtime.py",
+        f"{PACKAGE_LABEL}/hy_sdk_migration.py",
+    }
+)
+SDK_CLIENT_CONSTRUCTOR = f"{PACKAGE_LABEL}/hy_sdk_runtime.py"
 
 # Frozen migration debt: modules that still write the legacy memory/raw
 # authority.  The target count is zero.  A new module cannot join this set.
@@ -84,19 +83,37 @@ LEGACY_AUTHORITY_WRITE = re.compile(
     re.IGNORECASE,
 )
 
-# Existing accesses to SDK implementation details.  These are exact file /
-# attribute debts so deleting one is always green, while adding one fails.
-SDK_PRIVATE_ACCESS_DEBT: dict[str, frozenset[str]] = {
-    f"{PACKAGE_LABEL}/hy_lifecycle.py": frozenset({"_collection", "_vector_store"}),
-    f"{PACKAGE_LABEL}/hy_memory_bridge.py": frozenset(
-        {"_coding_store", "_graph_store", "_loop_thread", "_vector_store"}
-    ),
-    f"{PACKAGE_LABEL}/hy_response_guard.py": frozenset({"_loop_thread", "_registry"}),
-}
-
 SDK_PRIVATE_ATTRIBUTES = frozenset(
-    attribute for attributes in SDK_PRIVATE_ACCESS_DEBT.values() for attribute in attributes
+    {
+        "_coding_store",
+        "_coding_stores",
+        "_collection",
+        "_embed_service",
+        "_graph_store",
+        "_loop_thread",
+        "_readers",
+        "_reader",
+        "_reconciler",
+        "_registry",
+        "_run",
+        "_shared_embed_service",
+        "_spawn_search_dedup",
+        "_system2_writer",
+        "_vector_store",
+        "_writer",
+    }
 )
+SDK_PRIVATE_ACCESS_EXCEPTIONS = {
+    f"{PACKAGE_LABEL}/hy_graph_projection.py": frozenset(
+        {"_graph_store", "_loop_thread", "_run"}
+    ),
+    # Startup migration runs while Runtime writers are stopped. It is the one
+    # non-runtime path allowed to manipulate SDK storage directly so that
+    # migration-only mechanics never become public HyMemoryClient methods.
+    f"{PACKAGE_LABEL}/hy_sdk_migration.py": frozenset(
+        {"_embed_service", "_graph_store", "_loop_thread", "_vector_store"}
+    ),
+}
 
 
 def _relative(path: Path) -> str:
@@ -171,49 +188,26 @@ def test_adapters_use_only_the_readonly_host_sqlite_module() -> None:
     )
 
 
-def test_hy_sdk_is_confined_to_the_l1_adapter() -> None:
-    """A0-01: HY SDK stays behind HyMemoryPort; entries must not import it.
-
-    This is the executable form of "Function/MCP/scan/Hook must not call the
-    HY SDK directly". `hy_memory_bridge` is the current L1 adapter and the
-    permission stores quote HY types, so they are the allowed set; anything
-    else reaching for the SDK is a new bypass and must fail here.
-
-    Only `hy_memory_bridge` *calls* the SDK. The two permission stores merely
-    import HY datatypes (`CodingMemory`, `WorkingFact`, memory models) to type
-    what they persist; they hold no client and issue no writes. They are
-    listed here so the gate stays green on today's truth while still failing
-    on any new module that reaches for the SDK.
-    """
-    allowed = frozenset({HY_BRIDGE}) | HY_TYPE_ONLY
+def test_hy_sdk_is_confined_to_public_sdk_modules() -> None:
+    """Only the runtime and offline migration adapters may import the SDK."""
     offenders: list[str] = []
     for path in _all_module_files():
         relative = _relative(path)
-        if relative in allowed:
+        tree = ast.parse(path.read_text(encoding="utf-8"))
+        names = _module_names(tree)
+        if not any(_root(name) in HY_SDK_ROOTS for name in names):
             continue
-        names = _module_names(ast.parse(path.read_text(encoding="utf-8")))
-        if any(_root(name) in HY_SDK_ROOTS for name in names):
+        if relative not in SDK_MODULES:
             offenders.append(relative)
-    assert not offenders, f"HY SDK must stay in the L1 adapter: {offenders}"
+    assert not offenders, f"HY SDK imports must stay in {sorted(SDK_MODULES)}: {offenders}"
 
 
-def test_only_the_l1_adapter_constructs_a_hy_client() -> None:
-    """A0-01: type-only imports are tolerated; constructing a client is not.
-
-    A permission store quoting `CodingMemory` cannot write a memory. A module
-    that instantiates `HyMemoryClient` can. So the stricter check is on client
-    construction, and it must pass today rather than merely documenting an
-    aspiration.
-
-    The check is on the AST, not on the substring: a docstring or comment that
-    *mentions* `HyMemoryClient` builds nothing. Matching text would flag prose
-    and, worse, would keep passing while a real caller is added inside a
-    function the text search happened to like.
-    """
+def test_only_runtime_constructs_a_hy_client() -> None:
+    """Migration borrows a lease; only runtime constructs clients."""
     offenders: list[str] = []
     for path in _all_module_files():
         relative = _relative(path)
-        if relative == SDK_CALLER:
+        if relative == SDK_CLIENT_CONSTRUCTOR:
             continue
         tree = ast.parse(path.read_text(encoding="utf-8"))
         for node in ast.walk(tree):
@@ -233,7 +227,7 @@ def test_only_the_l1_adapter_constructs_a_hy_client() -> None:
                 if any(alias.name == "HyMemoryClient" for alias in node.names):
                     offenders.append(relative)
                     break
-    assert not offenders, f"only the L1 adapter may build a HY client: {offenders}"
+    assert not offenders, f"only {SDK_CLIENT_CONSTRUCTOR} may build a HY client: {offenders}"
 
 
 def test_no_new_module_writes_legacy_memory_authority() -> None:
@@ -247,41 +241,90 @@ def test_no_new_module_writes_legacy_memory_authority() -> None:
         if relative not in LEGACY_AUTHORITY_WRITERS:
             offenders.append(relative)
     assert not offenders, (
-        "new Evidence/Governance writers are forbidden; write memory/raw via "
-        f"the SDK bridge instead: {offenders}"
+        "new Evidence/Governance writers are forbidden; use public SDK client "
+        f"methods instead: {offenders}"
     )
 
 
-def test_no_new_sdk_private_store_access() -> None:
-    """Freeze direct SDK-store access until public client APIs replace it."""
+def test_no_sdk_private_implementation_access() -> None:
+    """Runtime uses public APIs; only Graph and stopped-writer migration are privileged."""
     offenders: list[str] = []
     for path in _all_module_files():
         tree = ast.parse(path.read_text(encoding="utf-8"))
         relative = _relative(path)
-        names = _module_names(tree)
-        if relative not in SDK_PRIVATE_ACCESS_DEBT and not any(
-            _root(name) in HY_SDK_ROOTS for name in names
-        ):
-            # Names such as `_registry` and `_collection` are common for
-            # MemHub-owned objects too.  They become SDK-private access only
-            # in a module that imports the SDK (or is frozen debt already).
-            continue
         used = {
             node.attr
             for node in ast.walk(tree)
-            if isinstance(node, ast.Attribute) and node.attr in SDK_PRIVATE_ATTRIBUTES
+            if isinstance(node, ast.Attribute)
+            and node.attr in SDK_PRIVATE_ATTRIBUTES
+            and (
+                relative.startswith(f"{PACKAGE_LABEL}/hy_")
+                or (
+                    isinstance(node.value, ast.Name)
+                    and node.value.id in {"client", "native", "concrete"}
+                )
+            )
         }
-        if not used:
-            continue
-        unexpected = sorted(used - SDK_PRIVATE_ACCESS_DEBT.get(relative, frozenset()))
-        if unexpected:
-            offenders.append(f"{relative}: {unexpected}")
+        if used:
+            unexpected = used - SDK_PRIVATE_ACCESS_EXCEPTIONS.get(
+                relative, frozenset()
+            )
+            if unexpected:
+                offenders.append(f"{relative}: {sorted(unexpected)}")
     assert not offenders, (
-        "new SDK private-store access is forbidden; add a public HyMemoryClient "
-        f"API and call it through hy_memory_bridge: {offenders}"
+        "SDK private implementation access is forbidden outside the isolated "
+        f"Graph and offline-migration boundaries: {offenders}"
     )
 
 
+def test_sdk_modules_import_only_the_public_package_root() -> None:
+    """Runtime imports only public SDK symbols; offline migration is privileged."""
+    offenders: list[str] = []
+    for path in _all_module_files():
+        relative = _relative(path)
+        if relative == f"{PACKAGE_LABEL}/hy_sdk_migration.py":
+            continue
+        tree = ast.parse(path.read_text(encoding="utf-8"))
+        for node in ast.walk(tree):
+            if isinstance(node, ast.Import) and any(
+                alias.name.startswith("hy_memory.") for alias in node.names
+            ):
+                offenders.append(relative)
+                break
+            if isinstance(node, ast.ImportFrom) and (
+                node.module or ""
+            ).startswith("hy_memory."):
+                offenders.append(relative)
+                break
+            if (
+                isinstance(node, ast.Call)
+                and isinstance(node.func, ast.Name)
+                and node.func.id == "import_module"
+                and node.args
+                and isinstance(node.args[0], ast.Constant)
+                and isinstance(node.args[0].value, str)
+                and node.args[0].value.startswith("hy_memory.")
+            ):
+                offenders.append(relative)
+                break
+    assert not offenders, f"import SDK symbols from `hy_memory` only: {offenders}"
+
+
+def test_memhub_never_patches_sdk_prompts() -> None:
+    offenders: list[str] = []
+    forbidden = (
+        "extract_system_prompt_en",
+        "extract_system_prompt_zh",
+        "hy_memory.agent.prompts",
+    )
+    for path in _all_module_files():
+        source = path.read_text(encoding="utf-8")
+        hits = [token for token in forbidden if token in source]
+        if hits:
+            offenders.append(f"{_relative(path)}: {hits}")
+    assert not offenders, f"SDK prompt patching is forbidden: {offenders}"
+
+
 @pytest.mark.parametrize("driver", sorted(MEMHUB_DRIVERS))
 def test_no_new_module_adopts_a_memhub_driver_outside_the_data_layer(driver: str) -> None:
     """A0-01: guard-rail against A1 sprawl, not a demand that today is clean.

memhub/apps/core/tests/test_cli_migration_error_codes.py

@@ -0,0 +1,101 @@
+"""The startup dialog can only act on the code it is handed.
+
+A full disk and a damaged backup are different next steps, so they must reach
+the person under different names. These tests pin those names.
+"""
+
+from __future__ import annotations
+
+import errno
+from pathlib import Path
+
+import pytest
+
+from one_memory.cli import migration_failure_code
+
+
+class _Coded(RuntimeError):
+    def __init__(self, code: str) -> None:
+        super().__init__(code)
+        self.code = code
+
+
+def test_running_out_of_disk_is_named_however_deep_it_is_wrapped() -> None:
+    error = OSError(errno.ENOSPC, "No space left on device")
+    wrapped: BaseException = error
+    for _ in range(3):
+        try:
+            raise RuntimeError("wrapper") from wrapped
+        except RuntimeError as caught:
+            wrapped = caught
+
+    assert migration_failure_code(wrapped) == "sdk_migration_disk_space_insufficient"
+    # The explicit context chain counts too: `raise ... from None` discards the
+    # cause but Python still keeps the original as __context__.
+    try:
+        try:
+            raise OSError(errno.ENOSPC, "No space left on device")
+        except OSError:
+            raise RuntimeError("sdk_snapshot_failed") from None
+    except RuntimeError as caught:
+        assert migration_failure_code(caught) == "sdk_migration_disk_space_insufficient"
+
+
+def test_quota_exhaustion_counts_as_no_space(tmp_path: Path) -> None:
+    quota = getattr(errno, "EDQUOT", errno.ENOSPC)
+    assert (
+        migration_failure_code(OSError(quota, "Disk quota exceeded"))
+        == "sdk_migration_disk_space_insufficient"
+    )
+
+
+def test_migration_state_codes_pass_through_because_they_are_already_actionable() -> None:
+    assert (
+        migration_failure_code(_Coded("sdk_data_plane_migration_bundle_conflict"))
+        == "sdk_data_plane_migration_bundle_conflict"
+    )
+    assert (
+        migration_failure_code(_Coded("sdk_migration_swap_state_ambiguous"))
+        == "sdk_migration_swap_state_ambiguous"
+    )
+
+
+def test_unknown_failures_keep_their_message_out_of_the_code() -> None:
+    error = RuntimeError("/Users/someone/private/path blew up")
+    assert migration_failure_code(error) == "sdk_migration_preflight_failed"
+    assert (
+        migration_failure_code(error, fallback="sdk_migration_bundle_copy_failed")
+        == "sdk_migration_bundle_copy_failed"
+    )
+
+
+def test_a_non_code_attribute_never_becomes_a_code() -> None:
+    class Odd(RuntimeError):
+        code = 17
+
+    assert migration_failure_code(Odd("x")) == "sdk_migration_preflight_failed"
+
+
+def test_the_chain_survives_a_cycle() -> None:
+    first = RuntimeError("first")
+    second = RuntimeError("second")
+    first.__cause__ = second
+    second.__cause__ = first
+
+    assert migration_failure_code(first) == "sdk_migration_preflight_failed"
+
+
+@pytest.mark.parametrize(
+    "code",
+    [
+        "sdk_migration_disk_space_insufficient",
+        "sdk_migration_preflight_failed",
+        "sdk_migration_bundle_copy_failed",
+        "sdk_migration_cutover_failed",
+    ],
+)
+def test_the_codes_the_desktop_translates_stay_in_sync(code: str) -> None:
+    """Desktop maps these exact names to sentences; renaming one blinds it."""
+    desktop = Path(__file__).resolve().parents[2] / "desktop" / "src" / "stable-runtime.ts"
+    assert desktop.is_file(), desktop
+    assert f'"{code}"' in desktop.read_text(encoding="utf-8")

memhub/apps/core/tests/test_cli_restore.py

@@ -1,10 +1,14 @@
+import asyncio
 import io
+import json
 import sqlite3
-from types import SimpleNamespace
 
 from one_memory import cli
 from one_memory.config import Settings
+from one_memory.memory_server.migration_runner import SqliteMigrationStore, migrate
+from one_memory.memory_server.migrations import MIGRATIONS
 from one_memory.runtime_client import RuntimeStatus
+from one_memory.sdk_data_plane_migration import SdkDataPlaneMigrationStore
 
 
 def test_restore_requires_a_source(tmp_path, monkeypatch) -> None:
@@ -110,56 +114,7 @@ def test_restore_sdk_bundle_uses_full_data_plane_restore(tmp_path, monkeypatch)
     assert '"restored_files":4' in document
 
 
-def test_migration_bundle_refuses_while_runtime_is_running(tmp_path, monkeypatch) -> None:
-    monkeypatch.setattr(cli, "load_settings", lambda: Settings(_env_file=None, data_dir=tmp_path))
-    monkeypatch.setattr(
-        cli,
-        "inspect_runtime",
-        lambda _data_dir: RuntimeStatus("running", "runtime_verified"),
-    )
-    output = io.StringIO()
-
-    code = cli.run(
-        [
-            "migration-bundle",
-            "--bundle-id",
-            "migration-1",
-            "--runtime-build",
-            "build-13",
-            "--yes",
-            "--json",
-        ],
-        output=output,
-    )
-    assert code == 4
-    assert "sdk_migration_bundle_runtime_running" in output.getvalue()
-
-
-def test_migration_bundle_previews_before_writing(tmp_path, monkeypatch) -> None:
-    monkeypatch.setattr(cli, "load_settings", lambda: Settings(_env_file=None, data_dir=tmp_path))
-    monkeypatch.setattr(
-        cli,
-        "inspect_runtime",
-        lambda _data_dir: RuntimeStatus("stopped", "runtime_receipt_missing"),
-    )
-    output = io.StringIO()
-
-    code = cli.run(
-        [
-            "migration-bundle",
-            "--bundle-id",
-            "migration-1",
-            "--runtime-build",
-            "build-13",
-            "--json",
-        ],
-        output=output,
-    )
-    assert code == 4
-    assert '"state": "preview"' in output.getvalue() or '"state":"preview"' in output.getvalue()
-
-
-def test_migration_bundle_success_reports_bundle_identity(tmp_path, monkeypatch) -> None:
+def _stopped(monkeypatch, tmp_path) -> None:
     monkeypatch.setattr(cli, "load_settings", lambda: Settings(_env_file=None, data_dir=tmp_path))
     monkeypatch.setattr(
         cli,
@@ -167,221 +122,22 @@ def test_migration_bundle_success_reports_bundle_identity(tmp_path, monkeypatch)
         lambda _data_dir: RuntimeStatus("stopped", "runtime_receipt_missing"),
     )
 
-    class Result:
-        bundle_id = "migration-1"
-        bundle_path = tmp_path / "backups" / "migration-1.sdk-bundle"
-        file_count = 4
-        total_bytes = 123
 
-    monkeypatch.setattr(
-        "one_memory.sdk_migration_bundle.create_sdk_migration_bundle",
-        lambda **_kwargs: Result(),
-    )
-    output = io.StringIO()
-    code = cli.run(
-        [
-            "migration-bundle",
-            "--bundle-id",
-            "migration-1",
-            "--runtime-build",
-            "build-13",
-            "--yes",
-            "--json",
-        ],
-        output=output,
-    )
-    assert code == 0
-    document = output.getvalue().replace(" ", "")
-    assert '"state":"created"' in document
-    assert '"bundle_id":"migration-1"' in document
-
-
-def test_migration_bundle_reuses_an_existing_verified_bundle(tmp_path, monkeypatch) -> None:
-    from one_memory.sdk_migration_bundle import MigrationBundleRejected
-
-    monkeypatch.setattr(cli, "load_settings", lambda: Settings(_env_file=None, data_dir=tmp_path))
-    monkeypatch.setattr(
-        cli,
-        "inspect_runtime",
-        lambda _data_dir: RuntimeStatus("stopped", "runtime_receipt_missing"),
-    )
-
-    def exists(**_kwargs):
-        raise MigrationBundleRejected("sdk_migration_bundle_exists")
-
-    monkeypatch.setattr(
-        "one_memory.sdk_migration_bundle.create_sdk_migration_bundle",
-        exists,
-    )
-    monkeypatch.setattr(
-        "one_memory.sdk_migration_bundle.inspect_sdk_migration_bundle",
-        lambda **_kwargs: {"file_count": 4, "total_bytes": 123},
-    )
-    output = io.StringIO()
-
-    code = cli.run(
-        [
-            "migration-bundle",
-            "--bundle-id",
-            "migration-1",
-            "--runtime-build",
-            "build-13",
-            "--yes",
-            "--json",
-        ],
-        output=output,
-    )
-
-    assert code == 0
-    assert '"state": "reused"' in output.getvalue()
-
-
-def test_sdk_migrate_preview_never_commits_cutover(tmp_path, monkeypatch) -> None:
-    monkeypatch.setattr(cli, "load_settings", lambda: Settings(_env_file=None, data_dir=tmp_path))
-    monkeypatch.setattr(
-        cli,
-        "inspect_runtime",
-        lambda _data_dir: RuntimeStatus("stopped", "runtime_receipt_missing"),
-    )
-    output = io.StringIO()
-    code = cli.run(
-        [
-            "sdk-migrate",
-            "--migration-id",
-            "migration-1",
-            "--bundle-id",
-            "bundle-1",
-            "--json",
-        ],
-        output=output,
-    )
-    assert code == 4
-    document = output.getvalue().replace(" ", "")
-    assert '"state":"preview"' in document
-    assert '"commits_cutover":false' in document
-
-
-def test_sdk_migrate_runs_offline_backfill_without_cutover(tmp_path, monkeypatch) -> None:
-    monkeypatch.setattr(cli, "load_settings", lambda: Settings(_env_file=None, data_dir=tmp_path))
-    monkeypatch.setattr(
-        cli,
-        "inspect_runtime",
-        lambda _data_dir: RuntimeStatus("stopped", "runtime_receipt_missing"),
-    )
-    monkeypatch.setattr(
-        "one_memory.sdk_migration_bundle.inspect_sdk_migration_bundle",
-        lambda **_kwargs: {"runtime_build": "v1.0.4"},
-    )
-
-    async def fake_migrate(*_args, **_kwargs):
-        return None
-
-    monkeypatch.setattr("one_memory.memory_server.migration_runner.migrate", fake_migrate)
-
-    profile = SimpleNamespace(
-        has_api_key=False,
-        embedding_provider_kind="openai",
-        embedding_model="embedding",
-        embedding_dimensions=1024,
-    )
-
-    class Profiles:
-        def __init__(self, _database):
-            pass
-
-        def active_identity(self):
-            return ("profile", 1)
-
-        def snapshot(self, *_identity):
-            return profile
-
-        def resolve_snapshot_secret(self, *_args):
-            return None
-
-    class Secrets:
-        pass
-
-    class Bridge:
-        def __init__(self, **_kwargs):
-            pass
-
-        async def close(self):
-            return None
-
-    async def backfill(**_kwargs):
-        return SimpleNamespace(
-            state="ready",
-            phase="reconciled",
-            migration_id="migration-1",
-            counts={"verified": 2},
-            memory_items=1,
-            raw_items=1,
+def _legacy_home(tmp_path) -> None:
+    database = tmp_path / "one-memory.sqlite3"
+    asyncio.run(migrate(SqliteMigrationStore(database), MIGRATIONS))
+    with sqlite3.connect(database) as connection:
+        connection.execute(
+            "INSERT INTO evidence ("
+            "evidence_id,idempotency_key,user_id,source_id,conversation_id,"
+            "content_kind,role,written_by,verbatim,content_hash,revision,"
+            "imported_at,updated_at"
+            ") VALUES ('e1','k','local','cursor','s','raw_conversation',"
+            "'user','person','hello','hash',1,'now','now')"
         )
 
-    monkeypatch.setattr("one_memory.model_profiles.ModelProfileStore", Profiles)
-    monkeypatch.setattr("one_memory.secrets.KeyringSecretStore", Secrets)
-    monkeypatch.setattr("one_memory.hy_memory_bridge.HyMemoryBridge", Bridge)
-    monkeypatch.setattr(
-        "one_memory.sdk_migration_runner.run_sdk_data_plane_backfill",
-        backfill,
-    )
-    output = io.StringIO()
-    code = cli.run(
-        [
-            "sdk-migrate",
-            "--migration-id",
-            "migration-1",
-            "--bundle-id",
-            "bundle-1",
-            "--yes",
-            "--json",
-        ],
-        output=output,
-    )
-    assert code == 0
-    document = output.getvalue().replace(" ", "")
-    assert '"state":"ready"' in document
-    assert '"commits_cutover":false' in document
 
-
-def test_sdk_cutover_is_offline_confirmed_and_returns_marker(tmp_path, monkeypatch) -> None:
-    monkeypatch.setattr(cli, "load_settings", lambda: Settings(_env_file=None, data_dir=tmp_path))
-    monkeypatch.setattr(
-        cli,
-        "inspect_runtime",
-        lambda _data_dir: RuntimeStatus("stopped", "runtime_receipt_missing"),
-    )
-    monkeypatch.setattr(
-        "one_memory.sdk_migration_runner.commit_reconciled_cutover",
-        lambda **_kwargs: SimpleNamespace(
-            migration_id="migration-1",
-            backup_bundle_id="bundle-1",
-            sdk_version="2.0.5.dev3",
-            completed_at="2026-10-21T00:00:00Z",
-        ),
-    )
-    output = io.StringIO()
-
-    code = cli.run(
-        [
-            "sdk-cutover",
-            "--migration-id",
-            "migration-1",
-            "--bundle-id",
-            "bundle-1",
-            "--yes",
-            "--json",
-        ],
-        output=output,
-    )
-
-    assert code == 0
-    document = output.getvalue().replace(" ", "")
-    assert '"state":"cutover"' in document
-    assert '"sdk_version":"2.0.5.dev3"' in document
-
-
-def test_sdk_cutover_never_runs_while_runtime_is_live(tmp_path, monkeypatch) -> None:
+def test_sdk_migrate_never_runs_while_runtime_is_live(tmp_path, monkeypatch) -> None:
     monkeypatch.setattr(cli, "load_settings", lambda: Settings(_env_file=None, data_dir=tmp_path))
     monkeypatch.setattr(
         cli,
@@ -389,35 +145,72 @@ def test_sdk_cutover_never_runs_while_runtime_is_live(tmp_path, monkeypatch) ->
         lambda _data_dir: RuntimeStatus("running", "healthy"),
     )
     output = io.StringIO()
+    code = cli.run(["sdk-migrate", "--runtime-build", "20", "--yes", "--json"], output=output)
+    assert code == 4
+    assert json.loads(output.getvalue())["code"] == "sdk_migration_runtime_running"
 
-    code = cli.run(
-        [
-            "sdk-cutover",
-            "--migration-id",
-            "migration-1",
-            "--bundle-id",
-            "bundle-1",
-            "--yes",
-            "--json",
-        ],
-        output=output,
+
+def test_sdk_migrate_previews_without_confirmation(tmp_path, monkeypatch) -> None:
+    _stopped(monkeypatch, tmp_path)
+    _legacy_home(tmp_path)
+    output = io.StringIO()
+    assert cli.run(["sdk-migrate", "--runtime-build", "20", "--json"], output=output) == 4
+    assert json.loads(output.getvalue())["state"] == "preview"
+    assert SdkDataPlaneMigrationStore(tmp_path / "one-memory.sqlite3").authority().value == (
+        "legacy"
     )
 
-    assert code == 4
-    assert '"code": "sdk_cutover_runtime_running"' in output.getvalue()
+
+def test_sdk_migrate_requires_the_runtime_build(tmp_path, monkeypatch) -> None:
+    _stopped(monkeypatch, tmp_path)
+    assert cli.run(["sdk-migrate", "--yes"], output=io.StringIO()) == 2
 
 
-def test_sdk_migration_status_distinguishes_empty_and_legacy_data(tmp_path, monkeypatch) -> None:
+def test_sdk_migrate_reports_each_step_then_the_result(tmp_path, monkeypatch) -> None:
+    _stopped(monkeypatch, tmp_path)
+    _legacy_home(tmp_path)
+    output = io.StringIO()
+
+    code = cli.run(["sdk-migrate", "--runtime-build", "20", "--yes", "--json"], output=output)
+
+    assert code == 0, output.getvalue()
+    lines = [json.loads(line) for line in output.getvalue().splitlines()]
+    assert [line["step"] for line in lines[:-1]] == ["bundle", "prepare", "cutover", "cleanup"]
+    assert lines[-1]["state"] == "cutover"
+    assert lines[-1]["bundle_id"] == f"{lines[-1]['migration_id']}-pre-cutover"
+    assert not (tmp_path / ".sdk-migration-capture-fence.json").exists()
+
+    again = io.StringIO()
+    assert cli.run(["sdk-migrate", "--runtime-build", "20", "--yes", "--json"], output=again) == 0
+    assert json.loads(again.getvalue())["state"] == "not_required"
+
+
+def test_sdk_migrate_failure_is_one_named_code(tmp_path, monkeypatch) -> None:
+    import errno
+
+    _stopped(monkeypatch, tmp_path)
+
+    def full_disk(**_kwargs):
+        raise OSError(errno.ENOSPC, "No space left on device")
+
+    monkeypatch.setattr("one_memory.sdk_migration_runner.run_startup_migration", full_disk)
+    output = io.StringIO()
+    assert cli.run(["sdk-migrate", "--runtime-build", "20", "--yes", "--json"], output=output) == 1
+    assert json.loads(output.getvalue()) == {
+        "schema_version": 1,
+        "state": "failed",
+        "code": "sdk_migration_disk_space_insufficient",
+    }
+
+
+def test_sdk_migration_status_says_whether_migrate_has_work(tmp_path, monkeypatch) -> None:
     monkeypatch.setattr(cli, "load_settings", lambda: Settings(_env_file=None, data_dir=tmp_path))
-    empty = io.StringIO()
-    assert cli.run(["sdk-migration-status", "--json"], output=empty) == 0
-    assert '"state": "empty"' in empty.getvalue()
 
-    database = tmp_path / "one-memory.sqlite3"
-    with sqlite3.connect(database) as connection:
-        connection.execute("CREATE TABLE evidence (evidence_id TEXT PRIMARY KEY)")
-        connection.execute("INSERT INTO evidence VALUES ('e1')")
-    required = io.StringIO()
-    assert cli.run(["sdk-migration-status", "--json"], output=required) == 0
-    assert '"state": "required"' in required.getvalue()
-    assert '"raw": 1' in required.getvalue()
+    def status() -> dict[str, object]:
+        output = io.StringIO()
+        assert cli.run(["sdk-migration-status", "--json"], output=output) == 0
+        return json.loads(output.getvalue())
+
+    assert status()["state"] == "not_required"
+    _legacy_home(tmp_path)
+    assert status() == {"schema_version": 1, "authority": "legacy", "state": "required"}

memhub/apps/core/tests/test_contract_parity.py

@@ -50,21 +50,3 @@ def test_governed_memory_carries_the_same_fields_on_both_sides() -> None:
     }
 
     assert typescript == set(GovernedMemory.model_fields)
-
-
-def test_an_agent_owned_memory_is_usable_and_never_passes_as_the_person(tmp_path) -> None:
-    """助手输出里的事实直接可用，但始终标成助手说的。
-
-    召回读到它时，origin 让别的 AI 应用知道这是模型的陈述、不是本人的判决。
-    已有的同内容记忆，也不会被模型输出改掉来源标记。
-    """
-    from tests.test_memory_processing import (
-        test_agent_output_keeps_the_origin_of_a_same_content_memory,
-        test_agent_owned_memories_land_usable_but_marked_as_the_assistant,
-    )
-
-    created, existing = tmp_path / "created", tmp_path / "existing"
-    created.mkdir()
-    existing.mkdir()
-    test_agent_owned_memories_land_usable_but_marked_as_the_assistant(created)
-    test_agent_output_keeps_the_origin_of_a_same_content_memory(existing, MemoryOrigin.AGENT)

memhub/apps/core/tests/test_conversation_search.py

@@ -1,10 +1,8 @@
 """Searching imported conversations before any generation model is configured.
 
 This search is literal, not semantic, and that is a correction rather than a
-shortcut. The vector store `semantic_search` reads is written only by
-extraction (`memory_processing.py` EXTRACT tasks), and extraction is precisely
-what a user without a generation model does not have: their store is empty, so
-a semantic path would return nothing no matter how it was wired.
+shortcut. Literal conversation search must remain available before a model is
+configured and independently of SDK semantic-memory extraction.
 """
 
 import sqlite3

memhub/apps/core/tests/test_conversation_turn_grouping.py

@@ -4,9 +4,9 @@ from __future__ import annotations
 
 from datetime import UTC, datetime
 
+from one_memory.conversation_processing import _conversation_chunks
 from one_memory.imports.models import EvidenceInput, EvidenceOrigin, OriginMetadata, WrittenBy
 from one_memory.imports.storage import EvidenceStore
-from one_memory.memory_processing import _conversation_chunks
 
 
 def _records(tmp_path, *speakers: tuple[str, str, str]) -> list:

memhub/apps/core/tests/test_credential_responsiveness.py

@@ -1,58 +1,16 @@
 import asyncio
 import threading
-import time
 
 import httpx
 import pytest
 from test_memory_qa import _grounded_setup, _ndjson_stream
-from test_onboarding_profile import FakeGraph, _app, _client, _setup, _supervisor
+from test_onboarding_profile import _app, _client
 
 from one_memory.memory_qa import MemoryAnswerer
 from one_memory.model_probe import ModelProber
 from one_memory.secrets import SecretStoreUnavailableError
 
 
-@pytest.mark.parametrize(
-    "operation,args",
-    [
-        ("digest", ()),
-        ("graph_snapshot", ()),
-        ("graph_width_mismatch", ()),
-        ("semantic_search", ("example",)),
-        ("update_hy_memory", ("id", "corrected")),
-        ("delete_hy_memory", ("id",)),
-        ("decide_graph_width", ("keep_existing",)),
-    ],
-)
-@pytest.mark.parametrize("phase", ["active", "secret"])
-def test_credentials_do_not_block_the_event_loop(tmp_path, monkeypatch, operation, args, phase):
-    stores = _setup(tmp_path)
-    supervisor = _supervisor(tmp_path, stores, FakeGraph(()))
-    method = "active_identity" if phase == "active" else "resolve_snapshot_secret"
-    original = getattr(supervisor.profiles, method)
-    called = threading.Event()
-
-    def waiting(*args, **kwargs):
-        called.set()
-        time.sleep(0.25)
-        return original(*args, **kwargs)
-
-    monkeypatch.setattr(supervisor.profiles, method, waiting)
-
-    async def scenario():
-        task = asyncio.create_task(getattr(supervisor, operation)(*args))
-        started = time.monotonic()
-        await asyncio.sleep(0.04)
-        elapsed = time.monotonic() - started
-        # Some graph operations have no fake implementation; those exceptions
-        # are irrelevant to whether a credential read stops the event loop.
-        await asyncio.gather(task, return_exceptions=True)
-        assert called.is_set()
-        assert elapsed < 0.18, f"Credential wait blocked the loop for {elapsed:.3f}s"
-
-    asyncio.run(scenario())
-
-
 def test_disabled_automatic_processing_does_not_digest_onboarding(tmp_path):
     async def scenario():
         app = _app(tmp_path)
@@ -64,7 +22,7 @@ def test_disabled_automatic_processing_does_not_digest_onboarding(tmp_path):
                 calls.append(True)
                 return {}
 
-            app.state.memory_processing_supervisor.digest = digest
+            app.state.sdk_memory_runtime.digest = digest
             response = await client.post("/api/v1/onboarding/complete")
             assert response.status_code == 200
             assert calls == []

memhub/apps/core/tests/test_detailed_logging.py

@@ -35,7 +35,7 @@ def test_live_log_level_applies_to_full_hy_and_provider_call_chain(
     set_diagnostic_log_level(tmp_path, "debug")
 
     assert diagnostic_log_level(tmp_path) == "debug"
-    assert logging.getLogger("one_memory.memory_processing").isEnabledFor(logging.DEBUG)
+    assert logging.getLogger("one_memory.sdk_write_processor").isEnabledFor(logging.DEBUG)
     assert logging.getLogger("hy_memory.runtime").isEnabledFor(logging.DEBUG)
     assert logging.getLogger("hy_memory.client").isEnabledFor(logging.DEBUG)
     assert logging.getLogger("hy_memory.agent.llm_provider").isEnabledFor(logging.DEBUG)

memhub/apps/core/tests/test_diagnostic_log.py

@@ -18,11 +18,13 @@ from one_memory.diagnostic_log import (
     diagnostic_exception,
     diagnostics_enabled,
     diagnostics_requested,
+    read_trace_events,
     record_database_error,
     reset_diagnostics,
     reset_trace_id,
     resource_snapshot,
 )
+from one_memory.exception_reporting import report_exception
 
 
 def test_debug_events_are_collected_only_at_debug_level(tmp_path, monkeypatch) -> None:
@@ -105,7 +107,31 @@ def test_structured_log_is_private_correlated_and_owner_only(tmp_path):
         assert log.path.stat().st_mode & 0o777 == 0o600
 
 
-def test_arbitrary_detail_and_absolute_stack_are_rejected(tmp_path):
+def test_arbitrary_valid_detail_keys_and_scalars_are_recorded(tmp_path):
+    log = DiagnosticLog(tmp_path)
+    details = {
+        "Custom.metric-v1": "ready",
+        "noneValue": None,
+        "boolValue": False,
+        "intValue": 7,
+        "floatValue": 1.5,
+    }
+    log.write(
+        level="info",
+        component="core.lifecycle",
+        event="core.started",
+        outcome="succeeded",
+        details=details,
+    )
+    assert log.flush()
+    assert json.loads(log.path.read_text())["details"] == details
+
+
+@pytest.mark.parametrize(
+    "key",
+    ["__proto__", "prototype", "constructor", "_private", "9startsWithDigit", "a" * 65],
+)
+def test_dangerous_or_malformed_detail_keys_are_rejected(tmp_path, key):
     log = DiagnosticLog(tmp_path)
     with pytest.raises(ValueError, match="diagnostic_detail_key_invalid"):
         log.write(
@@ -113,8 +139,76 @@ def test_arbitrary_detail_and_absolute_stack_are_rejected(tmp_path):
             component="hy.bridge",
             event="hy.failed",
             outcome="failed",
-            details={"prompt": "private"},
+            details={key: "value"},
         )
+
+
+@pytest.mark.parametrize("value", [{}, [], object()])
+def test_complex_detail_values_are_rejected(tmp_path, value):
+    log = DiagnosticLog(tmp_path)
+    with pytest.raises(ValueError, match="diagnostic_detail_value_invalid"):
+        log.write(
+            level="error",
+            component="hy.bridge",
+            event="hy.failed",
+            outcome="failed",
+            details={"payload": value},
+        )
+
+
+@pytest.mark.parametrize("value", [float("nan"), float("inf"), float("-inf")])
+def test_non_finite_detail_values_are_rejected(tmp_path, value):
+    log = DiagnosticLog(tmp_path)
+    with pytest.raises(ValueError, match="diagnostic_detail_value_invalid"):
+        log.write(
+            level="error",
+            component="hy.bridge",
+            event="hy.failed",
+            outcome="failed",
+            details={"metric": value},
+        )
+
+
+@pytest.mark.parametrize("value", [2**53, -(2**53), float(2**53), -float(2**53)])
+def test_numeric_detail_values_outside_json_safe_range_are_rejected(tmp_path, value):
+    log = DiagnosticLog(tmp_path)
+    with pytest.raises(ValueError, match="diagnostic_detail_value_invalid"):
+        log.write(
+            level="error",
+            component="hy.bridge",
+            event="hy.failed",
+            outcome="failed",
+            details={"count": value},
+        )
+
+
+@pytest.mark.parametrize("value", ["", "nul\0byte", "carriage\rreturn", "line\nfeed", "x" * 257])
+def test_invalid_detail_strings_are_rejected(tmp_path, value):
+    log = DiagnosticLog(tmp_path)
+    with pytest.raises(ValueError, match="diagnostic_detail_value_invalid"):
+        log.write(
+            level="error",
+            component="hy.bridge",
+            event="hy.failed",
+            outcome="failed",
+            details={"message": value},
+        )
+
+
+def test_more_than_32_detail_keys_are_rejected(tmp_path):
+    log = DiagnosticLog(tmp_path)
+    with pytest.raises(ValueError, match="diagnostic_detail_count_invalid"):
+        log.write(
+            level="error",
+            component="hy.bridge",
+            event="hy.failed",
+            outcome="failed",
+            details={f"key{index}": index for index in range(33)},
+        )
+
+
+def test_absolute_or_credential_bearing_stack_is_rejected(tmp_path):
+    log = DiagnosticLog(tmp_path)
     with pytest.raises(ValueError, match="diagnostic_stack_invalid"):
         log.write(
             level="error",
@@ -123,16 +217,90 @@ def test_arbitrary_detail_and_absolute_stack_are_rejected(tmp_path):
             outcome="failed",
             safe_stack=("/Users/me/private.py:1",),
         )
+    with pytest.raises(ValueError, match="diagnostic_stack_invalid"):
+        log.write(
+            level="error",
+            component="hy.bridge",
+            event="hy.failed",
+            outcome="failed",
+            safe_stack=("one_memory/sk-live-secret-token.py:1:call",),
+        )
 
 
-def test_a_rejected_payload_does_not_break_the_caller(tmp_path, monkeypatch) -> None:
-    """Rejection protects the log; it must not become the caller's outage.
+def test_safe_stack_uses_strict_shared_grammar_and_sixteen_line_limit(tmp_path):
+    log = DiagnosticLog(tmp_path)
+    for line in (
+        "arbitrary stack text",
+        "thread=worker|threading.py:12:wait",
+        "FILE:///Users/me/private.py:1:call",
+        "one_memory/worker.py:9:call /Users/private",
+        "one_memory/GOCSPX-privateCredential.py:9:call",
+        "packages/ASIA1234567890ABCDEF/file.ts:1:1",
+        "packages/Aa1Bb2Cc3Dd4Ee5Ff6Gg7Hh8/file.ts:1:1",
+    ):
+        with pytest.raises(ValueError, match="diagnostic_stack_invalid"):
+            log.write(
+                level="error",
+                component="hy.bridge",
+                event="hy.failed",
+                outcome="failed",
+                safe_stack=(line,),
+            )
 
-    Two of our own nodes shipped with unlisted `details` keys. Both sat on a
-    request path, so the rejection surfaced as a 500 to the person using the
-    app — including the sedimentation switch, whose every request failed
-    because logging the toggle raised before the endpoint could answer.
-    """
+    accepted = (
+        "apps/desktop/src/main.ts:80:1",
+        "one_memory/diagnostic_log.py:895:_safe_stack_line",
+        "threading.py:1045:_bootstrap_inner",
+        "thread|threading.py:1045:_bootstrap_inner > one_memory/worker.py:9:call",
+    )
+    log.write(
+        level="error",
+        component="hy.bridge",
+        event="hy.failed",
+        outcome="failed",
+        safe_stack=accepted,
+    )
+    with pytest.raises(ValueError, match="diagnostic_stack_invalid"):
+        log.write(
+            level="error",
+            component="hy.bridge",
+            event="hy.failed",
+            outcome="failed",
+            safe_stack=tuple(f"one_memory/worker.py:{index + 1}:call" for index in range(17)),
+        )
+    assert log.flush()
+    assert json.loads(log.path.read_text())["safe_stack"] == list(accepted)
+
+
+def test_invalid_level_is_strict_but_diagnostic_event_remains_fail_open(
+    tmp_path, monkeypatch
+) -> None:
+    log = DiagnosticLog(tmp_path)
+    with pytest.raises(ValueError, match="diagnostic_level_invalid"):
+        log.write(
+            level="fatal",  # type: ignore[arg-type]
+            component="core.lifecycle",
+            event="core.failed",
+            outcome="failed",
+        )
+
+    monkeypatch.setenv("ONE_MEMORY_DIAGNOSTICS", "1")
+    receipt = configure_diagnostics(tmp_path / "fail-open")
+    try:
+        diagnostic_event(
+            level="fatal",
+            component="core.lifecycle",
+            event="core.failed",
+            outcome="failed",
+        )
+    finally:
+        reset_diagnostics()
+    assert not receipt.path.exists()
+
+
+def test_a_rejected_payload_does_not_break_the_caller_or_later_events(
+    tmp_path, monkeypatch
+) -> None:
     monkeypatch.setenv("ONE_MEMORY_DIAGNOSTICS", "1")
     receipt = configure_diagnostics(tmp_path)
     try:
@@ -141,20 +309,122 @@ def test_a_rejected_payload_does_not_break_the_caller(tmp_path, monkeypatch) ->
             component="memory.sedimentation",
             event="memory_sedimentation.toggled",
             outcome="succeeded",
-            details={"prompt": "private"},
+            details={"payload": ["not", "scalar"]},
+        )
+        diagnostic_event(
+            level="info",
+            component="memory.sedimentation",
+            event="memory_sedimentation.toggled",
+            outcome="succeeded",
+            details={"custom.metric": 1},
         )
     finally:
         reset_diagnostics()
-    # Rejected, so nothing is written — but the call returned.
-    assert not receipt.path.exists() or receipt.path.read_text() == ""
+    events = [json.loads(line) for line in receipt.path.read_text().splitlines()]
+    assert [event["details"] for event in events] == [{"custom.metric": 1}]
+
+
+def test_trace_projection_does_not_expose_arbitrary_local_detail_keys(tmp_path):
+    log = DiagnosticLog(tmp_path)
+    log.write(
+        level="info",
+        component="core.api",
+        event="memory.request.completed",
+        outcome="succeeded",
+        request_id="request-1",
+        details={
+            "phase": "recall",
+            "reason": "/Users/private/query.txt",
+            "status": "sk-live-secret-token",
+            "custom.private": "local-only",
+        },
+    )
+    assert log.flush()
+    events = read_trace_events(tmp_path, request_id="request-1")
+    assert events[0]["details"] == {"phase": "recall"}
+
+
+def test_trace_projection_rejects_symlinked_directory_and_redacts_top_level_secrets(
+    tmp_path,
+) -> None:
+    request_id = "request-1"
+    logs = tmp_path / "logs"
+    logs.mkdir()
+    event = {
+        "timestamp": "2026-10-23T01:00:00Z",
+        "request_id": request_id,
+        "component": "secret.token",
+        "event": "credential.failed",
+        "outcome": "Aa1Bb2Cc3Dd4Ee5Ff6Gg7Hh8",
+        "error_code": "GOCSPX-clientSecret_A1",
+        "error_type": "ASIA1234567890ABCDEF",
+        "details": {
+            "phase": "recall",
+            "reason": "x_ASIA1234567890ABCDEF",
+            "mode": "x_GOCSPX-clientSecret_A1",
+            "status": "Aa1Bb2Cc3Dd4Ee5Ff6Gg7Hh8",
+            "failure_site": "AIzaSyExampleCredential1234567890",
+            "count": 1,
+        },
+    }
+    (logs / "diagnostic-core.jsonl").write_text(json.dumps(event) + "\n")
+    projected = read_trace_events(tmp_path, request_id=request_id)
+    assert projected == (
+        {
+            "timestamp": "2026-10-23T01:00:00Z",
+            "component": "diagnostic.redacted",
+            "event": "diagnostic.redacted",
+            "outcome": "redacted",
+            "details": {"phase": "recall", "count": 1},
+        },
+    )
+
+    external = tmp_path / "external"
+    external.mkdir()
+    (logs / "diagnostic-core.jsonl").unlink()
+    logs.rmdir()
+    logs.symlink_to(external, target_is_directory=True)
+    with pytest.raises(ValueError, match="diagnostic_directory_unsafe"):
+        read_trace_events(tmp_path, request_id=request_id)
+
+
+def test_credential_detection_is_conservative_and_matches_export_rules() -> None:
+    for value in (
+        "ASIA1234567890ABCDEF",
+        "GOCSPX-clientSecret_A1",
+        "Aa1Bb2Cc3Dd4Ee5Ff6Gg7Hh8",
+        "Aa1Bb2Cc3Dd4Ee5Ff6Gg7Hh8_",
+    ):
+        assert diagnostic_log._contains_credential_material(value)
+    for value in (
+        "11111111-1111-4111-8111-111111111111",
+        "abcdef1234567890abcdef1234567890abcdef12",
+        "cursor",
+    ):
+        assert not diagnostic_log._contains_credential_material(value)
+
+
+def test_trace_reader_never_follows_a_diagnostic_file_symlink(tmp_path) -> None:
+    logs = tmp_path / "logs"
+    logs.mkdir()
+    outside = tmp_path / "outside.jsonl"
+    outside.write_text(
+        json.dumps(
+            {
+                "timestamp": "2026-10-23T01:00:00Z",
+                "request_id": "request-1",
+                "component": "core.api",
+                "event": "memory.request.completed",
+                "outcome": "succeeded",
+            }
+        )
+        + "\n"
+    )
+    (logs / "diagnostic-core.jsonl").symlink_to(outside)
+    assert read_trace_events(tmp_path, request_id="request-1") == ()
 
 
 def test_the_sedimentation_toggle_payload_is_recorded(tmp_path, monkeypatch) -> None:
-    """`enabled` is a bounded boolean and has to be a legal detail key.
-
-    Omitting it from the allow-list is what made the switch's own endpoint
-    answer 500 while still saving the new value.
-    """
     monkeypatch.setenv("ONE_MEMORY_DIAGNOSTICS", "1")
     receipt = configure_diagnostics(tmp_path)
     try:
@@ -181,7 +451,7 @@ def test_exception_diagnostic_keeps_code_location_not_message():
             component="hy.bridge",
             event="hy.failed",
             outcome="failed",
-            details={"prompt": secret},
+            details={"prompt": f"{secret}\n"},
         )
     except ValueError as error:
         safe = diagnostic_exception(error)
@@ -191,6 +461,28 @@ def test_exception_diagnostic_keeps_code_location_not_message():
     assert str(__file__) not in json.dumps(safe)
 
 
+def test_report_exception_never_writes_the_exception_message_to_diagnostics(
+    tmp_path, monkeypatch
+) -> None:
+    monkeypatch.setenv("ONE_MEMORY_DIAGNOSTICS", "1")
+    receipt = configure_diagnostics(tmp_path)
+    secret = "sk-live-private-provider-response"
+    try:
+        report_exception(
+            "core.test",
+            "core.test.failed",
+            RuntimeError(f"provider failed with {secret} at /Users/private/query.txt"),
+        )
+    finally:
+        reset_diagnostics()
+    payload = receipt.path.read_text()
+    assert secret not in payload
+    assert "/Users/private" not in payload
+    event = json.loads(payload)
+    assert event["details"] == {"count": 1}
+    assert event["error_type"] == "RuntimeError"
+
+
 def test_old_logs_are_removed_but_symlinks_are_never_followed(tmp_path):
     log = DiagnosticLog(tmp_path)
     log.path.parent.mkdir(parents=True)

memhub/apps/core/tests/test_diagnostic_log_off_loop.py

@@ -449,7 +449,7 @@ def test_invalid_input_is_still_rejected_on_the_caller_side(tmp_path):
             component="hy.bridge",
             event="hy.failed",
             outcome="failed",
-            details={"prompt": "private"},
+            details={"invalid key": "private"},
         )
     log.close()