Compare commits

...
Author SHA1 Message Date
alvinfeiandClaude Opus 5.5 567a8b349d test(desktop): let the attach-race winner exist before the loop gives up
The test wrote the winning Core's receipt and token from inside the
spawn, without awaiting it, while the readiness loop ran with zero
delay. On a slow runner the loop finished first, the re-read found a
receipt without its token, and start() reported a false conflict. The
probe now waits for the winner's files, which is the order the test is
about.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-27 09:00:50 +08:00
alvinfeiandClaude Opus 5.5 d89afb3887 fix(memhub): make Core ready before it resumes unfinished work
A home with pending imports and write jobs kept /health closed while
the lifespan resumed them, so the Runtime hit its 60s readiness budget,
killed a healthy Core with core_start_timeout, and the next Core began
the same recovery again. The write drain also started before readiness
and held the single writer the rest of startup needed.

Readiness is now only "the stores are open and Core answers". After
core.ready a recovery task runs, step by step: the post-cutover
metadata repair, the durable write drain, raw retention (or the legacy
pending-dispatch replay), interrupted imports, live sync, source
reconcile and the legacy catch-up. A failed step is logged and named in
/health without stopping readiness or the steps after it. /health
reports recovery as pending, running, done or failed. Shutdown cancels
recovery first and still releases the content-backfill gate.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-26 23:59:38 +08:00
feitianxiang 0abf87d4b6 Merge pull request #180 from Tencent-Hunyuan/fix/memhub-sdk-only-migration-guard
fix(memhub): serve SDK only and stop on an unfinished upgrade migration
2026-10-26 22:34:53 +08:00
alvinfeiandClaude Opus 5.5 4e5a44b8dc fix(memhub): pass the lint gate
Format cli.test.ts, and mark the map's search form role as intentional:
biome's useSemanticElements also fails main (#179 merge), and a <search>
wrapper would add a box around the flex row .map-search styles.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-26 22:33:12 +08:00
alvinfeiandClaude Opus 5.5 bf36ba765b fix(memhub): serve SDK only and stop on an unfinished upgrade migration
Every 1.0.4 home must pass the offline SDK cutover, but the runtime
service could start Core on a home whose migration failed, so it served
legacy. Callers also asked Core for the memory authority before many
calls and kept v1 data-plane branches that only ever 409 on SDK.

Migration and startup:
- Write every migration step to logs/sdk-migration.log (content-free
  JSON lines) from Desktop, CLI, the runtime service and Core.
- Migrate before the runtime service spawns Core; Core refuses a legacy
  home with sdk_migration_required.
- Desktop shows a dialog that reveals the log and opens a GitHub issue,
  then quits. CLI install/upgrade no longer roll back to a version
  without the gate; start/restart/status/doctor name the log.
- Cycle the service when the migration status is unreadable so no
  pre-upgrade Runtime keeps serving.

SDK only:
- Web, host-bridge and CLI call v2 only; drop the authority query.
- Delete Core's v1 data plane, the Reader and /api/v2/memory-authority.
- Fix MCP memory write on v2, MemHub_conversations, memhub recall and
  status, v2 call events and WorkBuddy delivery reports.
- Remove dead features: matters, boundary peek/ack, memory plan/apply,
  the legacy library and its v1 operations.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-26 22:14:46 +08:00
feitianxiang e9cce69650 Merge pull request #179 from Tencent-Hunyuan/fix/memory-search-button
fix(memhub): search on submit and set SDK recall limits apart
2026-10-26 16:08:51 +08:00
alvinfeiandClaude Opus 5.5 2fed7adf6b fix(memhub): set the SDK context recall lanes apart
POST /api/v2/context/search passed the caller's `limit` as the normal,
schema and proactive intention top-k alike, so limit=5 ran three full
recall lanes including the intention lane the SDK keeps off by default.
The v2 request now takes `limit` (default 10), `schema_limit` (default
3) and `intention_limit` (default 1) and passes each through. The host
broker's default item cap follows v2's default of 10.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-26 15:48:48 +08:00
alvinfeiandClaude Opus 5.5 7308ed9df2 fix(memhub): search the memory list on submit, over 20 results
- The map's memory search no longer queries on every keystroke: typing
  edits the words and the search runs on the 搜索 button or Enter.
  Submitting the words already applied re-reads the page.
- Core asks SDK search for a 20-memory relevance window instead of 500;
  the larger window made a single search slow enough to freeze the page.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-26 15:27:20 +08:00
feitianxiang 1c0cbf6d05 Merge pull request #176 from Tencent-Hunyuan/fix/memory-page-sdk-consistency
fix(memhub): make the SDK memory page one consistent page
2026-10-26 09:10:22 +08:00
alvinfeiandClaude Opus 5.5 71854a3510 fix(memhub): make the SDK memory page one consistent page
Root cause: the page read its rows from an unfiltered limit=300 window
while its count, source labels and filter came from the paged library,
and the cutover backfill (import_exact) had written every legacy memory
with source "default" and session "default_session".

- Web: on SDK authority the list, count, source/time labels and graph
  all come from one library page and the graph for the same source and
  window, read together and polled together. "default" is a named,
  filterable 未识别来源 bucket; impossible conversation times are not
  shown as when a conversation happened; detail timestamps carry their
  year; empty states tell "no match" from "empty source"; a finished
  memory note re-reads the page at once.
- Core: library search keeps the window's rows that contain the query
  words (the Chroma store has no keyword channel), reports when the
  window was full, and the graph endpoint takes the list's source and
  offset and no longer chains "default_session" memories together.
- Repair v2 links SDK memories to legacy evidence through
  governed_memory_evidence and write-job ids across every pre-cutover
  bundle, and settles remaining future memory_at values to gmt_created.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-25 23:23:33 +08:00
feitianxiang fcb1bcd866 Merge pull request #174 from Tencent-Hunyuan/chore/bump-build-22
chore(release): bump MemHub build number to 22
2026-10-25 15:32:06 +08:00
alvinfeiandClaude Opus 5.5 6305e4cea8 chore(release): bump MemHub build number to 22
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-25 15:31:24 +08:00
feitianxiang a4b0969248 Merge pull request #173 from Tencent-Hunyuan/fix/memory-library-build23-acceptance
fix(memhub): repair Build 23 memory library issues
2026-10-25 15:28:12 +08:00
alvinfeiandClaude Opus 5.5 e0920311a1 fix(memhub): close gaps in the Build 23 memory library repair
- Cursor: resume WAL-backed stores from their rowid. The cursor identity
  now carries the WAL change token, but resume_checkpoint/touched_cursor
  still compared the bare dev:ino, so every change re-read the store from
  rowid 0 and reported a rewind.
- Library search: leave the proactive intention lane off, list only
  active hits, bound the per-hit client.get fan-out, and count skipped
  rows from the single page parse instead of re-validating the page.
- Metadata repair: pool evidence per SDK memory so disagreeing sources
  stay unknown instead of last-output-wins; close the legacy, coding and
  Chroma handles; register both openings in the main-database gate,
  which the previous commit left failing.
- Runtime: the same-build opt-in fails closed when the selected tree's
  manifest is unreadable, so it cannot bypass downgrade refusal.
- Detail: keep the heading within 72 characters and break English text
  at a word boundary.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-25 15:21:31 +08:00
alvinfeiandCursor e6426f6fd0 fix(memhub): repair SDK memory library acceptance issues
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-25 14:53:38 +08:00
feitianxiang 99e3c8d9d7 Merge pull request #172 from Tencent-Hunyuan/fix/sdk-memory-native-detail
fix(memhub): reuse the memory map for SDK-native details
2026-10-25 09:54:03 +08:00
alvinfeiandCursor 896a9c6c83 fix(memhub): render SDK memory details natively
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-25 09:46:34 +08:00
kristinhe 193fabeeb4 fix(memhub): open an SDK memory where the map put it
On SDK authority, clicking a memory switched the whole page to the SDK
library — a second screen made of status tabs and a ledger, where the memory
page's design has one list, one graph and one inspector. Those three already
exist: they are the components the mock-data screen was built from. This wires
the real SDK reads into them instead of standing a second screen beside them.

- opening a memory reads it through v2 and draws it in the map's inspector;
  edits and deletes go to updateSdkMemory / deleteSdkMemory, and a write
  re-reads the page instead of waiting for the poll
- the left list gets a real library page (50 a page, with the total and each
  memory's source and conversation time), so the count, the paging footer and
  the provenance line read the way the design reads
- a memory with no cited excerpt no longer renders an evidence block and a
  source line that can only say "not available"

The SDK library panel is left as it was; it is no longer on this path.
2026-10-25 09:36:02 +08:00
feitianxiang 0aeb3e8760 Merge pull request #167 from Tencent-Hunyuan/fix/simplify-sdk-migration
fix(memhub): migrate to SDK authority without copying memory
2026-10-24 23:18:32 +08:00
237 changed files with 8102 additions and 32901 deletions

No files matched your search

+22
View File
@@ -4,6 +4,28 @@ Hy Memory and MemHub are versioned and released independently.
## Unreleased
- The SDK memory page now reads its list, count, source and time labels and
graph from one page response for one source filter, so rows, totals and the
map no longer disagree. Memories with no recorded source are shown and
filterable as "unidentified source"; library search keeps only memories
that contain the words within the SDK's relevance window.
- Metadata repair v2 follows `governed_memory_evidence` across every
pre-cutover bundle, which restores the source of memories the cutover
backfilled without one, and moves impossible future conversation times back
to the write time.
- Reused the existing three-column memory map for SDK-authority list/detail
views and removed the retired "original evidence" surface. SDK-native status,
source, conversation time and creation time now render without fabricated
revisions; source and text filters are applied by the paged Core endpoint.
- SDK memory search now calls the SDK search contract instead of listing the
entire library in Core. Corrupt empty records are skipped and diagnosed
without mutating data or failing a page, and post-cutover metadata repair
restores source and original times from the verified migration bundle.
- Kept Cursor's WAL change token separate from the file mtime so a hash can no
longer appear as a conversation date centuries in the future.
- A trusted packaged Runtime now repairs a dirty same-build installation by
staging a clean replacement, and `runtime_service_not_ready` is rendered as
user-facing startup guidance instead of an internal code.
- Fixed legacy-to-SDK startup migration to adopt native Coding Experience and
Working Fact IDs instead of importing them as Chat layers. Raw migration now
rebuilds bounded user/assistant turns from the latest three days, excludes
+2
View File
@@ -15,6 +15,8 @@
3. MemHub **不存 memory 实体**;**可以存 `memory_id`** 做组织/回执。原文进入 **SDK raw**。
4. SDK raw 只保留滚动三天;一条 raw 对应一个完整 turn,只含真人 user 与
assistant,tool 不持久化。提取后的高层 memory 不受该窗口影响。
5. `source_raw_ids` 是 SDK 内部溯源关系,不是产品 Evidence 合同;记忆详情不提供
“原始依据”或按 raw ID 展开原文的入口。
生产入口通过共享 Runtime 租约取得公开 `HyMemoryClient`;MemHub 只在调用前执行
模型配置、密钥和 harness/model 门禁,不维护第二套 memory 正文权威。
+23 -48
View File
@@ -21,18 +21,21 @@ const TOKEN = "probe-token-not-a-real-credential";
const MEMORY = {
memory_id: "probe-1",
user_id: "local",
kind: "chat",
status: "confirmed",
content: "部署脚本要先跑 verify_merge_plan。",
scopes: ["all-apps"],
evidence_ids: ["probe-e1"],
origin: "extracted",
layer: "l2_fact",
status: "active",
is_latest: true,
supersedes: [],
superseded_by: [],
tags: [],
revision: 2,
created_at: "2026-09-19T00:00:00Z",
updated_at: "2026-09-19T00:00:00Z",
deleted_at: null,
owner: null,
source_raw_ids: ["probe-raw-1"],
user_id: "local",
agent_id: "codex",
session_id: "probe-session",
source: "codex",
memory_at: 1758240000,
gmt_created: 1758240000,
};
const ROUTES = new Map([
@@ -124,26 +127,6 @@ const ROUTES = new Map([
},
],
],
[
"/api/v1/memory-tasks",
[
{
task_id: "probe-task",
request_id: "probe-request",
user_id: "local",
kind: "extract",
input_ref: "probe",
status: "waiting_for_model",
profile_id: "probe-profile",
profile_revision: 1,
profile_snapshot: {},
attempts: 0,
last_error_code: "model_missing",
created_at: "2026-09-19T00:00:00Z",
updated_at: "2026-09-19T00:00:01Z",
},
],
],
[
"/api/v1/model-profiles",
[
@@ -159,28 +142,20 @@ const ROUTES = new Map([
},
],
],
["/api/v1/home-summary", { memory_count: 1, latest: null, observed_at: "T" }],
[
"/api/v1/memory-library",
{ total: 1, offset: 0, limit: 100, items: [MEMORY] },
],
[
"/api/v1/memories/probe-1",
"/api/v2/home-summary",
{
memory: MEMORY,
evidence: [
{
evidence_id: "probe-e1",
available: true,
source_id: "codex",
written_by: "user",
created_at: "2026-09-19T00:00:00Z",
verbatim: "部署前先跑 verify_merge_plan,别靠读 diff 猜冲突。",
},
],
history: [{ memory_id: "probe-1", revision: 2, action: "confirm" }],
schema_version: 2,
memory_count: 1,
latest: MEMORY,
observed_at: "2026-09-19T00:00:01Z",
},
],
[
"/api/v2/memory-library",
{ memories: [MEMORY], total: 1, offset: 0, limit: 100 },
],
["/api/v2/memories/probe-1", MEMORY],
]);
const seen = [];
+304 -38
View File
@@ -4,6 +4,7 @@ import path from "node:path";
import {
claimRuntimeServiceForApp,
readRuntimeServiceOwner,
recordSdkMigrationBlock,
resolveRuntimeCliConfig,
} from "@one-memory/runtime-control";
import { describe, expect, it, vi } from "vitest";
@@ -790,7 +791,7 @@ describe("public One Memory CLI", () => {
}
});
it("upgrade failure restarts the restored artifact with the same data identity", async () => {
it("upgrade restart failure keeps the new version and never restarts the old one", async () => {
vi.clearAllMocks();
runtimeMocks.planCliLink.mockResolvedValueOnce({
schema_version: 1,
@@ -832,25 +833,27 @@ describe("public One Memory CLI", () => {
command: "upgrade",
code: "restart failed",
});
expect(runtimeMocks.rollbackLocalInstall).toHaveBeenCalledOnce();
// The CLI manages the objective aliases the package does not own
// (`memhub` + `memhub-cli`); both are rolled back.
expect(aliasMocks.rollbackManagedCliLink).toHaveBeenCalledTimes(2);
expect(runtimeMocks.restartUserService).toHaveBeenCalledTimes(2);
for (const call of runtimeMocks.restartUserService.mock
.calls as unknown[][]) {
expect(call[0]).toMatchObject({
dataDir: "/tmp/custom data",
installRoot: "/tmp/install/current",
});
}
expect(JSON.parse(streams.stderr())).toMatchObject({
rolled_back: false,
});
// Product rule: the previous Runtime may predate the SDK migration gate,
// so it is never restored after the migration was handed the home.
expect(runtimeMocks.rollbackLocalInstall).not.toHaveBeenCalled();
expect(aliasMocks.rollbackManagedCliLink).not.toHaveBeenCalled();
expect(runtimeMocks.restartUserService).toHaveBeenCalledOnce();
expect(
(runtimeMocks.restartUserService.mock.calls as unknown[][])[0]?.[0],
).toMatchObject({
dataDir: "/tmp/custom data",
installRoot: "/tmp/install/current",
});
});
// O-22 (2026-09-29 C8 H17): `upgrade --yes` on a manifest-valid but
// crash-looping version reported exit 0 and left `current` on the bad
// version. A restart that merely returns is not proof the new Runtime came
// up, so the CLI now verifies health and rolls back if it never does.
it("rolls back and fails when the upgraded Runtime never becomes healthy", async () => {
it("fails without rolling back when the upgraded Runtime never becomes healthy", async () => {
vi.clearAllMocks();
runtimeMocks.planCliLink.mockResolvedValueOnce({
schema_version: 1,
@@ -890,9 +893,9 @@ describe("public One Memory CLI", () => {
command: "upgrade",
code: "upgrade_health_check_failed",
});
// Rolled back to the previous version and restarted it.
expect(runtimeMocks.rollbackLocalInstall).toHaveBeenCalledOnce();
expect(runtimeMocks.restartUserService).toHaveBeenCalledTimes(2);
// Kept on the new version; the old Runtime is not restarted.
expect(runtimeMocks.rollbackLocalInstall).not.toHaveBeenCalled();
expect(runtimeMocks.restartUserService).toHaveBeenCalledOnce();
});
const upgradeArguments = [
@@ -948,7 +951,7 @@ describe("public One Memory CLI", () => {
);
});
it("rolls back to the previous version when the migration does not finish", async () => {
it("stays on the new version, not serving, when the migration does not finish", async () => {
vi.clearAllMocks();
runtimeMocks.planCliLink.mockResolvedValueOnce(presentCliLink);
runtimeMocks.runSdkMigration.mockResolvedValueOnce({
@@ -967,11 +970,87 @@ describe("public One Memory CLI", () => {
streams.restore();
}
expect(code!).toBe(1);
expect(JSON.parse(streams.stderr())).toMatchObject({
const failure = JSON.parse(streams.stderr());
expect(failure).toMatchObject({
command: "upgrade",
code: "sdk_migration_disk_space_insufficient",
// The one file support needs, and where to send it.
log_file: "/tmp/custom data/logs/sdk-migration.log",
issues_url: "https://github.com/Tencent-Hunyuan/Hy-Memory/issues/new",
});
expect(failure.message).toContain(
"/tmp/custom data/logs/sdk-migration.log",
);
expect(failure.message).toContain("GitHub");
expect(failure.message).not.toContain("回滚到上一版本");
expect(runtimeMocks.runSdkMigration).toHaveBeenCalledWith(
expect.objectContaining({ caller: "cli-upgrade" }),
);
// No rollback: the old Runtime has no migration gate. The new service is
// (re)started so its gate, not the old Runtime, holds the data dir.
expect(runtimeMocks.rollbackLocalInstall).not.toHaveBeenCalled();
expect(aliasMocks.rollbackManagedCliLink).not.toHaveBeenCalled();
expect(runtimeMocks.restartUserService).toHaveBeenCalledOnce();
expect(
(runtimeMocks.restartUserService.mock.calls as unknown[][])[0]?.[0],
).toMatchObject({
installRoot: "/tmp/install/current",
});
});
it("stays on the new version when the migration itself throws", async () => {
vi.clearAllMocks();
runtimeMocks.planCliLink.mockResolvedValueOnce(presentCliLink);
runtimeMocks.runSdkMigration.mockRejectedValueOnce(
new Error("sdk_migration_fence_failed"),
);
const streams = captureStreams();
let code: number;
try {
code = await run(
upgradeArguments,
{},
{ waitForRuntimeReady: async () => true },
);
} finally {
streams.restore();
}
expect(code!).toBe(1);
expect(JSON.parse(streams.stderr())).toMatchObject({
command: "upgrade",
code: "sdk_migration_fence_failed",
log_file: "/tmp/custom data/logs/sdk-migration.log",
issues_url: "https://github.com/Tencent-Hunyuan/Hy-Memory/issues/new",
});
expect(runtimeMocks.rollbackLocalInstall).not.toHaveBeenCalled();
});
it("still rolls back a failure before the migration touched the home", async () => {
vi.clearAllMocks();
runtimeMocks.planCliLink.mockResolvedValueOnce(presentCliLink);
aliasMocks.installManagedCliLink.mockRejectedValueOnce(
new Error("cli_link_failed"),
);
const streams = captureStreams();
let code: number;
try {
code = await run(
upgradeArguments,
{},
{ waitForRuntimeReady: async () => true },
);
} finally {
streams.restore();
}
expect(code!).toBe(1);
expect(JSON.parse(streams.stderr())).toMatchObject({
command: "upgrade",
code: "cli_link_failed",
});
expect(runtimeMocks.runSdkMigration).not.toHaveBeenCalled();
expect(runtimeMocks.rollbackLocalInstall).toHaveBeenCalledOnce();
// The previous Runtime was never stopped; nothing is (re)started.
expect(runtimeMocks.restartUserService).not.toHaveBeenCalled();
});
it("keeps the new version when a migrated home is slow to come up", async () => {
@@ -1492,7 +1571,7 @@ describe("public One Memory CLI", () => {
expect(JSON.parse(output.value()).apply).toBe(true);
});
it("rolls back the distribution when service installation fails", async () => {
it("keeps the distribution when service installation fails after the migration ran", async () => {
runtimeMocks.applyLocalInstall.mockResolvedValueOnce({
schema_version: 1,
source: "/tmp/distribution",
@@ -1533,13 +1612,14 @@ describe("public One Memory CLI", () => {
command: "install",
status: "error",
code: "service failed",
rolled_back: false,
});
expect(runtimeMocks.rollbackLocalInstall).toHaveBeenCalledOnce();
expect(runtimeMocks.rollbackCliLink).toHaveBeenCalledOnce();
expect(aliasMocks.rollbackManagedCliLink).toHaveBeenCalledTimes(2);
expect(runtimeMocks.rollbackLocalInstall).not.toHaveBeenCalled();
expect(runtimeMocks.rollbackCliLink).not.toHaveBeenCalled();
expect(aliasMocks.rollbackManagedCliLink).not.toHaveBeenCalled();
});
it("upgrades only an existing managed CLI and rolls back a failed restart", async () => {
it("upgrades only an existing managed CLI and keeps it after a failed restart", async () => {
runtimeMocks.planCliLink.mockResolvedValueOnce({
schema_version: 1,
platform: "linux",
@@ -1579,7 +1659,7 @@ describe("public One Memory CLI", () => {
status: "error",
code: "restart failed",
});
expect(runtimeMocks.rollbackLocalInstall).toHaveBeenCalledOnce();
expect(runtimeMocks.rollbackLocalInstall).not.toHaveBeenCalled();
expect(runtimeMocks.installCliLink).not.toHaveBeenCalled();
});
@@ -2003,6 +2083,101 @@ describe("lifecycle commands accept --json", () => {
});
});
// A Runtime service that refuses to serve an unmigrated home must be reported
// as that, with the migration log and where to send it, not as a generic
// "not ready within budget".
describe("lifecycle commands report a migration block", () => {
async function blockedHome() {
const dataDir = await mkdtemp(path.join(tmpdir(), "memhub-blocked-"));
await recordSdkMigrationBlock(
dataDir,
"sdk_migration_disk_space_insufficient",
"runtime-service",
);
return dataDir;
}
// darwin: kickstart "restarts" but never becomes ready. Elsewhere: no
// service, so the direct spawn fails. Both must surface the block.
const blockedControls = () => ({
restartManagedRuntime: vi.fn(async () => "restarted" as const),
waitForRuntimeReady: vi.fn(async () => false),
stopRuntime: vi.fn(async () => "stopped" as const),
ensureRuntime: vi.fn(async () => {
throw new Error("Timed out waiting for MemHub Runtime");
}),
});
it("restart --json prints the block code, log file and issues URL", async () => {
const dataDir = await blockedHome();
const output = captureStdout();
let code: number;
try {
code = await run(
["restart", "--json"],
{ ONE_MEMORY_DATA_DIR: dataDir },
blockedControls(),
);
} finally {
output.restore();
await rm(dataDir, { recursive: true, force: true });
}
expect(code!).toBe(1);
const lines = output.value().trim().split("\n");
const document = JSON.parse(lines[lines.length - 1]!);
expect(document).toMatchObject({
command: "restart",
state: "blocked",
code: "sdk_migration_disk_space_insufficient",
log_file: path.join(dataDir, "logs", "sdk-migration.log"),
issues_url: "https://github.com/Tencent-Hunyuan/Hy-Memory/issues/new",
});
expect(output.value()).not.toContain("runtime_not_ready_within_budget");
});
it("restart in text mode names the log file and GitHub issues", async () => {
const dataDir = await blockedHome();
const output = captureStdout();
try {
expect(
await run(
["restart"],
{ ONE_MEMORY_DATA_DIR: dataDir },
blockedControls(),
),
).toBe(1);
} finally {
output.restore();
await rm(dataDir, { recursive: true, force: true });
}
expect(output.value()).toContain("sdk_migration_disk_space_insufficient");
expect(output.value()).toContain(
path.join(dataDir, "logs", "sdk-migration.log"),
);
expect(output.value()).toContain(
"https://github.com/Tencent-Hunyuan/Hy-Memory/issues/new",
);
});
it("ui --json reports the block instead of runtime_not_running", async () => {
const dataDir = await blockedHome();
const output = captureStdout();
let code: number;
try {
code = await run(["ui", "--json"], { ONE_MEMORY_DATA_DIR: dataDir });
} finally {
output.restore();
await rm(dataDir, { recursive: true, force: true });
}
expect(code!).not.toBe(0);
expect(JSON.parse(output.value())).toMatchObject({
command: "ui",
state: "blocked",
code: "sdk_migration_disk_space_insufficient",
log_file: path.join(dataDir, "logs", "sdk-migration.log"),
});
});
});
// The objective names `memhub host install <host>` (host as a positional). Both
// that and the `--host <id>` flag form must reach the same command; naming both
// at once is ambiguous and refused rather than guessed.
@@ -2112,7 +2287,7 @@ describe("install gates on Runtime readiness", () => {
});
}
it("fails and rolls back when the installed Runtime never becomes ready", async () => {
it("fails without rolling back when the installed Runtime never becomes ready", async () => {
vi.clearAllMocks();
stubInstallPlan();
const streams = captureStreams();
@@ -2141,10 +2316,10 @@ describe("install gates on Runtime readiness", () => {
}
expect(code!).toBe(1);
expect(streams.stderr()).toContain("install_runtime_not_ready");
expect(runtimeMocks.rollbackLocalInstall).toHaveBeenCalledOnce();
expect(runtimeMocks.rollbackLocalInstall).not.toHaveBeenCalled();
expect(
lifecycleMocks.rollbackPreparedServiceInstall,
).toHaveBeenCalledOnce();
).not.toHaveBeenCalled();
});
it("takes CLI ownership after a successful install over an App-owned Runtime", async () => {
@@ -2189,7 +2364,7 @@ describe("install gates on Runtime readiness", () => {
}
});
it("restores App ownership before relaunching the old Runtime after install failure", async () => {
it("never hands the home back to the App Runtime after the migration ran", async () => {
vi.clearAllMocks();
stubInstallPlan();
const root = await mkdtemp(path.join(tmpdir(), "memhub-cli-owner-"));
@@ -2199,14 +2374,6 @@ describe("install gates on Runtime readiness", () => {
appBundle: "/Applications/MemHub.app",
appExecutable: "/Applications/MemHub.app/Contents/MacOS/MemHub",
});
lifecycleMocks.rollbackPreparedServiceInstall.mockImplementationOnce(
async (_prepared: unknown, rollback: () => Promise<void>) => {
await rollback();
await expect(readRuntimeServiceOwner(dataDir)).resolves.toMatchObject({
owner: "app",
});
},
);
const streams = captureStreams();
try {
expect(
@@ -2229,15 +2396,114 @@ describe("install gates on Runtime readiness", () => {
{ waitForRuntimeReady: async () => false },
),
).toBe(1);
// The App Runtime may predate the SDK migration gate: CLI keeps it.
await expect(readRuntimeServiceOwner(dataDir)).resolves.toMatchObject({
owner: "app",
owner: "cli",
});
expect(
lifecycleMocks.rollbackPreparedServiceInstall,
).not.toHaveBeenCalled();
} finally {
streams.restore();
await rm(root, { recursive: true, force: true });
}
});
it("keeps the install, not serving, when its migration is deferred", async () => {
vi.clearAllMocks();
stubInstallPlan();
runtimeMocks.runSdkMigration.mockResolvedValueOnce({
state: "deferred",
code: "sdk_migration_memory_unreadable",
});
const waitForRuntimeReady = vi.fn(async () => true);
const streams = captureStreams();
let code: number;
try {
code = await run(
[
"install",
"--source",
"/tmp/dist",
"--home",
"/tmp/home",
"--install-root",
"/tmp/install",
"--bin-dir",
"/tmp/bin",
"--data-dir",
"/tmp/data",
"--yes",
],
{},
{ waitForRuntimeReady },
);
} finally {
streams.restore();
}
expect(code!).toBe(1);
const failure = JSON.parse(streams.stderr());
expect(failure).toMatchObject({
command: "install",
code: "sdk_migration_memory_unreadable",
log_file: "/tmp/data/logs/sdk-migration.log",
issues_url: "https://github.com/Tencent-Hunyuan/Hy-Memory/issues/new",
});
expect(failure.message).toContain("/tmp/data/logs/sdk-migration.log");
expect(runtimeMocks.runSdkMigration).toHaveBeenCalledWith(
expect.objectContaining({ caller: "cli-install" }),
);
// The new (gated) service owns the data dir; the previous registration is
// never re-bootstrapped and nothing is rolled back.
expect(lifecycleMocks.applyPreparedServiceInstall).toHaveBeenCalledWith(
expect.objectContaining({ restorePreviousOnFailure: false }),
);
expect(runtimeMocks.rollbackLocalInstall).not.toHaveBeenCalled();
expect(
lifecycleMocks.rollbackPreparedServiceInstall,
).not.toHaveBeenCalled();
expect(waitForRuntimeReady).not.toHaveBeenCalled();
});
it("still rolls back an install that failed before its migration ran", async () => {
vi.clearAllMocks();
stubInstallPlan();
runtimeMocks.installCliLink.mockRejectedValueOnce(
new Error("cli_link_failed"),
);
const streams = captureStreams();
let code: number;
try {
code = await run(
[
"install",
"--source",
"/tmp/dist",
"--home",
"/tmp/home",
"--install-root",
"/tmp/install",
"--bin-dir",
"/tmp/bin",
"--data-dir",
"/tmp/data",
"--yes",
],
{},
{ waitForRuntimeReady: async () => true },
);
} finally {
streams.restore();
}
expect(code!).toBe(1);
expect(JSON.parse(streams.stderr())).toMatchObject({
command: "install",
code: "cli_link_failed",
});
expect(runtimeMocks.runSdkMigration).not.toHaveBeenCalled();
expect(runtimeMocks.rollbackLocalInstall).toHaveBeenCalledOnce();
});
it("succeeds when the installed Runtime becomes ready", async () => {
vi.clearAllMocks();
stubInstallPlan();
+281 -266
View File
@@ -32,6 +32,7 @@ import {
processIsAlive,
type ResolvedRuntimeRelease,
RuntimeDiscoveryError,
readSdkMigrationBlock,
removeRuntimeServiceOwner,
resolveRuntimeCliConfig,
resolveRuntimeRelease,
@@ -42,6 +43,7 @@ import {
runSdkMigration,
runtimeInstallLayout,
type SdkMigrationResult,
sdkMigrationLogPath,
stopRuntimeProcess,
} from "@one-memory/runtime-control";
import axios from "axios";
@@ -177,21 +179,10 @@ import {
removeManagedCliLink,
rollbackManagedCliLink,
} from "./managed-cli-link.js";
import {
executeMemoryMutation,
inspectMemoryPlan,
type MemoryFlowDocument,
memoryFlowExitCode,
memoryPlanAcceptsContent,
parseMemoryContent,
planExistingMemoryMutation,
planMemoryAdd,
readMemoryMutationReceipt,
renderMemoryFlowText,
} from "./memory-flows.js";
import {
isMemoryId,
type MemoryQuery,
parseMemoryContent,
readMemoryDetail,
readMemoryPage,
renderMemoryDetailText,
@@ -1197,6 +1188,28 @@ async function runCommand(
: state === "already_running"
? "MemHub Runtime 已在运行。\n"
: "Runtime 服务已注册但未就绪;请运行 memhub logs 查看原因。\n";
// A Runtime that did not come up because its service refuses to serve an
// unmigrated home reports that cause, the migration log and where to send
// it, instead of a generic budget timeout.
const migrationBlocked = async (): Promise<string | null> => {
const blocked = await runtimeMigrationBlock(config.dataDir, command);
if (blocked === null) return null;
return asJson
? `${JSON.stringify({
schema_version: 1,
product: PRODUCT,
command,
state: "blocked",
code: blocked.code,
message: blocked.message,
log_file: blocked.log_file,
issues_url: blocked.issues_url,
})}\n`
: `${blocked.message}\n`;
};
const notReady = async () =>
(await migrationBlocked()) ??
lifecycle("not-ready", "runtime_not_ready_within_budget");
// A launchd-managed Runtime must be driven through launchctl. Spawning a
// second process while the service holds the data directory is exactly what
// produced "Runtime exited before becoming ready".
@@ -1241,16 +1254,12 @@ async function runCommand(
process.stdout.write(lifecycle("restarted", "runtime_verified"));
return EXIT_OK;
}
process.stdout.write(
lifecycle("not-ready", "runtime_not_ready_within_budget"),
);
process.stdout.write(await notReady());
await restoreEnableState();
return EXIT_FAILED;
}
if (restarted === "not-ready") {
process.stdout.write(
lifecycle("not-ready", "runtime_not_ready_within_budget"),
);
process.stdout.write(await notReady());
await restoreEnableState();
return EXIT_FAILED;
}
@@ -1306,16 +1315,12 @@ async function runCommand(
process.stdout.write(lifecycle("started", "runtime_verified"));
return EXIT_OK;
}
process.stdout.write(
lifecycle("not-ready", "runtime_not_ready_within_budget"),
);
process.stdout.write(await notReady());
await restoreEnableState();
return EXIT_FAILED;
}
if (startedManaged === "not-ready") {
process.stdout.write(
lifecycle("not-ready", "runtime_not_ready_within_budget"),
);
process.stdout.write(await notReady());
await restoreEnableState();
return EXIT_FAILED;
}
@@ -1344,6 +1349,11 @@ async function runCommand(
// A spawned Runtime that exits before ready is a failure of this command,
// not a crash: report it as a structured document (JSON) or a plain
// message, with a non-zero exit, instead of letting it escape.
const blocked = await migrationBlocked();
if (blocked !== null) {
process.stdout.write(blocked);
return EXIT_FAILED;
}
const message = error instanceof Error ? error.message : String(error);
process.stdout.write(
asJson
@@ -1540,6 +1550,24 @@ async function runCommand(
return uiExitCode(result);
};
if (attachment === null) {
const migration = await runtimeMigrationBlock(config.dataDir, "ui");
if (migration !== null) {
emit(
asJson
? `${JSON.stringify({
schema_version: 1,
product: PRODUCT,
command: "ui",
state: "blocked",
code: migration.code,
message: migration.message,
log_file: migration.log_file,
issues_url: migration.issues_url,
})}\n`
: `ui 未打开:${migration.code}\n${migration.message}\n`,
);
return uiExitCode({ state: "blocked", code: migration.code });
}
return emitUi(
{ state: "blocked", code: "runtime_not_running" },
`先运行 ${PRODUCT} start 启动 Runtime。`,
@@ -1644,7 +1672,7 @@ async function runCommand(
});
if (runtime.state !== "running") {
const document: ContextSearchDocument = {
schema_version: 1,
schema_version: 2,
product: PRODUCT,
state: "unavailable",
code: runtime.code,
@@ -1668,7 +1696,7 @@ async function runCommand(
timeout_ms: parsed.timeoutMs,
})
: ({
schema_version: 1,
schema_version: 2,
product: PRODUCT,
state: "unavailable",
code: attached.failure ?? "runtime_attach_failed",
@@ -2434,20 +2462,20 @@ async function runCommand(
};
}
if (blocked || !reader) {
if (
parsed.kind === "update" ||
parsed.kind === "delete" ||
parsed.kind === "plan-add" ||
parsed.kind === "plan-existing" ||
parsed.kind === "apply"
) {
const document: MemoryFlowDocument = {
schema_version: 1,
if (parsed.kind === "update" || parsed.kind === "delete") {
const document = {
schema_version: 2,
product: PRODUCT,
state: "unknown",
operation: parsed.kind,
memory_id: parsed.memoryId,
code: blocked?.code ?? "runtime_attach_failed",
};
emitMemoryFlow(document, asJson);
emit(
asJson
? `${JSON.stringify(document)}\n`
: `记忆操作未完成:${document.state}(${document.code})\n`,
);
return runtime.state === "stopped" ? 3 : 1;
}
const document =
@@ -2541,73 +2569,7 @@ async function runCommand(
);
return outcome.state === "committed" ? EXIT_OK : EXIT_FAILED;
}
if (parsed.kind === "plan-add") {
const content = await readMemoryContentFile(parsed.contentFile);
if (content === null) return usage();
const document = planMemoryAdd({
content,
memoryKind: parsed.memoryKind,
scopes: parsed.scopes,
evidenceIds: parsed.evidenceIds,
});
emitMemoryFlow(document, asJson);
return memoryFlowExitCode(document);
}
if (parsed.kind === "plan-existing") {
let content: string | undefined;
if (parsed.contentFile) {
const read = await readMemoryContentFile(parsed.contentFile);
if (read === null) return usage();
content = read;
}
const document = await planExistingMemoryMutation(reader, {
operation: parsed.operation,
memoryId: parsed.memoryId,
...(content === undefined ? {} : { content }),
});
emitMemoryFlow(document, asJson);
return memoryFlowExitCode(document);
}
if (!operations) {
const document: MemoryFlowDocument = {
schema_version: 1,
product: PRODUCT,
state: "unknown",
code: "runtime_attach_failed",
};
emitMemoryFlow(document, asJson);
return EXIT_FAILED;
}
const file = await readBoundedJson(parsed.planFile);
if (file === null) return usage();
const plan = unwrapPlan(file);
const identity = inspectMemoryPlan(plan);
if (!identity) return usage();
let content: string | undefined;
if (parsed.contentFile) {
const read = await readMemoryContentFile(parsed.contentFile);
if (read === null) return usage();
content = read;
}
if (
(identity.needsContent && content === undefined) ||
(!identity.needsContent && content !== undefined) ||
!memoryPlanAcceptsContent(plan, content)
) {
return usage();
}
const attempt = await claimPlanAttempt(parsed.planFile, {
schema_version: 1,
operation_id: identity.operationId,
plan_digest: identity.digest,
});
if (attempt === "invalid") return usage();
const document =
attempt === "prior"
? await readMemoryMutationReceipt(reader, plan)
: await executeMemoryMutation(plan, content, operations, reader);
emitMemoryFlow(document, asJson);
return memoryFlowExitCode(document);
return usage();
}
if (named === "calls" || named === "import-file") {
const parsed = parseUiWriteCommand(named, arguments_.slice(1));
@@ -2817,10 +2779,15 @@ async function runCommand(
if (!ready) throw new Error("runtime_service_activation_unverified");
return plan;
})
.catch((error: unknown) => {
.catch(async (error: unknown) => {
const migration =
error instanceof Error &&
error.message === "runtime_service_activation_unverified"
? await runtimeMigrationBlock(layout.dataDir, "service-install")
: null;
process.stderr.write(
`${JSON.stringify(
serviceErrorDocument("service-install", error),
migration ?? serviceErrorDocument("service-install", error),
null,
2,
)}\n`,
@@ -3155,6 +3122,9 @@ async function runCommand(
let installReceiptWritten = false;
let serviceApplied = false;
let migrated = false;
// `null` until the migration is handed this home; "failed" if it threw.
let migrationState: SdkMigrationResult["state"] | "failed" | null = null;
let deferredCode: string | null = null;
let previousOwner:
| Awaited<ReturnType<typeof claimCliServiceOwnership>>
| undefined;
@@ -3167,6 +3137,7 @@ async function runCommand(
previousOwner = await claimCliServiceOwnership(layout.dataDir);
// `install` over an existing home (reinstall, App-to-CLI takeover) must
// migrate it too, before the new Runtime starts serving it.
migrationState = "failed";
const migration = await (
controls.migrateSdkData ?? migrateSdkDataBeforeStart
)({
@@ -3174,15 +3145,24 @@ async function runCommand(
dataDir: layout.dataDir,
runtimeBuild: String(plan.build_number ?? plan.version),
asJson: true,
caller: "cli-install",
});
migrationState = migration.state;
// A deferred migration keeps this install: the new Runtime service is
// registered and its migration gate keeps Core down until the home is
// migrated. Rolling back would hand the home to the previous (App or CLI)
// Runtime, which has no gate and would serve the legacy data.
if (migration.state === "deferred") {
throw new SdkMigrationDeferred(
migration.code ?? "sdk_migration_incomplete",
);
deferredCode = migration.code ?? "sdk_migration_incomplete";
}
migrated =
migration.state === "cutover" || migration.state === "complete";
await applyPreparedServiceInstall(preparedService);
// From here the previous LaunchAgent is never re-bootstrapped: it may
// be an older Runtime with no migration gate.
await applyPreparedServiceInstall({
...preparedService,
restorePreviousOnFailure: false,
});
serviceApplied = true;
// The receipt exists so a later command can find the data dir without
// flags. An isolated install named its data dir explicitly, and writing
@@ -3198,6 +3178,7 @@ async function runCommand(
});
installReceiptWritten = true;
}
if (deferredCode !== null) throw new SdkMigrationDeferred(deferredCode);
// Registering the service is not the same as a working install: on a host
// where Core cold-start stalls, launchd reports a pid while status is
// still `runtime_service_running`. Gate on real readiness (same predicate
@@ -3233,10 +3214,38 @@ async function runCommand(
);
}
};
if (migrated) {
// An older Runtime refuses a migrated home: keep this install.
// Once the migration has been handed this home nothing rolls back: the
// previous version (App or CLI) may be an older Runtime without the
// migration gate, and restoring it could serve a home that is not on SDK
// authority. This install stays; its service keeps Core down until the
// home is migrated. Only a failure before the migration ran (the
// previous Runtime was never stopped and the service never touched) is
// rolled back below.
if (migrationState !== null) {
if (!serviceApplied && !migrated && deferredCode === null) {
// The migration threw. Hand the data dir to this version's gated
// service anyway so that the previous Runtime does not come back.
await applyPreparedServiceInstall({
...preparedService,
restorePreviousOnFailure: false,
}).catch(() => undefined);
}
process.stderr.write(
`${JSON.stringify(notReadyAfterMigration("install"), null, 2)}\n`,
`${JSON.stringify(
deferredCode !== null
? sdkMigrationFailure("install", deferredCode, layout.dataDir)
: migrated
? notReadyAfterMigration("install")
: migrationState === "failed"
? sdkMigrationFailure(
"install",
sdkMigrationErrorCode(error),
layout.dataDir,
)
: notReadyAfterUpgrade("install", error),
null,
2,
)}\n`,
);
return EXIT_FAILED;
}
@@ -3249,13 +3258,7 @@ async function runCommand(
await rollbackFilesAndOwner();
}
process.stderr.write(
`${JSON.stringify(
error instanceof SdkMigrationDeferred
? sdkMigrationFailure("install", error.code)
: serviceErrorDocument("install", error),
null,
2,
)}\n`,
`${JSON.stringify(serviceErrorDocument("install", error), null, 2)}\n`,
);
return EXIT_FAILED;
}
@@ -3618,9 +3621,17 @@ async function runCommand(
| Awaited<ReturnType<typeof claimCliServiceOwnership>>
| undefined;
let migrated = false;
const rollbackUpgrade = async (
dataDir: string | undefined,
): Promise<void> => {
// `null` until the migration is handed this home; "failed" if it threw.
let migrationState: SdkMigrationResult["state"] | "failed" | null = null;
/**
* Restore the previous version's files and ownership. Only used for a
* failure before the migration ran: the previous Runtime was never stopped
* and the service never restarted, so nothing new starts serving. After
* the migration has been handed the home the upgrade never rolls back:
* the previous Runtime may predate the migration gate and would serve a
* home that is not on SDK authority.
*/
const rollbackUpgrade = async (): Promise<void> => {
for (const receipt of memhubReceipts.reverse()) {
await rollbackManagedCliLink(receipt);
}
@@ -3630,25 +3641,29 @@ async function runCommand(
() => undefined,
);
}
await restartUserService(
runtimeInstallLayout({
platform: plan.platform,
home: parsed.home,
installRoot: canonicalManagedServiceRoot(
plan.platform,
plan.current_link,
),
dataDir,
...hourlyIsolationFromEnvironment(environment),
}),
serviceOptions,
).catch(() => undefined);
};
/**
* After a migration that did not finish, make sure the process holding the
* data dir is the new version's service (gated: it keeps Core down while
* the migration is deferred) and not the previous Runtime. The migration
* fence already stopped it unless the status probe itself failed.
*/
const keepUpgradedRuntimeOffLegacyHome = async (): Promise<void> => {
try {
await restartUserService(layout, serviceOptions);
} catch {
await (controls.stopRuntime ?? stopRuntimeProcess)({
dataDir: parsed.dataDir ?? layout.dataDir,
probeIdentity,
}).catch(() => undefined);
}
};
try {
for (const link of companionLinks) {
memhubReceipts.push(await installManagedCliLink(link));
}
previousOwner = await claimCliServiceOwnership(layout.dataDir);
migrationState = "failed";
const migration = await (
controls.migrateSdkData ?? migrateSdkDataBeforeStart
)({
@@ -3656,16 +3671,22 @@ async function runCommand(
dataDir: parsed.dataDir ?? layout.dataDir,
runtimeBuild: String(plan.build_number ?? plan.version),
asJson,
caller: "cli-upgrade",
});
migrationState = migration.state;
if (migration.state === "deferred") {
await rollbackUpgrade(parsed.dataDir);
// Stay on the new version. Its Runtime service refuses to spawn Core
// while the migration is deferred, so nothing serves the legacy home;
// rolling back would restart the old Runtime, which has no such gate.
await keepUpgradedRuntimeOffLegacyHome();
process.stderr.write(
`${JSON.stringify(sdkMigrationFailure("upgrade", migration.code ?? "sdk_migration_incomplete"), null, 2)}\n`,
`${JSON.stringify(sdkMigrationFailure("upgrade", migration.code ?? "sdk_migration_incomplete", parsed.dataDir ?? layout.dataDir), null, 2)}\n`,
);
return EXIT_FAILED;
}
// An older Runtime refuses a migrated home, so from here a failure
// keeps the new version instead of rolling back to one that cannot start.
// From here a failure keeps the new version: an older Runtime either
// refuses a migrated home or, predating the migration gate, would serve
// one that is not on SDK authority.
migrated =
migration.state === "cutover" || migration.state === "complete";
await restartUserService(layout, serviceOptions);
@@ -3673,8 +3694,8 @@ async function runCommand(
// accepted the command. O-22: the first `upgrade` implementation
// reported success even when the new artifact crash-looped, leaving
// `current` on a version that never came up. Verify the restarted
// Runtime actually becomes healthy; if it does not, roll back to the
// previously working version.
// Runtime actually becomes healthy; if it does not, fail loudly. It is
// not rolled back (product rule: SDK authority or not serving at all).
const dataDir = parsed.dataDir ?? layout.dataDir;
const ready = await (controls.waitForRuntimeReady ?? waitForRuntimeReady)(
dataDir,
@@ -3686,20 +3707,11 @@ async function runCommand(
return EXIT_FAILED;
}
if (!ready) {
await rollbackUpgrade(parsed.dataDir);
// Not rolled back: see `rollbackUpgrade`. If the service is blocked on
// the migration (it re-runs it before spawning Core), say so.
const blocked = await runtimeMigrationBlock(dataDir, "upgrade");
process.stderr.write(
`${JSON.stringify(
{
schema_version: 1,
product: PRODUCT,
command: "upgrade",
status: "error",
code: "upgrade_health_check_failed",
message: "新版本重启后未在限定时间内就绪,已回滚到上一版本。",
},
null,
2,
)}\n`,
`${JSON.stringify(blocked ?? notReadyAfterUpgrade("upgrade"), null, 2)}\n`,
);
return EXIT_FAILED;
}
@@ -3712,9 +3724,32 @@ async function runCommand(
});
}
} catch (error) {
if (!migrated) await rollbackUpgrade(parsed.dataDir);
if (migrationState === null) {
await rollbackUpgrade();
process.stderr.write(
`${JSON.stringify(serviceErrorDocument("upgrade", error), null, 2)}\n`,
);
return EXIT_FAILED;
}
if (migrated) {
// An older Runtime refuses a migrated home: keep the new version.
process.stderr.write(
`${JSON.stringify(notReadyAfterMigration("upgrade"), null, 2)}\n`,
);
return EXIT_FAILED;
}
if (migrationState === "failed") {
// The migration itself threw: its outcome is unknown, so treat the
// home as not migrated and keep the old Runtime off it.
await keepUpgradedRuntimeOffLegacyHome();
process.stderr.write(
`${JSON.stringify(sdkMigrationFailure("upgrade", sdkMigrationErrorCode(error), parsed.dataDir ?? layout.dataDir), null, 2)}\n`,
);
return EXIT_FAILED;
}
// `not_required`: the restart itself failed. Stay on the new version.
process.stderr.write(
`${JSON.stringify(migrated ? notReadyAfterMigration("upgrade") : serviceErrorDocument("upgrade", error), null, 2)}\n`,
`${JSON.stringify(notReadyAfterUpgrade("upgrade", error), null, 2)}\n`,
);
return EXIT_FAILED;
}
@@ -4100,11 +4135,13 @@ async function migrateSdkDataBeforeStart(options: {
dataDir: string;
runtimeBuild: string;
asJson: boolean;
caller?: "cli-install" | "cli-upgrade";
}): Promise<SdkMigrationResult> {
return runSdkMigration({
dataDir: options.dataDir,
core: { command: options.coreCommand },
runtimeBuild: options.runtimeBuild,
...(options.caller === undefined ? {} : { caller: options.caller }),
onProgress: (step) => {
process.stderr.write(
options.asJson
@@ -4133,33 +4170,105 @@ function notReadyAfterMigration(command: "install" | "upgrade") {
};
}
/**
* The new version did not come up and was not rolled back: the previous one
* may predate the SDK migration gate. `error` carries the underlying code
* (a failed service restart), never free text from a child process.
*/
function notReadyAfterUpgrade(command: "install" | "upgrade", error?: unknown) {
return {
schema_version: 1,
product: PRODUCT,
command,
status: "error",
code:
error !== undefined
? serviceErrorDocument(command, error).code
: command === "upgrade"
? "upgrade_health_check_failed"
: "install_runtime_not_ready",
rolled_back: false,
message:
"新版本 Runtime 没有启动或没有在限定时间内就绪。为避免旧版本打开记忆数据,已保留新版本,没有回滚。请运行 memhub restart 重试,或运行 memhub doctor 查看原因;如果仍然失败,请在 GitHub 提交反馈:" +
MIGRATION_ISSUES_URL,
issues_url: MIGRATION_ISSUES_URL,
};
}
const SDK_MIGRATION_FAILURES: Record<string, string> = {
sdk_migration_disk_space_insufficient:
"磁盘空间不足,记忆数据迁移没有完成,已回滚到上一版本。请腾出与记忆数据同等大小的空间后重试;原有记忆数据不会丢失。",
"磁盘空间不足,记忆数据迁移没有完成。请腾出与记忆数据同等大小的空间后运行 memhub restart 重试。",
sdk_migration_memory_unreadable:
"核对发现新版本还读不到你的全部记忆,没有切换数据源,已回滚到上一版本。请把这个错误码反馈给我们;原有记忆数据不会丢失。",
"核对发现新版本还读不到你的全部记忆,没有切换数据源。",
sdk_migration_model_profile_required:
"迁移需要先有可用的模型配置,用来核对新版本能否读回记忆。已回滚到上一版本;请先用 memhub models setup 完成配置后再升级。",
"迁移需要先有可用的模型配置,用来核对新版本能否读回记忆。请先用 memhub models setup 完成配置,再运行 memhub restart 重试。",
};
function sdkMigrationFailure(command: "install" | "upgrade", code: string) {
const MIGRATION_ISSUES_URL =
"https://github.com/Tencent-Hunyuan/Hy-Memory/issues/new";
/**
* Why MemHub is not serving: the home has not reached SDK authority.
*
* Product rule: after an upgrade the home is on SDK authority or MemHub does
* not serve it at all. Nothing here rolls back to an older Runtime (it has no
* migration gate and would serve the legacy home); the user gets the code, the
* migration log and where to report it.
*/
function sdkMigrationFailure(
command: string,
code: string,
dataDir: string,
context: "upgrade" | "blocked" = "upgrade",
) {
const logFile = sdkMigrationLogPath(dataDir);
const reason =
SDK_MIGRATION_FAILURES[code] ??
"记忆数据迁移没有完成。可以运行 memhub restart 重试。";
const lead =
context === "upgrade"
? `${reason}已保留新版本;在迁移完成前 MemHub 不会启动,也不会用旧版本打开未迁移的数据。`
: `MemHub 没有启动:记忆数据还没有迁移到新版本(${code})。${reason}`;
return {
schema_version: 1,
product: PRODUCT,
command,
status: "error",
code,
message:
SDK_MIGRATION_FAILURES[code] ??
"记忆数据迁移没有完成,已回滚到上一版本。请重试;如果仍然失败,请把这个错误码反馈给我们。原有记忆数据不会丢失。",
message: `${lead}原有记忆数据不会丢失。迁移日志:${logFile}。请在 GitHub 提交反馈并附上这个日志文件:${MIGRATION_ISSUES_URL}`,
log_file: logFile,
issues_url: MIGRATION_ISSUES_URL,
};
}
/** A stable code from a thrown migration error, never its free text. */
function sdkMigrationErrorCode(error: unknown): string {
const message = error instanceof Error ? error.message : "";
return /^[a-z][a-z0-9_]{0,95}$/.test(message)
? message
: "sdk_migration_incomplete";
}
/**
* The Runtime service refuses to spawn Core while this home's migration is
* deferred and leaves a block note; report that instead of a generic
* "not ready within budget".
*/
async function runtimeMigrationBlock(
dataDir: string,
command: string,
): Promise<ReturnType<typeof sdkMigrationFailure> | null> {
const block = await readSdkMigrationBlock(dataDir).catch(() => null);
if (block === null) return null;
return sdkMigrationFailure(command, block.code, dataDir, "blocked");
}
async function waitForRuntimeReady(
dataDir: string,
timeoutMs = 45_000,
): Promise<boolean> {
const deadline = Date.now() + timeoutMs;
const started = Date.now();
const deadline = started + timeoutMs;
while (Date.now() < deadline) {
const runtime = await inspectRuntimeStatus({
dataDir,
@@ -4167,6 +4276,11 @@ async function waitForRuntimeReady(
requireServiceReceipt: true,
}).catch(() => null);
if (runtimeIsReady(runtime)) return true;
// The service deferred its migration during this wait and will not spawn
// Core: stop waiting. An older note may predate a retry still running, so
// only a note written after the wait began ends it early.
const blocked = await readSdkMigrationBlock(dataDir).catch(() => null);
if (blocked !== null && Date.parse(blocked.at) >= started) return false;
await new Promise((resolve) => setTimeout(resolve, 500));
}
return false;
@@ -6086,21 +6200,7 @@ type MemoryCommand =
contentFile: string;
confirmed: boolean;
}
| { kind: "delete"; memoryId: string; confirmed: boolean }
| {
kind: "plan-add";
contentFile: string;
memoryKind: string;
scopes: string[];
evidenceIds: string[];
}
| {
kind: "plan-existing";
operation: "correct" | "delete";
memoryId: string;
contentFile?: string;
}
| { kind: "apply"; planFile: string; contentFile?: string };
| { kind: "delete"; memoryId: string; confirmed: boolean };
async function parseMemoryCommand(
rest: readonly string[],
@@ -6140,31 +6240,6 @@ async function parseMemoryCommand(
? { kind: "delete", memoryId, confirmed }
: null;
}
if (action === "plan" || action === "apply") return null;
if (action === "plan") return parseMemoryPlanCommand(tokens.slice(1));
if (action === "apply") {
let planFile: string | undefined;
let contentFile: string | undefined;
let confirmed = false;
for (let index = 1; index < tokens.length; index += 1) {
const flag = tokens[index];
if (flag === "--confirm") {
if (confirmed) return null;
confirmed = true;
continue;
}
const value = tokens[index + 1];
if (value === undefined || value.startsWith("--")) return null;
if (flag === "--plan-file" && planFile === undefined) planFile = value;
else if (flag === "--content-file" && contentFile === undefined)
contentFile = value;
else return null;
index += 1;
}
return planFile && confirmed
? { kind: "apply", planFile, ...(contentFile ? { contentFile } : {}) }
: null;
}
if (action !== "list") return null;
let status: string | undefined;
let offset = 0;
@@ -6191,73 +6266,11 @@ async function parseMemoryCommand(
return { kind: "list", query: { status, offset, limit } };
}
function parseMemoryPlanCommand(
tokens: readonly string[],
): MemoryCommand | null {
const operation = tokens[0];
const add = operation === "add";
if (!add && !["correct", "delete"].includes(operation)) {
return null;
}
const memoryId = add ? undefined : tokens[1];
if (!add && (!memoryId || !isMemoryId(memoryId))) return null;
let contentFile: string | undefined;
let memoryKind = "chat";
let kindSet = false;
const scopes: string[] = [];
const evidenceIds: string[] = [];
for (let index = add ? 1 : 2; index < tokens.length; index += 1) {
const flag = tokens[index];
const value = tokens[index + 1];
if (value === undefined || value.startsWith("--")) return null;
if (flag === "--content-file" && contentFile === undefined)
contentFile = value;
else if (flag === "--kind" && add && !kindSet) {
memoryKind = value;
kindSet = true;
} else if (flag === "--scope" && add && !scopes.includes(value))
scopes.push(value);
else if (flag === "--evidence" && add && !evidenceIds.includes(value))
evidenceIds.push(value);
else return null;
index += 1;
}
if (add) {
return contentFile && scopes.length > 0 && evidenceIds.length > 0
? {
kind: "plan-add",
contentFile,
memoryKind,
scopes,
evidenceIds,
}
: null;
}
if (
(operation === "correct" && !contentFile) ||
(operation === "delete" && (contentFile !== undefined || scopes.length > 0))
) {
return null;
}
return {
kind: "plan-existing",
operation: operation as "correct" | "delete",
memoryId: memoryId as string,
...(contentFile ? { contentFile } : {}),
};
}
async function readMemoryContentFile(file: string): Promise<string | null> {
const bytes = await readBoundedFile(file, 100_000 * 4 + 2);
return bytes ? parseMemoryContent(bytes) : null;
}
function emitMemoryFlow(document: MemoryFlowDocument, asJson: boolean): void {
emit(
asJson ? `${JSON.stringify(document)}\n` : renderMemoryFlowText(document),
);
}
function emitModelWriteDocument(
document: ModelWriteDocument,
asJson: boolean,
@@ -6430,6 +6443,8 @@ function serviceErrorDocument(
"服务文件不属于当前 MemHub 安装;请保留文件并确认其来源后再处理。",
runtime_service_managed_upgrade_failed:
"新服务启动失败,旧 MemHub 服务已恢复;检查新安装日志后重试。",
runtime_service_replacement_failed:
"新服务启动失败;记忆数据尚未处于新版本可用的状态,所以没有恢复旧服务。请运行 memhub doctor 检查服务状态。",
runtime_service_managed_upgrade_rollback_failed:
"新服务启动失败且旧服务恢复失败;请运行 memhub doctor 检查服务状态。",
runtime_service_file_modified:
+2 -2
View File
@@ -387,7 +387,7 @@ export const COMMANDS: readonly CommandDescriptor[] = [
{
flag: "--content-file <path>",
summary:
"从有界普通文件读取新增/纠正正文;计划只保存摘要,执行时再次校验。",
"从有界普通文件读取 update 的新正文;预览只显示摘要,--confirm 时才写入。",
},
DIRECT_CONFIRM_OPTION,
JSON_OPTION,
@@ -485,7 +485,7 @@ export const COMMANDS: readonly CommandDescriptor[] = [
{
name: "recall",
summary:
"从当前授权的原文与治理记忆统一检索上下文;本地只读,不调用模型、不触发补索引。",
"从当前授权的原文与已沉淀记忆统一检索上下文;本地只读,不调用模型、不触发补索引。",
usage:
"memhub recall <query> [--kind <raw|memory>] [--source <id>] [--limit <n>] [--character-budget <n>] [--timeout-ms <n>] [--json]\n memhub recall --query-file <path> [...]\n memhub recall --stdin [...]",
options: [
+70 -135
View File
@@ -43,61 +43,35 @@ const ATTACHED = {
};
const RESPONSE = {
schema_version: 1,
schema_version: 2,
request_id: "context-request-1",
state: "grounded",
paths: {
raw: {
status: "ok",
reason: null,
elapsed_ms: 2,
returned_count: 1,
truncated: false,
truncated_by: [],
},
memory: {
status: "ok",
reason: null,
elapsed_ms: 3,
returned_count: 1,
truncated: false,
truncated_by: [],
},
},
items: [
{
kind: "raw",
source_id: "codex",
conversation_id: "conversation-1",
anchor_evidence_id: "evidence-1",
messages: [
{
evidence_id: "evidence-1",
revision: 1,
role: "user",
written_by: "user",
created_at: "2026-09-19T00:00:00Z",
text: "Synthetic raw citation.",
truncated: false,
},
],
raw_id: "raw-1",
content: "Synthetic raw citation.",
source: "codex",
session_id: "session-1",
memory_at: "2026-09-19T00:00:00Z",
score: 0.8,
truncated: false,
},
{
kind: "memory",
memory_id: "memory-1",
revision: 2,
content: "Synthetic governed memory.",
memory_kind: "working",
content: "Synthetic SDK memory.",
layer: "l2_fact",
status: "active",
source: "codex",
updated_at: "2026-09-19T00:00:01Z",
evidence_ids: ["evidence-1"],
score: 0.7,
truncated: false,
},
],
used_characters: 49,
used_characters: 44,
truncated: false,
total_ms: 5,
adoption: "unknown",
};
beforeEach(() => {
@@ -115,42 +89,17 @@ beforeEach(() => {
});
describe("unified context search", () => {
it("accepts an empty tool message in a valid partial context response", async () => {
it("accepts an empty raw item in a valid partial context response", async () => {
transport.post.mockResolvedValue({
status: 200,
data: {
...RESPONSE,
state: "partial",
paths: {
raw: {
...RESPONSE.paths.raw,
status: "degraded",
reason: "lexical_only",
returned_count: 1,
},
memory: {
...RESPONSE.paths.memory,
status: "empty",
returned_count: 0,
},
},
items: [
{
kind: "raw",
source_id: "workbuddy",
conversation_id: "conversation-1",
anchor_evidence_id: "evidence-1",
messages: [
{
evidence_id: "evidence-1",
revision: 1,
role: "tool",
written_by: "tool",
created_at: "2026-10-05T20:48:12Z",
text: "",
truncated: true,
},
],
...RESPONSE.items[0],
source: "workbuddy",
content: "",
truncated: true,
},
],
@@ -165,8 +114,9 @@ describe("unified context search", () => {
streams.restore();
}
expect(JSON.parse(streams.stdout())).toMatchObject({
schema_version: 2,
state: "partial",
items: [{ kind: "raw", messages: [{ text: "" }] }],
items: [{ kind: "raw", content: "" }],
});
});
@@ -178,12 +128,12 @@ describe("unified context search", () => {
streams.restore();
}
const output = streams.stdout();
expect(output).toContain("Synthetic governed memory.");
expect(output).toContain("Synthetic SDK memory.");
expect(output).toContain("Synthetic raw citation.");
expect(output).not.toMatch(/^- raw |memory-1|evidence-1|采用状态/m);
expect(output).not.toMatch(/^- raw |memory-1|raw-1|采用状态/m);
});
it("queries the current read-only context endpoint and preserves citations", async () => {
it("queries the SDK context endpoint directly and preserves item ids", async () => {
const streams = captureStreams();
try {
expect(
@@ -209,7 +159,7 @@ describe("unified context search", () => {
streams.restore();
}
expect(transport.post).toHaveBeenCalledWith(
"/api/v1/context/search",
"/api/v2/context/search",
{
query: "synthetic question",
kinds: ["raw", "memory"],
@@ -222,12 +172,18 @@ describe("unified context search", () => {
);
const output = JSON.parse(streams.stdout());
expect(output).toMatchObject({
schema_version: 2,
state: "grounded",
request_id: "context-request-1",
adoption: "unknown",
});
expect(output.items[0].messages[0].evidence_id).toBe("evidence-1");
expect(output.items[1].evidence_ids).toEqual(["evidence-1"]);
expect(output).not.toHaveProperty("paths");
expect(output.items[0]).toMatchObject({ kind: "raw", raw_id: "raw-1" });
expect(output.items[1]).toMatchObject({
kind: "memory",
memory_id: "memory-1",
layer: "l2_fact",
});
// No authority pre-query: released Core always serves the SDK data plane.
expect(transport.get).not.toHaveBeenCalled();
expect(transport.put).not.toHaveBeenCalled();
expect(transport.patch).not.toHaveBeenCalled();
@@ -242,19 +198,8 @@ describe("unified context search", () => {
status: 200,
data: {
...RESPONSE,
paths: {
...RESPONSE.paths,
raw: {
status: "not_requested",
reason: "not_requested",
elapsed_ms: 0,
returned_count: 0,
truncated: false,
truncated_by: [],
},
},
items: [RESPONSE.items[1]],
used_characters: 26,
used_characters: 21,
},
});
const streams = captureStreams();
@@ -394,29 +339,7 @@ describe("unified context search", () => {
status: 200,
data: {
...RESPONSE,
paths: {
...RESPONSE.paths,
memory: {
...RESPONSE.paths.memory,
status: "empty",
returned_count: 0,
},
},
items: [
{
...(RESPONSE.items[0] as Record<string, unknown>),
messages: [
{
...(
RESPONSE.items[0] as {
messages: Array<Record<string, unknown>>;
}
).messages[0],
text,
},
],
},
],
items: [{ ...RESPONSE.items[0], content: text }],
used_characters: 10,
},
});
@@ -429,7 +352,7 @@ describe("unified context search", () => {
expect(JSON.parse(streams.stdout())).toMatchObject({
state: "grounded",
used_characters: 10,
items: [{ kind: "raw", messages: [{ text }] }],
items: [{ kind: "raw", content: text }],
});
});
@@ -441,29 +364,7 @@ describe("unified context search", () => {
status: 200,
data: {
...RESPONSE,
paths: {
...RESPONSE.paths,
memory: {
...RESPONSE.paths.memory,
status: "empty",
returned_count: 0,
},
},
items: [
{
...(RESPONSE.items[0] as Record<string, unknown>),
messages: [
{
...(
RESPONSE.items[0] as {
messages: Array<Record<string, unknown>>;
}
).messages[0],
text,
},
],
},
],
items: [{ ...RESPONSE.items[0], content: text }],
used_characters: 12,
},
});
@@ -479,6 +380,40 @@ describe("unified context search", () => {
});
});
it.each([
[
"a raw item from a source that was not asked for",
["--source", "codex"],
{ ...RESPONSE.items[0], source: "cursor" },
],
[
"a raw item when only memory was asked for",
["--kind", "memory"],
RESPONSE.items[0],
],
["a score outside 0..1", [], { ...RESPONSE.items[1], score: 1.5 }],
])("rejects %s as an invalid response", async (_label, flags, item) => {
transport.post.mockResolvedValue({
status: 200,
data: {
...RESPONSE,
items: [item],
used_characters: Array.from(String(item.content)).length,
},
});
const streams = captureStreams();
try {
expect(await run(["recall", "query", ...flags, "--json"])).toBe(1);
} finally {
streams.restore();
}
expect(JSON.parse(streams.stdout())).toMatchObject({
schema_version: 2,
state: "invalid",
code: "response_invalid",
});
});
it("reports endpoint denial without exposing the response body", async () => {
transport.post.mockResolvedValue({
status: 403,
+43 -162
View File
@@ -3,31 +3,6 @@ import { PRODUCT } from "./version.js";
export const CONTEXT_QUERY_MAX_CHARACTERS = 2_000;
const CONTEXT_KINDS = ["raw", "memory"] as const;
const PATH_STATES = [
"ok",
"empty",
"not_requested",
"not_authorized",
"unavailable",
"degraded",
] as const;
const PATH_REASONS = [
"not_requested",
"permission_required",
"source_revoked",
"read_unavailable",
"timeout_budget",
"evidence_changed",
"memory_changed",
"lexical_only",
] as const;
const LIMIT_REASONS = [
"count_budget",
"character_budget",
"timeout_budget",
"metadata_budget",
] as const;
export interface ContextSearchInput {
query: string;
kinds: Array<(typeof CONTEXT_KINDS)[number]>;
@@ -37,8 +12,9 @@ export interface ContextSearchInput {
timeout_ms: number;
}
/** The CLI's JSON document mirrors Core's SDK-native context schema v2. */
export interface ContextSearchDocument {
schema_version: 1;
schema_version: 2;
product: string;
state:
| "grounded"
@@ -50,12 +26,10 @@ export interface ContextSearchDocument {
| "invalid";
code?: string;
request_id?: string;
paths?: Record<string, unknown>;
items: Array<Record<string, unknown>>;
used_characters?: number;
truncated?: boolean;
total_ms?: number;
adoption?: "unknown";
}
export async function searchContext(
@@ -67,7 +41,7 @@ export async function searchContext(
const validated = validateContextResponse(outcome.data, input);
return (
validated ?? {
schema_version: 1,
schema_version: 2,
product: PRODUCT,
state: "invalid",
code: "response_invalid",
@@ -131,13 +105,7 @@ export function renderContextText(document: ContextSearchDocument): string {
: "查到以下相关内容:",
];
for (const item of document.items) {
if (item.kind === "memory") {
lines.push(`- ${String(item.content)}`);
} else {
for (const message of item.messages as Array<Record<string, unknown>>) {
lines.push(`- ${String(message.text)}`);
}
}
lines.push(`- ${String(item.content)}`);
}
if (document.truncated) lines.push("结果已按请求预算截断。");
return `${lines.join("\n")}\n`;
@@ -149,84 +117,37 @@ function validateContextResponse(
): ContextSearchDocument | null {
const response = record(value);
if (
response?.schema_version !== 1 ||
response?.schema_version !== 2 ||
!oneOf(response.state, ["grounded", "empty", "partial", "unavailable"]) ||
!boundedText(response.request_id, 64) ||
!record(response.paths) ||
!Array.isArray(response.items) ||
response.items.length > request.limit ||
!integer(response.used_characters, 0, request.character_budget) ||
typeof response.truncated !== "boolean" ||
!number(response.total_ms, 0) ||
response.adoption !== "unknown"
!number(response.total_ms, 0)
) {
return null;
}
const paths = response.paths as Record<string, unknown>;
const rawPath = validatePath(paths.raw);
const memoryPath = validatePath(paths.memory);
if (!rawPath || !memoryPath) return null;
const requested = new Set(request.kinds);
if (
(!requested.has("raw") && rawPath.status !== "not_requested") ||
(!requested.has("memory") && memoryPath.status !== "not_requested")
) {
return null;
}
const items: Array<Record<string, unknown>> = [];
let characters = 0;
for (const itemValue of response.items) {
const item = validateItem(itemValue, request);
if (!item || !requested.has(item.kind as "raw" | "memory")) return null;
items.push(item);
characters +=
item.kind === "memory"
? characterCount(item.content)
: (item.messages as Array<Record<string, unknown>>).reduce(
(sum, message) => sum + characterCount(message.text),
0,
);
characters += characterCount(item.content);
}
if (characters !== response.used_characters) return null;
if (response.state === "empty" && items.length !== 0) return null;
return {
schema_version: 1,
schema_version: 2,
product: PRODUCT,
state: response.state,
request_id: response.request_id as string,
paths: { raw: rawPath, memory: memoryPath },
items,
used_characters: response.used_characters as number,
truncated: response.truncated,
total_ms: response.total_ms as number,
adoption: "unknown",
};
}
function validatePath(value: unknown): Record<string, unknown> | null {
const path = record(value);
if (
!path ||
!oneOf(path.status, PATH_STATES) ||
!number(path.elapsed_ms, 0) ||
!integer(path.returned_count, 0, 20) ||
typeof path.truncated !== "boolean" ||
!enumList(path.truncated_by, LIMIT_REASONS) ||
!(
path.reason === null ||
path.reason === undefined ||
oneOf(path.reason, PATH_REASONS)
)
) {
return null;
}
return {
status: path.status,
reason: path.reason ?? null,
elapsed_ms: path.elapsed_ms,
returned_count: path.returned_count,
truncated: path.truncated,
truncated_by: path.truncated_by,
};
}
@@ -235,88 +156,65 @@ function validateItem(
request: ContextSearchInput,
): Record<string, unknown> | null {
const item = record(value);
if (!item || !oneOf(item.kind, CONTEXT_KINDS)) return null;
if (
!item ||
!oneOf(item.kind, CONTEXT_KINDS) ||
boundedText(item.content, 20_000, true) === null ||
!number(item.score, 0) ||
(item.score as number) > 1 ||
typeof item.truncated !== "boolean"
) {
return null;
}
if (item.kind === "memory") {
if (
!boundedText(item.memory_id, 512) ||
!integer(item.revision, 1) ||
boundedText(item.content, 20_000, true) === null ||
boundedText(item.memory_kind, 64, true) === null ||
!boundedText(item.updated_at, 64) ||
!textList(item.evidence_ids, 128, 512) ||
typeof item.truncated !== "boolean"
boundedText(item.layer, 64, true) === null ||
boundedText(item.status, 32, true) === null ||
!optionalText(item.source, 128) ||
!optionalText(item.updated_at, 64)
) {
return null;
}
return {
kind: "memory",
memory_id: item.memory_id,
revision: item.revision,
content: item.content,
memory_kind: item.memory_kind,
updated_at: item.updated_at,
evidence_ids: item.evidence_ids,
layer: item.layer,
status: item.status,
source: item.source ?? null,
updated_at: item.updated_at ?? null,
score: item.score,
truncated: item.truncated,
};
}
if (
!boundedText(item.source_id, 128) ||
!boundedText(item.raw_id, 512) ||
!boundedText(item.source, 128) ||
(request.raw_source_ids !== undefined &&
!request.raw_source_ids.includes(item.source_id as string)) ||
!boundedText(item.conversation_id, 512) ||
!boundedText(item.anchor_evidence_id, 512) ||
!Array.isArray(item.messages) ||
item.messages.length < 1 ||
item.messages.length > 3 ||
typeof item.truncated !== "boolean"
!request.raw_source_ids.includes(item.source as string)) ||
!boundedText(item.session_id, 512) ||
!optionalText(item.memory_at, 64)
) {
return null;
}
const messages = item.messages.map(validateMessage);
if (messages.some((message) => message === null)) return null;
return {
kind: "raw",
source_id: item.source_id,
conversation_id: item.conversation_id,
anchor_evidence_id: item.anchor_evidence_id,
messages,
raw_id: item.raw_id,
content: item.content,
source: item.source,
session_id: item.session_id,
memory_at: item.memory_at ?? null,
score: item.score,
truncated: item.truncated,
};
}
function validateMessage(value: unknown): Record<string, unknown> | null {
const message = record(value);
if (
!message ||
!boundedText(message.evidence_id, 512) ||
!integer(message.revision, 1) ||
boundedText(message.role, 32, true) === null ||
boundedText(message.written_by, 32, true) === null ||
!(
message.created_at === null ||
boundedText(message.created_at, 64) !== null
) ||
boundedText(message.text, 20_000, true) === null ||
typeof message.truncated !== "boolean"
) {
return null;
}
return {
evidence_id: message.evidence_id,
revision: message.revision,
role: message.role,
written_by: message.written_by,
created_at: message.created_at,
text: message.text,
truncated: message.truncated,
};
}
function failed(
outcome: Exclude<ReadOutcome<unknown>, { state: "ok" }>,
): ContextSearchDocument {
return {
schema_version: 1,
schema_version: 2,
product: PRODUCT,
state: outcome.state,
code: outcome.code,
@@ -374,27 +272,10 @@ function oneOf<const T extends readonly string[]>(
return typeof value === "string" && values.includes(value as T[number]);
}
function enumList(
value: unknown,
values: readonly string[],
): value is string[] {
function optionalText(value: unknown, maximum: number): boolean {
return (
Array.isArray(value) &&
value.length <= 4 &&
new Set(value).size === value.length &&
value.every((item) => oneOf(item, values))
);
}
function textList(
value: unknown,
maximumItems: number,
maximumText: number,
): value is string[] {
return (
Array.isArray(value) &&
value.length <= maximumItems &&
new Set(value).size === value.length &&
value.every((item) => boundedText(item, maximumText) !== null)
value === null ||
value === undefined ||
boundedText(value, maximum) !== null
);
}
+4 -11
View File
@@ -8,14 +8,12 @@ import axios from "axios";
*/
const READ_ONLY_PATHS = [
"/api/v1/call-events",
"/api/v1/home-summary",
"/api/v1/import-jobs",
"/api/v1/live-sync",
"/api/v1/memory-library",
"/api/v1/memory-processing",
"/api/v1/memory-tasks",
"/api/v1/model-profiles",
"/api/v1/sources",
"/api/v2/home-summary",
"/api/v2/memory-library",
"/api/v2/user-profile",
] as const;
@@ -67,13 +65,10 @@ export function createUiSessionRequest(attachment: {
};
}
const MEMORY_DETAIL = /^\/api\/v1\/memories\/[A-Za-z0-9][\w.-]{0,127}$/;
const IMPORT_JOB =
/^\/api\/v1\/import-jobs\/[A-Za-z0-9][\w.-]{0,127}(?:\/issues)?$/;
const OPERATION_RECEIPT =
/^\/api\/v1\/(?:source-connections|model-profile-creations|model-probe-operations)\/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/;
const MEMORY_MUTATION_RECEIPT =
/^\/api\/v1\/memory-mutations\/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/;
/**
* Core registers these static routes before `/{memory_id}`, so a memory whose
* id happens to be one of them would reach a different handler: `graph`
@@ -173,7 +168,7 @@ export function createCoreContextQuery(attachment: {
let response: { status: number; data: unknown };
try {
response = await client.post(
"/api/v1/context/search",
"/api/v2/context/search",
body,
options?.signal ? { signal: options.signal } : undefined,
);
@@ -197,10 +192,8 @@ export function isReadOnlyPath(path: string): boolean {
if ((READ_ONLY_PATHS as readonly string[]).includes(path)) return true;
if (IMPORT_JOB.test(path)) return true;
if (OPERATION_RECEIPT.test(path)) return true;
if (MEMORY_MUTATION_RECEIPT.test(path)) return true;
if (SDK_MEMORY_DETAIL.test(path)) return true;
if (!MEMORY_DETAIL.test(path)) return false;
const segment = path.slice("/api/v1/memories/".length);
if (!SDK_MEMORY_DETAIL.test(path)) return false;
const segment = path.slice("/api/v2/memories/".length);
return !(RESERVED_MEMORY_SEGMENTS as readonly string[]).includes(segment);
}
+24 -212
View File
@@ -1,18 +1,20 @@
import type {
GovernedMemory,
HomeSummary,
ImportIssue,
ImportJob,
LiveSyncHealth,
MemoryDetail,
MemoryEvidenceExcerpt,
MemoryLibraryPage,
MemoryProcessingSummary,
ModelProfile,
SourceResponse,
} from "@one-memory/contracts";
/** The only memory states a response may carry, and the only ones a filter may ask for. */
export const MEMORY_STATUSES = ["confirmed", "deleted", "superseded"] as const;
/** SDK memory states `memhub memories list --status` may ask for. */
export const MEMORY_STATUSES = [
"active",
"superseded",
"shadow",
"negated",
"conflicted",
"archived",
] as const;
/**
* HTTP 200 is not a contract. Everything here re-checks the shape and the
@@ -23,7 +25,6 @@ export const MEMORY_STATUSES = ["confirmed", "deleted", "superseded"] as const;
const ID_MAX = 128;
const LABEL_MAX = 256;
const TEXT_MAX = 100_000;
const LIST_MAX = 500;
const LIVE_SYNC_STATES = [
"idle",
@@ -39,14 +40,6 @@ const PROVIDER_KINDS = [
"local_onnx",
] as const;
const CREDENTIAL_STATES = ["available", "missing", "unavailable"] as const;
const MEMORY_KINDS = [
"chat",
"working",
"coding",
"schema",
"intention",
] as const;
const MEMORY_ORIGINS = ["extracted", "answered", "agent", "manual"] as const;
const IMPORT_STATUSES = [
"pending",
"running",
@@ -57,16 +50,6 @@ const IMPORT_STATUSES = [
"cancelled",
] as const;
const SCAN_PHASES = ["discover", "read", "emit", "done"] as const;
const MEMORY_TASK_KINDS = ["extract", "system2", "ultra", "sweeper"] as const;
const MEMORY_TASK_STATUSES = [
"pending",
"running",
"waiting_for_model",
"completed",
"failed",
"cancelled",
"deferred_outside_window",
] as const;
const CAPABILITY_STATES = ["not_implemented", "implemented"] as const;
const VERIFICATION_STATES = ["fixture_only", "real_replayed"] as const;
const SYNC_COMPLETENESS = [
@@ -87,16 +70,6 @@ const LOCATION_STATES = [
] as const;
const LIVE_SYNC_MODES = ["polling", "watch_and_polling"] as const;
export interface ValidatedMemoryTask {
task_id: string;
kind: (typeof MEMORY_TASK_KINDS)[number];
status: (typeof MEMORY_TASK_STATUSES)[number];
attempts: number;
last_error_code: string | null;
created_at: string;
updated_at: string;
}
export interface SourceConnectionCandidate {
source_id: string;
display_name: string;
@@ -352,45 +325,6 @@ export function validateImportIssues(
});
}
export function validateMemoryTasks(
value: unknown,
): ValidatedMemoryTask[] | null {
return list(value, (item) => {
const task = record(item);
if (!task) return null;
const taskId = text(task.task_id, ID_MAX);
if (
task.last_error_code !== null &&
text(task.last_error_code, LABEL_MAX) === null
) {
return null;
}
const error = task.last_error_code as string | null;
const createdAt = text(task.created_at, LABEL_MAX);
const updatedAt = text(task.updated_at, LABEL_MAX);
const attempts = count(task.attempts);
if (
taskId === null ||
createdAt === null ||
updatedAt === null ||
attempts === null ||
!oneOf(task.kind, MEMORY_TASK_KINDS) ||
!oneOf(task.status, MEMORY_TASK_STATUSES)
) {
return null;
}
return {
task_id: taskId,
kind: task.kind,
status: task.status,
attempts,
last_error_code: error,
created_at: createdAt,
updated_at: updatedAt,
};
});
}
export function validateProcessing(
value: unknown,
): MemoryProcessingSummary | null {
@@ -444,150 +378,32 @@ export function validateModelProfiles(value: unknown): ModelProfile[] | null {
});
}
export function validateHomeSummary(value: unknown): HomeSummary | null {
/** `GET /api/v2/home-summary`: the SDK-authority home card. */
export interface SdkHomeSummary {
schema_version: 2;
memory_count: number;
latest: { memory_id: string } | null;
observed_at: string;
}
export function validateHomeSummary(value: unknown): SdkHomeSummary | null {
const summary = record(value);
if (
!summary ||
summary.schema_version !== 2 ||
count(summary.memory_count) === null ||
text(summary.observed_at, LABEL_MAX) === null
) {
return null;
}
return summary as unknown as HomeSummary;
}
/** The page must be the page that was asked for, not merely a page. */
export function validateMemoryPage(
value: unknown,
request: { offset: number; limit: number },
): MemoryLibraryPage | null {
const page = record(value);
if (!page) return null;
const total = count(page.total);
if (
total === null ||
page.offset !== request.offset ||
page.limit !== request.limit
) {
return null;
}
const items = list(page.items, validateGovernedMemory);
if (items === null || items.length > request.limit) return null;
if (items.length > 0 && total < request.offset + items.length) return null;
return { items, total, offset: page.offset, limit: page.limit };
}
export function validateMemoryDetail(
value: unknown,
memoryId: string,
): MemoryDetail | null {
const detail = record(value);
if (!detail) return null;
const memory = validateGovernedMemory(detail.memory);
if (memory === null || memory.memory_id !== memoryId) return null;
const cited = new Set(memory.evidence_ids);
const returned = new Set<string>();
const evidence = list(detail.evidence ?? [], (item) => {
const excerpt = record(item);
if (!excerpt) return null;
const evidenceId = text(excerpt.evidence_id, ID_MAX);
// Core builds this list from the memory's own citations; anything else is
// another record's excerpt and must not be shown under this memory.
if (
evidenceId === null ||
!cited.has(evidenceId) ||
returned.has(evidenceId) ||
typeof excerpt.available !== "boolean"
) {
const latest = summary.latest;
if (latest !== null) {
const memory = record(latest);
if (!memory || typeof memory.memory_id !== "string" || !memory.memory_id) {
return null;
}
returned.add(evidenceId);
const available = excerpt.available === true;
const verbatim = available ? text(excerpt.verbatim ?? "", TEXT_MAX) : "";
if (verbatim === null) return null;
return {
evidence_id: evidenceId,
available,
source_id: optional(excerpt.source_id, ID_MAX),
written_by: optional(excerpt.written_by, LABEL_MAX),
created_at: optional(excerpt.created_at, LABEL_MAX),
// An excerpt the user cannot read has no text, whatever the server sent.
verbatim,
} as unknown as MemoryEvidenceExcerpt;
});
if (evidence === null || evidence.length !== memory.evidence_ids.length) {
return null;
}
const history = list(detail.history ?? [], (item) => {
const version = record(item);
if (
!version ||
typeof version.revision !== "number" ||
!Number.isSafeInteger(version.revision) ||
version.revision < 1 ||
text(version.action, LABEL_MAX) === null ||
!oneOf(version.status, MEMORY_STATUSES) ||
text(version.content, TEXT_MAX) === null ||
identifiers(version.scopes) === null ||
text(version.created_at, LABEL_MAX) === null
) {
return null;
}
return {
revision: version.revision,
action: version.action,
status: version.status,
content: version.content,
scopes: identifiers(version.scopes) as string[],
created_at: version.created_at,
};
});
if (history === null) return null;
return {
memory,
evidence,
history,
index_sync: null,
} as unknown as MemoryDetail;
}
export function validateGovernedMemory(value: unknown): GovernedMemory | null {
const memory = record(value);
if (
!memory ||
!text(memory.memory_id, ID_MAX) ||
!text(memory.user_id, LABEL_MAX) ||
!text(memory.content, TEXT_MAX) ||
!oneOf(memory.kind, MEMORY_KINDS) ||
!oneOf(memory.origin, MEMORY_ORIGINS) ||
typeof memory.revision !== "number" ||
!Number.isSafeInteger(memory.revision) ||
memory.revision < 1 ||
!oneOf(memory.status, MEMORY_STATUSES) ||
!text(memory.created_at, LABEL_MAX) ||
!text(memory.updated_at, LABEL_MAX)
) {
return null;
}
const scopes = identifiers(memory.scopes);
const evidenceIds = identifiers(memory.evidence_ids);
const tags = identifiers(memory.tags ?? []);
if (
scopes === null ||
scopes.length < 1 ||
evidenceIds === null ||
evidenceIds.length < 1 ||
tags === null ||
tags.length > 32
) {
return null;
}
return {
...memory,
scopes,
evidence_ids: evidenceIds,
tags,
} as unknown as GovernedMemory;
return summary as unknown as SdkHomeSummary;
}
function identifiers(value: unknown): string[] | null {
@@ -618,10 +434,6 @@ function text(value: unknown, max: number): string | null {
return typeof value === "string" && value.length <= max ? value : null;
}
function optional(value: unknown, max: number): string | undefined {
return typeof value === "string" && value.length <= max ? value : undefined;
}
function count(value: unknown): number | null {
return typeof value === "number" && Number.isSafeInteger(value) && value >= 0
? value
@@ -134,11 +134,6 @@ function progress(): ProgressDocument {
generation: {
summary_state: { state: "ok" },
summary: { waiting_for_model: 1, failed: 0 },
tasks_state: { state: "ok" },
tasks: [{ task_id: "t1", status: "waiting_for_model" }],
returned: 1,
validated: 1,
output_truncated: false,
},
};
}
+21 -12
View File
@@ -65,6 +65,9 @@ export function renderDoctorText(document: DoctorDocument): string {
model: "模型",
pending_operations: "待处理操作",
};
const migration = document.checks.find(
(check) => check.id === "core" && check.details?.log_file,
);
return [
`MemHub 检查:${doctorState(document.state)}`,
...document.checks.map(
@@ -73,7 +76,13 @@ export function renderDoctorText(document: DoctorDocument): string {
),
...(document.state === "healthy"
? []
: ["下一步:打开 MemHub 的“帮助与诊断”查看并处理异常。"]),
: migration
? [
`升级迁移没有完成(错误码:${migration.code}),MemHub 不会启动。`,
`下一步:把迁移日志附在 GitHub 反馈里发给我们:${migration.details?.log_file}`,
`反馈地址:${migration.details?.issues_url}`,
]
: ["下一步:打开 MemHub 的“帮助与诊断”查看并处理异常。"]),
"",
].join("\n");
}
@@ -107,6 +116,14 @@ function coreCheck(status: StatusDocument): DoctorCheck {
id: "core",
state: status.runtime.state === "stopped" ? "warn" : "fail",
code: status.runtime.code,
...(status.runtime.log_file
? {
details: {
log_file: status.runtime.log_file,
issues_url: status.runtime.issues_url ?? "",
},
}
: {}),
};
}
@@ -200,8 +217,7 @@ function modelCheck(status: StatusDocument): DoctorCheck {
function pendingCheck(progress: ProgressDocument | null): DoctorCheck {
if (
progress?.imports.state !== "ok" ||
progress.generation.summary_state.state !== "ok" ||
progress.generation.tasks_state.state !== "ok"
progress.generation.summary_state.state !== "ok"
) {
return {
id: "pending_operations",
@@ -212,27 +228,20 @@ function pendingCheck(progress: ProgressDocument | null): DoctorCheck {
const jobAttention = progress.imports.jobs.filter((item) =>
["paused", "interrupted", "failed"].includes(text(item.status) ?? ""),
).length;
const taskAttention = progress.generation.tasks.filter((item) =>
["waiting_for_model", "failed"].includes(text(item.status) ?? ""),
).length;
const summary = progress.generation.summary;
const waiting = summary ? number(summary.waiting_for_model) : 0;
const failed = summary ? number(summary.failed) : 0;
const attention = jobAttention + taskAttention + waiting + failed;
const attention = jobAttention + waiting + failed;
return {
id: "pending_operations",
state: attention > 0 ? "warn" : "pass",
code: attention > 0 ? "operations_need_attention" : "pending_operations_ok",
details: {
import_jobs_validated: progress.imports.validated,
generation_tasks_validated: progress.generation.validated,
jobs_need_attention: jobAttention,
tasks_need_attention: taskAttention,
waiting_for_model: waiting,
failed,
output_truncated:
progress.imports.output_truncated ||
progress.generation.output_truncated,
output_truncated: progress.imports.output_truncated,
},
};
}
-700
View File
@@ -1,700 +0,0 @@
import { createHash, randomUUID, timingSafeEqual } from "node:crypto";
import type { GovernedMemory } from "@one-memory/contracts";
import {
buildMemoryActionRequest,
buildMemoryCorrectRequest,
buildMemoryCreateRequest,
parseMemoryReceipt,
} from "@one-memory/operations";
import type { CoreReader } from "./core-client.js";
import type { CoreOperations } from "./core-operations.js";
import { validateMemoryDetail } from "./core-responses.js";
import { isMemoryId } from "./memory-report.js";
import { PRODUCT } from "./version.js";
const MEMORY_KINDS = [
"chat",
"working",
"coding",
"schema",
"intention",
] as const;
const OPERATIONS = ["add", "correct", "delete"] as const;
const MEMORY_ORIGINS = ["extracted", "answered", "agent", "manual"] as const;
const MUTATION_RECEIPT_KEYS = new Set([
"schema_version",
"request_id",
"state",
"operation",
"memory_id",
"result_revision",
"result_status",
"memory_state",
"migrated_from",
]);
const CONTENT_MAX_CHARACTERS = 100_000;
const ID = /^[A-Za-z0-9][\w.-]{0,127}$/;
const UUID = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/;
interface PlanBody {
schema_version: 1;
kind: "memory_mutation";
operation: (typeof OPERATIONS)[number];
request_id: string;
memory_id?: string;
expected_revision?: number;
expected_status?: string;
expected_origin?: string;
memory_kind?: (typeof MEMORY_KINDS)[number];
scopes?: string[];
evidence_ids?: string[];
content_sha256?: string;
content_characters?: number;
current?: {
status: string;
origin: string;
revision: number;
scopes: string[];
evidence: Array<{
evidence_id: string;
available: boolean;
source_id?: string;
written_by?: string;
}>;
};
}
export type MemoryMutationPlan = PlanBody & { digest: string };
export type MemoryFlowDocument =
| {
schema_version: 1;
product: string;
state: "preview";
plan: MemoryMutationPlan;
}
| {
schema_version: 1;
product: string;
state: "committed" | "recovered";
operation_id: string;
receipt: Record<string, unknown>;
}
| {
schema_version: 1;
product: string;
state: "rejected" | "unknown" | "invalid";
operation_id?: string;
code: string;
};
export function parseMemoryContent(bytes: Buffer): string | null {
if (bytes.byteLength > CONTENT_MAX_CHARACTERS * 4 + 2) return null;
let decoded: string;
try {
decoded = new TextDecoder("utf-8", { fatal: true }).decode(bytes);
} catch {
return null;
}
const content = decoded.replace(/\r?\n$/, "");
return content.trim().length > 0 &&
[...content].length <= CONTENT_MAX_CHARACTERS &&
![...content].some((character) => {
const code = character.charCodeAt(0);
return code <= 8 || (code >= 11 && code <= 31) || code === 127;
})
? content
: null;
}
export function planMemoryAdd(input: {
content: string;
memoryKind: string;
scopes: string[];
evidenceIds: string[];
}): MemoryFlowDocument {
if (
!validContent(input.content) ||
!oneOf(input.memoryKind, MEMORY_KINDS) ||
!validScopes(input.scopes) ||
!validIds(input.evidenceIds, 500)
) {
return invalid("memory_add_input_invalid");
}
return preview(
withDigest({
schema_version: 1,
kind: "memory_mutation",
operation: "add",
request_id: randomUUID(),
memory_kind: input.memoryKind,
scopes: input.scopes,
evidence_ids: input.evidenceIds,
...contentIdentity(input.content),
}),
);
}
export async function planExistingMemoryMutation(
reader: CoreReader,
input: {
operation: Exclude<(typeof OPERATIONS)[number], "add">;
memoryId: string;
content?: string;
},
): Promise<MemoryFlowDocument> {
if (
!isMemoryId(input.memoryId) ||
(input.operation === "correct" && input.content === undefined) ||
(input.content !== undefined && !validContent(input.content))
) {
return invalid("memory_plan_input_invalid");
}
const outcome = await reader<unknown>(`/api/v1/memories/${input.memoryId}`);
if (outcome.state !== "ok") {
return invalid(`memory_${outcome.state}:${outcome.code}`);
}
const detail = validateMemoryDetail(outcome.data, input.memoryId);
if (!detail) return invalid("memory_response_invalid");
const memory = detail.memory;
if (memory.status !== "confirmed") {
return invalid("memory_operation_not_allowed");
}
const body: PlanBody = {
schema_version: 1,
kind: "memory_mutation",
operation: input.operation,
request_id: randomUUID(),
memory_id: memory.memory_id,
expected_revision: memory.revision,
expected_status: memory.status,
expected_origin: memory.origin,
...(input.content ? contentIdentity(input.content) : {}),
current: {
status: memory.status,
origin: memory.origin,
revision: memory.revision,
scopes: [...memory.scopes],
evidence: detail.evidence.map((item) => ({
evidence_id: item.evidence_id,
available: item.available,
...(item.source_id ? { source_id: item.source_id } : {}),
...(item.written_by ? { written_by: item.written_by } : {}),
})),
},
};
return preview(withDigest(body));
}
export function inspectMemoryPlan(
value: unknown,
): { operationId: string; digest: string; needsContent: boolean } | null {
const plan = validatePlan(value);
return plan
? {
operationId: plan.request_id,
digest: plan.digest,
needsContent: plan.operation === "add" || plan.operation === "correct",
}
: null;
}
export function memoryPlanAcceptsContent(
value: unknown,
content: string | undefined,
): boolean {
const plan = validatePlan(value);
if (!plan) return false;
if (plan.operation !== "add" && plan.operation !== "correct") {
return content === undefined;
}
if (content === undefined) return false;
const identity = contentIdentity(content);
return (
identity.content_sha256 === plan.content_sha256 &&
identity.content_characters === plan.content_characters
);
}
export async function executeMemoryMutation(
value: unknown,
content: string | undefined,
operations: CoreOperations,
reader?: CoreReader,
): Promise<MemoryFlowDocument> {
const plan = validatePlan(value);
if (!plan) return invalid("memory_plan_invalid");
if (!memoryPlanAcceptsContent(plan, content)) {
return invalid("memory_content_mismatch");
}
const { path, body } = operationRequest(plan, content);
const outcome = await operations.post<unknown>(path, body);
if (outcome.state === "rejected") {
return {
schema_version: 1,
product: PRODUCT,
state: "rejected",
operation_id: plan.request_id,
code: outcome.code,
};
}
if (outcome.state === "unknown") {
return reader
? readMemoryMutationReceipt(reader, plan)
: pending(plan.request_id, outcome.code);
}
const memory = validateWriteResponse(outcome.data, plan);
if (!memory || !matchesReceipt(memory, plan, content)) {
return reader
? readMemoryMutationReceipt(reader, plan)
: pending(plan.request_id, "memory_receipt_invalid");
}
return {
schema_version: 1,
product: PRODUCT,
state: "committed",
operation_id: plan.request_id,
receipt: {
memory_id: memory.memory_id,
status: memory.status,
origin: memory.origin,
revision: memory.revision,
scopes: memory.scopes,
evidence_ids: memory.evidence_ids,
content_sha256: contentIdentity(memory.content).content_sha256,
content_characters: [...memory.content].length,
created_at: memory.created_at,
updated_at: memory.updated_at,
},
};
}
/**
* Validate the write response through the shared `parseMemoryReceipt`, which
* owns the GovernedMemory shape and the revision/status binding Core must
* satisfy. The CLI keeps only its own plan binding (`matchesReceipt`, which
* also checks `origin`, `kind` and the echoed content), so the two surfaces
* judge a write response identically and no shape rule is duplicated here.
*
* `add` creates the first revision and expects `confirmed`; the shared parser
* derives that from `expectedRevision: 0`. Every other operation advances the
* revision the caller acted against and expects `confirmed` (correct/scope)
* or `deleted` (delete).
*/
function validateWriteResponse(
value: unknown,
plan: MemoryMutationPlan,
): GovernedMemory | null {
const expectedStatus = plan.operation === "delete" ? "deleted" : "confirmed";
// For `add` there is no id or prior revision to bind yet, so the parser's
// own receipt is the only source for both; the plan binding that follows is
// what proves the response belongs to this request.
const memoryId =
plan.operation === "add"
? String((value as { memory_id?: unknown } | null)?.memory_id ?? "")
: (plan.memory_id as string);
if (memoryId === "") return null;
return parseMemoryReceipt(value, {
memoryId,
expectedRevision:
plan.operation === "add" ? 0 : (plan.expected_revision as number),
expectedStatus,
});
}
export async function readMemoryMutationReceipt(
reader: CoreReader,
value: unknown,
): Promise<MemoryFlowDocument> {
const plan = validatePlan(value);
if (!plan) return invalid("memory_plan_invalid");
const outcome = await reader<unknown>(
`/api/v1/memory-mutations/${encodeURIComponent(plan.request_id)}`,
);
if (outcome.state !== "ok") {
return pending(plan.request_id, `memory_mutation_receipt_${outcome.code}`);
}
const receipt = validateMutationReceipt(outcome.data, plan);
if (receipt === "not_found") {
return pending(
plan.request_id,
"memory_mutation_receipt_not_found_after_attempt",
);
}
if (!receipt) {
return pending(plan.request_id, "memory_mutation_receipt_invalid");
}
return {
schema_version: 1,
product: PRODUCT,
state: "recovered",
operation_id: plan.request_id,
receipt,
};
}
export function memoryFlowExitCode(document: MemoryFlowDocument): number {
return ["preview", "committed", "recovered"].includes(document.state) ? 0 : 1;
}
export function renderMemoryFlowText(document: MemoryFlowDocument): string {
if (document.state === "preview") {
const plan = document.plan;
const lines = [
`记忆操作预览:${plan.operation}`,
`request_id:${plan.request_id}`,
];
if (plan.memory_id) lines.push(`memory_id:${plan.memory_id}`);
if (plan.expected_revision) {
lines.push(
`当前版本:${plan.expected_revision} 状态:${plan.expected_status} 来源方式:${plan.expected_origin}`,
);
}
if (plan.scopes) lines.push(`使用范围:${plan.scopes.join(",")}`);
if (plan.current) {
for (const evidence of plan.current.evidence) {
lines.push(
`证据 ${evidence.evidence_id}:source=${evidence.source_id ?? "unknown"} author=${evidence.written_by ?? "unknown"} available=${evidence.available}`,
);
}
}
if (plan.content_characters !== undefined) {
lines.push(
`正文:${plan.content_characters} 字符,sha256=${plan.content_sha256}`,
);
}
lines.push("尚未写入;保存 JSON 计划后用 memories apply --confirm 执行。");
return `${lines.join("\n")}\n`;
}
if (document.state === "committed" || document.state === "recovered") {
return `记忆操作${document.state === "recovered" ? "已从治理回执恢复" : "已提交"}:${document.operation_id}\n${JSON.stringify(document.receipt)}\n`;
}
return `记忆操作未完成:${document.state}(${"code" in document ? document.code : "memory_operation_state_invalid"})${document.operation_id ? ` request_id=${document.operation_id}` : ""}\n`;
}
function operationRequest(
plan: MemoryMutationPlan,
content: string | undefined,
): { path: string; body: Record<string, unknown> } {
if (plan.operation === "add") {
// The shared create builder is the Web console's manual-add body, which
// always creates `chat` (the Web has no kind picker). The CLI exposes
// `--kind`, so it builds through the same builder and then sets the kind,
// keeping every other field (user_id/scope/evidence/origin) shared.
const request = buildMemoryCreateRequest(
{
content: content as string,
scopes: plan.scopes ?? [],
evidence_ids: plan.evidence_ids ?? [],
origin: "manual",
},
plan.request_id,
);
const draft = request.body.draft as Record<string, unknown>;
draft.kind = plan.memory_kind;
return { path: request.path, body: request.body };
}
if (plan.operation === "correct") {
return buildMemoryCorrectRequest(
plan.memory_id as string,
plan.expected_revision as number,
content as string,
undefined,
plan.request_id,
);
}
return buildMemoryActionRequest(
plan.memory_id as string,
plan.operation,
plan.expected_revision as number,
plan.request_id,
);
}
function matchesReceipt(
memory: GovernedMemory,
plan: MemoryMutationPlan,
content: string | undefined,
): boolean {
if (plan.operation === "add") {
return (
memory.status === "confirmed" &&
memory.origin === "manual" &&
memory.revision === 1 &&
memory.kind === plan.memory_kind &&
same(memory.scopes, plan.scopes ?? []) &&
same(memory.evidence_ids, plan.evidence_ids ?? []) &&
memory.content === content
);
}
if (
memory.memory_id !== plan.memory_id ||
memory.revision !== (plan.expected_revision as number) + 1 ||
memory.origin !== plan.expected_origin
) {
return false;
}
if (plan.operation === "delete" && memory.status !== "deleted") return false;
if (plan.operation === "correct" && memory.status !== plan.expected_status) {
return false;
}
if (plan.operation === "correct" && memory.content !== content) return false;
const expectedScopes =
plan.scopes ?? (plan.current?.scopes as string[] | undefined);
return expectedScopes ? same(memory.scopes, expectedScopes) : true;
}
function validateMutationReceipt(
value: unknown,
plan: MemoryMutationPlan,
): Record<string, unknown> | "not_found" | null {
const receipt = record(value);
if (
!receipt ||
Object.keys(receipt).length !== MUTATION_RECEIPT_KEYS.size ||
Object.keys(receipt).some((key) => !MUTATION_RECEIPT_KEYS.has(key)) ||
receipt.schema_version !== 2 ||
receipt.request_id !== plan.request_id
) {
return null;
}
if (receipt.state === "not_found") {
return [
receipt.operation,
receipt.memory_id,
receipt.result_revision,
receipt.result_status,
receipt.memory_state,
receipt.migrated_from,
].every((item) => item === null)
? "not_found"
: null;
}
const expectedOperation =
plan.operation === "add" ? "create" : plan.operation;
const expectedRevision =
plan.operation === "add" ? 1 : (plan.expected_revision as number) + 1;
const expectedStatus = plan.operation === "delete" ? "deleted" : "confirmed";
// A migrated receipt records an operation from before v2; no plan made
// today can have produced it.
if (
receipt.state !== "committed" ||
receipt.migrated_from !== null ||
receipt.operation !== expectedOperation ||
typeof receipt.memory_id !== "string" ||
!isMemoryId(receipt.memory_id) ||
(plan.memory_id !== undefined && receipt.memory_id !== plan.memory_id) ||
receipt.result_revision !== expectedRevision ||
receipt.result_status !== expectedStatus ||
!["present", "deleted"].includes(String(receipt.memory_state))
) {
return null;
}
return {
schema_version: 2,
request_id: plan.request_id,
state: "committed",
operation: expectedOperation,
memory_id: receipt.memory_id,
result_revision: expectedRevision,
result_status: expectedStatus,
memory_state: receipt.memory_state,
proof_scope: "governance_transaction_only",
downstream_sync_verified: false,
payload_fingerprint_verified: false,
};
}
function validatePlan(value: unknown): MemoryMutationPlan | null {
const plan = record(value);
if (
plan?.schema_version !== 1 ||
plan.kind !== "memory_mutation" ||
!oneOf(plan.operation, OPERATIONS) ||
typeof plan.request_id !== "string" ||
!UUID.test(plan.request_id) ||
typeof plan.digest !== "string"
) {
return null;
}
const body = { ...plan };
delete body.digest;
const expected = withDigest(body as unknown as PlanBody);
if (!safeEqual(expected.digest, plan.digest)) return null;
if (plan.operation === "add") {
if (
!oneOf(plan.memory_kind, MEMORY_KINDS) ||
!validScopes(plan.scopes) ||
!validIds(plan.evidence_ids, 500) ||
!contentFields(plan)
) {
return null;
}
} else if (
typeof plan.memory_id !== "string" ||
!isMemoryId(plan.memory_id) ||
!Number.isSafeInteger(plan.expected_revision) ||
(plan.expected_revision as number) < 1 ||
plan.expected_status !== "confirmed" ||
!oneOf(plan.expected_origin, MEMORY_ORIGINS) ||
!validCurrent(plan) ||
(plan.operation === "correct" && !contentFields(plan))
) {
return null;
}
return plan as unknown as MemoryMutationPlan;
}
function contentFields(plan: Record<string, unknown>): boolean {
return (
typeof plan.content_sha256 === "string" &&
/^[0-9a-f]{64}$/.test(plan.content_sha256) &&
Number.isSafeInteger(plan.content_characters) &&
(plan.content_characters as number) >= 1 &&
(plan.content_characters as number) <= CONTENT_MAX_CHARACTERS
);
}
function validCurrent(plan: Record<string, unknown>): boolean {
const current = record(plan.current);
if (
!current ||
current.status !== plan.expected_status ||
current.origin !== plan.expected_origin ||
current.revision !== plan.expected_revision ||
!validScopes(current.scopes) ||
!Array.isArray(current.evidence) ||
current.evidence.length < 1 ||
current.evidence.length > 500
) {
return false;
}
return current.evidence.every((value) => {
const evidence = record(value);
return (
evidence !== null &&
typeof evidence.evidence_id === "string" &&
ID.test(evidence.evidence_id) &&
typeof evidence.available === "boolean" &&
(evidence.source_id === undefined ||
(typeof evidence.source_id === "string" &&
evidence.source_id.length <= 128)) &&
(evidence.written_by === undefined ||
(typeof evidence.written_by === "string" &&
evidence.written_by.length <= 256))
);
});
}
function contentIdentity(content: string): {
content_sha256: string;
content_characters: number;
} {
return {
content_sha256: createHash("sha256").update(content, "utf8").digest("hex"),
content_characters: [...content].length,
};
}
function validContent(content: string): boolean {
return (
content.trim().length > 0 &&
[...content].length <= CONTENT_MAX_CHARACTERS &&
![...content].some((character) => {
const code = character.charCodeAt(0);
return code <= 8 || (code >= 11 && code <= 31) || code === 127;
})
);
}
function validScopes(value: unknown): value is string[] {
if (
!Array.isArray(value) ||
value.length < 1 ||
value.length > 128 ||
new Set(value).size !== value.length ||
value.some(
(scope) =>
typeof scope !== "string" ||
scope.trim() !== scope ||
scope.length < 1 ||
scope.length > 512 ||
scope === "*",
)
) {
return false;
}
return !value.includes("private") || value.length === 1;
}
function validIds(value: unknown, maximum: number): value is string[] {
return (
Array.isArray(value) &&
value.length >= 1 &&
value.length <= maximum &&
new Set(value).size === value.length &&
value.every((item) => typeof item === "string" && ID.test(item))
);
}
function withDigest(body: PlanBody): MemoryMutationPlan {
return {
...body,
digest: createHash("sha256")
.update(JSON.stringify(body), "utf8")
.digest("hex"),
};
}
function preview(plan: MemoryMutationPlan): MemoryFlowDocument {
return {
schema_version: 1,
product: PRODUCT,
state: "preview",
plan,
};
}
function invalid(code: string): MemoryFlowDocument {
return {
schema_version: 1,
product: PRODUCT,
state: "invalid",
code,
};
}
function pending(operationId: string, code: string): MemoryFlowDocument {
return {
schema_version: 1,
product: PRODUCT,
state: "unknown",
operation_id: operationId,
code,
};
}
function same(left: readonly string[], right: readonly string[]): boolean {
return (
left.length === right.length &&
left.every((value, index) => value === right[index])
);
}
function safeEqual(left: string, right: string): boolean {
if (left.length !== right.length) return false;
return timingSafeEqual(Buffer.from(left), Buffer.from(right));
}
function oneOf<const T extends readonly string[]>(
value: unknown,
values: T,
): value is T[number] {
return typeof value === "string" && values.includes(value as T[number]);
}
function record(value: unknown): Record<string, unknown> | null {
return value !== null && typeof value === "object" && !Array.isArray(value)
? (value as Record<string, unknown>)
: null;
}
@@ -1,211 +0,0 @@
import { readFileSync } from "node:fs";
import { describe, expect, it } from "vitest";
import type { CoreReader } from "./core-client.js";
import {
planExistingMemoryMutation,
planMemoryAdd,
readMemoryMutationReceipt,
} from "./memory-flows.js";
// Written by a real Core and checked against it on the Core side, so these are
// the receipts the CLI meets in use rather than ones shaped to fit the parser.
const FIXTURE = JSON.parse(
readFileSync(
new URL(
"../../../packages/contracts/fixtures/memory-mutation-receipts.v2.json",
import.meta.url,
),
"utf8",
),
) as { receipts: Record<"add" | "correct" | "delete", Receipt> };
interface Receipt {
schema_version: number;
request_id: string;
memory_id: string;
result_revision: number;
result_status: string;
operation: string;
}
function reader(data: unknown): CoreReader {
return async <T>() => ({ state: "ok", data: data as T });
}
function detail(sample: Receipt) {
const revision = sample.result_revision - 1;
return {
memory: {
memory_id: sample.memory_id,
user_id: "local",
kind: "chat",
status: "confirmed",
content: "Synthetic memory.",
scopes: ["all-apps"],
evidence_ids: ["evidence-1"],
origin: "manual",
tags: [],
revision,
created_at: "2026-09-19T00:00:00Z",
updated_at: "2026-09-19T00:00:01Z",
deleted_at: null,
},
evidence: [
{
evidence_id: "evidence-1",
available: true,
source_id: "fixture",
written_by: "person",
created_at: "2026-09-19T00:00:00Z",
verbatim: "Synthetic source text.",
},
],
history: [
{
revision,
action: "create",
status: "confirmed",
content: "Synthetic memory.",
scopes: ["all-apps"],
created_at: "2026-09-19T00:00:01Z",
},
],
index_sync: null,
};
}
describe("memory mutation receipt v2 from Core", () => {
it("recovers an add from the receipt Core returns for a create", async () => {
const sample = FIXTURE.receipts.add;
expect(sample.schema_version).toBe(2);
const preview = planMemoryAdd({
content: "Synthetic memory.",
memoryKind: "chat",
scopes: ["all-apps"],
evidenceIds: ["evidence-1"],
});
if (preview.state !== "preview") throw new Error("preview expected");
// The request id is the one binding a receipt to its plan; the plan picks it.
const result = await readMemoryMutationReceipt(
reader({ ...sample, request_id: preview.plan.request_id }),
preview.plan,
);
expect(result).toMatchObject({
state: "recovered",
receipt: {
schema_version: 2,
operation: "create",
memory_id: sample.memory_id,
result_revision: 1,
result_status: "confirmed",
memory_state: "present",
},
});
});
it.each<["correct" | "delete", { content?: string }]>([
["correct", { content: "Corrected synthetic memory." }],
["delete", {}],
])(
"recovers %s from the receipt Core returns for it",
async (operation, extra) => {
const sample = FIXTURE.receipts[operation];
const preview = await planExistingMemoryMutation(reader(detail(sample)), {
operation,
memoryId: sample.memory_id,
...extra,
});
if (preview.state !== "preview") {
throw new Error(`preview expected: ${JSON.stringify(preview)}`);
}
const result = await readMemoryMutationReceipt(
reader({ ...sample, request_id: preview.plan.request_id }),
preview.plan,
);
expect(result).toMatchObject({
state: "recovered",
receipt: {
schema_version: 2,
operation: sample.operation,
memory_id: sample.memory_id,
result_revision: sample.result_revision,
result_status: sample.result_status,
},
});
},
);
it("does not take a v1 receipt for a v2 one", async () => {
const sample = FIXTURE.receipts.delete;
const preview = await planExistingMemoryMutation(reader(detail(sample)), {
operation: "delete",
memoryId: sample.memory_id,
});
if (preview.state !== "preview") throw new Error("preview expected");
const { migrated_from: _dropped, ...v1 } = sample as Receipt & {
migrated_from: null;
};
// As v1 was, and with v2's fields but v1's number: either alone is refused.
for (const receipt of [
{ ...v1, schema_version: 1 },
{ ...sample, schema_version: 1 },
]) {
const result = await readMemoryMutationReceipt(
reader({ ...receipt, request_id: preview.plan.request_id }),
preview.plan,
);
expect(result).toMatchObject({
state: "unknown",
code: "memory_mutation_receipt_invalid",
});
}
});
it("does not take a not-found receipt that names a migrated operation", async () => {
const sample = FIXTURE.receipts.delete;
const preview = await planExistingMemoryMutation(reader(detail(sample)), {
operation: "delete",
memoryId: sample.memory_id,
});
if (preview.state !== "preview") throw new Error("preview expected");
const notFound = {
schema_version: 2,
request_id: preview.plan.request_id,
state: "not_found",
operation: null,
memory_id: null,
result_revision: null,
result_status: null,
memory_state: null,
};
expect(
await readMemoryMutationReceipt(
reader({ ...notFound, migrated_from: null }),
preview.plan,
),
).toMatchObject({
code: "memory_mutation_receipt_not_found_after_attempt",
});
expect(
await readMemoryMutationReceipt(
reader({ ...notFound, migrated_from: "reject" }),
preview.plan,
),
).toMatchObject({ code: "memory_mutation_receipt_invalid" });
});
it("plans nothing on a memory that is not in effect", async () => {
const sample = FIXTURE.receipts.delete;
const superseded = detail(sample);
superseded.memory.status = "superseded";
superseded.history[0].status = "superseded";
const result = await planExistingMemoryMutation(reader(superseded), {
operation: "delete",
memoryId: sample.memory_id,
});
expect(result).toMatchObject({
state: "invalid",
code: "memory_operation_not_allowed",
});
});
});
+23
View File
@@ -15,6 +15,29 @@ export function isMemoryId(value: string): boolean {
);
}
const CONTENT_MAX_CHARACTERS = 100_000;
/** A memory body read from `--content-file`: UTF-8, non-blank, bounded, and
* free of control characters other than tab and newline. */
export function parseMemoryContent(bytes: Buffer): string | null {
if (bytes.byteLength > CONTENT_MAX_CHARACTERS * 4 + 2) return null;
let decoded: string;
try {
decoded = new TextDecoder("utf-8", { fatal: true }).decode(bytes);
} catch {
return null;
}
const content = decoded.replace(/\r?\n$/, "");
return content.trim().length > 0 &&
[...content].length <= CONTENT_MAX_CHARACTERS &&
![...content].some((character) => {
const code = character.charCodeAt(0);
return code <= 8 || (code >= 11 && code <= 31) || code === 127;
})
? content
: null;
}
export interface MemoryQuery {
query?: string;
status?: string;
+39 -16
View File
@@ -184,8 +184,16 @@ const CORE = new Map<string, { status: number; data: unknown }>([
},
],
[
"/api/v1/home-summary",
{ status: 200, data: { memory_count: 42, latest: null, observed_at: "T" } },
"/api/v2/home-summary",
{
status: 200,
data: {
schema_version: 2,
memory_count: 42,
latest: null,
observed_at: "T",
},
},
],
]);
@@ -194,9 +202,7 @@ const READ_ONLY_PATHS = [
"/api/v1/live-sync",
"/api/v1/memory-processing",
"/api/v1/model-profiles",
"/api/v1/home-summary",
"/api/v1/memory-library",
"/api/v1/memories/",
"/api/v2/home-summary",
"/api/v2/memory-library",
"/api/v2/memories/",
"/api/v2/user-profile",
@@ -816,7 +822,7 @@ describe("CLI-21 memory reads", () => {
expect(report.history).toEqual([]);
});
// Core registers /api/v1/memories/graph before /{memory_id}, and that static
// Core registers /api/v2/memories/graph before /{memory_id}, and that static
// handler initializes the HY bridge. A memory id that collides with a
// reserved segment must never become a request.
it("refuses a memory id that collides with a static Core route", async () => {
@@ -830,18 +836,15 @@ describe("CLI-21 memory reads", () => {
expect(streams.stdout()).toBe("");
}
expect(transport.get).not.toHaveBeenCalled();
expect(isReadOnlyPath("/api/v1/memories/graph")).toBe(false);
expect(isReadOnlyPath("/api/v1/memories/ask")).toBe(false);
expect(isReadOnlyPath("/api/v1/memories/m1")).toBe(true);
expect(isReadOnlyPath("/api/v2/memory-library")).toBe(true);
expect(isReadOnlyPath("/api/v2/memories/graph")).toBe(false);
expect(isReadOnlyPath("/api/v2/memories/ask")).toBe(false);
expect(isReadOnlyPath("/api/v2/memories/m1")).toBe(true);
expect(isReadOnlyPath("/api/v2/memory-library")).toBe(true);
expect(isReadOnlyPath("/api/v2/home-summary")).toBe(true);
// The retired v1 data plane is no longer a CLI read.
expect(isReadOnlyPath("/api/v1/memories/m1")).toBe(false);
expect(isReadOnlyPath("/api/v1/home-summary")).toBe(false);
expect(isReadOnlyPath("/api/v1/model-content-permission")).toBe(false);
expect(
isReadOnlyPath(
"/api/v1/memory-mutations/123e4567-e89b-12d3-a456-426614174000",
),
).toBe(true);
expect(isReadOnlyPath("/api/v1/memory-mutations/../sources")).toBe(false);
});
it("refuses a memory id that would escape the memories path", async () => {
@@ -861,6 +864,26 @@ describe("CLI-21 memory reads", () => {
// was cast to the expected type, so a wrong record, an unavailable excerpt, an
// oversized page and a non-string model field all reached stdout as success.
describe("malformed success responses", () => {
it("treats a v1-shaped home summary as invalid, not as a memory count", async () => {
transport.get.mockImplementation(async (url: string) =>
url === "/api/v2/home-summary"
? {
status: 200,
data: { memory_count: 7, latest: null, observed_at: "T" },
}
: (CORE.get(url) ?? { status: 404, data: { detail: "not_found" } }),
);
const streams = captureStreams();
try {
await run(["status", "--json"], {});
} finally {
streams.restore();
}
expect(JSON.parse(streams.stdout()).sections.memories).toMatchObject({
state: "invalid",
});
});
it("refuses a detail that is not the memory that was asked for", async () => {
transport.get.mockResolvedValue({
status: 200,
@@ -315,6 +315,40 @@ describe("macOS service lifecycle", () => {
expect(deps.verifyRegistrationSnapshot).toHaveBeenCalledTimes(2);
});
it("does not re-bootstrap a previous registration that may serve a legacy home", async () => {
const f = fixture();
const oldPath = "/Users/example/Library/LaunchAgents/old-runtime.plist";
const runCommand = vi
.fn()
.mockResolvedValueOnce(undefined)
.mockRejectedValueOnce(new Error("bootstrap failed"));
const deps = dependencies({
inspectRegistration: vi.fn(async () => ({
kind: "managed" as const,
plistPath: oldPath,
content: "old managed plist",
})),
runCommand,
});
const prepared = await prepareServiceInstall(f.layout, f.options, deps);
await expect(
applyPreparedServiceInstall(
{ ...prepared, restorePreviousOnFailure: false },
deps,
),
).rejects.toThrow("runtime_service_replacement_failed");
// bootout + the failed bootstrap only; the old plist is not bootstrapped.
expect(runCommand).toHaveBeenCalledTimes(2);
expect(runCommand).not.toHaveBeenCalledWith([
"launchctl",
"bootstrap",
"gui/501",
oldPath,
]);
});
it("removes the replacement plist and restores the previous registration", async () => {
const f = fixture();
const oldPath = "/Users/example/Library/LaunchAgents/old-runtime.plist";
+16 -1
View File
@@ -72,6 +72,13 @@ export type PreparedServiceInstall = {
mode: "direct" | "bootstrap-existing" | "restart" | "replace";
previous?: ManagedSnapshot;
previouslyEnabled: boolean;
/**
* Re-bootstrap the previous (possibly older) LaunchAgent when the new one
* fails to bootstrap. `install` turns this off once the home has been handed
* to this version's SDK migration: an older Runtime has no migration gate and
* would serve a home that is not on SDK authority.
*/
restorePreviousOnFailure?: boolean;
};
const defaultDependencies: ServiceLifecycleDependencies = {
@@ -191,7 +198,8 @@ export async function applyPreparedServiceInstall(
if (!previous) throw new Error("runtime_service_upgrade_state_invalid");
await verifyPreparedRegistration(prepared, dependencies);
const domain = launchdDomain(options.uid);
let replacementFailure: "upgrade" | "rollback" | null = null;
let replacementFailure: "upgrade" | "rollback" | "not-restored" | null =
null;
const replaceCommand: ServiceCommandRunner = async (command) => {
if (command[0] !== "launchctl" || command[1] !== "bootstrap") {
await dependencies.runCommand(command);
@@ -208,6 +216,10 @@ export async function applyPreparedServiceInstall(
await dependencies.runCommand(command);
} catch {
replacementFailure = "upgrade";
if (prepared.restorePreviousOnFailure === false) {
replacementFailure = "not-restored";
throw new Error("runtime_service_replacement_failed");
}
try {
await dependencies.verifyRegistrationSnapshot(previous);
await dependencies.runCommand([
@@ -234,6 +246,9 @@ export async function applyPreparedServiceInstall(
if (replacementFailure === "upgrade") {
throw new Error("runtime_service_managed_upgrade_failed");
}
if (replacementFailure === "not-restored") {
throw new Error("runtime_service_replacement_failed");
}
throw error;
}
} catch (error) {
+4 -35
View File
@@ -7,12 +7,10 @@ import type {
} from "@one-memory/contracts";
import type { CoreReader, ReadOutcome } from "./core-client.js";
import {
type ValidatedMemoryTask,
validateImportIssues,
validateImportJob,
validateImportJobs,
validateLiveSync,
validateMemoryTasks,
validateProcessing,
validateSources,
} from "./core-responses.js";
@@ -70,11 +68,6 @@ export interface ProgressDocument {
generation: {
summary_state: ReadState;
summary: Record<string, unknown> | null;
tasks_state: ReadState;
tasks: Array<Record<string, unknown>>;
returned: number;
validated: number;
output_truncated: boolean;
};
}
@@ -134,12 +127,13 @@ export async function readProgress(
const importsPath = options.jobId
? `/api/v1/import-jobs/${options.jobId}`
: "/api/v1/import-jobs";
const [importsRead, sourcesRead, processingRead, tasksRead, issuesRead] =
// Generation progress comes from `memory-processing` alone: under SDK
// authority the per-task list (`/api/v1/memory-tasks`) is retired and empty.
const [importsRead, sourcesRead, processingRead, issuesRead] =
await Promise.all([
reader<unknown>(importsPath),
reader<unknown>("/api/v1/sources"),
reader<unknown>("/api/v1/memory-processing"),
reader<unknown>("/api/v1/memory-tasks"),
options.jobId && options.issues
? reader<unknown>(`/api/v1/import-jobs/${options.jobId}/issues`)
: Promise.resolve(null),
@@ -153,7 +147,6 @@ export async function readProgress(
: validated(importsRead, validateImportJobs);
const processing = validated(processingRead, validateProcessing);
const sources = validated(sourcesRead, validateSources);
const tasks = validated(tasksRead, validateMemoryTasks);
const issues =
options.jobId && options.issues && issuesRead
? validated(issuesRead, (value) =>
@@ -162,25 +155,19 @@ export async function readProgress(
: null;
const jobs = imports.state === "ok" ? imports.data : [];
const taskItems = tasks.state === "ok" ? tasks.data : [];
const issueItems = issues?.state === "ok" ? issues.data : [];
const importState = stateOnly(imports);
const summaryState = stateOnly(processing);
const tasksState = stateOnly(tasks);
const hasReadFailure =
importState.state !== "ok" ||
sources.state !== "ok" ||
summaryState.state !== "ok" ||
tasksState.state !== "ok" ||
(issues !== null && issues.state !== "ok");
const hasProblem =
jobs.some((job) =>
["paused", "interrupted", "failed"].includes(job.status),
) ||
taskItems.some((task) =>
["waiting_for_model", "failed"].includes(task.status),
) ||
(processing.state === "ok" &&
(processing.data.failed > 0 ||
processing.data.waiting_for_model > 0 ||
@@ -188,7 +175,6 @@ export async function readProgress(
issueItems.length > 0;
const hasActive =
jobs.some((job) => ["pending", "running"].includes(job.status)) ||
taskItems.some((task) => ["pending", "running"].includes(task.status)) ||
(processing.state === "ok" &&
(processing.data.pending > 0 || processing.data.running > 0));
@@ -229,11 +215,6 @@ export async function readProgress(
summary_state: summaryState,
summary:
processing.state === "ok" ? projectProcessing(processing.data) : null,
tasks_state: tasksState,
tasks: taskItems.slice(0, options.limit).map(projectTask),
returned: Math.min(taskItems.length, options.limit),
validated: taskItems.length,
output_truncated: taskItems.length > options.limit,
},
};
}
@@ -289,10 +270,7 @@ export function renderProgressText(document: ProgressDocument): string {
? `记忆整理:待处理 ${document.generation.summary.pending} · 进行中 ${document.generation.summary.running} · 等待模型 ${document.generation.summary.waiting_for_model} · 失败 ${document.generation.summary.failed}`
: "记忆整理:暂时无法读取",
);
if (
document.imports.output_truncated ||
document.generation.output_truncated
) {
if (document.imports.output_truncated) {
lines.push("这里只显示汇总;完整记录请使用 --json。");
}
lines.push("");
@@ -353,11 +331,6 @@ export function unreadProgress(
generation: {
summary_state: failed,
summary: null,
tasks_state: failed,
tasks: [],
returned: 0,
validated: 0,
output_truncated: false,
},
};
}
@@ -465,7 +438,3 @@ function projectProcessing(
})),
};
}
function projectTask(task: ValidatedMemoryTask): Record<string, unknown> {
return { ...task };
}
+6 -48
View File
@@ -129,18 +129,6 @@ const RESPONSES = new Map<string, { status: number; data: unknown }>([
},
},
],
[
"/api/v1/memory-tasks",
{
status: 200,
data: [
memoryTask("task-pending", "pending"),
memoryTask("task-running", "running"),
memoryTask("task-model", "waiting_for_model", "model_missing"),
memoryTask("task-failed", "failed", "provider_unavailable"),
],
},
],
[
"/api/v1/import-jobs/job-running",
{ status: 200, data: importJob("job-running", "running") },
@@ -282,9 +270,9 @@ describe("CLI-19 progress and issue reads", () => {
imported_messages: 20,
totals_state: { state: "ok" },
});
expect(
report.generation.tasks.map((task: { status: string }) => task.status),
).toEqual(["pending", "running", "waiting_for_model", "failed"]);
// The retired per-task list is no longer read or reported.
expect(report.generation).not.toHaveProperty("tasks");
expect(requestedPaths()).not.toContain("/api/v1/memory-tasks");
expect(streams.stdout()).not.toMatch(/percent|percentage|%/i);
});
@@ -374,7 +362,7 @@ describe("CLI-19 progress and issue reads", () => {
expect(streams.stdout()).not.toContain("SYNTHETIC_FOREIGN_ISSUE");
});
it("bounds each task list and emits an explicit truncation receipt", async () => {
it("bounds the import job list and emits an explicit truncation receipt", async () => {
const streams = captureStreams();
try {
expect(await run(["progress", "--limit", "1", "--json"], {})).toBe(1);
@@ -388,12 +376,6 @@ describe("CLI-19 progress and issue reads", () => {
validated: 3,
output_truncated: true,
});
expect(report.generation.tasks).toHaveLength(1);
expect(report.generation).toMatchObject({
returned: 1,
validated: 4,
output_truncated: true,
});
});
it("streams bounded progress snapshots as serial NDJSON", async () => {
@@ -428,7 +410,6 @@ describe("CLI-19 progress and issue reads", () => {
"/api/v1/import-jobs",
"/api/v1/sources",
"/api/v1/memory-processing",
"/api/v1/memory-tasks",
]) {
expect(requestedPaths().filter((item) => item === path)).toHaveLength(2);
}
@@ -500,7 +481,7 @@ describe("CLI-19 progress and issue reads", () => {
it("preserves denied, unavailable and malformed sections in one partial report", async () => {
transport.get.mockImplementation(async (url: string) => {
if (url === "/api/v1/import-jobs") return { status: 503, data: {} };
if (url === "/api/v1/memory-tasks") return { status: 403, data: {} };
if (url === "/api/v1/sources") return { status: 403, data: {} };
if (url === "/api/v1/memory-processing") {
return { status: 200, data: { pending: { secret: "nested" } } };
}
@@ -521,7 +502,7 @@ describe("CLI-19 progress and issue reads", () => {
state: "invalid",
code: "response_invalid",
});
expect(report.generation.tasks_state).toEqual({
expect(report.imports.totals_state).toEqual({
state: "denied",
code: "http_403",
});
@@ -574,7 +555,6 @@ describe("CLI-19 progress and issue reads", () => {
"/api/v1/live-sync",
"/api/v1/import-jobs",
"/api/v1/memory-processing",
"/api/v1/memory-tasks",
].some((allowed) => path === allowed || path.startsWith(`${allowed}/`)),
),
).toBe(true);
@@ -634,28 +614,6 @@ function importJob(
};
}
function memoryTask(
taskId: string,
status: string,
error: string | null = null,
) {
return {
task_id: taskId,
request_id: `request-${taskId}`,
user_id: "local",
kind: "extract",
input_ref: "synthetic",
status,
profile_id: "profile-1",
profile_revision: 1,
profile_snapshot: {},
attempts: status === "failed" ? 2 : 0,
last_error_code: error,
created_at: "T1",
updated_at: "T2",
};
}
function requestedPaths(): string[] {
return (transport.get.mock.calls as [string][]).map(([path]) => path);
}
+43 -1
View File
@@ -1,6 +1,11 @@
import { describe, expect, it } from "vitest";
import { renderStatusText, type StatusDocument } from "./status-report.js";
import { composeDoctorReport, renderDoctorText } from "./doctor-report.js";
import {
composeStatusReport,
renderStatusText,
type StatusDocument,
} from "./status-report.js";
describe("status default output", () => {
it("formats blocked task objects recursively instead of printing object coercions", () => {
@@ -35,3 +40,40 @@ describe("status default output", () => {
);
});
});
describe("status and doctor on an unfinished upgrade migration", () => {
it("names the migration log and the issues page", async () => {
const report = await composeStatusReport({
dataDir: "/tmp/isolated/.one-memory",
runtime: { state: "stopped", code: "sdk_migration_required" },
background: null,
reader: null,
});
expect(report.runtime.log_file).toBe(
"/tmp/isolated/.one-memory/logs/sdk-migration.log",
);
expect(report.runtime.issues_url).toBe(
"https://github.com/Tencent-Hunyuan/Hy-Memory/issues/new",
);
const text = renderStatusText(report, "MemHub");
expect(text).toContain("sdk_migration_required");
expect(text).toContain("/tmp/isolated/.one-memory/logs/sdk-migration.log");
const doctor = renderDoctorText(composeDoctorReport(report, null));
expect(doctor).toContain(
"/tmp/isolated/.one-memory/logs/sdk-migration.log",
);
expect(doctor).not.toContain("帮助与诊断");
});
it("adds no log path for an ordinary stopped runtime", async () => {
const report = await composeStatusReport({
dataDir: "/tmp/isolated/.one-memory",
runtime: { state: "stopped", code: "runtime_not_running" },
background: null,
reader: null,
});
expect(report.runtime.log_file).toBeUndefined();
});
});
+23 -3
View File
@@ -1,5 +1,4 @@
import type {
HomeSummary,
LiveSyncHealth,
MemoryProcessingSummary,
ModelProfile,
@@ -9,8 +8,10 @@ import type {
BackgroundHealth,
RuntimeStatus,
} from "@one-memory/runtime-control";
import { sdkMigrationLogPath } from "@one-memory/runtime-control";
import type { CoreReader, ReadOutcome } from "./core-client.js";
import {
type SdkHomeSummary,
validateHomeSummary,
validateLiveSync,
validateModelProfiles,
@@ -19,6 +20,9 @@ import {
} from "./core-responses.js";
import { PRODUCT } from "./version.js";
const MIGRATION_ISSUES_URL =
"https://github.com/Tencent-Hunyuan/Hy-Memory/issues/new";
export type SectionState =
| "ok"
| "denied"
@@ -42,6 +46,9 @@ export interface StatusDocument {
host?: string;
port?: number;
version?: string;
/** Set only when an unfinished upgrade migration is why nothing serves. */
log_file?: string;
issues_url?: string;
};
bridge: {
state: BackgroundHealth["state"] | "not_checked";
@@ -102,7 +109,7 @@ export async function composeStatusReport(options: {
reader<unknown>("/api/v1/live-sync"),
reader<unknown>("/api/v1/memory-processing"),
reader<unknown>("/api/v1/model-profiles"),
reader<unknown>("/api/v1/home-summary"),
reader<unknown>("/api/v2/home-summary"),
]);
sections.sources = section(sources, validateSources, describeSources);
sections.sync = section(sync, validateLiveSync, describeSync);
@@ -153,6 +160,12 @@ export async function composeStatusReport(options: {
host: options.runtime.host,
port: options.runtime.port,
version: options.runtime.version,
...(options.runtime.code.startsWith("sdk_migration_")
? {
log_file: sdkMigrationLogPath(options.dataDir),
issues_url: MIGRATION_ISSUES_URL,
}
: {}),
},
bridge,
sections,
@@ -177,6 +190,13 @@ export function renderStatusText(
): string {
const lines = [
`${heading}:${runtimeLine(report)}`,
...(report.runtime.log_file
? [
`升级迁移没有完成(错误码:${report.runtime.code}),MemHub 不会启动。`,
`请把迁移日志附在 GitHub 反馈里发给我们:${report.runtime.log_file}`,
`反馈地址:${report.runtime.issues_url}`,
]
: []),
`资料空间:${report.data_dir}`,
`应用连接:${BRIDGE_LABELS[report.bridge.state]}`,
];
@@ -347,7 +367,7 @@ function describeModel(profiles: ModelProfile[]): Record<string, unknown> {
};
}
function describeMemories(summary: HomeSummary): Record<string, unknown> {
function describeMemories(summary: SdkHomeSummary): Record<string, unknown> {
return {
memory_count: summary.memory_count,
};
File diff suppressed because it is too large. Load diff
@@ -15,7 +15,7 @@ from one_memory.call_event_store import (
CallTool,
HostId,
)
from one_memory.context_models import ContextSearchResponse, MemoryContextItem, RawContextItem
from one_memory.context_models import SdkContextSearchResponse
from one_memory.external_memory_access import ExternalMemoryAccess
from one_memory.recall import RecallReceipt
@@ -41,75 +41,27 @@ def _host_id(access: ExternalMemoryAccess | None) -> HostId | None:
return cast(HostId, host) if host in HOSTS else None
def context_call_event(
result: ContextSearchResponse,
def sdk_context_call_event(
result: SdkContextSearchResponse,
access: ExternalMemoryAccess | None,
*,
memory_searched: bool,
timed_out: bool,
) -> CallEvent:
references: list[CallReference] = []
seen: set[tuple[str, str, int]] = set()
for item in result.items:
if isinstance(item, RawContextItem):
candidates = [
CallReference(
kind="raw",
id=message.evidence_id,
revision=message.revision,
source_id=item.source_id,
)
for message in item.messages
]
elif isinstance(item, MemoryContextItem):
candidates = [
CallReference(
kind="memory", id=item.memory_id, revision=item.revision, source_id=None
)
]
else:
continue
for reference in candidates:
identity = (reference.kind, reference.id, reference.revision)
if identity not in seen:
seen.add(identity)
references.append(reference)
timed_out = any(
path.reason == "timeout_budget" for path in (result.paths.raw, result.paths.memory)
# SDK items carry no Evidence/Governance revision, and the reference
# resolver reads the legacy store, so the event records counts only.
raw_count = sum(item.kind == "raw" for item in result.items)
memory_count = sum(item.kind == "memory" for item in result.items)
state: CallState = "timed_out" if timed_out else result.state
return _metadata_event(
tool="om_recall" if memory_searched else "om_conversations",
state=state,
elapsed_ms=result.total_ms,
access=access,
raw_count=raw_count,
memory_count=memory_count,
event_id=UUID(result.request_id),
)
state: CallState = (
"partial" if timed_out and references else "timed_out" if timed_out else result.state
)
return CallEvent.model_validate(
{
"event_id": UUID(result.request_id),
"occurred_at": datetime.now(UTC),
"host_id": _host_id(access),
"installation_id": UUID(access.installation_id) if access else None,
"tool": _context_tool(result),
"state": state,
"stage": "core_result",
"delivery": "unknown",
"adoption": "unknown",
"host_session_id": None,
"injection_elapsed_ms": None,
"elapsed_ms": result.total_ms,
"raw_count": sum(isinstance(item, RawContextItem) for item in result.items),
"memory_count": sum(isinstance(item, MemoryContextItem) for item in result.items),
"references": tuple(references),
}
)
def record_context_result(
store: CallEventStore,
result: ContextSearchResponse,
access: ExternalMemoryAccess | None,
) -> None:
store.record(context_call_event(result, access))
def _context_tool(result: ContextSearchResponse) -> CallTool:
if result.paths.memory.status not in {"not_requested", "not_authorized"}:
return "om_recall"
return "om_conversations"
def _metadata_event(
@@ -142,26 +94,6 @@ def _metadata_event(
)
def conversation_call_event(
result: object,
access: ExternalMemoryAccess | None,
elapsed_ms: float,
) -> CallEvent:
hits = tuple(getattr(result, "hits", ()) or ())
conversations = {
conversation_id
for hit in hits
if (conversation_id := getattr(hit, "conversation_id", None))
}
return _metadata_event(
tool="om_conversations",
state="grounded" if hits else "empty",
elapsed_ms=elapsed_ms,
access=access,
raw_count=min(len(conversations), 20),
)
def ask_call_event(
result: object,
access: ExternalMemoryAccess | None,
+240 -83
View File
@@ -1,6 +1,5 @@
import os
import signal
import socket
import sys
# A handed-off capture ends itself: the hook that gave it the prompt has
@@ -29,10 +28,13 @@ def _end_by(seconds: str | None) -> None:
if __name__ == "__main__" and sys.argv[1:2] == ["capture"]:
_end_by(os.environ.get("ONE_MEMORY_CAPTURE_DEADLINE_S"))
import contextlib # noqa: E402
import errno # noqa: E402
import json # noqa: E402
import re # noqa: E402
import time # noqa: E402
from collections.abc import Iterator, Sequence # noqa: E402
from pathlib import Path # noqa: E402
from typing import TextIO # noqa: E402
from one_memory.config import load_settings # noqa: E402
@@ -68,6 +70,9 @@ def migration_failure_code(
return "sdk_migration_disk_space_insufficient"
for item in chain:
code = getattr(item, "code", "")
# `sdk_data_plane_write_job_input_missing:<job_id>`: the name is the
# code; the id is for the log, not for the dialog.
code = code.split(":", 1)[0] if isinstance(code, str) else code
if not isinstance(code, str) or not _SAFE_MIGRATION_CODE.match(code):
continue
if code.startswith(("sdk_data_plane_", "sdk_migration_")):
@@ -103,12 +108,8 @@ def run(arguments: Sequence[str], *, output: TextIO | None = None) -> int:
if not arguments or list(arguments) in (
["serve"],
["serve", "--maintenance"],
["serve", "--reader"],
):
serve(
maintenance=list(arguments) == ["serve", "--maintenance"],
reader=list(arguments) == ["serve", "--reader"],
)
serve(maintenance=list(arguments) == ["serve", "--maintenance"])
return 0
command, *options = arguments
if command == "restore":
@@ -117,29 +118,17 @@ def run(arguments: Sequence[str], *, output: TextIO | None = None) -> int:
return sdk_migrate(options, stream)
if command == "sdk-migration-status":
return sdk_migration_status(options, stream)
if command == "boundary" and not options:
from one_memory.workbuddy_boundary import write_hook
return write_hook(stream, sys.stdin)
if command == "boundary" and options == ["--peek"]:
from one_memory.workbuddy_boundary import write_peek
return write_peek(stream, sys.stdin)
if command == "boundary" and len(options) == 2 and options[0] == "--ack":
from one_memory.workbuddy_boundary import run_confirm
return run_confirm(options[1])
if command == "capture" and len(options) == 1:
from one_memory.live_capture import capture_stdio
return capture_stdio(options[0], sys.stdin, stream)
if command not in {"status", "doctor"} or options not in ([], ["--json"]):
stream.write(
"Usage: one-memory [serve [--maintenance|--reader]|status [--json]|doctor [--json]"
"Usage: one-memory [serve [--maintenance]|status [--json]|doctor [--json]"
"|restore --source <managed-backup> --yes [--json]"
"|sdk-migrate --runtime-build <build> --yes [--json]"
"|sdk-migration-status [--json]"
"|boundary [--peek|--ack TOKEN]|capture SOURCE]\n"
"|capture SOURCE]\n"
)
return 2
@@ -312,7 +301,22 @@ def sdk_migrate(options: Sequence[str], stream: TextIO) -> int:
stream.write("one-memory: sdk-migrate 需要 --runtime-build <build>\n")
return 2
from one_memory.sdk_migration_log import authority_snapshot, write_migration_log
settings = load_settings()
database_path = settings.data_dir / "one-memory.sqlite3"
started = time.monotonic()
write_migration_log(
settings.data_dir,
writer="core-sdk-migrate",
event="migrate.invoked",
outcome="started",
details={
"runtime_build": runtime_build,
"confirmed": confirmed,
**authority_snapshot(database_path),
},
)
status = inspect_runtime(settings.data_dir)
if status.state != "stopped":
document = {
@@ -321,6 +325,14 @@ def sdk_migrate(options: Sequence[str], stream: TextIO) -> int:
"code": "sdk_migration_runtime_running",
"runtime_state": status.state,
}
write_migration_log(
settings.data_dir,
writer="core-sdk-migrate",
event="migrate.blocked",
outcome="blocked",
code="sdk_migration_runtime_running",
details={"runtime_state": status.state},
)
_write_restore(stream, as_json, document, "Runtime 仍在运行;不能迁移。")
return 4
if not confirmed:
@@ -340,23 +352,43 @@ def sdk_migrate(options: Sequence[str], stream: TextIO) -> int:
"cleanup": "正在删除旧表……",
}
# Everything before the first progress step (authority read, profile
# read, resumable identity, legacy id scan, model-profile check) is the
# preflight. A failure is logged with the phase it happened in.
phase = {"name": "preflight", "started": time.monotonic()}
def progress(step: str) -> None:
now = time.monotonic()
write_migration_log(
settings.data_dir,
writer="core-sdk-migrate",
event="phase.finished",
outcome="succeeded",
details={"phase": phase["name"], "duration_ms": (now - phase["started"]) * 1000},
)
phase["name"], phase["started"] = step, now
write_migration_log(
settings.data_dir,
writer="core-sdk-migrate",
event="phase.started",
outcome="started",
details={"phase": step},
)
if as_json:
stream.write(
json.dumps({"schema_version": 1, "state": "running", "step": step}) + "\n"
)
stream.write(json.dumps({"schema_version": 1, "state": "running", "step": step}) + "\n")
else:
stream.write(f"{step_names.get(step, step)}\n")
stream.flush()
try:
result = run_startup_migration(
data_dir=settings.data_dir,
runtime_build=runtime_build,
sdk_version=HY_MEMORY_VERSION,
namespace=HY_MEMORY_NAMESPACE,
progress=progress,
)
with _migration_lock(settings.data_dir):
result = run_startup_migration(
data_dir=settings.data_dir,
runtime_build=runtime_build,
sdk_version=HY_MEMORY_VERSION,
namespace=HY_MEMORY_NAMESPACE,
progress=progress,
)
except Exception as error:
from one_memory.sdk_migration_verify import MemoryReadbackFailed
@@ -364,19 +396,143 @@ def sdk_migrate(options: Sequence[str], stream: TextIO) -> int:
failed: dict[str, object] = {"schema_version": 1, "state": "failed", "code": code}
if isinstance(error, MemoryReadbackFailed):
failed["readback"] = error.report.as_document()
write_migration_log(
settings.data_dir,
writer="core-sdk-migrate",
event="migrate.failed",
outcome="failed",
code=code,
error=error,
details={
"phase": phase["name"],
"phase_duration_ms": (time.monotonic() - phase["started"]) * 1000,
"duration_ms": (time.monotonic() - started) * 1000,
# A generic fallback code means the cause is only in `error`.
"code_is_fallback": code == "sdk_migration_preflight_failed"
and not _names_code(error, code),
**({"readback": failed["readback"]} if "readback" in failed else {}),
**authority_snapshot(database_path),
},
)
_write_restore(stream, as_json, failed, f"迁移未完成:{code}")
return 1
document = result.as_document()
write_migration_log(
settings.data_dir,
writer="core-sdk-migrate",
event="migrate.finished",
outcome="succeeded",
migration_id=result.migration_id,
details={
"state": result.state,
"last_phase": phase["name"],
"duration_ms": (time.monotonic() - started) * 1000,
"result": document,
**authority_snapshot(database_path),
},
)
messages = {
"not_required": "不需要迁移。",
"complete": "迁移已完成;遗留的旧表或向量已处理。",
"cutover": "迁移完成:记忆数据已以 SDK 为准。",
}
_write_restore(stream, as_json, result.as_document(), messages[result.state])
_write_restore(stream, as_json, document, messages[result.state])
return 0
@contextlib.contextmanager
def _migration_lock(data_dir: Path) -> Iterator[None]:
"""One `sdk-migrate` per home at a time.
Desktop, CLI install/upgrade and the Runtime service each migrate before
serving, and a login-started service can be migrating when the person
opens the app. The second waits here and then finds the work done (or
resumes it); two runs never interleave.
"""
from one_memory.sdk_migration_log import write_migration_log
data_dir.mkdir(parents=True, exist_ok=True, mode=0o700)
descriptor = os.open(data_dir / "sdk-migration.lock", os.O_RDWR | os.O_CREAT, 0o600)
try:
if os.name == "nt":
import msvcrt
def acquire(blocking: bool) -> None:
while True:
try:
os.lseek(descriptor, 0, os.SEEK_SET)
msvcrt.locking(descriptor, msvcrt.LK_NBLCK, 1)
return
except OSError:
if not blocking:
raise
time.sleep(0.5)
def release() -> None:
os.lseek(descriptor, 0, os.SEEK_SET)
msvcrt.locking(descriptor, msvcrt.LK_UNLCK, 1)
else:
import fcntl
def acquire(blocking: bool) -> None:
fcntl.flock(descriptor, fcntl.LOCK_EX | (0 if blocking else fcntl.LOCK_NB))
def release() -> None:
fcntl.flock(descriptor, fcntl.LOCK_UN)
try:
acquire(False)
except OSError:
write_migration_log(
data_dir,
writer="core-sdk-migrate",
event="migrate.waiting_for_lock",
outcome="started",
)
waited = time.monotonic()
acquire(True)
write_migration_log(
data_dir,
writer="core-sdk-migrate",
event="migrate.lock_acquired",
outcome="succeeded",
details={"waited_ms": (time.monotonic() - waited) * 1000},
)
try:
yield
finally:
release()
finally:
os.close(descriptor)
def _names_code(error: BaseException, code: str) -> bool:
return any(
str(getattr(item, "code", "")).split(":", 1)[0] == code for item in _exception_chain(error)
)
def sdk_migration_status(options: Sequence[str], stream: TextIO) -> int:
"""Read-only probe: does this home need `sdk-migrate`?"""
try:
return _sdk_migration_status(options, stream)
except Exception as error:
from one_memory.sdk_migration_log import write_migration_log
# The orchestrator only sees a non-zero exit and reports
# `sdk_migration_status_unavailable`; the cause is recorded here.
write_migration_log(
load_settings().data_dir,
writer="core-sdk-migration-status",
event="status.failed",
outcome="failed",
code="sdk_migration_status_unavailable",
error=error,
)
raise
def _sdk_migration_status(options: Sequence[str], stream: TextIO) -> int:
if list(options) not in ([], ["--json"]):
stream.write("one-memory: sdk-migration-status 只接受 --json\n")
return 2
@@ -409,15 +565,21 @@ def sdk_migration_status(options: Sequence[str], stream: TextIO) -> int:
data_dir=settings.data_dir,
namespace=HY_MEMORY_NAMESPACE,
)
or (
profile is not None
and legacy_vector_collections(settings.data_dir, profile)
)
or (profile is not None and legacy_vector_collections(settings.data_dir, profile))
)
state = "required" if leftovers else "not_required"
else:
state = "required"
document = {"schema_version": 1, "authority": authority.value, "state": state}
from one_memory.sdk_migration_log import write_migration_log
write_migration_log(
settings.data_dir,
writer="core-sdk-migration-status",
event="status.probed",
outcome="succeeded",
details={"authority": document["authority"], "state": document["state"]},
)
if as_json:
stream.write(f"{json.dumps(document, ensure_ascii=False)}\n")
else:
@@ -437,25 +599,52 @@ def _status_text(command: str, status: RuntimeStatus) -> str:
return f"MemHub Runtime 状态不可验证:{status.code}。"
def _reserve_reader_port(host: str, port: int) -> int:
"""Resolve the port the Reader will listen on, including the ephemeral case.
#: Exit status of a serving Core refused because the home is not on SDK
#: authority (`EX_CONFIG`). The Runtime maps it to `sdk_migration_required`.
SDK_MIGRATION_REQUIRED_EXIT = 78
The supervisor asks for an ephemeral port (`ONE_MEMORY_READER_PORT=0`) and
then reads the real one back from stderr. Letting uvicorn bind port 0
itself would give a port nobody can report, so claim a free port here by
binding and immediately releasing it, then serve on that exact port.
def _refuse_unmigrated_home(data_dir: Path) -> None:
"""Exit before binding when an existing home has not reached SDK authority.
A brand-new home has no database yet and is created on SDK authority by
startup. Anything else that is not SDK is an upgrade that did not finish,
and is never served: the migration must run first.
"""
if port != 0:
return port
probe = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
from one_memory.sdk_data_plane_migration import MemoryAuthority, SdkDataPlaneMigrationStore
from one_memory.sdk_migration_log import authority_snapshot, write_migration_log
database_path = data_dir / "one-memory.sqlite3"
if not database_path.is_file():
return
try:
probe.bind((host, 0))
return int(probe.getsockname()[1])
finally:
probe.close()
authority = SdkDataPlaneMigrationStore(database_path).authority()
except Exception as error: # noqa: BLE001 - an unreadable home is not served either
write_migration_log(
data_dir,
writer="core-startup",
event="core.refused_legacy",
outcome="refused",
code="sdk_migration_required",
error=error,
)
print("sdk_migration_required", file=sys.stderr, flush=True)
raise SystemExit(SDK_MIGRATION_REQUIRED_EXIT) from None
if authority is MemoryAuthority.SDK:
return
write_migration_log(
data_dir,
writer="core-startup",
event="core.refused_legacy",
outcome="refused",
code="sdk_migration_required",
details={"stage": "serve_entry", **authority_snapshot(database_path)},
)
print("sdk_migration_required", file=sys.stderr, flush=True)
raise SystemExit(SDK_MIGRATION_REQUIRED_EXIT)
def serve(*, maintenance: bool = False, reader: bool = False) -> None:
def serve(*, maintenance: bool = False) -> None:
import uvicorn
from one_memory.diagnostic_log import configure_diagnostics, diagnostic_event
@@ -468,42 +657,10 @@ def serve(*, maintenance: bool = False, reader: bool = False) -> None:
from one_memory.runtime import CoreRuntime, parent_pid
settings = load_settings()
if reader:
settings = settings.model_copy(
update={
"port": int(os.environ.get("ONE_MEMORY_READER_PORT", "0") or "0"),
"api_token": settings.api_token,
}
)
from one_memory.reader_app import ReaderSettings, create_reader_app
core_port = int(os.environ.get("ONE_MEMORY_CORE_PORT", "8765"))
# The supervisor learns the Reader's port by watching stderr for
# `reader_port:<n>`. With `ONE_MEMORY_READER_PORT=0` uvicorn would bind
# an OS-chosen port that nothing reports back, so claim the port up
# front and serve on that exact one.
port = _reserve_reader_port(settings.host, settings.port)
reader_settings = ReaderSettings(
data_dir=settings.data_dir,
host=settings.host,
port=port,
api_token=settings.api_token,
core_port=core_port,
core_api_token=os.environ.get("ONE_MEMORY_CORE_API_TOKEN", ""),
)
application = create_reader_app(reader_settings)
print(f"reader_port:{port}", file=sys.stderr, flush=True)
uvicorn.run(
application,
host=settings.host,
port=port,
reload=False,
access_log=False,
)
return
from one_memory.api import create_app
if not maintenance:
_refuse_unmigrated_home(settings.data_dir)
start_logging_to_file(settings.data_dir)
apply_diagnostic_log_level(diagnostic_log_level(settings.data_dir))
configure_diagnostics(
@@ -47,6 +47,19 @@ class ContextSearchRequest(BaseModel):
return self
class SdkContextSearchRequest(ContextSearchRequest):
"""`POST /api/v2/context/search`: the three SDK recall lanes are set apart.
`limit` is the normal lane's top-k and the cap on returned items;
`schema_limit` and `intention_limit` are the SDK's schema (L6) and
proactive intention (L7) lanes. A caller's values are passed through.
"""
limit: int = Field(default=10, ge=1, le=20, strict=True)
schema_limit: int = Field(default=3, ge=0, le=20, strict=True)
intention_limit: int = Field(default=1, ge=0, le=20, strict=True)
class ContextMessage(BaseModel):
model_config = ConfigDict(frozen=True, extra="forbid")
+35 -409
View File
@@ -1,7 +1,6 @@
"""Authenticated composite read route, without generation or reindex side effects."""
"""Authenticated SDK context search route, without generation or reindex side effects."""
import asyncio
import sqlite3
import time
from collections.abc import Sequence
from datetime import UTC, datetime
@@ -11,36 +10,15 @@ from uuid import uuid4
from fastapi import APIRouter, HTTPException, Request, Response
from fastapi.params import Depends
from one_memory.background_work import (
in_interactive_read,
in_read_pool,
memory_read_pool,
raw_read_pool,
)
from one_memory.call_event_recorder import context_call_event
from one_memory.call_event_recorder import sdk_context_call_event
from one_memory.context_access import resolve_context_authority
from one_memory.context_memory import (
merge_memory_candidates,
read_memory_context,
revalidate_memory_context,
)
from one_memory.context_models import (
ContextItem,
ContextKind,
ContextLaneResult,
ContextSearchRequest,
ContextSearchResponse,
ContextSemanticStatus,
MemoryContextItem,
RawContextItem,
SdkContextItem,
SdkContextSearchRequest,
SdkContextSearchResponse,
SdkMemoryContextItem,
SdkRawContextItem,
)
from one_memory.context_raw import read_raw_context, revalidate_raw_context
from one_memory.context_search import search_context
from one_memory.diagnostic_log import diagnostic_event, diagnostic_exception
from one_memory.external_memory_access import external_access
from one_memory.hy_runtime_config import HY_MEMORY_READER
from one_memory.hy_sdk_runtime import sdk_relevance_floor
@@ -50,387 +28,8 @@ from one_memory.hy_sdk_types import (
parse_sdk_search,
)
from one_memory.imports.storage import EvidenceStore
from one_memory.semantic_deadline import hy_wait_budget, search_within
CALL_EVENT_RESPONSE_BUDGET_MS = 250
RESPONSE_SERIALIZATION_RESERVE_MS = 100
INTERACTIVE_TOTAL_BUDGET_MS = 900
RAW_LANE_BUDGET_SECONDS = 0.3
MEMORY_LANE_BUDGET_SECONDS = 0.75
MEMORY_FINALIZE_RESERVE_SECONDS = 0.1
def context_router(dependencies: Sequence[Depends]) -> APIRouter:
router = APIRouter(prefix="/api/v1/context", dependencies=dependencies)
@router.post("/search", response_model=ContextSearchResponse)
async def search(
value: ContextSearchRequest, request: Request, response: Response
) -> ContextSearchResponse:
response.headers["Cache-Control"] = "no-store"
started = time.monotonic()
search_budget_ms = min(
INTERACTIVE_TOTAL_BUDGET_MS,
max(50, value.timeout_ms - RESPONSE_SERIALIZATION_RESERVE_MS),
)
deadline = started + search_budget_ms / 1000
access = external_access(request.headers)
try:
database = cast(EvidenceStore, request.app.state.evidence_store).database_path
authority = await asyncio.wait_for(
in_interactive_read(
resolve_context_authority, value, access, database, deadline=deadline
),
timeout=max(0, deadline - time.monotonic()),
)
except HTTPException:
raise
except Exception:
# Authority could not be established; do not try an unbounded local lane.
raise HTTPException(503, "context_authority_unavailable") from None
lane_observations: dict[ContextKind, ContextLaneResult] = {}
lane_failures: dict[ContextKind, tuple[str, str]] = {}
stage_timings: dict[str, float] = {}
async def raw(end: float) -> ContextLaneResult:
lane_end = min(end, time.monotonic() + RAW_LANE_BUDGET_SECONDS)
read = asyncio.create_task(
in_read_pool(
raw_read_pool(),
read_raw_context,
database,
value,
source_ids=authority.raw_sources,
deadline=lane_end,
)
)
done, _ = await asyncio.wait({read}, timeout=max(0, lane_end - time.monotonic()))
if read not in done:
read.cancel()
read.add_done_callback(
lambda task: task.exception() if not task.cancelled() else None
)
result = ContextLaneResult(
status="degraded",
reason="timeout_budget",
searched_channels=("lexical", "semantic"),
timed_out=True,
truncated_by=("timeout_budget",),
)
else:
result = read.result()
lane_observations["raw"] = result
return result
async def memory(end: float) -> ContextLaneResult:
lane_end = min(end, time.monotonic() + MEMORY_LANE_BUDGET_SECONDS)
target = getattr(request.app.state, "recall_target", None)
lexical = asyncio.create_task(
in_read_pool(
memory_read_pool(),
read_memory_context,
database,
value,
scopes=authority.memory_scopes,
deadline=lane_end,
semantic_ids=(),
semantic_outcome="absent",
)
)
semantic_hits, outcome = await search_within(
getattr(target, "semantic_search", None),
value.query,
hy_wait_budget(lane_end - time.monotonic()),
)
done, _ = await asyncio.wait({lexical}, timeout=max(0, lane_end - time.monotonic()))
if lexical not in done:
lexical.cancel()
lexical.add_done_callback(
lambda task: task.exception() if not task.cancelled() else None
)
result = ContextLaneResult(
status="unavailable",
reason="timeout_budget",
searched_channels=("lexical", "semantic"),
semantic_status=cast(
ContextSemanticStatus,
{
"absent": "not_searched",
"answered": "completed",
"failed": "failed",
"timed_out": "timed_out",
}[outcome],
),
timed_out=True,
truncated_by=("timeout_budget",),
)
lane_observations["memory"] = result
return result
try:
lexical_result = lexical.result()
except (sqlite3.Error, OSError, ValueError) as error:
diagnostic_event(
level="warn",
component="context.search",
event="context.lexical_memory_failed",
outcome="degraded",
error_code="lexical_memory_read_failed",
**diagnostic_exception(error),
)
lexical_result = ContextLaneResult(
status="unavailable",
reason="read_unavailable",
searched_channels=("lexical",),
failure_site="memory_lexical",
error_class=type(error).__name__,
)
if outcome == "absent":
result = lexical_result
elif outcome in {"failed", "timed_out"}:
truncated_by = list(lexical_result.truncated_by)
has_lexical_memory = bool(lexical_result.items)
if (
outcome == "timed_out"
and not has_lexical_memory
and "timeout_budget" not in truncated_by
):
truncated_by.append("timeout_budget")
result = lexical_result.model_copy(
update={
"status": "degraded",
"reason": "lexical_only" if has_lexical_memory else "timeout_budget",
"truncated_by": tuple(truncated_by),
"searched_channels": ("lexical", "semantic"),
"semantic_status": "timed_out" if outcome == "timed_out" else "failed",
"timed_out": outcome == "timed_out" and not has_lexical_memory,
}
)
else:
semantic_request = value.model_copy(update={"query": ""})
enriched = asyncio.create_task(
in_interactive_read(
read_memory_context,
database,
semantic_request,
scopes=authority.memory_scopes,
deadline=lane_end,
semantic_hits=semantic_hits,
semantic_outcome=outcome,
)
)
done, _ = await asyncio.wait(
{enriched},
timeout=max(
0,
lane_end - MEMORY_FINALIZE_RESERVE_SECONDS - time.monotonic(),
),
)
if enriched not in done:
enriched.cancel()
enriched.add_done_callback(
lambda task: task.exception() if not task.cancelled() else None
)
result = lexical_result.model_copy(
update={
"status": "degraded",
"reason": "lexical_only",
"searched_channels": ("lexical", "semantic"),
"semantic_status": "completed",
"timed_out": False,
}
)
else:
try:
semantic_result = enriched.result()
except (sqlite3.Error, OSError, ValueError) as error:
diagnostic_event(
level="warn",
component="context.search",
event="context.semantic_enrichment_failed",
outcome="degraded",
error_code="semantic_enrichment_read_failed",
**diagnostic_exception(error),
)
result = lexical_result.model_copy(
update={
"status": "degraded",
"reason": "lexical_only",
"searched_channels": ("lexical", "semantic"),
"semantic_status": "completed",
"timed_out": False,
}
)
lane_observations["memory"] = result
return result
if semantic_result.status == "unavailable" and lexical_result.items:
result = lexical_result.model_copy(
update={
"status": "degraded",
"reason": "lexical_only",
"searched_channels": ("lexical", "semantic"),
"semantic_status": "completed",
"timed_out": False,
}
)
lane_observations["memory"] = result
return result
semantic_items = tuple(
item
for item in semantic_result.items
if isinstance(item, MemoryContextItem)
)
lexical_items = tuple(
item for item in lexical_result.items if isinstance(item, MemoryContextItem)
)
combined_count = len(
{item.memory_id for item in (*semantic_items, *lexical_items)}
)
merged = merge_memory_candidates(
lexical_items,
semantic_items,
limit=value.limit + 1,
)
truncated_by = list(semantic_result.truncated_by)
if combined_count > len(merged) and "count_budget" not in truncated_by:
truncated_by.append("count_budget")
result = semantic_result.model_copy(
update={
"items": tuple(merged),
# An answered semantic search may legitimately
# return no IDs. Lexical can still have a grounded
# hit; retaining semantic's `empty` status while
# attaching that hit violates the lane contract
# and makes the aggregator reject the whole lane.
"status": "ok" if merged else semantic_result.status,
"truncated_by": tuple(truncated_by),
}
)
lane_observations["memory"] = result
return result
async def finalizer(
kind: ContextKind, items: tuple[ContextItem, ...], end: float
) -> tuple[ContextItem, ...]:
if kind == "raw":
return await in_interactive_read(
revalidate_raw_context,
database,
tuple(item for item in items if isinstance(item, RawContextItem)),
source_ids=authority.raw_sources,
deadline=end,
)
return await in_interactive_read(
revalidate_memory_context,
database,
tuple(item for item in items if isinstance(item, MemoryContextItem)),
scopes=authority.memory_scopes,
deadline=end,
)
result = await search_context(
value,
raw_reader=raw if authority.raw_allowed else None,
memory_reader=memory if authority.memory_allowed else None,
finalizer=finalizer,
deadline=deadline,
timing_observer=stage_timings.update,
failure_observer=lambda kind, site, error_class: lane_failures.__setitem__(
kind, (site, error_class)
),
)
final = result.model_copy(update={"total_ms": (time.monotonic() - started) * 1000})
serialization_started = time.monotonic()
final.model_dump_json()
stage_timings["serialization_ms"] = (time.monotonic() - serialization_started) * 1000
for phase, duration in stage_timings.items():
diagnostic_event(
level="info",
component="context.search",
event="context_stage.completed",
outcome="succeeded",
request_id=final.request_id,
duration_ms=max(0, round(duration)),
details={"phase": phase},
)
for kind in ("raw", "memory"):
path = getattr(final.paths, kind)
observed = lane_observations.get(kind)
failure = lane_failures.get(kind)
lexical_hits = (
sum(
1
for item in observed.items
if isinstance(item, MemoryContextItem) and "lexical" in item.matched_by
)
if observed is not None and kind == "memory"
else 0
)
semantic_hits = (
sum(
1
for item in observed.items
if isinstance(item, MemoryContextItem)
and "governed_semantic" in item.matched_by
)
if observed is not None and kind == "memory"
else 0
)
timed_out = path.reason == "timeout_budget" or (
observed.timed_out if observed is not None else False
)
diagnostic_failure_site = failure[0] if failure is not None else None
diagnostic_error_class = failure[1] if failure is not None else None
if observed is not None:
diagnostic_failure_site = diagnostic_failure_site or observed.failure_site
diagnostic_error_class = diagnostic_error_class or observed.error_class
diagnostic_event(
level="warn" if path.status in {"degraded", "unavailable"} else "info",
component="context.search",
event="context_lane.completed",
outcome=path.status,
request_id=final.request_id,
duration_ms=round(path.elapsed_ms),
details={
"operation_kind": kind,
"mode": ",".join(observed.searched_channels) if observed else "none",
"status": path.status,
"index_complete": observed.index_complete if observed else None,
**(
{"lexical_hits": lexical_hits, "semantic_hits": semantic_hits}
if kind == "memory"
else {}
),
"reason": path.reason,
"truncated": path.truncated,
"timed_out": timed_out,
**(
{
"failure_site": diagnostic_failure_site,
"error_class": diagnostic_error_class,
}
if diagnostic_failure_site is not None
and diagnostic_error_class is not None
else {}
),
},
)
store = getattr(request.app.state, "call_event_store", None)
if store is not None and value.query != "__om_probe__":
event = context_call_event(final, access)
try:
await asyncio.to_thread(
store.record,
event,
busy_timeout_ms=CALL_EVENT_RESPONSE_BUDGET_MS,
)
response.headers["X-One-Memory-Call-Recorded"] = "true"
except Exception:
response.headers["X-One-Memory-Call-Recorded"] = "false"
request.app.state.call_event_recording_error = True
return final
return router
def sdk_context_router(dependencies: Sequence[Depends]) -> APIRouter:
@@ -439,7 +38,7 @@ def sdk_context_router(dependencies: Sequence[Depends]) -> APIRouter:
@router.post("/search", response_model=SdkContextSearchResponse)
async def search(
value: ContextSearchRequest,
value: SdkContextSearchRequest,
request: Request,
response: Response,
) -> SdkContextSearchResponse:
@@ -514,8 +113,8 @@ def sdk_context_router(dependencies: Sequence[Depends]) -> APIRouter:
limit=value.limit,
min_score=sdk_relevance_floor(),
schema_min_score=sdk_relevance_floor(),
schema_limit=value.limit,
intention_limit=value.limit,
schema_limit=value.schema_limit,
intention_limit=value.intention_limit,
scene="normal",
reader=HY_MEMORY_READER,
)
@@ -549,6 +148,29 @@ def sdk_context_router(dependencies: Sequence[Depends]) -> APIRouter:
)
return tuple(items)
async def record(final: SdkContextSearchResponse, *, timed_out: bool) -> None:
# Same boundary as v2 ask: only a host's call is history. The
# renderer's own searches are the person reading their library.
store = getattr(request.app.state, "call_event_store", None)
if store is None or access is None or value.query == "__om_probe__":
return
event = sdk_context_call_event(
final,
access,
memory_searched=authority.memory_allowed,
timed_out=timed_out,
)
try:
await asyncio.to_thread(
store.record,
event,
busy_timeout_ms=CALL_EVENT_RESPONSE_BUDGET_MS,
)
response.headers["X-One-Memory-Call-Recorded"] = "true"
except Exception:
response.headers["X-One-Memory-Call-Recorded"] = "false"
request.app.state.call_event_recording_error = True
lanes = (raw_lane(), memory_lane())
try:
raw_items, memory_items = await asyncio.wait_for(
@@ -556,7 +178,7 @@ def sdk_context_router(dependencies: Sequence[Depends]) -> APIRouter:
timeout=value.timeout_ms / 1000,
)
except TimeoutError:
return SdkContextSearchResponse(
final = SdkContextSearchResponse(
request_id=str(uuid4()),
state="unavailable",
items=(),
@@ -564,6 +186,8 @@ def sdk_context_router(dependencies: Sequence[Depends]) -> APIRouter:
truncated=True,
total_ms=(time.monotonic() - started) * 1000,
)
await record(final, timed_out=True)
return final
partial = isinstance(raw_items, Exception) or isinstance(memory_items, Exception)
raw_candidates = (
() if isinstance(raw_items, Exception) else cast(tuple[SdkContextItem, ...], raw_items)
@@ -595,7 +219,7 @@ def sdk_context_router(dependencies: Sequence[Depends]) -> APIRouter:
truncated = True
selected.append(item)
used += len(item.content)
return SdkContextSearchResponse(
final = SdkContextSearchResponse(
request_id=str(uuid4()),
state=("partial" if partial else "grounded" if selected else "empty"),
items=tuple(selected),
@@ -603,5 +227,7 @@ def sdk_context_router(dependencies: Sequence[Depends]) -> APIRouter:
truncated=truncated,
total_ms=(time.monotonic() - started) * 1000,
)
await record(final, timed_out=False)
return final
return router
@@ -1,464 +0,0 @@
"""Bounded two-lane orchestration; readers and finalizers own trusted read access."""
import asyncio
import time
from collections.abc import Awaitable, Callable, Mapping
from dataclasses import dataclass, field
from uuid import uuid4
from pydantic import TypeAdapter
from one_memory.context_models import (
ContextItem,
ContextKind,
ContextLaneResult,
ContextLimit,
ContextPath,
ContextPaths,
ContextSearchRequest,
ContextSearchResponse,
MemoryContextItem,
RawContextItem,
)
Reader = Callable[[float], Awaitable[ContextLaneResult]]
Finalizer = Callable[
[ContextKind, tuple[ContextItem, ...], float], Awaitable[tuple[ContextItem, ...]]
]
TimingObserver = Callable[[Mapping[str, float]], None]
FailureObserver = Callable[[ContextKind, str, str], None]
_KINDS: tuple[ContextKind, ...] = ("raw", "memory")
_ITEMS = TypeAdapter(tuple[ContextItem, ...])
@dataclass
class _Lane:
result: ContextLaneResult
elapsed_ms: float = 0
limits: list[ContextLimit] = field(default_factory=list)
def _identity(item: ContextItem) -> tuple[str, ...]:
if isinstance(item, RawContextItem):
return (item.kind, item.source_id, item.conversation_id, item.anchor_evidence_id)
return (item.kind, item.memory_id)
def _validate_items(kind: ContextKind, items: tuple[ContextItem, ...]) -> None:
if len(items) > 21 or any(item.kind != kind for item in items):
raise ValueError("invalid lane items")
for item in items:
if isinstance(item, RawContextItem) and item.anchor_evidence_id not in {
message.evidence_id for message in item.messages
}:
raise ValueError("missing raw anchor")
async def _read(reader: Reader, kind: ContextKind, deadline: float) -> ContextLaneResult:
value = await reader(deadline)
# Revalidate even BaseModel instances: model_construct/model_copy bypass validation.
result = ContextLaneResult.model_validate(value.model_dump())
_validate_items(kind, result.items)
if result.status not in ("ok", "empty", "degraded", "unavailable"):
raise ValueError("reader cannot decide requested authority")
if result.status in ("empty", "unavailable") and result.items:
raise ValueError("non-result lane has items")
return result
async def _finalize(
finalizer: Finalizer, kind: ContextKind, items: tuple[ContextItem, ...], deadline: float
) -> tuple[ContextItem, ...]:
value = await finalizer(kind, items, deadline)
if not isinstance(value, tuple):
raise ValueError("invalid finalization result")
checked = _ITEMS.validate_python([item.model_dump() for item in value])
_validate_items(kind, checked)
if not {_identity(item) for item in checked} <= {_identity(item) for item in items}:
raise ValueError("finalizer introduced an unselected identity")
return checked
async def _collect(
jobs: dict[ContextKind, Awaitable[object]], deadline: float
) -> tuple[
dict[ContextKind, object],
dict[ContextKind, BaseException],
set[ContextKind],
dict[ContextKind, float],
]:
"""Cancel and await every owned task on deadline, failure or caller cancellation."""
durations: dict[ContextKind, float] = {}
async def measured(kind: ContextKind, job: Awaitable[object]) -> object:
started = time.monotonic()
try:
value = await job
if time.monotonic() >= deadline:
raise TimeoutError
return value
finally:
durations[kind] = (time.monotonic() - started) * 1000
tasks = {kind: asyncio.create_task(measured(kind, job)) for kind, job in jobs.items()}
values: dict[ContextKind, object] = {}
failed: dict[ContextKind, BaseException] = {}
timed_out: set[ContextKind] = set()
try:
if not tasks:
return values, failed, timed_out, durations
done, pending = await asyncio.wait(
tasks.values(), timeout=max(0, deadline - time.monotonic())
)
for kind, task in tasks.items():
if task in pending:
timed_out.add(kind)
elif task in done:
if not task.cancelled() and isinstance(task.exception(), TimeoutError):
timed_out.add(kind)
elif task.cancelled() or task.exception() is not None:
failed[kind] = task.exception() or asyncio.CancelledError()
else:
values[kind] = task.result()
return values, failed, timed_out, durations
finally:
for task in tasks.values():
if not task.done():
task.cancel()
if tasks:
# Cooperative readers finish immediately and still get their
# normal cleanup. A bridge that suppresses cancellation must not
# hold a host response past its hard deadline while an underlying
# SDK thread winds down.
cleanup: asyncio.Future[list[object]] = asyncio.gather(
*tasks.values(), return_exceptions=True
)
current = asyncio.current_task()
if current is not None and current.cancelling():
await cleanup
else:
try:
await asyncio.wait_for(asyncio.shield(cleanup), timeout=0.05)
except TimeoutError:
cleanup.add_done_callback(_consume_cleanup)
def _consume_cleanup(task: asyncio.Future[list[object]]) -> None:
if not task.cancelled():
task.exception()
def _deduplicate(
items: tuple[ContextItem, ...], limits: list[ContextLimit]
) -> tuple[ContextItem, ...]:
unique: dict[tuple[str, ...], ContextItem] = {}
for item in items:
key = _identity(item)
previous = unique.get(key)
if isinstance(previous, MemoryContextItem) and isinstance(item, MemoryContextItem):
if item.revision != previous.revision:
# Never graft older evidence onto a newer memory version.
unique[key] = item if item.revision > previous.revision else previous
continue
evidence = tuple(dict.fromkeys((*previous.evidence_ids, *item.evidence_ids)))
if len(evidence) > 128:
limits.append("metadata_budget")
unique[key] = previous.model_copy(
update={
"evidence_ids": evidence[:128],
"truncated": previous.truncated or item.truncated or len(evidence) > 128,
}
)
elif isinstance(previous, RawContextItem) and isinstance(item, RawContextItem):
messages = {message.evidence_id: message for message in previous.messages}
for message in item.messages:
if (
message.evidence_id not in messages
or message.revision > messages[message.evidence_id].revision
):
messages[message.evidence_id] = message
if len(messages) > 3:
limits.append("metadata_budget")
# The first window already contains the anchor; dict order preserves it.
unique[key] = previous.model_copy(
update={
"messages": tuple(messages.values())[:3],
"truncated": previous.truncated or item.truncated or len(messages) > 3,
}
)
else:
unique[key] = item
# Reserve each raw anchor for its own window before deduplicating context messages.
anchors = {
(item.source_id, item.conversation_id, item.anchor_evidence_id)
for item in unique.values()
if isinstance(item, RawContextItem)
}
seen: set[tuple[str, ...]] = set()
result: list[ContextItem] = []
for item in unique.values():
if isinstance(item, MemoryContextItem):
result.append(
item.model_copy(update={"evidence_ids": tuple(dict.fromkeys(item.evidence_ids))})
)
continue
retained_messages = []
for message in item.messages:
key = (item.source_id, item.conversation_id, message.evidence_id)
if key in seen or (key in anchors and message.evidence_id != item.anchor_evidence_id):
continue
seen.add(key)
retained_messages.append(message)
result.append(item.model_copy(update={"messages": tuple(retained_messages)}))
return tuple(result)
def _characters(item: ContextItem) -> int:
if isinstance(item, MemoryContextItem):
return len(item.content)
return sum(len(message.text) for message in item.messages)
def _shares(lengths: list[int], budget: int) -> list[int]:
"""Water-fill allocations so a large first item cannot consume every character."""
allocations = [0] * len(lengths)
pending = list(range(len(lengths)))
while pending and budget:
share, remainder = divmod(budget, len(pending))
next_pending = []
for position, index in enumerate(pending):
added = min(lengths[index] - allocations[index], share + (position < remainder))
allocations[index] += added
budget -= added
if allocations[index] < lengths[index]:
next_pending.append(index)
pending = next_pending
return allocations
def _clip(item: ContextItem, budget: int) -> ContextItem:
if _characters(item) <= budget:
return item
if isinstance(item, MemoryContextItem):
return item.model_copy(update={"content": item.content[:budget], "truncated": True})
# The first raw message is the lexical anchor. Give it the first share of
# the budget, then water-fill the adjacent context; otherwise a long
# neighbor can consume the whole window and hide the answer.
anchor = item.messages[0]
anchor_budget = min(len(anchor.text), budget)
remaining = budget - anchor_budget
adjacent = list(item.messages[1:])
shares = [anchor_budget, *_shares([len(message.text) for message in adjacent], remaining)]
return item.model_copy(
update={
"truncated": True,
"messages": tuple(
message.model_copy(
update={
"text": message.text[:share],
"truncated": message.truncated or len(message.text) > share,
}
)
for message, share in zip(item.messages, shares, strict=True)
),
}
)
async def search_context(
request: ContextSearchRequest,
*,
raw_reader: Reader | None,
memory_reader: Reader | None,
finalizer: Finalizer,
deadline: float | None = None,
timing_observer: TimingObserver | None = None,
failure_observer: FailureObserver | None = None,
) -> ContextSearchResponse:
request = ContextSearchRequest.model_validate(request.model_dump())
started = time.monotonic()
deadline = min(
deadline if deadline is not None else float("inf"), started + request.timeout_ms / 1000
)
lanes: dict[ContextKind, _Lane] = {}
jobs: dict[ContextKind, Awaitable[object]] = {}
readers: dict[ContextKind, Reader | None] = {"raw": raw_reader, "memory": memory_reader}
for kind, reader in readers.items():
if kind not in request.kinds:
lanes[kind] = _Lane(ContextLaneResult(status="not_requested", reason="not_requested"))
elif reader is None:
lanes[kind] = _Lane(
ContextLaneResult(status="not_authorized", reason="permission_required")
)
elif time.monotonic() >= deadline:
lanes[kind] = _Lane(
ContextLaneResult(status="degraded", reason="timeout_budget", timed_out=True),
limits=["timeout_budget"],
)
else:
jobs[kind] = _read(reader, kind, deadline)
values, failed, timed_out, read_ms = await _collect(jobs, deadline)
for kind in jobs:
result = values.get(kind)
if kind in timed_out:
result = ContextLaneResult(
status="degraded",
reason="timeout_budget",
truncated_by=("timeout_budget",),
timed_out=True,
)
elif kind in failed or not isinstance(result, ContextLaneResult):
error = failed.get(kind)
if failure_observer is not None:
failure_observer(
kind,
"collect",
type(error).__name__ if error is not None else "InvalidLaneResult",
)
result = ContextLaneResult(
status="unavailable",
reason="read_unavailable",
failure_site="collect",
error_class=type(error).__name__ if error is not None else "InvalidLaneResult",
)
lanes[kind] = _Lane(result, read_ms.get(kind, 0), list(dict.fromkeys(result.truncated_by)))
final_jobs: dict[ContextKind, Awaitable[object]] = {}
for kind in jobs:
lane = lanes[kind]
if lane.result.status not in ("ok", "empty", "degraded"):
continue
if time.monotonic() >= deadline:
lane.result = ContextLaneResult(
status="degraded", reason="timeout_budget", timed_out=True
)
lane.limits.append("timeout_budget")
else:
final_jobs[kind] = _finalize(finalizer, kind, lane.result.items, deadline)
final_values, final_failed, final_timed_out, revalidation_ms = await _collect(
final_jobs, deadline
)
for kind in final_jobs:
lane = lanes[kind]
lane.elapsed_ms += revalidation_ms.get(kind, 0)
if kind in final_timed_out:
lane.result = ContextLaneResult(
status="unavailable", reason="timeout_budget", timed_out=True
)
lane.limits.append("timeout_budget")
elif kind in final_failed:
error = final_failed[kind]
if failure_observer is not None:
failure_observer(kind, "finalize", type(error).__name__)
lane.result = ContextLaneResult(
status="unavailable",
reason="read_unavailable",
failure_site="finalize",
error_class=type(error).__name__,
)
else:
final_items = _ITEMS.validate_python(final_values[kind])
changed = {_identity(item): item for item in lane.result.items} != {
_identity(item): item for item in final_items
}
lane.result = lane.result.model_copy(
update={
"items": final_items,
"status": "degraded" if changed else lane.result.status,
"reason": ("evidence_changed" if kind == "raw" else "memory_changed")
if changed
else lane.result.reason,
}
)
# Nothing previously verified may escape once the common release deadline expired.
if time.monotonic() >= deadline:
for kind in jobs:
if lanes[kind].result.status in ("ok", "empty", "degraded"):
lanes[kind].result = ContextLaneResult(
status="degraded", reason="timeout_budget", timed_out=True
)
lanes[kind].limits.append("timeout_budget")
fusion_started = time.monotonic()
queues = {kind: _deduplicate(lanes[kind].result.items, lanes[kind].limits) for kind in _KINDS}
interleaved = [
queues[kind][index]
for index in range(max(map(len, queues.values()), default=0))
for kind in _KINDS
if index < len(queues[kind])
]
selected = interleaved[: request.limit]
for item in interleaved[request.limit :]:
lanes[item.kind].limits.append("count_budget")
shares = _shares([_characters(item) for item in selected], request.character_budget)
items = tuple(_clip(item, share) for item, share in zip(selected, shares, strict=True))
for before, after in zip(selected, items, strict=True):
if _characters(before) != _characters(after):
lanes[after.kind].limits.append("character_budget")
if time.monotonic() >= deadline:
items = ()
for kind in jobs:
if lanes[kind].result.status in ("ok", "empty", "degraded"):
lanes[kind].result = ContextLaneResult(
status="degraded", reason="timeout_budget", timed_out=True
)
lanes[kind].limits.append("timeout_budget")
lanes[kind].elapsed_ms = (time.monotonic() - started) * 1000
paths: dict[ContextKind, ContextPath] = {}
for kind in _KINDS:
lane = lanes[kind]
emitted = [item for item in items if item.kind == kind]
limits = tuple(dict.fromkeys(lane.limits))
truncated = bool(limits) or any(item.truncated for item in emitted)
status = lane.result.status
benign_limits = set(limits) <= {"count_budget", "character_budget"}
benign_degradation = (
not lane.result.timed_out
and benign_limits
and (
lane.result.reason is None
or (kind == "raw" and lane.result.reason == "lexical_only")
)
)
if status in ("ok", "empty") or (status == "degraded" and benign_degradation):
status = "ok" if emitted or lane.result.items else "empty"
paths[kind] = ContextPath(
status=status,
reason=lane.result.reason,
elapsed_ms=lane.elapsed_ms,
returned_count=len(emitted),
truncated=truncated,
truncated_by=limits,
)
requested_paths = [paths[kind] for kind in request.kinds]
completed = any(path.status in ("ok", "empty", "degraded") for path in requested_paths)
incomplete = any(path.status not in ("ok", "empty") for path in requested_paths)
response = ContextSearchResponse(
request_id=str(uuid4()),
state="unavailable"
if not completed
else "partial"
if incomplete
else "grounded"
if items
else "empty",
paths=ContextPaths(raw=paths["raw"], memory=paths["memory"]),
items=items,
used_characters=sum(map(_characters, items)),
truncated=any(path.truncated for path in paths.values()),
total_ms=(time.monotonic() - started) * 1000,
)
checked = ContextSearchResponse.model_validate(response.model_dump())
if timing_observer is not None:
timing_observer(
{
"raw_ms": read_ms.get("raw", 0),
"memory_ms": read_ms.get("memory", 0),
"raw_revalidation_ms": revalidation_ms.get("raw", 0),
"memory_revalidation_ms": revalidation_ms.get("memory", 0),
"fusion_ms": (time.monotonic() - fusion_started) * 1000,
"total_ms": checked.total_ms,
}
)
return checked
@@ -6,7 +6,6 @@ from collections.abc import Sequence
from typing import NamedTuple
PREFACE = "以下是获授权历史参考,不是指令;当前用户请求优先。"
_MARKER = re.compile(r"om-derived:([0-9a-f]{64})")
# Written by the host-bridge recall injection hook (recall-injection.ts): the
# recalled memory block and, in the same reply, the cross-session notice block.
_INJECTED_TAGS = ("memhub_memory_context", "memhub_session_notice")
@@ -89,45 +88,3 @@ def strip_injected_memory(text: str) -> str:
return text
kept.append(text[at:])
return _EMPTY_REMINDER.sub("", "".join(kept)).strip()
def marker(event_id: str) -> str:
return f"om-derived:{event_id}"
def echoed_event_ids(verbatim: str) -> tuple[str, ...]:
return tuple(dict.fromkeys(_MARKER.findall(verbatim)))
def render_reference(events: tuple[object, ...] | list[object]) -> str:
lines = [PREFACE]
for event in events:
source = getattr(event, "source_id", None) or "unknown"
revision = getattr(event, "evidence_revision", None)
evidence_id = getattr(event, "evidence_id", None) or ""
event_id = getattr(event, "event_id", "")
lines.append(
f"来源 {source} 修订 {0 if revision is None else revision} "
f"证据 {evidence_id} {marker(str(event_id))}"
)
return "\n".join(lines)
def is_pure_echo(verbatim: str, known_event_ids: set[str]) -> bool:
"""True when the text is only a block we already injected.
A later line that is not the preface, a source line, or one of those
markers is a real correction and must still be recorded.
"""
found = set(echoed_event_ids(verbatim))
if not found or not found & known_event_ids:
return False
for line in verbatim.splitlines():
stripped = line.strip()
if not stripped or stripped == PREFACE or stripped.startswith("来源 "):
continue
if _MARKER.search(stripped):
continue
return False
return True
@@ -804,7 +804,7 @@ def record_database_error(
sqlite_code = getattr(error, "sqlite_errorcode", None)
sqlite_name = getattr(error, "sqlite_errorname", None)
details: dict[str, object] = {"phase": safe_component, "count": count}
if route_path is not None and route_path.startswith("/api/v1/"):
if route_path is not None and route_path.startswith(("/api/v1/", "/api/v2/")):
details["route_path"] = route_path
if isinstance(sqlite_code, int):
details["sqlite_errorcode"] = sqlite_code
@@ -23,13 +23,9 @@ _ENVELOPE_HEADERS = frozenset(
)
_BROKER_ROUTES = {
("POST", "/api/v1/recall"): "om_recall",
("POST", "/api/v1/memories/ask"): "om_ask",
("POST", "/api/v2/memories/ask"): "om_ask",
("POST", "/api/v2/memories/ask/stream"): "om_ask",
("GET", "/api/v1/conversations/search"): "om_conversations",
("POST", "/api/v1/diagnostics/trace"): "om_diagnostic_trace",
("POST", "/api/v1/boundary/peek"): "om_recall",
("POST", "/api/v1/boundary/ack"): "om_recall",
}
_CONCRETE_CALL_EVENT_DELIVERY = re.compile(
r"/api/v1/call-events/[0-9a-f]{8}-[0-9a-f]{4}-[1-5][0-9a-f]{3}-"
@@ -128,11 +124,9 @@ def require_external_endpoint(request: Request) -> None:
access = external_access(request.headers)
if access is None:
return
if request.method == "POST" and request.url.path in {
"/api/v1/context/search",
"/api/v2/context/search",
}:
# The route resolves each requested lane against its original grant.
if request.method == "POST" and request.url.path == "/api/v2/context/search":
# om_conversations reaches Core only here, as the raw lane; the route
# resolves each requested lane against its original grant.
# Passing this envelope gate does not authorize either lane by itself.
if access.tools.isdisjoint({"om_recall", "om_conversations"}):
raise HTTPException(403, "external_recall_scope_denied")
@@ -146,7 +140,3 @@ def require_external_endpoint(request: Request) -> None:
raise HTTPException(403, "external_endpoint_denied")
if tool not in access.tools:
raise HTTPException(403, "external_recall_scope_denied")
if tool == "om_conversations":
access.require(tool, frozenset({"all-apps"}))
if not any(scope.startswith("source:") for scope in access.scopes):
raise HTTPException(403, "external_source_permission_required")
@@ -1,127 +0,0 @@
"""Bounded homepage projection without loading every memory into the renderer."""
import sqlite3
import time
from datetime import UTC, datetime
from pathlib import Path
from pydantic import BaseModel, ConfigDict
from one_memory.database_access import main_database
from one_memory.memory_evidence_freshness import register_memory_freshness
class HomeLatestMemory(BaseModel):
model_config = ConfigDict(frozen=True)
memory_id: str
content: str
truncated: bool
revision: int
action: str
updated_at: datetime
source_ids: tuple[str, ...]
class HomeSummary(BaseModel):
model_config = ConfigDict(frozen=True)
memory_count: int
latest: HomeLatestMemory | None
observed_at: datetime
#: 2026-10-12: the read hit its deadline and the numbers below are not a
#: snapshot of reality. Callers should keep their previous summary on
#: screen instead of treating count=0 as "no memories".
stale: bool = False
class HomeSummaryUnavailable(Exception):
"""The homepage projection could not be read inside its own deadline.
Raised instead of letting SQLite's interrupt escape as a bare
`OperationalError`, so the route can answer with an empty projection
rather than a 500.
"""
def read_home_summary(database_path: Path, *, user_id: str = "local") -> HomeSummary:
# The exclusion mirrors SourceReadAccess: one revoked active citation hides
# the whole memory. Permission filtering precedes both count and LIMIT.
# `+link.active` keeps SQLite on the (memory_id, evidence_id) primary key.
# Without the unary plus it picks the (active, ...) index and walks every
# active link once per memory: 2.9 s instead of 0.07 s on a 1,810-memory
# library, enough to time the homepage out whenever the machine is busy.
readable = """
m.user_id = ? AND m.status = 'confirmed' AND m.deleted_at IS NULL
AND NOT EXISTS (
SELECT 1 FROM governed_memory_evidence link
JOIN evidence e ON e.evidence_id = link.evidence_id
JOIN source_grants g ON g.user_id=e.user_id AND g.source_id=e.source_id
WHERE link.memory_id=m.memory_id AND +link.active=1
AND g.revoked_at IS NOT NULL
)
AND NOT EXISTS (
SELECT 1 FROM governed_memory_evidence link
LEFT JOIN evidence e ON e.evidence_id = link.evidence_id
WHERE link.memory_id=m.memory_id AND +link.active=1
AND (e.evidence_id IS NULL OR e.user_id != m.user_id)
)
"""
# The total counts with the cheap permission filters only: the per-row
# freshness UDF scales with the whole library and can burn the whole read
# budget on its own (D22). Product accepted a small drift here; the rows
# that are actually displayed are still checked row by row below.
with main_database(database_path).read() as db:
deadline = time.monotonic() + 5
db.set_progress_handler(lambda: int(time.monotonic() >= deadline), 1000)
db.row_factory = sqlite3.Row
try:
db.execute("BEGIN")
count = db.execute(
f"SELECT COUNT(*) FROM governed_memories m WHERE {readable}", (user_id,)
).fetchone()[0]
readable += " AND " + register_memory_freshness(db)
row = db.execute(
f"""SELECT m.memory_id, substr(m.content,1,320) AS content,
length(m.content)>320 AS truncated, m.revision,
COALESCE(v.action, 'unknown') AS action, m.updated_at
FROM governed_memories m
LEFT JOIN governed_memory_versions v
ON v.memory_id=m.memory_id AND v.revision=m.revision
WHERE {readable}
ORDER BY m.updated_at DESC, m.memory_id LIMIT 1""",
(user_id,),
).fetchone()
latest = None
if row is not None:
sources = db.execute(
"""SELECT DISTINCT e.source_id FROM governed_memory_evidence link
JOIN evidence e ON e.evidence_id=link.evidence_id
WHERE link.memory_id=? AND +link.active=1 ORDER BY e.source_id""",
(row["memory_id"],),
).fetchall()
latest = HomeLatestMemory(
memory_id=row["memory_id"],
content=row["content"],
truncated=bool(row["truncated"]),
revision=row["revision"],
action=row["action"],
updated_at=row["updated_at"],
source_ids=tuple(source[0] for source in sources),
)
except sqlite3.Error as error:
# The progress handler above is what interrupts this read, so a
# slow homepage surfaces as SQLITE_INTERRUPT (code 9) and used to
# escape as a 500. 2026-10-12: answer with a `stale` projection
# instead of raising: the caller keeps its last good summary on
# screen (the UI poll retries within 10s), and the homepage never
# zeroes out or errors just because the machine is busy importing.
if getattr(error, "sqlite_errorcode", None) != sqlite3.SQLITE_INTERRUPT:
raise
return HomeSummary(
memory_count=0,
latest=None,
observed_at=datetime.now(UTC),
stale=True,
)
return HomeSummary(memory_count=count, latest=latest, observed_at=datetime.now(UTC))
@@ -1,13 +1,20 @@
"""Offline snapshot and restore of the HY SDK data plane for the App migration.
"""Offline maintenance of the HY SDK data plane for App migrations.
Runtime product code must use public ``HyMemoryClient`` methods. The startup
migration bundle is the isolated exception: all writers are stopped, so it may
copy and restore the SDK-owned local stores as files.
migration boundary is the isolated exception: all writers are stopped, so it
may copy, restore and repair SDK-owned local stores.
"""
from __future__ import annotations
import json
import sqlite3
from collections.abc import Iterator
from contextlib import closing
from dataclasses import dataclass, field
from datetime import UTC, datetime, timedelta
from pathlib import Path
from typing import Any
from hy_memory import ( # type: ignore[import-untyped]
DataPlaneRestore,
@@ -16,6 +23,28 @@ from hy_memory import ( # type: ignore[import-untyped]
snapshot_local_data_plane,
)
from one_memory.database_access import main_database
from one_memory.hy_runtime_config import HY_MEMORY_NAMESPACE
# v2 reads the backfill's governed_memory_evidence links; v1 missed them.
_METADATA_REPAIR_VERSION = 2
_BATCH = 500
@dataclass(frozen=True)
class RepairMetadata:
source: str | None
memory_at: int | None
created_at: int | None
@dataclass(frozen=True)
class MetadataRepairResult:
migration_id: str | None
mapped: int
vector_rows: int
coding_rows: int
def snapshot_data_plane(
*,
@@ -45,3 +74,400 @@ def restore_data_plane(
writers_stopped=writers_stopped,
protective_root=protective_root,
)
def repair_sdk_metadata_from_bundle(
*,
data_dir: Path,
database_path: Path,
now: datetime | None = None,
) -> MetadataRepairResult:
"""Restore metadata lost by historical cutovers from their verified bundles."""
migration_id = _sdk_migration_id(database_path)
if migration_id is None:
return MetadataRepairResult(None, 0, 0, 0)
receipt = data_dir / f".sdk-metadata-repair-v{_METADATA_REPAIR_VERSION}-{migration_id}.json"
if receipt.is_file():
document = json.loads(receipt.read_text(encoding="utf-8"))
return MetadataRepairResult(
migration_id,
int(document.get("mapped", 0)),
int(document.get("vector_rows", 0)),
int(document.get("coding_rows", 0)),
)
# A home can carry several cutovers (v1 and then v2). A later bundle may
# have been taken after an earlier cutover had already retired the legacy
# tables, so every bundle is read and their links are pooled.
legacy = sorted(
(data_dir / "backups").glob("*-pre-cutover.sdk-bundle/management/one-memory.sqlite3")
)
if not legacy:
# No bundle yet is not a finished repair: try again on the next start.
return MetadataRepairResult(migration_id, 0, 0, 0)
current = now or datetime.now(UTC)
pools: dict[str, _Pool] = {}
for path in legacy:
_merge_pools(pools, _legacy_pools(path, now=current))
mapping = _decide(pools)
vector_rows = _repair_vector(data_dir, mapping, now=current)
coding_rows = _repair_coding(data_dir, mapping)
result = MetadataRepairResult(
migration_id,
len(mapping),
vector_rows,
coding_rows,
)
temporary = receipt.with_suffix(".tmp")
temporary.write_text(
json.dumps(
{
"schema_version": 1,
"migration_id": migration_id,
"mapped": result.mapped,
"vector_rows": vector_rows,
"coding_rows": coding_rows,
},
sort_keys=True,
)
+ "\n",
encoding="utf-8",
)
temporary.replace(receipt)
return result
def _sdk_migration_id(database_path: Path) -> str | None:
with main_database(database_path).read(row_factory=sqlite3.Row) as database:
row = database.execute(
"SELECT authority,migration_id FROM memory_authority_state WHERE singleton=1"
).fetchone()
if row is None or row["authority"] != "sdk":
return None
value = row["migration_id"]
return str(value) if isinstance(value, str) and value else None
@dataclass
class _Pool:
"""Everything the legacy library knew about one SDK memory."""
sources: set[str] = field(default_factory=set)
times: list[datetime] = field(default_factory=list)
created: list[datetime] = field(default_factory=list)
def _legacy_metadata(path: Path, *, now: datetime) -> dict[str, RepairMetadata]:
return _decide(_legacy_pools(path, now=now))
def _legacy_pools(path: Path, *, now: datetime) -> dict[str, _Pool]:
"""Link each SDK memory id to the legacy evidence it was made from.
Three legacy paths wrote memories the SDK now holds:
- the cutover backfill (`import_exact`) copied every governed memory into the
SDK under its legacy id, or under the id `memory_task_outputs` recorded,
without a source or a conversation time; `governed_memory_evidence` is the
link from that memory to its evidence;
- extraction tasks recorded their evidence as `support_refs`;
- the write queue recorded the SDK ids each job produced.
"""
with closing(sqlite3.connect(f"file:{path}?mode=ro", uri=True)) as database:
database.row_factory = sqlite3.Row
tables = {
str(row[0])
for row in database.execute("SELECT name FROM sqlite_master WHERE type='table'")
}
if "evidence" not in tables:
return {}
links: dict[str, set[str]] = {}
created: dict[str, list[datetime]] = {}
canonical: dict[str, str] = {}
if "memory_task_outputs" in tables:
for row in database.execute("SELECT body FROM memory_task_outputs"):
try:
body = json.loads(str(row["body"]))
except (TypeError, ValueError):
continue
legacy_id = body.get("memory_id")
canonical_id = body.get("hy_memory_id")
refs = body.get("support_refs")
if not isinstance(legacy_id, str) or not isinstance(canonical_id, str):
continue
canonical[legacy_id] = canonical_id
links.setdefault(canonical_id, set()).update(
str(ref["evidence_id"])
for ref in (refs if isinstance(refs, list) else ())
if isinstance(ref, dict) and isinstance(ref.get("evidence_id"), str)
)
if "governed_memories" in tables:
for row in database.execute(
"SELECT memory_id,created_at,updated_at FROM governed_memories"
):
legacy_id = str(row["memory_id"])
canonical_id = canonical.get(legacy_id, legacy_id)
links.setdefault(canonical_id, set())
legacy_created = _plausible_time(
row["created_at"], fallback=row["updated_at"], now=now
)
if legacy_created is not None:
created.setdefault(canonical_id, []).append(legacy_created)
if "governed_memory_evidence" in tables:
for row in database.execute(
"SELECT memory_id,evidence_id FROM governed_memory_evidence"
):
legacy_id = str(row["memory_id"])
links.setdefault(canonical.get(legacy_id, legacy_id), set()).add(
str(row["evidence_id"])
)
if "memory_write_jobs" in tables and "sdk_memory_ids_json" in _columns(
database, "memory_write_jobs"
):
job_evidence: dict[str, set[str]] = {}
if "memory_write_job_evidence" in tables:
for row in database.execute(
"SELECT job_id,evidence_id FROM memory_write_job_evidence"
):
job_evidence.setdefault(str(row["job_id"]), set()).add(str(row["evidence_id"]))
for row in database.execute(
"SELECT job_id,sdk_memory_ids_json FROM memory_write_jobs "
"WHERE sdk_memory_ids_json IS NOT NULL"
):
try:
memory_ids = json.loads(str(row["sdk_memory_ids_json"]))
except (TypeError, ValueError):
continue
for memory_id in memory_ids if isinstance(memory_ids, list) else ():
if isinstance(memory_id, str) and memory_id:
links.setdefault(memory_id, set()).update(
job_evidence.get(str(row["job_id"]), ())
)
evidence: dict[str, tuple[str, datetime | None]] = {}
wanted = sorted({identifier for refs in links.values() for identifier in refs})
for batch in _batches(wanted):
placeholders = ",".join("?" for _ in batch)
for row in database.execute(
"SELECT evidence_id,source_id,created_at,imported_at "
f"FROM evidence WHERE evidence_id IN ({placeholders})",
batch,
):
evidence[str(row["evidence_id"])] = (
str(row["source_id"]),
_plausible_time(row["created_at"], fallback=row["imported_at"], now=now),
)
pools: dict[str, _Pool] = {}
for canonical_id, refs in links.items():
pool = pools.setdefault(canonical_id, _Pool())
for identifier in refs:
if identifier not in evidence:
continue
source, timestamp = evidence[identifier]
if source and source != "default":
pool.sources.add(source)
if timestamp is not None:
pool.times.append(timestamp)
pool.created.extend(created.get(canonical_id, ()))
return pools
def _merge_pools(into: dict[str, _Pool], other: dict[str, _Pool]) -> None:
for canonical_id, pool in other.items():
merged = into.setdefault(canonical_id, _Pool())
merged.sources |= pool.sources
merged.times.extend(pool.times)
merged.created.extend(pool.created)
def _decide(pools: dict[str, _Pool]) -> dict[str, RepairMetadata]:
# Several legacy memories can fold into one SDK memory. Their evidence is
# pooled before deciding, so a disagreeing source stays unknown instead of
# whichever link happened to come last.
return {
canonical_id: RepairMetadata(
source=next(iter(pool.sources)) if len(pool.sources) == 1 else None,
memory_at=int(max(pool.times).timestamp()) if pool.times else None,
created_at=int(min(pool.created).timestamp()) if pool.created else None,
)
for canonical_id, pool in pools.items()
}
def _columns(database: sqlite3.Connection, table: str) -> set[str]:
return {str(row[1]) for row in database.execute(f"PRAGMA table_info({table})")}
def _plausible_time(
value: object,
*,
fallback: object,
now: datetime,
) -> datetime | None:
for candidate in (value, fallback):
if not isinstance(candidate, str) or not candidate:
continue
try:
parsed = datetime.fromisoformat(candidate)
except ValueError:
continue
if parsed.tzinfo is None:
parsed = parsed.replace(tzinfo=UTC)
parsed = parsed.astimezone(UTC)
if parsed <= now + timedelta(days=1):
return parsed
return None
def _repair_vector(
data_dir: Path,
mapping: dict[str, RepairMetadata],
*,
now: datetime,
) -> int:
directory = data_dir / "hy-memory" / HY_MEMORY_NAMESPACE / "vector"
if not (directory / "chroma.sqlite3").is_file():
return 0
import chromadb
from chromadb.config import Settings
client = chromadb.PersistentClient(
path=str(directory),
settings=Settings(anonymized_telemetry=False),
)
# The SDK client is not open yet (startup has not created its handle), so
# this is the only writer; release the store before the SDK takes it.
try:
repaired = 0
ids = tuple(mapping)
for collection in client.list_collections():
for batch in _batches(ids):
page = collection.get(ids=list(batch), include=["metadatas"])
updated_ids: list[str] = []
updated_metadata: list[dict[str, object]] = []
for memory_id, metadata in zip(
page.get("ids") or (),
page.get("metadatas") or (),
strict=True,
):
if not isinstance(metadata, dict):
continue
desired = mapping.get(str(memory_id))
if desired is None:
continue
next_metadata = dict(metadata)
changed = False
if desired.source and next_metadata.get("source") in {
None,
"",
"default",
}:
next_metadata["source"] = desired.source
changed = True
if (
desired.memory_at is not None
and next_metadata.get("memory_at") != desired.memory_at
):
next_metadata["memory_at"] = desired.memory_at
changed = True
if (
desired.created_at is not None
and next_metadata.get("gmt_created") != desired.created_at
):
next_metadata["gmt_created"] = desired.created_at
changed = True
if changed:
updated_ids.append(str(memory_id))
updated_metadata.append(next_metadata)
if updated_ids:
collection.update(ids=updated_ids, metadatas=updated_metadata)
repaired += len(updated_ids)
repaired += _settle_future_times(collection, now=now)
finally:
client.close()
return repaired
def _settle_future_times(collection: Any, *, now: datetime) -> int:
"""Move conversation times that are still in the future back to the write time.
A historical Cursor bug stored a file hash as the message time (years like
2254). The SDK lists newest first by `memory_at`, so those rows hold the
first page. Rows the bundles could date were fixed above; for the rest the
write time is the closest true moment that is known.
"""
cutoff = int((now + timedelta(days=1)).timestamp())
page = collection.get(where={"memory_at": {"$gt": cutoff}}, include=["metadatas"])
updated_ids: list[str] = []
updated_metadata: list[dict[str, object]] = []
for memory_id, metadata in zip(page.get("ids") or (), page.get("metadatas") or (), strict=True):
if not isinstance(metadata, dict):
continue
created = metadata.get("gmt_created")
settled = (
created
if isinstance(created, int) and not isinstance(created, bool) and created <= cutoff
else int(now.timestamp())
)
updated_ids.append(str(memory_id))
updated_metadata.append({**metadata, "memory_at": settled})
if updated_ids:
collection.update(ids=updated_ids, metadatas=updated_metadata)
return len(updated_ids)
def _repair_coding(
data_dir: Path,
mapping: dict[str, RepairMetadata],
) -> int:
root = data_dir / "hy-memory" / HY_MEMORY_NAMESPACE
repaired = 0
for name in ("coding.sqlite3", "coding_cv.sqlite3"):
path = root / name
if not path.is_file():
continue
with closing(sqlite3.connect(path)) as database:
tables = {
str(row[0])
for row in database.execute("SELECT name FROM sqlite_master WHERE type='table'")
}
for table, identifier in (
("coding_memory_meta", "experience_id"),
("working_fact", "fact_id"),
):
if table not in tables:
continue
rows = database.execute(
f"SELECT {identifier},source,created_at FROM {table}"
).fetchall()
for memory_id, source, created_at in rows:
desired = mapping.get(str(memory_id))
if desired is None:
continue
updates: list[str] = []
values: list[object] = []
if desired.source and source in {None, "", "default"}:
updates.append("source=?")
values.append(desired.source)
if desired.created_at is not None:
desired_created_at = (
datetime.fromtimestamp(desired.created_at, tz=UTC)
.replace(tzinfo=None)
.isoformat()
)
if created_at != desired_created_at:
updates.append("created_at=?")
values.append(desired_created_at)
if not updates:
continue
values.append(memory_id)
database.execute(
f"UPDATE {table} SET {','.join(updates)} WHERE {identifier}=?",
values,
)
repaired += 1
database.commit()
return repaired
def _batches[T](values: list[T] | tuple[T, ...]) -> Iterator[tuple[T, ...]]:
for start in range(0, len(values), _BATCH):
yield tuple(values[start : start + _BATCH])
@@ -92,9 +92,35 @@ class HySdkMemoryPage(BaseModel):
total: int = Field(ge=0)
limit: int = Field(ge=1)
offset: int = Field(ge=0)
# Set on a search whose relevance window was full: matches were looked for
# only among this many most relevant memories, so `total` may undercount.
search_window: int | None = None
@classmethod
def from_sdk(cls, raw: object, *, limit: int, offset: int) -> HySdkMemoryPage:
page, _skipped = cls._parse(raw, limit=limit, offset=offset, skip_invalid=False)
return page
@classmethod
def from_sdk_skipping_invalid(
cls, raw: object, *, limit: int, offset: int
) -> tuple[HySdkMemoryPage, int]:
"""Parse a page, leaving out rows outside the public record contract.
Returns the page and how many rows were left out, so the caller can
report them. The rows themselves are never touched.
"""
return cls._parse(raw, limit=limit, offset=offset, skip_invalid=True)
@classmethod
def _parse(
cls,
raw: object,
*,
limit: int,
offset: int,
skip_invalid: bool,
) -> tuple[HySdkMemoryPage, int]:
if not isinstance(raw, Mapping):
raise HySdkReadError("hy_memory_list_shape_unknown")
rows: list[object] = []
@@ -115,15 +141,27 @@ class HySdkMemoryPage(BaseModel):
total += bucket_total
rows.extend(bucket[rows_name])
deduplicated: dict[str, HySdkMemoryRecord] = {}
skipped = 0
for row in rows:
record = HySdkMemoryRecord.from_sdk(row)
try:
record = HySdkMemoryRecord.from_sdk(row)
except HySdkReadError:
if not skip_invalid:
raise
skipped += 1
continue
deduplicated.setdefault(record.memory_id, record)
return cls(
page = cls(
memories=tuple(deduplicated.values()),
# `total` is the SDK's stable result-set count. Subtracting only
# invalid rows observed on this page would make it change as the
# caller paged. Until the SDK exposes a quarantined count, keep the
# authoritative total and skip malformed rows only in this page.
total=total,
limit=limit,
offset=offset,
)
return page, skipped
class HySdkRawRecord(BaseModel):
@@ -289,6 +327,38 @@ def parse_sdk_search(raw: object, *, error_code: str) -> tuple[SemanticMemoryHit
return tuple(found)
def library_keyword_hits(raw: object, query: str) -> tuple[tuple[str, ...], int]:
"""Ids in an SDK search window whose text contains every query term.
The local SDK store has no keyword channel (Chroma's keyword search is
empty), so search is pure vector similarity: a word in the library can
score below the floor, and a word in none of it still scores above. The
library search asks the SDK for its most relevant window and keeps only the
rows that actually contain the words, most relevant first. Returns those ids
and how many distinct rows the window held.
"""
rows = _reply_rows(raw, "hy_memory_search_shape_unknown")
terms = query.casefold().split()
seen: set[str] = set()
matched: list[str] = []
for item in sorted(rows, key=_score, reverse=True):
memory_id = item.get("memory_id") or item.get("id")
if not isinstance(memory_id, str) or not memory_id or memory_id in seen:
continue
seen.add(memory_id)
content = item.get("content")
tags = item.get("tags")
text = " ".join(
[
content if isinstance(content, str) else "",
*(tag for tag in (tags if isinstance(tags, list) else ()) if isinstance(tag, str)),
]
).casefold()
if all(term in text for term in terms):
matched.append(memory_id)
return tuple(matched), len(seen)
def parse_sdk_raw_search(raw: object) -> tuple[HySdkRawHit, ...]:
return tuple(
HySdkRawHit.from_sdk_search(row)
@@ -50,6 +50,10 @@ class AuthorizedSourceFile:
size: int
modified_ns: int
changed_ns: int
# Source-specific change token (for example Cursor's WAL identity). It is
# cursor metadata only; modified_ns remains the real filesystem mtime used
# as a fallback conversation timestamp.
cursor_version: str | None = None
root_identity: str | None = None
requested_root: Path | None = None
access_check: Callable[[], None] | None = field(default=None, repr=False, compare=False)
@@ -644,7 +644,7 @@ def is_unchanged(source_file: AuthorizedSourceFile, cursors: dict[str, SourceCur
return (
previous is not None
and previous.read_to_end
and previous.file_identity == source_file.file_identity
and previous.file_identity == _cursor_file_identity(source_file)
and previous.observed_size == source_file.size
and previous.observed_modified_ns == source_file.modified_ns
# Chmod alone leaves size, mtime and inode untouched, so a file that
@@ -691,7 +691,7 @@ def touched_cursor(
previous = options.cursors.get(source_file.safe_id)
keep = (
previous is not None
and previous.file_identity == source_file.file_identity
and _same_file(previous, source_file)
and previous.observed_size <= source_file.size
)
# A kept cursor means the next scan resumes exactly where this one ended:
@@ -871,7 +871,7 @@ def resume_checkpoint(
previous = options.cursors.get(source_file.safe_id)
if previous is None:
return None
if previous.file_identity != source_file.file_identity:
if not _same_file(previous, source_file):
return None
if source_file.size < previous.observed_size:
return None
@@ -1046,7 +1046,7 @@ def source_cursor(
) -> SourceCursor:
return SourceCursor(
file_key=source_file.safe_id,
file_identity=source_file.file_identity,
file_identity=_cursor_file_identity(source_file),
checkpoint=checkpoint,
checkpoint_anchor=checkpoint_anchor,
observed_size=source_file.size,
@@ -1054,6 +1054,27 @@ def source_cursor(
)
def _cursor_file_identity(source_file: AuthorizedSourceFile) -> str:
return (
f"{source_file.file_identity}{_VERSION_SEPARATOR}{source_file.cursor_version}"
if source_file.cursor_version
else source_file.file_identity
)
_VERSION_SEPARATOR = ":v:"
def _same_file(previous: SourceCursor, source_file: AuthorizedSourceFile) -> bool:
"""Whether a stored cursor still points into this file.
The stored identity may carry a change token (Cursor's WAL version). The
token says the file changed, which `is_unchanged` handles; it does not mean
the file was replaced, so resuming compares the bare `dev:ino` only.
"""
return previous.file_identity.partition(_VERSION_SEPARATOR)[0] == source_file.file_identity
_POSITION = re.compile(r"(?<=:)\d+(?=:|$)")
@@ -533,8 +533,8 @@ def _with_write_ahead_log(database: AuthorizedSourceFile) -> AuthorizedSourceFil
version = hashlib.blake2b(
f"{database.modified_ns}:{journal.st_mtime_ns}:{journal.st_size}:{journal.st_ino}".encode(),
digest_size=8,
).digest()
return replace(database, modified_ns=int.from_bytes(version, "big") & ((1 << 63) - 1))
).hexdigest()
return replace(database, cursor_version=version)
def _live_baseline(listing: _CursorListing) -> dict[str, SourceCursor]:
@@ -79,9 +79,9 @@ SDK_MIGRATION_CAPTURE_FENCE = ".sdk-migration-capture-fence.json"
def database_path() -> Path:
from one_memory.workbuddy_boundary import database_path as boundary_database
from one_memory.config import load_settings
return boundary_database()
return load_settings().data_dir / "one-memory.sqlite3"
def capture_stdio(source_id: str, stdin: TextIO, output: TextIO) -> int:
@@ -1,342 +0,0 @@
"""Read-only local library projection over the existing governed memory store."""
import json
import os
import sqlite3
import time
from pathlib import Path
from typing import Literal
from pydantic import BaseModel, ConfigDict, Field
from one_memory.database_access import main_database
from one_memory.legacy_lifecycle_types import HyLifecycle
from one_memory.memory_evidence_freshness import register_memory_freshness
from one_memory.memory_governance import (
GovernedMemory,
MemoryGovernanceStore,
MemoryStatus,
lifecycle_predicate,
)
#: Read budget for one library page. 2026-10-12: raised from 5s (which starved
#: under a concurrent history import and surfaced as 503) to 15s, still well
#: under the 30s client timeout; past it the read degrades to a `stale` page.
READ_BUDGET_SECONDS = 15
class MemorySourceTime(BaseModel):
"""When the conversation a memory came from happened (D96).
`basis` is `conversation` when the cited messages carry their own time,
`import` when only the import time is known; the UI words them apart.
"""
model_config = ConfigDict(frozen=True, extra="forbid")
at: str
basis: Literal["conversation", "import"]
class MemoryLibraryPage(BaseModel):
model_config = ConfigDict(frozen=True, extra="forbid")
items: list[GovernedMemory]
total: int = Field(ge=0)
offset: int = Field(ge=0)
limit: int = Field(ge=1, le=500)
source_ids_by_memory: dict[str, list[str]] = Field(default_factory=dict)
source_time_by_memory: dict[str, MemorySourceTime] = Field(default_factory=dict)
#: True when the read budget ran out mid-page (typically during a history
#: import): the counts and rows above are a partial snapshot, not an
#: authoritative view. Clients should show "importing" instead of an error.
stale: bool = False
def _sort_by_conversation() -> bool:
"""D99: the list reads as a timeline of the conversations, newest first.
A memory generated today from a three-week-old dialogue belongs next to
that week, not at the top. ONE_MEMORY_LIBRARY_SORT=updated brings back the
old "newest generated first" order without a code change.
"""
return os.environ.get("ONE_MEMORY_LIBRARY_SORT", "").strip().lower() != "updated"
def _origin_join() -> str:
"""The newest cited message per memory, as one joined aggregate.
A correlated subquery would re-scan the link table for every candidate row;
this is a single pass. Memories with no readable citation keep their old
position through COALESCE.
"""
if not _sort_by_conversation():
return ""
return """LEFT JOIN (
SELECT link.memory_id AS memory_id,
MAX(COALESCE(e.created_at, e.imported_at)) AS origin_at
FROM governed_memory_evidence link
JOIN evidence e ON e.evidence_id = link.evidence_id
WHERE link.active = 1
GROUP BY link.memory_id
) origin ON origin.memory_id = m.memory_id"""
def _order_by() -> str:
if not _sort_by_conversation():
return "m.updated_at DESC, m.memory_id ASC"
# Conversations that share a timestamp keep the generated order between
# themselves; ties would otherwise fall back to the id and look arbitrary.
return "COALESCE(origin.origin_at, m.updated_at) DESC, m.updated_at DESC, m.memory_id ASC"
class _LibraryQuery(BaseModel):
model_config = ConfigDict(frozen=True, extra="forbid")
user_id: str = Field(min_length=1, max_length=256)
query: str | None = None
status: MemoryStatus | None = None
active_only: bool = False
source: str | None = Field(default=None, min_length=1)
offset: int = Field(default=0, ge=0, le=2**63 - 1, strict=True)
limit: int = Field(default=100, ge=1, le=500, strict=True)
def read_conversation_memories(
database_path: Path,
*,
user_id: str,
source_id: str,
conversation_id: str,
workspace: str | None,
limit: int = 100,
) -> list[GovernedMemory]:
"""Current memories backed by any active Evidence in one conversation."""
with main_database(database_path).read() as db:
db.row_factory = sqlite3.Row
freshness = register_memory_freshness(db)
rows = db.execute(
f"""
SELECT DISTINCT m.* FROM governed_memories m
JOIN governed_memory_evidence link ON link.memory_id=m.memory_id AND link.active=1
JOIN evidence e ON e.evidence_id=link.evidence_id
WHERE m.user_id=? AND m.status='confirmed' AND m.deleted_at IS NULL
AND e.user_id=? AND e.source_id=? AND e.conversation_id=?
AND e.workspace IS ? AND {freshness}
ORDER BY m.updated_at DESC, m.memory_id
LIMIT ?
""",
(user_id, user_id, source_id, conversation_id, workspace, limit),
).fetchall()
return MemoryGovernanceStore.memories_from_rows(db, rows)
def read_memory_library(
database_path: Path,
*,
user_id: str = "local",
query: str | None = None,
status: MemoryStatus | None = None,
active_only: bool = False,
source: str | None = None,
offset: int = 0,
limit: int = 100,
lifecycle: HyLifecycle | None = None,
) -> MemoryLibraryPage:
"""Read one page and its exact readable total from the same SQLite snapshot.
This is the local UI policy, not an external agent authorization entry point.
Missing grants do not revoke already stored content; explicit revocations do.
Read/parse failures propagate rather than masquerading as an empty library.
Which memories are current is HY's `lifecycle` when given, as in
`MemoryGovernanceStore.list`.
"""
request = _LibraryQuery(
user_id=user_id,
query=query,
status=status,
active_only=active_only,
source=source,
offset=offset,
limit=limit,
)
if request.active_only and request.status is not None:
raise ValueError("active_only_conflicts_with_status")
clauses = ["m.user_id = ?"]
values: list[str | int] = [request.user_id]
selected, selected_values = lifecycle_predicate(
lifecycle, MemoryStatus.CONFIRMED if request.active_only else request.status, alias="m"
)
if selected is not None:
clauses.append(selected)
values.extend(selected_values)
if request.status != MemoryStatus.DELETED:
clauses.append("m.deleted_at IS NULL")
# SourceReadAccess hides the whole memory if any active citation is revoked.
# Do this before COUNT/LIMIT; a readable citation cannot rescue a mixed memory.
clauses.append("""NOT EXISTS (
SELECT 1 FROM governed_memory_evidence link
JOIN evidence e ON e.evidence_id = link.evidence_id
JOIN source_grants g ON g.user_id = e.user_id AND g.source_id = e.source_id
WHERE link.memory_id = m.memory_id AND +link.active = 1
AND g.revoked_at IS NOT NULL
)""")
# Governance normally guarantees ownership and referential integrity. Fail
# closed for broken active links instead of exposing another user's citation.
clauses.append("""NOT EXISTS (
SELECT 1 FROM governed_memory_evidence link
LEFT JOIN evidence e ON e.evidence_id = link.evidence_id
WHERE link.memory_id = m.memory_id AND +link.active = 1
AND (e.evidence_id IS NULL OR e.user_id != m.user_id)
)""")
if request.query:
clauses.append("m.content LIKE ? ESCAPE '\\'")
escaped = request.query.replace("\\", "\\\\").replace("%", "\\%").replace("_", "\\_")
values.append(f"%{escaped}%")
if request.source is not None:
# Start from the compact active-link index, then resolve each Evidence
# identity by primary key. The old correlated EXISTS let SQLite scan
# every Evidence row for every candidate memory on large libraries.
clauses.append("""m.memory_id IN (
SELECT link.memory_id FROM governed_memory_evidence link
JOIN evidence e ON e.evidence_id = link.evidence_id
WHERE link.active = 1 AND e.user_id = ? AND e.source_id = ?
)""")
values.extend((request.user_id, request.source))
predicate = " AND ".join(clauses)
def _interrupted(error: Exception) -> bool:
return isinstance(error, sqlite3.OperationalError) and "interrupted" in str(error).lower()
with main_database(database_path).read() as db:
# 2026-10-12: 5s starved the page whenever a history import saturated
# SQLite, and the caller turned the deadline abort into a 503, so the
# memory page read as broken for the whole import. 15s stays well under
# the 30s client timeout; past it we return a `stale` page instead of
# an error — an incomplete list is honest, an error is not.
deadline = time.monotonic() + READ_BUDGET_SECONDS
db.set_progress_handler(lambda: int(time.monotonic() >= deadline), 1000)
db.row_factory = sqlite3.Row
db.execute("BEGIN")
# The total counts with the cheap permission filters only: the per-row
# freshness UDF scales with the whole library and can burn the whole
# read budget on its own (D22). Product accepted a small drift here;
# the rows that are actually displayed are still checked row by row.
try:
total = db.execute(
f"SELECT COUNT(*) FROM governed_memories m WHERE {predicate}", values
).fetchone()[0]
except sqlite3.OperationalError as error:
if not _interrupted(error):
raise
return MemoryLibraryPage(
items=[],
total=0,
offset=request.offset,
limit=request.limit,
stale=True,
)
predicate += " AND " + register_memory_freshness(db)
try:
if request.query:
current = db.execute(
f"""SELECT m.* FROM governed_memories m {_origin_join()}
WHERE {predicate}
ORDER BY {_order_by()}""",
values,
).fetchall()
rows = current[request.offset : request.offset + request.limit]
else:
rows = db.execute(
f"""SELECT m.* FROM governed_memories m {_origin_join()}
WHERE {predicate}
ORDER BY {_order_by()} LIMIT ? OFFSET ?""",
[*values, request.limit, request.offset],
).fetchall()
except sqlite3.OperationalError as error:
if not _interrupted(error):
raise
# The count already succeeded: report it, flag the page stale.
return MemoryLibraryPage(
items=[],
total=total,
offset=request.offset,
limit=request.limit,
stale=True,
)
# Static read helper only: constructing the Store would initialize schema.
items = MemoryGovernanceStore.memories_from_rows(db, rows, lifecycle=lifecycle)
# Only the authorized page participates; source filters select memories,
# not a subset of their provenance. Keep this in the count/page snapshot.
source_ids_by_memory: dict[str, list[str]] = {item.memory_id: [] for item in items}
# D96: the list shows when the cited conversation happened, not when
# the memory was generated (which is "today" for a fresh import). Read
# in the same query as the sources, so the page costs no extra round
# trip and only this page's IDs are looked at, never the whole library.
conversation_at: dict[str, str] = {}
imported_at: dict[str, str] = {}
if items:
try:
for link in db.execute(
# D104: the conversation time is when the person spoke, not
# when the turn ended. A memory's evidence covers the whole
# round (question, tool calls, reply), so MAX() lands on the
# reply and reads as a later, contradictory time next to the
# cited source. Prefer the earliest person-side evidence; only
# fall back to the earliest of any kind when the round has no
# person-authored row at all.
"""SELECT link.memory_id, e.source_id,
COALESCE(
MIN(
CASE WHEN e.role = 'user'
AND e.role = 'user'
THEN e.created_at END
),
MIN(e.created_at)
) AS conversation_at,
MAX(e.imported_at) AS imported_at
FROM governed_memory_evidence link
JOIN evidence e ON e.evidence_id = link.evidence_id
WHERE link.active = 1 AND e.user_id = ?
AND link.memory_id IN (SELECT value FROM json_each(?))
GROUP BY link.memory_id, e.source_id
ORDER BY link.memory_id, e.source_id""",
(request.user_id, json.dumps(list(source_ids_by_memory))),
):
memory_id = link["memory_id"]
source_ids_by_memory[memory_id].append(link["source_id"])
for seen, value in (
(conversation_at, link["conversation_at"]),
(imported_at, link["imported_at"]),
):
if value and str(value) > seen.get(memory_id, ""):
seen[memory_id] = str(value)
except sqlite3.OperationalError as error:
if not _interrupted(error):
raise
# Page rows are already read; ship them without source maps
# rather than failing the whole request at the last step.
return MemoryLibraryPage(
items=items,
total=total,
offset=request.offset,
limit=request.limit,
source_ids_by_memory=source_ids_by_memory,
stale=True,
)
source_time_by_memory = {
memory_id: MemorySourceTime(at=conversation_at[memory_id], basis="conversation")
if memory_id in conversation_at
else MemorySourceTime(at=imported_at[memory_id], basis="import")
for memory_id in source_ids_by_memory
if memory_id in conversation_at or memory_id in imported_at
}
return MemoryLibraryPage(
items=items,
total=total,
offset=request.offset,
limit=request.limit,
source_ids_by_memory=source_ids_by_memory,
source_time_by_memory=source_time_by_memory,
)
@@ -1,180 +0,0 @@
"""Bounded, permission-aware receipt projection; never initialize a write store."""
import sqlite3
from pathlib import Path
from typing import Literal
from pydantic import BaseModel, ConfigDict
from one_memory.database_access import main_database
#: `legacy` is a confirmation recorded before memories stopped waiting for
#: one; it changed nothing that still exists, and `migrated_from` says so.
Operation = Literal["create", "correct", "set_scopes", "undo", "delete", "hy_reconcile", "legacy"]
ResultStatus = Literal["confirmed", "deleted"]
MigratedFrom = Literal["confirm", "reject"]
OPERATIONS = {"create", "correct", "set_scopes", "undo", "delete", "hy_reconcile", "legacy"}
STATUSES = {"confirmed", "deleted"}
#: Which operation each pre-v2 one now reads as.
MIGRATED_OPERATIONS = {"confirm": "legacy", "reject": "delete"}
#: A memory may move on to these after an earlier receipt was written; the
#: receipt still reports what that request committed, not the current status.
CURRENT_STATUSES = STATUSES | {"superseded"}
class MemoryMutationReceiptForbidden(RuntimeError):
def __init__(self) -> None:
super().__init__("memory_mutation_receipt_forbidden")
class MemoryMutationReceiptUnavailable(RuntimeError):
def __init__(self) -> None:
super().__init__("memory_mutation_receipt_unavailable")
class MemoryMutationReceipt(BaseModel):
model_config = ConfigDict(frozen=True, extra="forbid")
schema_version: Literal[2] = 2
request_id: str
state: Literal["not_found", "committed"]
operation: Operation | None = None
memory_id: str | None = None
result_revision: int | None = None
result_status: ResultStatus | None = None
memory_state: Literal["present", "deleted"] | None = None
#: The operation as it was stored before v2, when it has no v2 name of its own.
migrated_from: MigratedFrom | None = None
def _identity(value: str, maximum: int) -> bool:
return isinstance(value, str) and 0 < len(value) <= maximum
def _read_snapshot(
database_path: Path, request_id: str, user_id: str
) -> tuple[MemoryMutationReceipt, tuple[object, ...]]:
with main_database(database_path).read(row_factory=sqlite3.Row) as db:
db.execute("PRAGMA query_only=ON")
db.execute("BEGIN")
rows = db.execute(
"SELECT user_id,request_id,memory_id,operation,result_revision "
"FROM governed_memory_mutations WHERE user_id=? AND request_id=? LIMIT 2",
(user_id, request_id),
).fetchall()
if not rows:
return MemoryMutationReceipt(request_id=request_id, state="not_found"), ()
if len(rows) != 1:
raise MemoryMutationReceiptUnavailable()
mutation = rows[0]
memory_id, operation, revision = (
mutation["memory_id"],
mutation["operation"],
mutation["result_revision"],
)
if (
not _identity(memory_id, 512)
or operation not in OPERATIONS
or type(revision) is not int
or not 1 <= revision <= 2**53 - 1
):
raise MemoryMutationReceiptUnavailable()
current = db.execute(
"SELECT user_id,revision,status,deleted_at FROM governed_memories WHERE memory_id=?",
(memory_id,),
).fetchone()
if (
current is None
or current["user_id"] != user_id
or type(current["revision"]) is not int
or current["revision"] < revision
or current["status"] not in CURRENT_STATUSES
or (current["status"] == "deleted") != (current["deleted_at"] is not None)
):
raise MemoryMutationReceiptUnavailable()
# Versions have no historical evidence snapshot. Check every retained link,
# not only active ones, before disclosing even operation/ID metadata.
links = db.execute(
"SELECT l.evidence_id,l.active,e.user_id,e.source_id,e.revision,"
"g.granted_at,g.revoked_at FROM governed_memory_evidence l "
"LEFT JOIN evidence e ON e.evidence_id=l.evidence_id "
"LEFT JOIN source_grants g ON g.user_id=e.user_id AND g.source_id=e.source_id "
"WHERE l.memory_id=? ORDER BY l.evidence_id LIMIT 10001",
(memory_id,),
).fetchall()
if not links or len(links) > 10000:
raise MemoryMutationReceiptUnavailable()
for link in links:
if (
link["user_id"] != user_id
or link["source_id"] is None
or link["active"] not in (0, 1)
):
raise MemoryMutationReceiptUnavailable()
if link["revoked_at"] is not None:
raise MemoryMutationReceiptForbidden()
versions = db.execute(
"SELECT memory_id,revision,action,status,migrated_from FROM governed_memory_versions "
"WHERE memory_id=? AND revision=? LIMIT 2",
(memory_id, revision),
).fetchall()
if len(versions) != 1:
raise MemoryMutationReceiptUnavailable()
version = versions[0]
if version["action"] != operation or version["status"] not in STATUSES:
raise MemoryMutationReceiptUnavailable()
migrated_from = version["migrated_from"]
if migrated_from is not None and MIGRATED_OPERATIONS.get(migrated_from) != operation:
raise MemoryMutationReceiptUnavailable()
if (operation == "legacy") != (migrated_from == "confirm"):
raise MemoryMutationReceiptUnavailable()
required_status = {
"create": "confirmed",
"legacy": "confirmed",
"delete": "deleted",
}
if operation in required_status and version["status"] != required_status[operation]:
raise MemoryMutationReceiptUnavailable()
if (operation == "create") != (revision == 1):
raise MemoryMutationReceiptUnavailable()
if (
operation in {"correct", "set_scopes", "hy_reconcile", "undo"}
and version["status"] != "confirmed"
):
raise MemoryMutationReceiptUnavailable()
receipt = MemoryMutationReceipt(
request_id=request_id,
state="committed",
operation=operation,
memory_id=memory_id,
result_revision=revision,
result_status=version["status"],
memory_state="deleted" if current["status"] == "deleted" else "present",
migrated_from=migrated_from,
)
return receipt, (
tuple(mutation),
tuple(current),
tuple(version),
tuple(tuple(link) for link in links),
)
def read_memory_mutation_receipt(
database_path: Path, *, request_id: str, user_id: str = "local"
) -> MemoryMutationReceipt:
"""History only, never current authorization, HY cleanup or safe-retry proof."""
if not _identity(request_id, 256):
raise ValueError("invalid_memory_mutation_request_id")
if not _identity(user_id, 256):
raise ValueError("invalid_memory_mutation_user_id")
try:
first = _read_snapshot(database_path, request_id, user_id)
current = _read_snapshot(database_path, request_id, user_id)
if first != current:
raise MemoryMutationReceiptUnavailable()
return current[0]
except MemoryMutationReceiptForbidden:
raise
except (sqlite3.Error, ValueError, TypeError, KeyError, IndexError, OSError):
raise MemoryMutationReceiptUnavailable() from None
@@ -67,20 +67,6 @@ class MemoryNoteReceipt(BaseModel):
error: str | None = None
class SavedMemoryNoteEvidence(BaseModel):
model_config = ConfigDict(frozen=True, extra="forbid")
available: Literal[True] = True
evidence_id: str
source_id: str
revision: int
imported_at: datetime
role: str
written_by: WrittenBy
created_at: datetime | None
verbatim: str
class LegacyMemoryNoteReader:
"""Read pre-cutover note receipts without retaining any write runtime."""
@@ -441,48 +427,6 @@ def memory_note_router(dependencies: Sequence[Depends]) -> APIRouter:
except ValueError as error:
raise HTTPException(409, str(error)) from error
@router.get("/{evidence_id}/evidence", response_model=SavedMemoryNoteEvidence)
def evidence(evidence_id: str, request: Request) -> SavedMemoryNoteEvidence:
try:
notes = store(request)
if notes.sdk_authority:
raise HTTPException(409, "memory_note_evidence_retired")
with notes.evidence._connect() as connection:
record = notes.evidence._get_with_connection(connection, evidence_id)
metadata = record.origin_meta.get(EvidenceOrigin.MANUAL_IMPORT) if record else None
if (
record is None
or record.user_id != "local"
or record.source_id != SOURCE_ID
or metadata is None
or metadata.raw_meta.get("input_mode") != "direct_memory_note"
):
raise KeyError(evidence_id)
grant = connection.execute(
"SELECT revoked_at FROM source_grants WHERE user_id='local' AND source_id=?",
(SOURCE_ID,),
).fetchone()
if grant is None or grant[0] is not None:
raise ValueError("memory_note_source_revoked")
result = SavedMemoryNoteEvidence(
evidence_id=record.evidence_id,
source_id=record.source_id,
revision=record.revision,
imported_at=record.imported_at,
role=record.role,
written_by=record.written_by,
created_at=record.created_at,
verbatim=record.verbatim,
)
# The returned body and reference share one snapshot. Recheck authority
# after that read so a revoked source is not released from a stale view.
notes.require(evidence_id)
return result
except KeyError as error:
raise HTTPException(404, "memory_note_not_found") from error
except ValueError as error:
raise HTTPException(409, str(error)) from error
@router.post("/{evidence_id}/process", response_model=MemoryNoteReceipt)
async def process(evidence_id: str, request: Request) -> MemoryNoteReceipt:
notes = store(request)
@@ -1,7 +1,7 @@
"""A3-01: the HTTP entry to the unified write path.
`POST /api/v1/memory-writes` accepts a write and returns a request id;
`GET /api/v1/memory-writes/{id}` reports its outcome.
`POST /api/v2/memory-writes` accepts OpenAI-style messages and returns a
request id; `GET /api/v2/memory-writes/{id}` reports its outcome.
Two rules shape this endpoint, both consequences of A0-03's measurement that HY
has no idempotency key:
@@ -45,17 +45,6 @@ from one_memory.memory_server.registry import WriteRegistry, WriteState
from one_memory.memory_server.submit import WriteConflictError
class MemoryWriteInput(BaseModel):
model_config = ConfigDict(extra="forbid")
internal_session_id: str = Field(min_length=1, max_length=128)
first_round: int = Field(ge=1)
last_round: int = Field(ge=1)
evidence_ids: list[str] = Field(min_length=1)
priority: Literal["live", "normal"] = "normal"
idempotency_key: str | None = Field(default=None, max_length=256)
class MemoryWriteReceipt(BaseModel):
request_id: str
state: WriteState
@@ -121,67 +110,6 @@ def _registry(
return cast(WriteRegistry | DurableWriteRegistry, registry)
def router(dependencies: list[Any] | None = None) -> APIRouter:
instance = APIRouter(
prefix="/api/v1/memory-writes",
tags=["memory-writes"],
dependencies=dependencies or [],
)
@instance.post("", response_model=MemoryWriteReceipt, status_code=202)
def submit(payload: MemoryWriteInput, request: Request) -> MemoryWriteReceipt:
registry = _registry(request)
batch = BatchIdentity(
internal_session_id=payload.internal_session_id,
first_round=payload.first_round,
last_round=payload.last_round,
)
request_contract = WriteRequest(
internal_session_id=payload.internal_session_id,
batch_identity=batch,
evidence_ids=payload.evidence_ids,
idempotency_key=payload.idempotency_key or batch.batch_id,
priority=payload.priority,
)
try:
record, enqueued = registry.submit(request_contract)
except WriteConflictError as error:
raise HTTPException(409, str(error)) from error
except LookupError as error:
# Evidence-first: the write points at Evidence the business
# database does not hold. Rejecting it here is a caller error, not
# a server fault, and it must not create a queue row.
raise HTTPException(422, str(error)) from error
return MemoryWriteReceipt(
request_id=record.request_id,
state=record.state,
created_at=record.created_at,
updated_at=record.updated_at,
detail="" if enqueued else "already accepted; not enqueued again",
)
@instance.get("/{request_id}", response_model=MemoryWriteReceipt)
def status(request_id: str, request: Request) -> MemoryWriteReceipt:
registry = _registry(request, write=False)
try:
record = registry.get(request_id)
except DurableQueueSchemaUnavailableError as error:
raise HTTPException(503, str(error)) from error
if record is None:
raise HTTPException(404, "unknown request id")
return MemoryWriteReceipt(
request_id=record.request_id,
state=record.state,
created_at=record.created_at,
updated_at=record.updated_at,
detail=record.detail,
raw_ids=record.raw_ids,
memory_ids=record.memory_ids,
)
return instance
def sdk_router(dependencies: list[Any] | None = None) -> APIRouter:
instance = APIRouter(
prefix="/api/v2/memory-writes",
@@ -1,623 +0,0 @@
"""Read-only process: serves the same read routes as Core without write access.
Opens the database in `mode=ro` with `PRAGMA query_only=ON`; every store
operation here goes through `main_database().read()`, which already enforces
this. No HY SDK, no Chroma, no ONNX runtime, no keychain: the imports that
would pull them in are simply not reached, and the routes below never call
into a write path.
Reads that need HY (semantic search, graph overlay) are delegated to the
Core's internal endpoints; the Core is the only process that owns the
vector stores. Side effects (call events, feedback) are queued and sent
asynchronously, never blocking the read response.
"""
from __future__ import annotations
import asyncio
import builtins
import contextlib
import logging
import os
import sqlite3
import time
from collections.abc import AsyncIterator, Collection, Mapping, Sequence
from contextlib import asynccontextmanager
from datetime import UTC, datetime
from pathlib import Path
from typing import Any, cast
from fastapi import Depends, FastAPI, HTTPException, Query, Request
from pydantic import BaseModel, ConfigDict, SecretStr
from one_memory.database_access import main_database
from one_memory.home_summary import (
HomeSummary,
HomeSummaryUnavailable,
read_home_summary,
)
from one_memory.memory_governance import GovernedMemory, MemoryStatus
from one_memory.memory_library import MemoryLibraryPage, read_memory_library
from one_memory.recall import RecallRequest, _KeywordCandidates
from one_memory.semantic_deadline import SemanticMemoryHit
from one_memory.source_read_access import SourceReadAccess
logger = logging.getLogger(__name__)
MAX_READER_RSS_BYTES = 250 * 1024 * 1024
_PROBE_QUERY = "__om_probe__"
def _check_imports() -> None:
"""Fail fast if any forbidden module is already loaded."""
forbidden = ("hy_memory", "chromadb", "onnxruntime", "keyring")
import sys
loaded = [name for name in forbidden if name in sys.modules]
if loaded:
raise ImportError(f"reader_process_loaded_forbidden_modules: {', '.join(loaded)}")
def _memory_readable_sql() -> str:
return """
m.user_id = ? AND m.status = 'confirmed' AND m.deleted_at IS NULL
AND NOT EXISTS (
SELECT 1 FROM governed_memory_evidence link
JOIN evidence e ON e.evidence_id = link.evidence_id
JOIN source_grants g ON g.user_id=e.user_id AND g.source_id=e.source_id
WHERE link.memory_id=m.memory_id AND +link.active=1
AND g.revoked_at IS NOT NULL
)
AND NOT EXISTS (
SELECT 1 FROM governed_memory_evidence link
LEFT JOIN evidence e ON e.evidence_id = link.evidence_id
WHERE link.memory_id=m.memory_id AND +link.active=1
AND (e.evidence_id IS NULL OR e.user_id != m.user_id)
)
"""
class ReaderSettings(BaseModel):
model_config = ConfigDict(frozen=True)
data_dir: Path
host: str = "127.0.0.1"
port: int = 0 # 0 = ephemeral
api_token: SecretStr
core_port: int
core_api_token: str
class SideEffectForwarder:
"""Queues read-path side effects and forwards them to Core.
Reads in the Reader must stay read-only: the database is opened with
`query_only`, so a call event cannot be written here. Instead each read
that would have recorded one enqueues it, and a background task posts it
to Core's internal endpoint. The response never waits for it; a Core that
is unavailable drops it and counts the drop.
"""
def __init__(self, settings: ReaderSettings) -> None:
self.settings = settings
self.queue: asyncio.Queue[dict[str, object]] = asyncio.Queue(maxsize=256)
self.dropped = 0
self.task: asyncio.Task[None] | None = None
def enqueue(self, event: dict[str, object]) -> None:
try:
self.queue.put_nowait(event)
except asyncio.QueueFull:
self.dropped += 1
def start(self) -> None:
if self.task is None or self.task.done():
self.task = asyncio.ensure_future(self._drain())
def stop(self) -> None:
if self.task is not None and not self.task.done():
self.task.cancel()
async def _drain(self) -> None:
import httpx
while True:
event = await self.queue.get()
try:
async with httpx.AsyncClient(
headers={
"Authorization": f"Bearer {self.settings.core_api_token}",
"Content-Type": "application/json",
},
timeout=5.0,
proxy=None,
) as client:
await client.post(
f"http://127.0.0.1:{self.settings.core_port}/internal/call-event",
json=event,
)
except Exception:
# A dropped side effect is never a failed read: recall already
# answered. Counted so a persistently unavailable Core is
# visible rather than silent.
self.dropped += 1
async def _core_get(
settings: ReaderSettings,
path: str,
timeout_s: float = 5.0,
params: Mapping[
str, str | int | float | bool | None | Sequence[str | int | float | bool | None]
]
| None = None,
) -> Any:
"""Call the Core's internal read endpoint for data only Core can serve.
`params` is passed to httpx rather than interpolated into the path: a
query containing `&`, `#` or a space would otherwise be cut short or
read as a fragment, silently searching for the wrong thing.
"""
import httpx
url = f"http://127.0.0.1:{settings.core_port}{path}"
async with httpx.AsyncClient(
headers={"Authorization": f"Bearer {settings.core_api_token}"},
timeout=timeout_s,
proxy=None,
) as client:
response = await client.get(url, params=params)
response.raise_for_status()
return response.json()
class _ReaderGovernance:
"""Minimal read interface over the governance store for recall."""
def __init__(self, database_path: Path) -> None:
self.database_path = database_path
def list(
self,
*,
user_id: str,
status: MemoryStatus | None = None,
query: str | None = None,
limit: int = 100,
offset: int = 0,
) -> list[GovernedMemory]:
"""One page of memories, read through the Core's own read function.
`MemoryGovernanceStore.list` cannot be used directly: constructing the
store runs schema DDL on the write connection, and the Reader opens
with `query_only`. `read_memory_library` is the same read expressed as
a static SQL read (revoked sources excluded, per-row freshness checked
for the rows it returns, cheap total for the rest), so the Reader and
the Core cannot drift on what "readable" means.
"""
page = read_memory_library(
self.database_path,
user_id=user_id,
query=query,
status=status,
offset=offset,
limit=limit,
)
return list(page.items)
def find_confirmed_matching_terms(
self,
*,
user_id: str,
terms: Collection[str],
limit: int = 200,
scopes: Collection[str] | None = None,
) -> tuple[builtins.list[GovernedMemory], bool]:
if not 1 <= limit <= 2_000:
raise ValueError("invalid candidate limit")
cleaned = tuple(dict.fromkeys(term.strip() for term in terms if term.strip()))
if not cleaned or scopes is not None and not scopes:
return [], False
clauses = ["m.user_id=?", "m.status='confirmed'", "m.deleted_at IS NULL"]
values: builtins.list[object] = [user_id]
matches = []
for term in cleaned[:32]:
matches.append("m.content LIKE ? ESCAPE '\\'")
escaped = term.replace("\\", "\\\\").replace("%", "\\%").replace("_", "\\_")
values.append(f"%{escaped}%")
clauses.append(f"({' OR '.join(matches)})")
if scopes is not None:
clauses.append(
"EXISTS (SELECT 1 FROM json_each(m.scopes_json) "
f"WHERE json_each.value IN ({','.join('?' for _ in scopes)}))"
)
values.extend(scopes)
values.append(limit + 1)
with main_database(self.database_path).read(row_factory=sqlite3.Row) as db:
rows = db.execute(
f"SELECT m.* FROM governed_memories m WHERE {' AND '.join(clauses)} "
"ORDER BY m.updated_at DESC,m.memory_id LIMIT ?",
values,
).fetchall()
items = self._memories_from_rows(db, rows[:limit])
return items, len(rows) > limit or len(cleaned) > 32
def get(self, memory_id: str, *, user_id: str) -> GovernedMemory | None:
memory = _get_memory_direct(self.database_path, memory_id, user_id)
return memory if isinstance(memory, GovernedMemory) else None
def sources_behind_memories(
self,
*,
user_id: str,
memory_ids: Collection[str],
) -> dict[str, frozenset[str]]:
if not memory_ids:
return {}
identifiers = tuple(memory_ids)
placeholders = ",".join("?" for _ in identifiers)
with main_database(self.database_path).read(row_factory=sqlite3.Row) as db:
rows = db.execute(
"SELECT link.memory_id,e.source_id FROM governed_memory_evidence link "
"JOIN evidence e ON e.evidence_id=link.evidence_id "
f"WHERE link.active=1 AND e.user_id=? AND link.memory_id IN ({placeholders})",
(user_id, *identifiers),
).fetchall()
found: dict[str, set[str]] = {}
for row in rows:
found.setdefault(str(row["memory_id"]), set()).add(str(row["source_id"]))
return {key: frozenset(values) for key, values in found.items()}
@staticmethod
def _memories_from_rows(
db: sqlite3.Connection, rows: Sequence[sqlite3.Row]
) -> builtins.list[GovernedMemory]:
from one_memory.memory_governance import MemoryGovernanceStore
return MemoryGovernanceStore.memories_from_rows(db, builtins.list(rows))
def _conversation_hit(row: Any) -> dict[str, Any]:
"""One stored row as a search hit.
The author travels with the row. A client that is not told who wrote a row
has to invent an author, and the invention is always "the user", which
files an assistant turn under the person who was talking to it.
"""
return {
"evidence_id": row["evidence_id"],
"conversation_id": row["conversation_id"],
"source_id": row["source_id"],
"workspace": row["workspace"],
"project": row["project"],
"created_at": row["created_at"],
"excerpt": row["verbatim"][:200] if row["verbatim"] else "",
# Both columns are NOT NULL on `evidence`, so a stored row always
# knows; a summary of a whole conversation has no single author and
# never reaches here.
"role": row["role"],
"written_by": row["written_by"],
}
def create_reader_app(settings: ReaderSettings) -> FastAPI:
"""A read-only FastAPI application for the Reader process."""
_check_imports()
database_path = settings.data_dir / "one-memory.sqlite3"
if not database_path.exists():
raise FileNotFoundError(f"reader_database_missing: {database_path}")
# Validate schema version without writing.
from one_memory.memory_server.migrations import SCHEMA_TABLE
from one_memory.schema_gate import BUILD_SCHEMA_VERSION
with main_database(database_path).read() as db:
tables = {
str(row[0]) for row in db.execute("SELECT name FROM sqlite_master WHERE type='table'")
}
if SCHEMA_TABLE in tables:
row = db.execute(f"SELECT MAX(version) FROM {SCHEMA_TABLE}").fetchone()
found = int(row[0]) if row and row[0] is not None else 0
if found > BUILD_SCHEMA_VERSION:
raise RuntimeError(f"reader_schema_too_new: {found} > {BUILD_SCHEMA_VERSION}")
elif not tables:
raise RuntimeError("reader_database_empty")
access = SourceReadAccess(database_path)
forwarder = SideEffectForwarder(settings)
governance = _ReaderGovernance(database_path)
@asynccontextmanager
async def lifespan(_: FastAPI) -> AsyncIterator[None]:
monitor = asyncio.get_running_loop().create_task(_memory_monitor())
forwarder.start()
yield
monitor.cancel()
forwarder.stop()
async def _memory_monitor() -> None:
import resource
import sys
while True:
await asyncio.sleep(30)
usage = resource.getrusage(resource.RUSAGE_SELF).ru_maxrss
rss_bytes = usage if sys.platform == "darwin" else usage * 1024
if rss_bytes > MAX_READER_RSS_BYTES:
logger.error(
"reader_memory_limit_exceeded: %d bytes > %d",
rss_bytes,
MAX_READER_RSS_BYTES,
)
os._exit(1)
app = FastAPI(title="MemHub Reader", lifespan=lifespan)
app.state.settings = settings
app.state.database_path = database_path
expected_token = settings.api_token.get_secret_value()
def require_auth(request: Request) -> None:
import hmac
origin = request.headers.get("origin")
if origin is not None and origin not in (
"one-memory://app",
"http://localhost:5173",
):
raise HTTPException(403, "origin_not_allowed")
authorization = request.headers.get("authorization", "")
scheme, _, token = authorization.partition(" ")
if not (scheme.lower() == "bearer" and hmac.compare_digest(token, expected_token)):
raise HTTPException(401, "invalid_local_token")
protected = [Depends(require_auth)]
@app.get("/api/v1/home-summary", response_model=HomeSummary, dependencies=protected)
def home_summary() -> HomeSummary:
try:
return read_home_summary(database_path)
except HomeSummaryUnavailable:
# Belt and braces: read_home_summary answers stale itself, but if
# that ever changes, never let the homepage 500.
return HomeSummary(
memory_count=0, latest=None, observed_at=datetime.now(UTC), stale=True
)
@app.get("/api/v1/memory-library", response_model=MemoryLibraryPage, dependencies=protected)
def memory_library(
query: str | None = Query(default=None, max_length=512),
status: str | None = None,
active_only: bool = False,
source: str | None = Query(default=None, min_length=1),
offset: int = Query(default=0, ge=0),
limit: int = Query(default=100, ge=1, le=500),
) -> MemoryLibraryPage:
from one_memory.memory_governance import MemoryStatus as MS
ms = MS(status) if status else None
return read_memory_library(
database_path,
user_id="local",
query=query,
status=ms,
active_only=active_only,
source=source,
offset=offset,
limit=limit,
)
@app.get("/api/v1/memories", dependencies=protected)
def list_memories(
status: str | None = None,
query: str | None = None,
limit: int = Query(default=100, ge=1, le=500),
offset: int = Query(default=0, ge=0),
) -> list[dict[str, Any]]:
from one_memory.memory_governance import MemoryStatus as MS
ms = MS(status) if status else None
memories = governance.list(
user_id="local", status=ms, query=query, limit=limit, offset=offset
)
visible = access.memories(memories)
return [m.model_dump(mode="json") for m in visible]
@app.get("/api/v1/memories/graph", dependencies=protected)
async def memory_graph(
limit: int = Query(default=160, ge=1, le=300),
) -> dict[str, Any]:
memories = governance.list(user_id="local", status=MemoryStatus.CONFIRMED, limit=500)
projected = access.memories(memories[:limit])
nodes: dict[str, dict[str, Any]] = {}
edges: list[dict[str, Any]] = []
source_ids: set[str] = set()
with main_database(database_path).read(row_factory=sqlite3.Row) as db:
for memory in projected:
node_id = f"memory:{memory.memory_id}"
nodes[node_id] = {
"node_id": node_id,
"node_type": "memory",
"label": memory.content,
"memory_id": memory.memory_id,
"kind": memory.kind.value
if hasattr(memory.kind, "value")
else str(memory.kind),
"status": memory.status.value
if hasattr(memory.status, "value")
else str(memory.status),
}
evidence_ids = list(memory.evidence_ids)
if evidence_ids:
placeholders = ",".join("?" for _ in evidence_ids)
sql = (
f"SELECT DISTINCT source_id FROM evidence"
f" WHERE evidence_id IN ({placeholders})"
)
for row in db.execute(
sql,
evidence_ids,
):
sid = str(row["source_id"])
source_ids.add(sid)
for sid in sorted(source_ids):
source_node_id = f"source:{sid}"
nodes[source_node_id] = {
"node_id": source_node_id,
"node_type": "source",
"label": sid,
"source_id": sid,
}
for memory in projected:
if any(_source_for_evidence(db, eid) == sid for eid in memory.evidence_ids):
memory_node_id = f"memory:{memory.memory_id}"
edges.append(
{
"edge_id": f"{source_node_id}->{memory_node_id}",
"source_node_id": source_node_id,
"target_node_id": memory_node_id,
}
)
# Fetch HY graph overlay from Core (not available in Reader).
hy_overlay = None
with contextlib.suppress(Exception):
hy_overlay = await _core_get(
settings,
"/internal/graph-snapshot",
timeout_s=5.0,
params={"limit": limit},
)
if hy_overlay and not hy_overlay.get("consulted", False):
hy_overlay = None
return {
"limit": limit,
"total": len(projected),
"nodes": tuple(nodes.values()),
"edges": edges,
**({"hy_overlay": hy_overlay} if hy_overlay else {}),
}
@app.get("/api/v1/memories/{memory_id}", dependencies=protected)
def get_memory(memory_id: str) -> dict[str, Any]:
memory = _get_memory_direct(database_path, memory_id, "local")
if memory is None:
raise HTTPException(404, "memory_not_found")
if not access.memories([memory]):
raise HTTPException(403, "memory_source_revoked")
return cast(dict[str, Any], memory.model_dump(mode="json"))
@app.post("/api/v1/recall", response_model=None, dependencies=protected)
async def recall(request: RecallRequest) -> dict[str, Any]:
from one_memory.recall import GovernedRecallTarget
started = time.monotonic()
target = GovernedRecallTarget(
governance=governance,
semantic_search=None,
resolve_workspace=None,
filter_memories=access.memories,
)
if request.query == _PROBE_QUERY:
return target.recall(request, user_id="local").model_dump(mode="json")
deadline = time.monotonic() + request.timeout_ms / 1_000
semantic_ids: tuple[str, ...] = ()
try:
remaining = max(0.1, deadline - time.monotonic())
result = await asyncio.wait_for(
_core_get(
settings,
"/internal/semantic-ids",
timeout_s=remaining,
params={"query": request.query, "limit": request.limit},
),
timeout=remaining,
)
semantic_ids = tuple(str(value) for value in result.get("ids", []) if value)
except Exception:
pass
readable = target._readable_scopes(request)
# Keyword candidates from read-only SQL.
keywords = target._keyword_candidates(
request, user_id="local", readable=readable, deadline=deadline
)
receipt = target._rank(
request,
user_id="local",
readable=readable,
semantic_hits={
memory_id: SemanticMemoryHit(
memory_id,
1.0 - index / max(1, len(semantic_ids)),
)
for index, memory_id in enumerate(semantic_ids)
},
keywords=keywords
if isinstance(keywords, _KeywordCandidates)
else _KeywordCandidates((), False, False),
deadline=deadline,
timed_out=False,
)
forwarder.enqueue(
{
"tool": "om_recall",
"query": request.query,
"scope": request.scope,
"state": receipt.state.value,
"memory_count": len(receipt.hits),
"elapsed_ms": (time.monotonic() - started) * 1000,
}
)
return receipt.model_dump(mode="json")
@app.get("/api/v1/conversations/search", dependencies=protected)
def search_conversations(
query: str = Query(min_length=1, max_length=512),
limit: int = Query(default=20, ge=1, le=100),
) -> dict[str, Any]:
if not database_path.exists():
raise HTTPException(503, "conversation_search_unavailable")
with main_database(database_path).read(row_factory=sqlite3.Row) as db:
try:
rows = db.execute(
"""SELECT * FROM evidence
WHERE user_id = 'local' AND verbatim LIKE ? ESCAPE '\\'
ORDER BY imported_at DESC LIMIT ?""",
(f"%{query}%", limit),
).fetchall()
except Exception:
raise HTTPException(503, "conversation_search_unavailable") from None
hits = [_conversation_hit(row) for row in rows]
return {
"hits": hits,
"empty_reason": None if hits else "no_matches",
"imported_messages": len(hits),
}
@app.get("/health")
def health() -> dict[str, str]:
return {"status": "ok", "service": "one-memory-reader"}
return app
def _get_memory_direct(database_path: Path, memory_id: str, user_id: str) -> Any:
from one_memory.memory_governance import _memory_from_row, _memory_links
with main_database(database_path).read(row_factory=sqlite3.Row) as db:
row = db.execute(
"SELECT * FROM governed_memories WHERE memory_id = ? AND user_id = ?",
(memory_id, user_id),
).fetchone()
if row is None:
return None
links = _memory_links(db, [memory_id])
return _memory_from_row(row, *(tuple(vals) for vals in links[memory_id]))
def _source_for_evidence(db: sqlite3.Connection, evidence_id: str) -> str | None:
row = db.execute(
"SELECT source_id FROM evidence WHERE evidence_id = ?", (evidence_id,)
).fetchone()
return str(row["source_id"]) if row else None
@@ -41,6 +41,20 @@ class MigrationProgress:
phase: str
def _ledger_log(database_path: Path, event: str, migration_id: str, **details: object) -> None:
"""Every ledger transition goes to `sdk-migration.log`; never raises."""
from one_memory.sdk_migration_log import write_migration_log
write_migration_log(
database_path.parent,
writer="core-sdk-migrate",
event=event,
outcome="succeeded",
migration_id=migration_id,
details=details,
)
class SdkDataPlaneMigrationStore:
def __init__(self, database_path: Path) -> None:
self.database_path = database_path
@@ -151,7 +165,16 @@ class SdkDataPlaneMigrationStore:
elif backup_bundle_id and existing["backup_bundle_id"] != backup_bundle_id:
raise DataPlaneMigrationError("sdk_data_plane_migration_bundle_conflict")
connection.execute("COMMIT")
return self.progress(migration_id)
progress = self.progress(migration_id)
_ledger_log(
self.database_path,
"ledger.started",
migration_id,
created=existing is None,
state=progress.state,
phase=progress.phase,
)
return progress
def set_phase(self, migration_id: str, *, phase: str, state: str = "running") -> None:
with main_database(self.database_path).write(autocommit=True) as connection:
@@ -164,6 +187,7 @@ class SdkDataPlaneMigrationStore:
if cursor.rowcount != 1:
raise DataPlaneMigrationError("sdk_data_plane_migration_not_mutable")
connection.execute("COMMIT")
_ledger_log(self.database_path, "ledger.phase_set", migration_id, phase=phase, state=state)
def progress(self, migration_id: str) -> MigrationProgress:
with main_database(self.database_path).read(row_factory=sqlite3.Row) as connection:
@@ -257,6 +281,7 @@ class SdkDataPlaneMigrationStore:
(migration_id, now),
)
connection.execute("COMMIT")
_ledger_log(self.database_path, "ledger.authority_committed", migration_id, authority="sdk")
marker = self.cutover()
if marker is None:
raise DataPlaneMigrationError("sdk_data_plane_cutover_missing")
@@ -0,0 +1,243 @@
"""The one log support asks for when an upgrade did not reach SDK authority.
`<data_dir>/logs/sdk-migration.log` is JSON Lines, shared with the TypeScript
orchestrator (`packages/runtime-control/src/sdk-migration-log.ts`), which writes
the same line shape. A person attaches this single file to a report, so every
line says who wrote it, which build, which migration and what happened.
It must never carry user content: no memory or conversation text, no provider
response, no exception message (a message can quote either). Errors are logged
as their type, a sanitised code and traceback frames as `file:line:function`.
Detail values are numbers, booleans and short identifier-shaped strings only;
anything else is dropped rather than escaped.
Writing is synchronous and flushed: a migration is rare and the line that
explains a failure is worth more than the milliseconds. A write failure is
swallowed, because logging must never be the reason a migration or a startup
fails.
"""
from __future__ import annotations
import json
import os
import re
import traceback
from collections.abc import Iterator, Mapping
from datetime import UTC, datetime
from pathlib import Path
from typing import Any
LOG_NAME = "sdk-migration.log"
#: Rotated to `sdk-migration.log.1` beyond this; one generation is kept.
MAX_BYTES = 4 * 1024 * 1024
_SAFE_TEXT = re.compile(r"^[A-Za-z0-9_.:/@+\-]{0,160}$")
_CODE = re.compile(r"^[a-z][a-z0-9]*(?:[._-][a-z0-9]+){0,15}$")
_SUBJECT = re.compile(r"^[A-Za-z0-9_.\-]{1,96}$")
_MAX_ITEMS = 32
_MAX_DEPTH = 4
_MAX_FRAMES = 24
_FRAME = re.compile(r"^[A-Za-z0-9_.\-]{1,96}:\d{1,7}:<?[A-Za-z0-9_]{1,96}>?$")
def log_path(data_dir: Path) -> Path:
return data_dir / "logs" / LOG_NAME
def _safe(value: object, depth: int = 0) -> Any:
if value is None or isinstance(value, bool):
return value
if isinstance(value, int):
return value
if isinstance(value, float):
return round(value, 3) if value == value and abs(value) != float("inf") else None
if isinstance(value, str):
return value if _SAFE_TEXT.fullmatch(value) else "<redacted>"
if depth >= _MAX_DEPTH:
return "<nested>"
if isinstance(value, Mapping):
result: dict[str, Any] = {}
for index, (key, item) in enumerate(value.items()):
if index >= _MAX_ITEMS:
result["<truncated>"] = len(value) - _MAX_ITEMS
break
if isinstance(key, str) and _SAFE_TEXT.fullmatch(key):
result[key] = _safe(item, depth + 1)
return result
if isinstance(value, (list, tuple, set, frozenset)):
items = list(value)
safe = [_safe(item, depth + 1) for item in items[:_MAX_ITEMS]]
if len(items) > _MAX_ITEMS:
safe.append(f"<+{len(items) - _MAX_ITEMS}>")
return safe
return "<unsupported>"
def _chain(error: BaseException) -> Iterator[BaseException]:
seen: set[int] = set()
current: BaseException | None = error
while current is not None and id(current) not in seen:
seen.add(id(current))
yield current
current = current.__cause__ or current.__context__
def describe_error(error: BaseException) -> dict[str, object]:
"""Type, code and frames of an exception chain; never its message."""
chain: list[dict[str, object]] = []
for item in _chain(error):
name = type(item).__name__
entry: dict[str, object] = {"type": name if _SAFE_TEXT.fullmatch(name) else "<unknown>"}
code = getattr(item, "code", None)
if isinstance(code, str):
# `<code>:<opaque id>` names the row that failed, e.g. a write job.
name, _, subject = code.partition(":")
if _CODE.fullmatch(name):
entry["code"] = name
if subject and _SUBJECT.fullmatch(subject):
entry["subject"] = subject
message = str(item)
# A message that is itself a stable code (the codebase's convention
# for domain errors) is safe to keep; anything else is content.
if _CODE.fullmatch(message):
entry["code"] = entry.get("code") or message
errno = getattr(item, "errno", None)
if isinstance(errno, int):
entry["errno"] = errno
if isinstance(item, OSError) and item.filename is not None:
# A path names a file, not its contents, but home directories can
# name a person: keep only the file name.
entry["file"] = Path(str(item.filename)).name
frames = traceback.extract_tb(item.__traceback__)[-_MAX_FRAMES:]
# Code locations, not content: built here from file names, line
# numbers and function names (`<module>`, `<lambda>` included).
entry["frames"] = [
location
for frame in frames
if _FRAME.fullmatch(
location := f"{Path(frame.filename).name}:{frame.lineno}:{frame.name}"
)
]
chain.append(entry)
if len(chain) >= 6:
break
return {"chain": chain}
def _build() -> dict[str, object]:
from one_memory import __version__
build: dict[str, object] = {"core_version": __version__}
for key, name in (
("ONE_MEMORY_VERSION", "app_version"),
("ONE_MEMORY_BUILD_NUMBER", "build_number"),
("ONE_MEMORY_COMMIT", "build_commit"),
("ONE_MEMORY_MIGRATION_CALLER", "caller"),
("ONE_MEMORY_MIGRATION_RUNTIME_BUILD", "runtime_build"),
):
value = os.environ.get(key, "").strip()
if value:
build[name] = value
return _safe(build)
def write_migration_log(
data_dir: Path,
*,
writer: str,
event: str,
outcome: str,
migration_id: str | None = None,
details: Mapping[str, object] | None = None,
error: BaseException | None = None,
code: str | None = None,
) -> None:
"""Append one line. Never raises."""
try:
line: dict[str, object] = {
"schema_version": 1,
"ts": datetime.now(UTC).isoformat(),
"pid": os.getpid(),
"writer": writer,
"event": event,
"outcome": outcome,
**_build(),
}
if migration_id is not None:
line["migration_id"] = migration_id
if code is not None:
line["code"] = code
if details:
line["details"] = dict(details)
safe_line = _safe(line)
if not isinstance(safe_line, dict):
return
if error is not None:
# Already content-free by construction; `_safe` would redact the
# `<module>` frames.
safe_line["error"] = describe_error(error)
destination = log_path(data_dir)
destination.parent.mkdir(parents=True, exist_ok=True, mode=0o700)
try:
if destination.stat().st_size > MAX_BYTES:
os.replace(destination, destination.with_name(f"{LOG_NAME}.1"))
except FileNotFoundError:
pass
payload = (json.dumps(safe_line, ensure_ascii=False) + "\n").encode("utf-8")
descriptor = os.open(destination, os.O_WRONLY | os.O_APPEND | os.O_CREAT, 0o600)
try:
os.write(descriptor, payload)
os.fsync(descriptor)
finally:
os.close(descriptor)
except Exception: # noqa: BLE001 - logging must never break a migration
return
def migration_rows(database_path: Path) -> list[dict[str, object]]:
"""The recorded migration attempts, newest first; ids, states and phases only."""
import sqlite3
try:
connection = sqlite3.connect(f"file:{database_path}?mode=ro", uri=True, timeout=1)
except sqlite3.Error:
return []
try:
rows = connection.execute(
"SELECT migration_id,state,phase,created_at,updated_at "
"FROM sdk_data_plane_migrations ORDER BY created_at DESC LIMIT 8"
).fetchall()
except sqlite3.Error:
return []
finally:
connection.close()
return [
{
"migration_id": row[0],
"state": row[1],
"phase": row[2],
# Older builds stored `YYYY-MM-DD HH:MM:SS`; keep it identifier-shaped.
"created_at": str(row[3]).replace(" ", "T"),
"updated_at": str(row[4]).replace(" ", "T"),
}
for row in rows
]
def authority_snapshot(database_path: Path) -> dict[str, object]:
"""Authority and recorded attempts, read without changing the database."""
if not database_path.is_file():
return {"database": "absent"}
snapshot: dict[str, object] = {"database": "present"}
try:
from one_memory.sdk_data_plane_migration import SdkDataPlaneMigrationStore
authority, authority_migration = SdkDataPlaneMigrationStore(database_path).authority_state()
snapshot["authority"] = authority.value
snapshot["authority_migration_id"] = authority_migration
except Exception as error: # noqa: BLE001 - a snapshot must not fail its caller
snapshot["authority"] = "unreadable"
snapshot["authority_error"] = type(error).__name__
snapshot["migrations"] = migration_rows(database_path)
return snapshot
@@ -14,6 +14,7 @@ import json
import os
import shutil
import sqlite3
import time
from collections.abc import Callable, Iterator
from contextlib import AbstractContextManager, contextmanager
from dataclasses import dataclass
@@ -63,6 +64,71 @@ from one_memory.target_schema import drop_legacy_tables, legacy_tables_present
PREPARED_PHASE = "prepared"
class _StepLog:
"""Each runner step, its duration, its counts and how it ended.
The CLI only sees the exception that escapes `run_startup_migration` and
names it with a generic code when it is not one of ours; these lines say
which step raised it. Content-free: only names, counts and codes.
"""
def __init__(self, data_dir: Path) -> None:
self.data_dir = data_dir
self.migration_id: str | None = None
def check(self, step: str, code: str | None) -> None:
"""One preflight check that is a decision, not a call: its result code."""
from one_memory.sdk_migration_log import write_migration_log
write_migration_log(
self.data_dir,
writer="core-sdk-migrate",
event="step.failed" if code else "step.finished",
outcome="failed" if code else "succeeded",
migration_id=self.migration_id,
code=code,
details={"step": step, "result": code or "passed"},
)
@contextmanager
def step(self, name: str) -> Iterator[dict[str, object]]:
from one_memory.sdk_migration_log import write_migration_log
started = time.monotonic()
details: dict[str, object] = {}
try:
yield details
except BaseException as error:
code = getattr(error, "code", None)
write_migration_log(
self.data_dir,
writer="core-sdk-migrate",
event="step.failed",
outcome="failed",
migration_id=self.migration_id,
code=code if isinstance(code, str) else None,
error=error,
details={
"step": name,
"duration_ms": (time.monotonic() - started) * 1000,
**details,
},
)
raise
write_migration_log(
self.data_dir,
writer="core-sdk-migrate",
event="step.finished",
outcome="succeeded",
migration_id=self.migration_id,
details={
"step": name,
"duration_ms": (time.monotonic() - started) * 1000,
**details,
},
)
@dataclass(frozen=True)
class CutoverPreparation:
"""What the cutover did to legacy work, for the receipt and the dialog."""
@@ -83,8 +149,7 @@ class CutoverPreparation:
def _tables(database: sqlite3.Connection) -> set[str]:
return {
str(row[0])
for row in database.execute("SELECT name FROM sqlite_master WHERE type='table'")
str(row[0]) for row in database.execute("SELECT name FROM sqlite_master WHERE type='table'")
}
@@ -128,9 +193,7 @@ def prepare_legacy_write_queue_for_cutover(database_path: Path) -> tuple[int, in
if "evidence" not in tables:
return (0, 0)
jobs = (
tuple(
database.execute("SELECT job_id,state,payload_json FROM memory_write_jobs")
)
tuple(database.execute("SELECT job_id,state,payload_json FROM memory_write_jobs"))
if "memory_write_jobs" in tables
else ()
)
@@ -281,15 +344,27 @@ def abandoned_legacy_work(database_path: Path) -> dict[str, int]:
return abandoned
def prepare_legacy_work_for_cutover(database_path: Path) -> CutoverPreparation:
"""Make the surviving queue self-contained. Safe to repeat."""
def prepare_legacy_work_for_cutover(
database_path: Path, trace: dict[str, object] | None = None
) -> CutoverPreparation:
"""Make the surviving queue self-contained. Safe to repeat.
`trace` (the migration log's step details) learns which sub-step is
running, so a failure names it.
"""
trace = {} if trace is None else trace
trace["sub_step"] = "write_queue"
frozen, moved = prepare_legacy_write_queue_for_cutover(database_path)
trace["sub_step"] = "pending_dispatches"
dispatches = move_pending_dispatches(database_path)
trace["sub_step"] = "abandoned_work"
abandoned = abandoned_legacy_work(database_path)
trace.pop("sub_step", None)
return CutoverPreparation(
frozen_jobs=frozen,
moved_tasks=moved,
moved_dispatches=dispatches,
abandoned=abandoned_legacy_work(database_path),
abandoned=abandoned,
)
@@ -478,14 +553,25 @@ def run_startup_migration(
return open_client(profile)
return runtime_sdk_client(database_path, data_dir, profile)
log = _StepLog(data_dir)
store = SdkDataPlaneMigrationStore(database_path)
authority, authority_migration = store.authority_state()
profile = read_active_profile(database_path)
with log.step("preflight.authority_read") as step:
authority, authority_migration = store.authority_state()
step["authority"] = authority.value
log.migration_id = authority_migration
with log.step("preflight.profile_read") as step:
profile = read_active_profile(database_path)
step["profile_present"] = profile is not None
if authority is MemoryAuthority.SDK:
vectors_waiting = profile is not None and bool(legacy_vector_collections(data_dir, profile))
leftovers = bool(legacy_tables_present(database_path)) or abandoned_staging_present(
data_dir=data_dir, namespace=namespace
)
with log.step("preflight.leftovers_scan") as step:
vectors_waiting = profile is not None and bool(
legacy_vector_collections(data_dir, profile)
)
leftovers = bool(legacy_tables_present(database_path)) or abandoned_staging_present(
data_dir=data_dir, namespace=namespace
)
step["vectors_waiting"] = vectors_waiting
step["leftovers"] = leftovers
if not vectors_waiting and not leftovers:
return StartupMigrationResult(state="not_required")
reattached: tuple[ReattachedCollection, ...] = ()
@@ -493,20 +579,28 @@ def run_startup_migration(
if vectors_waiting:
assert profile is not None
progress("vectors")
reattached = reattach_legacy_vectors(data_dir, profile)
legacy = legacy_memory_ids_from_bundle(data_dir, authority_migration)
with log.step("vectors.reattach") as step:
reattached = reattach_legacy_vectors(data_dir, profile)
step["collections"] = len(reattached)
with log.step("verify.legacy_ids_from_bundle") as step:
legacy = legacy_memory_ids_from_bundle(data_dir, authority_migration)
step["bundle_ids_present"] = legacy is not None
if legacy is not None and not legacy.empty:
# Authority already moved, so nothing is left to protect by
# refusing; record what the repaired home reads back.
progress("verify")
with client_for(profile) as client:
with log.step("verify.readback") as step, client_for(profile) as client:
readback = verify_readback(client, legacy)
step.update(readback.as_document())
dropped: tuple[str, ...] = ()
if leftovers:
progress("cleanup")
dropped = resume_legacy_cleanup(data_dir=data_dir, database_path=database_path)
with log.step("cleanup.drop_legacy_tables") as step:
dropped = resume_legacy_cleanup(data_dir=data_dir, database_path=database_path)
step["dropped_tables"] = len(dropped)
if dropped:
_vacuum(database_path)
with log.step("cleanup.vacuum"):
_vacuum(database_path)
return StartupMigrationResult(
state="complete",
migration_id=authority_migration,
@@ -520,63 +614,89 @@ def run_startup_migration(
from one_memory.memory_server.migration_runner import SqliteMigrationStore, migrate
from one_memory.memory_server.migrations import MIGRATIONS
migration_id, bundle_id = _resumable_identity(database_path)
legacy = legacy_memory_ids_from(database_path)
with log.step("preflight.resumable_identity") as step:
migration_id, bundle_id = _resumable_identity(database_path)
step["resumed"] = migration_id != f"{MIGRATION_PREFIX}-{database_generation(database_path)}"
log.migration_id = migration_id
with log.step("preflight.legacy_ids_scan") as step:
legacy = legacy_memory_ids_from(database_path)
step["chat"] = len(legacy.chat)
step["productivity"] = len(legacy.productivity)
step["users"] = len(legacy.users)
if profile is None and not legacy.empty:
# Without the embedding profile the Runtime's collections cannot be
# named, so neither reattached nor read back. Legacy stays intact.
log.check("preflight.model_profile_check", "sdk_migration_model_profile_required")
raise DataPlaneMigrationError("sdk_migration_model_profile_required")
log.check("preflight.model_profile_check", None)
with capture_fence(data_dir):
progress("bundle")
manifest = _ensure_bundle(
data_dir=data_dir,
bundle_id=bundle_id,
runtime_build=runtime_build,
)
with log.step("bundle.ensure") as step:
manifest = _ensure_bundle(
data_dir=data_dir,
bundle_id=bundle_id,
runtime_build=runtime_build,
)
step["bundle_id"] = bundle_id
progress("prepare")
asyncio.run(migrate(SqliteMigrationStore(database_path), MIGRATIONS))
close_main_databases()
recover_abandoned_staging(
data_dir=data_dir,
database_path=database_path,
namespace=namespace,
)
store.start(
migration_id=migration_id,
source_runtime_build=str(manifest.get("runtime_build") or ""),
target_sdk_version=sdk_version,
source_schema_version=MIGRATIONS[-1].version,
namespace_fingerprint=hashlib.sha256(namespace.encode()).hexdigest(),
backend_fingerprint=hashlib.sha256(b"chroma:kuzu:sqlite").hexdigest(),
backup_bundle_id=bundle_id,
)
preparation = prepare_legacy_work_for_cutover(database_path)
with log.step("prepare.schema_migrate") as step:
asyncio.run(migrate(SqliteMigrationStore(database_path), MIGRATIONS))
close_main_databases()
step["schema_version"] = MIGRATIONS[-1].version
with log.step("prepare.recover_abandoned_staging"):
recover_abandoned_staging(
data_dir=data_dir,
database_path=database_path,
namespace=namespace,
)
with log.step("prepare.ledger_start"):
store.start(
migration_id=migration_id,
source_runtime_build=str(manifest.get("runtime_build") or ""),
target_sdk_version=sdk_version,
source_schema_version=MIGRATIONS[-1].version,
namespace_fingerprint=hashlib.sha256(namespace.encode()).hexdigest(),
backend_fingerprint=hashlib.sha256(b"chroma:kuzu:sqlite").hexdigest(),
backup_bundle_id=bundle_id,
)
with log.step("prepare.legacy_work") as step:
preparation = prepare_legacy_work_for_cutover(database_path, step)
step.update(preparation.as_document())
reattached = ()
readback = None
if profile is not None:
progress("vectors")
reattached = reattach_legacy_vectors(data_dir, profile)
with log.step("vectors.reattach") as step:
reattached = reattach_legacy_vectors(data_dir, profile)
step["collections"] = len(reattached)
if not legacy.empty:
assert profile is not None
progress("verify")
with client_for(profile) as client:
readback = verify_readback(client, legacy)
if not readback.passed:
store.set_phase(migration_id, phase="readback_failed", state="blocked")
raise MemoryReadbackFailed(readback)
store.set_phase(migration_id, phase=PREPARED_PHASE, state="ready")
with log.step("verify.readback") as step:
with client_for(profile) as client:
readback = verify_readback(client, legacy)
step.update(readback.as_document())
if not readback.passed:
store.set_phase(migration_id, phase="readback_failed", state="blocked")
raise MemoryReadbackFailed(readback)
with log.step("prepare.mark_prepared"):
store.set_phase(migration_id, phase=PREPARED_PHASE, state="ready")
progress("cutover")
commit_sdk_cutover(
data_dir=data_dir,
database_path=database_path,
migration_id=migration_id,
bundle_id=bundle_id,
sdk_version=sdk_version,
)
with log.step("cutover.commit"):
commit_sdk_cutover(
data_dir=data_dir,
database_path=database_path,
migration_id=migration_id,
bundle_id=bundle_id,
sdk_version=sdk_version,
)
progress("cleanup")
dropped = resume_legacy_cleanup(data_dir=data_dir, database_path=database_path)
with log.step("cleanup.drop_legacy_tables") as step:
dropped = resume_legacy_cleanup(data_dir=data_dir, database_path=database_path)
step["dropped_tables"] = len(dropped)
if dropped:
_vacuum(database_path)
with log.step("cleanup.vacuum"):
_vacuum(database_path)
return StartupMigrationResult(
state="cutover",
migration_id=migration_id,
@@ -1,518 +0,0 @@
"""Per-session incremental read. A pull or an ack is not adoption."""
import json
import secrets
import sqlite3
from collections.abc import Iterator
from contextlib import contextmanager
from dataclasses import dataclass
from pathlib import Path
#: Schema owned by `memory_server.migrations` (migration 11). No local copy.
from one_memory.change_outbox import evidence_event_id
from one_memory.database_access import main_database
from one_memory.derived_echo import echoed_event_ids, is_pure_echo
from one_memory.memory_server.migrations import session_inbox_ddl
from one_memory.session_registry import SessionRegistry
_SCHEMA = session_inbox_ddl("sqlite")
class CursorRejected(Exception):
pass
@dataclass(frozen=True)
class InboxEvent:
event_id: str
evidence_id: str | None
evidence_revision: int | None
source_id: str | None
@dataclass(frozen=True)
class InboxPage:
batch_id: str
cursor: str
state: str
adoption: str
events: tuple[InboxEvent, ...]
@dataclass(frozen=True)
class InboxPosition:
"""Where a cursor would stand once the events handed over are confirmed."""
event_id: str | None
permission_epoch: int
handed_event_ids: frozenset[str] = frozenset()
receipt_id: str | None = None
@dataclass(frozen=True)
class InboxPeek:
events: tuple[InboxEvent, ...]
position: InboxPosition
@dataclass(frozen=True)
class InboxAck:
batch_id: str
state: str
adoption: str
@dataclass(frozen=True)
class InboxSignal:
signal_id: str
class SessionInbox:
def __init__(self, database_path: Path) -> None:
self._path = database_path
self._registry = SessionRegistry(database_path)
def initialize(self) -> None:
with main_database(self._path).write() as db, db:
db.executescript(_SCHEMA)
def subscribe(
self,
session_key: str,
sources: tuple[str, ...],
permission_epoch: int,
) -> None:
self._registry.require(session_key)
with main_database(self._path).write() as db, db:
db.execute(
"""
INSERT INTO session_subscriptions (
session_key, granted_sources, permission_epoch
) VALUES (?, ?, ?)
ON CONFLICT(session_key) DO UPDATE SET
granted_sources = excluded.granted_sources,
permission_epoch = excluded.permission_epoch
""",
(session_key, json.dumps(list(sources)), permission_epoch),
)
def pull(self, session_key: str, cursor: str | None, limit: int) -> InboxPage:
self._registry.require(session_key)
with self._snapshot() as db:
start_after, epoch = self._cursor_position(session_key, cursor, db)
granted = self._granted(session_key, db)
events = self._without_host_echo(self._events(db), db)
current = self._epoch(session_key, db)
selected, found = self._scan(granted, start_after, limit, events)
state = "delivered"
# A cursor whose event is gone, or whose permission epoch no longer
# matches, must not look like "nothing new". Rebuild from the start.
if cursor is not None and (not found or epoch != current):
selected, _found = self._scan(granted, None, limit, events)
state = "resync-required"
epoch = current
if selected:
scanned_id = selected[-1].event_id
elif state == "resync-required":
scanned_id = None
else:
scanned_id = start_after
return self._store_page(session_key, scanned_id, epoch, state, selected)
def peek(self, session_key: str, limit: int) -> InboxPeek:
"""What the next pull from the latest cursor would return, without moving it."""
self._registry.require(session_key)
# Grants, epoch and events are read in one snapshot: a revoke landing
# between them would pair the old grants with the new epoch.
with self._snapshot() as db:
cursor = self._latest_cursor(session_key, db)
start_after, epoch = self._cursor_position(session_key, cursor, db)
granted = self._granted(session_key, db)
events = self._without_host_echo(self._events(db), db)
current = self._epoch(session_key, db)
selected, found = self._scan(granted, start_after, limit, events)
if cursor is not None and (not found or epoch != current):
selected, _found = self._scan(granted, None, limit, events)
start_after = None
end = selected[-1].event_id if selected else start_after
receipt_id = None
if selected:
receipt_id = secrets.token_hex(16)
receipt = self._new_receipt(
session_key,
position_event_id=str(end),
permission_epoch=current,
handed_event_ids=tuple(event.event_id for event in selected),
)
with main_database(self._path).write() as db, db:
db.execute(
"""
INSERT INTO session_snapshots (
snapshot_token, session_key, frozen_ids
)
VALUES (?, ?, ?)
""",
(receipt_id, session_key, receipt),
)
return InboxPeek(
tuple(selected),
InboxPosition(
end,
current,
frozenset(event.event_id for event in selected),
receipt_id,
),
)
@staticmethod
def _new_receipt(
session_key: str,
*,
position_event_id: str,
permission_epoch: int,
handed_event_ids: tuple[str, ...],
) -> str:
return json.dumps(
{
"kind": "peek",
"session_key": session_key,
"event_id": position_event_id,
"permission_epoch": permission_epoch,
"handed_event_ids": list(handed_event_ids),
},
separators=(",", ":"),
)
def advance_to(self, session_key: str, position: InboxPosition) -> bool | None:
"""Move the latest cursor forward to a peeked position, never back.
Two rounds may peek overlapping pages and confirm in either order. The
cursor only ever moves to the furthest position actually handed over.
"""
self._registry.require(session_key)
if position.event_id is None:
return False
if position.receipt_id is None:
raise CursorRejected("missing_peek_receipt")
# IMMEDIATE takes the file's write lock at once: the confirming process
# is not the one that peeked, so the in-process writer queue is not enough.
with main_database(self._path).write() as db, db:
db.execute("BEGIN IMMEDIATE")
events = self._without_host_echo(self._events(db), db)
order = [event.event_id for event in events]
by_id = {event.event_id: event for event in events}
receipt = self._peek_receipt(session_key, position, db)
row = db.execute(
"SELECT permission_epoch FROM session_subscriptions WHERE session_key = ?",
(session_key,),
).fetchone()
epoch = int(row[0]) if row is not None else 0
handed = receipt
if any(event_id not in order for event_id in handed):
raise CursorRejected("handed_event_missing")
granted = self._granted(session_key, db)
target = by_id.get(position.event_id)
if target is None or target.source_id not in granted:
raise CursorRejected("target_not_selectable")
if any(by_id[event_id].source_id not in granted for event_id in handed):
raise CursorRejected("handed_event_not_selectable")
if position.permission_epoch != epoch:
return None
latest_token = self._latest_cursor(session_key, db)
cursor_position = -1
if latest_token is not None:
start_after, _cursor_epoch = self._cursor_position(session_key, latest_token, db)
if start_after in order:
cursor_position = order.index(str(start_after))
target_position = order.index(position.event_id)
if cursor_position >= target_position:
return None
required = [
event.event_id
for event in events[cursor_position + 1 : target_position + 1]
if event.source_id in granted
]
handed_after_cursor = [
event_id for event_id in handed if order.index(event_id) > cursor_position
]
if handed_after_cursor != required:
raise CursorRejected("incomplete_handed_range")
latest = db.execute(
"""
SELECT scanned_event_id, permission_epoch FROM session_inbox_cursors
WHERE session_key = ? ORDER BY rowid DESC LIMIT 1
""",
(session_key,),
).fetchone()
if (
latest is not None
and int(latest[1]) == epoch
and latest[0] in order
and order.index(latest[0]) > order.index(position.event_id)
):
return None
db.execute(
"""
INSERT INTO session_inbox_cursors (
cursor_token, session_key, scanned_event_id, permission_epoch
) VALUES (?, ?, ?, ?)
""",
(secrets.token_hex(32), session_key, position.event_id, epoch),
)
return True
@staticmethod
def _peek_receipt(
session_key: str, position: InboxPosition, db: sqlite3.Connection
) -> tuple[str, ...]:
row = db.execute(
"""
SELECT frozen_ids FROM session_snapshots
WHERE snapshot_token = ? AND session_key = ?
""",
(position.receipt_id, session_key),
).fetchone()
if row is None:
raise CursorRejected("invalid_peek_receipt")
try:
receipt = json.loads(str(row[0]))
except json.JSONDecodeError as error:
raise CursorRejected("invalid_peek_receipt") from error
if not isinstance(receipt, dict) or receipt.get("kind") != "peek":
raise CursorRejected("invalid_peek_receipt")
if receipt.get("session_key") != session_key:
raise CursorRejected("peek_receipt_session_mismatch")
if receipt.get("event_id") != position.event_id:
raise CursorRejected("peek_receipt_target_mismatch")
if receipt.get("permission_epoch") != position.permission_epoch:
raise CursorRejected("peek_receipt_epoch_mismatch")
handed = receipt.get("handed_event_ids")
if not isinstance(handed, list) or any(not isinstance(item, str) for item in handed):
raise CursorRejected("invalid_peek_receipt")
loaded = tuple(handed)
if frozenset(loaded) != position.handed_event_ids:
raise CursorRejected("peek_receipt_handed_mismatch")
return loaded
def latest_cursor(self, session_key: str) -> str | None:
self._registry.require(session_key)
with main_database(self._path).read() as db:
return self._latest_cursor(session_key, db)
@staticmethod
def _latest_cursor(session_key: str, db: sqlite3.Connection) -> str | None:
row = db.execute(
"""
SELECT cursor_token FROM session_inbox_cursors
WHERE session_key = ? ORDER BY rowid DESC LIMIT 1
""",
(session_key,),
).fetchone()
return None if row is None else str(row[0])
@contextmanager
def _snapshot(self) -> Iterator[sqlite3.Connection]:
"""One read transaction; under WAL every read in it sees the same state."""
# A pooled reader does not open a transaction for a SELECT on its own;
# the transaction is rolled back when the connection is handed back.
with main_database(self._path).read() as db:
db.execute("BEGIN")
yield db
def ack(self, session_key: str, batch_id: str) -> InboxAck:
self._registry.require(session_key)
with main_database(self._path).write() as db, db:
row = db.execute(
"SELECT session_key FROM session_inbox_batches WHERE batch_id = ?",
(batch_id,),
).fetchone()
if row is None or row[0] != session_key:
raise CursorRejected(batch_id)
db.execute(
"""
UPDATE session_inbox_batches SET state = 'acknowledged'
WHERE batch_id = ?
""",
(batch_id,),
)
return InboxAck(batch_id, "acknowledged", "unknown")
def begin_snapshot(self, session_key: str) -> str:
self._registry.require(session_key)
frozen = json.dumps([event.event_id for event in self._events()])
token = secrets.token_hex(16)
with main_database(self._path).write() as db, db:
db.execute(
"""
INSERT INTO session_snapshots (snapshot_token, session_key, frozen_ids)
VALUES (?, ?, ?)
""",
(token, session_key, frozen),
)
return token
def finish_snapshot(self, session_key: str, token: str) -> InboxPage:
self._registry.require(session_key)
with main_database(self._path).read() as db:
db.row_factory = sqlite3.Row
row = db.execute(
"SELECT session_key FROM session_snapshots WHERE snapshot_token = ?",
(token,),
).fetchone()
if row is None or row["session_key"] != session_key:
raise CursorRejected(token)
# Events committed before finish belong in this page. The cursor stops
# at the last event actually returned, so a late insert is not skipped.
return self.pull(session_key, cursor=None, limit=128)
def signal(self, session_key: str) -> InboxSignal:
self._registry.require(session_key)
return InboxSignal(secrets.token_hex(16))
def _cursor_position(
self, session_key: str, cursor: str | None, db: sqlite3.Connection
) -> tuple[str | None, int]:
if cursor is None:
return None, self._epoch(session_key, db)
row = db.execute(
"""
SELECT session_key, scanned_event_id, permission_epoch
FROM session_inbox_cursors WHERE cursor_token = ?
""",
(cursor,),
).fetchone()
if row is None or row[0] != session_key:
raise CursorRejected(cursor)
return row[1], int(row[2])
@staticmethod
def _scan(
granted: set[str],
start_after: str | None,
limit: int,
events: tuple[InboxEvent, ...],
) -> tuple[list[InboxEvent], bool]:
found = start_after is None
selected: list[InboxEvent] = []
for event in events:
if not found:
if event.event_id == start_after:
found = True
continue
if event.source_id not in granted:
continue
if len(selected) >= limit:
break
selected.append(event)
return selected, found
def _store_page(
self,
session_key: str,
scanned_id: str | None,
epoch: int,
state: str,
selected: list[InboxEvent],
) -> InboxPage:
token = secrets.token_hex(32)
batch_id = secrets.token_hex(16)
with main_database(self._path).write() as db, db:
db.execute(
"""
INSERT INTO session_inbox_cursors (
cursor_token, session_key, scanned_event_id, permission_epoch
) VALUES (?, ?, ?, ?)
""",
(token, session_key, scanned_id, epoch),
)
db.execute(
"""
INSERT INTO session_inbox_batches (batch_id, session_key, state)
VALUES (?, ?, ?)
""",
(batch_id, session_key, state),
)
return InboxPage(batch_id, token, state, "unknown", tuple(selected))
@staticmethod
def _epoch(session_key: str, db: sqlite3.Connection) -> int:
row = db.execute(
"SELECT permission_epoch FROM session_subscriptions WHERE session_key = ?",
(session_key,),
).fetchone()
return int(row[0]) if row is not None else 0
@staticmethod
def _granted(session_key: str, db: sqlite3.Connection) -> set[str]:
row = db.execute(
"SELECT granted_sources FROM session_subscriptions WHERE session_key = ?",
(session_key,),
).fetchone()
if row is None:
return set()
loaded = json.loads(row[0])
return set(loaded) if isinstance(loaded, list) else set()
def _events(self, db: sqlite3.Connection | None = None) -> tuple[InboxEvent, ...]:
if db is None:
with main_database(self._path).read() as own:
return self._events(own)
events: dict[str, tuple[str, InboxEvent]] = {}
for evidence_id, revision, source_id, created_at in db.execute(
"SELECT evidence_id, revision, source_id, updated_at FROM evidence"
):
event_id = evidence_event_id(str(evidence_id), int(revision))
events[event_id] = (
str(created_at),
InboxEvent(event_id, str(evidence_id), int(revision), str(source_id)),
)
for (
event_id,
evidence_id,
revision,
source_id,
created_at,
) in db.execute(
"""
SELECT o.event_id, o.evidence_id, o.evidence_revision, e.source_id, o.created_at
FROM change_outbox o
LEFT JOIN evidence e ON e.evidence_id = o.evidence_id
"""
):
events.setdefault(
str(event_id),
(str(created_at), InboxEvent(str(event_id), evidence_id, revision, source_id)),
)
return tuple(
event
for _occurred_at, event in sorted(
events.values(), key=lambda item: (item[0], item[1].event_id)
)
)
@staticmethod
def _without_host_echo(
events: tuple[InboxEvent, ...], db: sqlite3.Connection
) -> tuple[InboxEvent, ...]:
"""The host can save our notice back; that is not a new decision."""
known = {event.event_id for event in events}
delivered: set[str] = set()
kept: list[InboxEvent] = []
for event in events:
if event.evidence_id is None:
kept.append(event)
continue
row = db.execute(
"SELECT verbatim FROM evidence WHERE evidence_id = ?",
(event.evidence_id,),
).fetchone()
verbatim = "" if row is None else str(row[0])
if is_pure_echo(verbatim, known):
delivered.update(echoed_event_ids(verbatim))
continue
kept.append(event)
return tuple(event for event in kept if event.event_id not in delivered)
@@ -1,140 +0,0 @@
"""Online means an unexpired lease, not a session file that is still on disk."""
import sqlite3
from collections.abc import Iterator
from contextlib import contextmanager
from datetime import datetime, timedelta
from pathlib import Path
from typing import Any
from one_memory.database_access import main_database
#: Schema owned by `memory_server.migrations` (migration 5). No local copy,
#: so the migration cannot drift from what this module actually creates.
from one_memory.memory_server.migrations import session_leases_ddl
from one_memory.memory_server.sql_compat import (
execute,
execute_script,
is_mysql,
row_to_dict,
)
from one_memory.memory_server.store_binding import store_for
from one_memory.session_registry import SessionRegistry
#: Both renderings, chosen at use time (same reason as every migrated module:
#: the sqlite rendering carries SQLite types and index forms MySQL rejects).
SCHEMA_BY_ENGINE = {False: session_leases_ddl("sqlite"), True: session_leases_ddl("mysql")}
def schema_for(mysql: bool) -> str:
return SCHEMA_BY_ENGINE[mysql]
#: The heartbeat upsert, per engine.
#:
#: `ON CONFLICT ... DO UPDATE` (SQLite) vs `ON DUPLICATE KEY UPDATE` (MySQL).
#: Wrong either way and a session's lease silently keeps its old expiry, which
#: reads as "session went offline" rather than as a database error.
BEAT_SQLITE = """
INSERT INTO session_leases (session_key, pid, generation, expires_at)
VALUES (?, ?, ?, ?)
ON CONFLICT(session_key) DO UPDATE SET
pid = excluded.pid,
generation = excluded.generation,
expires_at = excluded.expires_at
"""
BEAT_MYSQL = """
INSERT INTO session_leases (session_key, pid, generation, expires_at)
VALUES (%s, %s, %s, %s)
ON DUPLICATE KEY UPDATE
pid = VALUES(pid),
generation = VALUES(generation),
expires_at = VALUES(expires_at)
"""
SELECT_SQLITE = "SELECT generation, expires_at FROM session_leases WHERE session_key = ?"
SELECT_MYSQL = "SELECT generation, expires_at FROM session_leases WHERE session_key = %s"
class SessionLease:
def __init__(self, database_path: Path) -> None:
self._path = database_path
self._store: Any = store_for(database_path)
self._registry = SessionRegistry(database_path)
def initialize(self) -> None:
mysql = is_mysql(self._store)
with self._connect(writing=True) as db:
execute_script(db, mysql, schema_for(mysql))
def beat(
self,
session_key: str,
*,
pid: int,
generation: int,
now: datetime,
ttl_seconds: int,
) -> None:
self._registry.require(session_key)
expires = (now + timedelta(seconds=ttl_seconds)).isoformat()
mysql = is_mysql(self._store)
with self._connect(writing=True) as db:
execute(
db,
mysql,
BEAT_MYSQL if mysql else BEAT_SQLITE,
(session_key, pid, generation, expires),
)
def online(
self,
session_key: str,
*,
now: datetime,
generation: int,
source_file: Path | None = None,
) -> bool:
del source_file
self._registry.require(session_key)
mysql = is_mysql(self._store)
with self._connect() as db:
row = execute(
db, mysql, SELECT_MYSQL if mysql else SELECT_SQLITE, (session_key,)
).fetchone()
if row is None:
return False
# Named access only works on `sqlite3.Row`; MySQL gives a tuple, so the
# row is normalised to a mapping on both engines first.
values = row_to_dict(row, ("generation", "expires_at"), mysql)
if int(values["generation"]) != generation:
return False
return now < _as_datetime(values["expires_at"], now)
@contextmanager
def _connect(self, *, writing: bool = False) -> Iterator[Any]:
"""A checkout on whichever engine is in force."""
if is_mysql(self._store):
checkout = self._store.write() if writing else self._store.read()
with checkout as connection:
yield connection
return
database = main_database(self._path)
checkout = database.write() if writing else database.read(row_factory=sqlite3.Row)
with checkout as connection, connection:
yield connection
def _as_datetime(value: object, reference: datetime) -> datetime:
"""Parse the stored expiry, matching the caller's awareness.
SQLite returns the stored ISO string so it parses aware. MySQL `DATETIME`
returns a naive `datetime`, and comparing that with an aware `now` raises
`TypeError` — a crash on a perfectly ordinary read. The stored value is the
same instant, so the zone is taken from the caller's clock rather than
assumed, which keeps the comparison meaningful either way.
"""
if isinstance(value, datetime):
if value.tzinfo is None and reference.tzinfo is not None:
return value.replace(tzinfo=reference.tzinfo)
return value
return datetime.fromisoformat(str(value))
@@ -1,137 +0,0 @@
"""Trusted host sessions. A directory is a relevance hint, not an identity."""
import hashlib
import sqlite3
from dataclasses import dataclass
from pathlib import Path
from one_memory.call_event_store import HostId
from one_memory.database_access import main_database
#: Schema owned by `memory_server.migrations` (migration 6). No local copy,
#: so the migration cannot drift from what this module actually creates.
from one_memory.memory_server.migrations import trusted_sessions_ddl
_SCHEMA = trusted_sessions_ddl("sqlite")
class SessionIdentityError(Exception):
pass
class AmbiguousWorkspace(SessionIdentityError):
pass
class ForgedSession(SessionIdentityError):
pass
@dataclass(frozen=True)
class TrustedSession:
session_key: str
installation_id: str
host_id: HostId
native_session_id: str
runtime_instance: str
workspace: str | None
class SessionRegistry:
def __init__(self, database_path: Path) -> None:
self._path = database_path
def initialize(self) -> None:
with main_database(self._path).write() as db, db:
db.execute(_SCHEMA)
def open(
self,
*,
installation_id: str,
host_id: HostId,
native_session_id: str,
runtime_instance: str,
workspace: str | None,
) -> TrustedSession:
if not installation_id or not native_session_id or not runtime_instance:
raise SessionIdentityError("incomplete_session")
key = _identity_key(installation_id, host_id, native_session_id, runtime_instance)
with main_database(self._path).write() as db, db:
row = db.execute(
"SELECT session_key FROM trusted_sessions WHERE session_key = ?",
(key,),
).fetchone()
if row is None:
db.execute(
"""
INSERT INTO trusted_sessions (
session_key, installation_id, host_id, native_session_id,
runtime_instance, workspace
) VALUES (?, ?, ?, ?, ?, ?)
""",
(key, installation_id, host_id, native_session_id, runtime_instance, workspace),
)
return self.require(key)
def require(self, session_key: str) -> TrustedSession:
with main_database(self._path).read() as db:
db.row_factory = sqlite3.Row
row = db.execute(
"SELECT * FROM trusted_sessions WHERE session_key = ?",
(session_key,),
).fetchone()
if row is None:
raise ForgedSession(session_key)
return _session(row)
def in_workspace(self, workspace: str) -> tuple[TrustedSession, ...]:
with main_database(self._path).read() as db:
db.row_factory = sqlite3.Row
rows = db.execute(
"SELECT * FROM trusted_sessions WHERE workspace = ? ORDER BY session_key",
(workspace,),
).fetchall()
return tuple(_session(row) for row in rows)
def matching_native(
self, host_id: HostId, native_session_id: str
) -> tuple[TrustedSession, ...]:
with main_database(self._path).read() as db:
db.row_factory = sqlite3.Row
rows = db.execute(
"""
SELECT * FROM trusted_sessions
WHERE host_id = ? AND native_session_id = ?
ORDER BY session_key
""",
(host_id, native_session_id),
).fetchall()
return tuple(_session(row) for row in rows)
def sole_session(self, workspace: str) -> TrustedSession:
found = self.in_workspace(workspace)
if len(found) == 1:
return found[0]
raise AmbiguousWorkspace(workspace)
def _identity_key(
installation_id: str,
host_id: str,
native_session_id: str,
runtime_instance: str,
) -> str:
material = "\n".join((installation_id, host_id, native_session_id, runtime_instance))
return hashlib.sha256(material.encode()).hexdigest()
def _session(row: sqlite3.Row) -> TrustedSession:
return TrustedSession(
session_key=row["session_key"],
installation_id=row["installation_id"],
host_id=row["host_id"],
native_session_id=row["native_session_id"],
runtime_instance=row["runtime_instance"],
workspace=row["workspace"],
)
@@ -1,180 +0,0 @@
"""WorkBuddy SessionStart and UserPromptSubmit. No idle turn is started.
The notice stream is the current Evidence revision projection in
`SessionInbox`; `change_outbox` is retained there only as a legacy event source.
"""
import base64
import binascii
import json
import sqlite3
from pathlib import Path
from typing import TextIO
from one_memory.derived_echo import render_reference
from one_memory.session_inbox import CursorRejected, InboxPosition, SessionInbox
from one_memory.session_registry import SessionIdentityError, SessionRegistry
BOUNDARY_EVENTS = frozenset({"SessionStart", "UserPromptSubmit"})
def database_path() -> Path:
from one_memory.config import load_settings
return load_settings().data_dir / "one-memory.sqlite3"
PAGE_LIMIT = 8
def _boundary_session(payload: object, database: Path) -> tuple[str, str] | None:
"""The hook event and the one trusted session it belongs to.
Identity is the payload's own session id. A parent id is not a substitute.
More than one matching trusted session is ambiguous, so nothing is injected.
"""
if not isinstance(payload, dict):
return None
event = payload.get("hook_event_name") or payload.get("hookEventName")
if event not in BOUNDARY_EVENTS:
return None
native = payload.get("session_id") or payload.get("sessionId")
if not isinstance(native, str) or not native or not database.is_file():
return None
matches = SessionRegistry(database).matching_native("workbuddy", native)
if len(matches) != 1:
return None
return str(event), matches[0].session_key
def hook_output(payload: object, database: Path) -> dict[str, object]:
"""The JSON WorkBuddy reads on the next session or prompt boundary.
Reading moves the cursor, so a notice lost before it reaches the host is
not sent again. Kept for installs whose hook predates `--peek`.
"""
silent: dict[str, object] = {"continue": True}
found = _boundary_session(payload, database)
if found is None:
return silent
event, session_key = found
inbox = SessionInbox(database)
page = inbox.pull(session_key, inbox.latest_cursor(session_key), PAGE_LIMIT)
if not page.events:
return silent
return {
"continue": True,
"hookSpecificOutput": {
"hookEventName": event,
"additionalContext": render_reference(page.events),
},
}
def peek_output(payload: object, database: Path) -> dict[str, object]:
"""The same notice without moving the cursor, plus the token that confirms it.
Until `confirm` receives the token the notice is sent again next round.
"""
found = _boundary_session(payload, database)
if found is None:
return {}
event, session_key = found
peeked = SessionInbox(database).peek(session_key, PAGE_LIMIT)
if not peeked.events:
return {}
return {
"hookSpecificOutput": {
"hookEventName": event,
"additionalContext": render_reference(peeked.events),
},
"one_memory_ack": _token(session_key, peeked.position),
}
def confirm(token: str, database: Path) -> bool | None:
"""Move the session's cursor up to what the token's peek handed over.
Raises ValueError for a token this module did not issue.
"""
session_key, position = _untoken(token)
return SessionInbox(database).advance_to(session_key, position)
def _token(session_key: str, position: InboxPosition) -> str:
body = json.dumps(
{
"s": session_key,
"e": position.event_id,
"p": position.permission_epoch,
"h": sorted(position.handed_event_ids),
"r": position.receipt_id,
},
separators=(",", ":"),
)
return base64.urlsafe_b64encode(body.encode()).decode().rstrip("=")
def _untoken(token: str) -> tuple[str, InboxPosition]:
try:
raw = base64.urlsafe_b64decode(token + "=" * (-len(token) % 4))
body = json.loads(raw)
except (binascii.Error, ValueError) as error:
raise ValueError("not a boundary token") from error
if not isinstance(body, dict):
raise ValueError("not a boundary token")
session_key, event_id, epoch, handed, receipt_id = (
body.get("s"),
body.get("e"),
body.get("p"),
body.get("h", []),
body.get("r"),
)
if not isinstance(session_key, str) or not isinstance(event_id, str):
raise ValueError("not a boundary token")
if not isinstance(epoch, int) or isinstance(epoch, bool):
raise ValueError("not a boundary token")
if not isinstance(handed, list) or any(not isinstance(item, str) for item in handed):
raise ValueError("not a boundary token")
if not isinstance(receipt_id, str) or not receipt_id:
raise ValueError("not a boundary token")
return session_key, InboxPosition(event_id, epoch, frozenset(handed), receipt_id)
def _payload(stdin: TextIO) -> object:
try:
return json.loads(stdin.read() or "{}")
except json.JSONDecodeError:
return {}
def write_hook(stream: TextIO, stdin: TextIO) -> int:
"""Print the reply only when it carries a notice.
WorkBuddy reads stdout without `hookSpecificOutput` as context text, so a
bare `{"continue": true}` would reach the model. Empty stdout also continues.
"""
body = hook_output(_payload(stdin), database_path())
if "hookSpecificOutput" in body:
stream.write(f"{json.dumps(body, ensure_ascii=False)}\n")
return 0
def write_peek(stream: TextIO, stdin: TextIO) -> int:
body = peek_output(_payload(stdin), database_path())
if body:
stream.write(f"{json.dumps(body, ensure_ascii=False)}\n")
return 0
def run_confirm(token: str) -> int:
try:
confirm(token, database_path())
except (ValueError, CursorRejected, SessionIdentityError, sqlite3.Error):
return 1
return 0
+4 -15
View File
@@ -63,17 +63,15 @@
| `test_context_memory.py` | `contract` | 覆盖 permissions precede candidate limit and do not leak counts、scope user status and active source isolation、denied scope does not open database 等 60 个场景。 |
| `test_context_raw.py` | `contract` | 覆盖 empty authority and unrequested path do not open database、permissions precede candidate limit and no model needed、explicit source filter never broadens authority 等 37 个场景。 |
| `test_context_raw_scale_benchmark.py` | `contract` | 覆盖 object scale probe has exact positive control and plan、raw scale probe uses fts and has a nonzero positive control 场景。 |
| `test_context_search.py` | `contract` | 覆盖 authorization and requested paths never invoke other lane、empty partial zero unavailable and safe failures、parallel common deadline and finalizers wait for both reads 等 24 个场景。 |
| `test_context_search_both_paths.py` | `integration` | The objective names 'recall' and requires BOTH paths: raw and memory. |
| `test_context_v2_contract_parity.py` | `unit` | Mechanical parity checks for the shared v2 coverage receipt contract. |
| `test_contract_parity.py` | `contract` | Python 的枚举和 TypeScript 里那份手写的联合类型必须一致。 |
| `test_conversation_search.py` | `integration` | Searching imported conversations before any generation model is configured. |
| `test_conversation_seed_ids.py` | `contract` | 覆盖 seed ids read metadata only without hydrating、seed eligibility keeps person and memory content only、latest activity includes nonseed turns and import time fallback 等 13 个场景。 |
| `test_conversation_turn_grouping.py` | `contract` | A proposal and the turn that adopts it must reach HY as one turn. |
| `test_core_ready_before_recovery.py` | `integration` | Readiness never waits for picking up work an earlier run left unfinished. |
| `test_core_test_reporting.py` | `contract` | 覆盖 report writes sanitized machine readable views、file and marker aggregates are stable、classification suggestions never guess unit contract or integration 等 6 个场景。 |
| `test_credential_responsiveness.py` | `integration` | 覆盖 credentials do not block the event loop、disabled automatic processing does not digest onboarding、answering reads credentials off loop 等 5 个场景。 |
| `test_cursor_default_root_windows.py` | `contract` | R-29 剩余:Cursor 默认根路径不能写死成 macOS。 |
| `test_d93_same_process_load.py` | `integration` | D93 test 2: read latency while pure-Python work holds the GIL. |
| `test_daemon_contract.py` | `unit` | A5-01/A5-03: process ownership, and the three grants kept apart. |
| `test_daemon_scheduler.py` | `contract` | A5-02/A5-04: the daemon's scan scheduler and composed health. |
| `test_data_safety.py` | `contract` | A5-05: backup usability/rotation, crash recovery, log retention. |
@@ -108,7 +106,6 @@
| `test_harness_contract_suite.py` | `contract` | A4-09: one contract, ten harnesses. |
| `test_health.py` | `contract` | 覆盖 a backfill that fails says so rather than disappearing、a failed catch up leaves a traceback behind、configured model without content consent does not start backfill 等 15 个场景。 |
| `test_history_listing_rotation.py` | `contract` | 覆盖 history listing alternates started and untouched files、history listing drains whichever cohort is longer 场景。 |
| `test_home_summary.py` | `integration` | 覆盖 summary counts beyond page limit and excludes ineligible rows、summary revocation excludes whole mixed memory not only citation、latest is bounded and other users revocations do not hide local data 等 6 个场景。 |
| `test_host_capabilities.py` | `unit` | A plugin does not become realtime on every host or every unknown version. |
| `test_hourly_c1_peak_replay_contract.py` | `contract` | Guard the package peak-replay probe's error accounting. |
| `test_hourly_c1_recall_stage_probe_contract.py` | `contract` | Guard the packaged recall stage probe's command and error semantics. |
@@ -159,8 +156,6 @@
| `test_memory_content_import.py` | `integration` | 覆盖 existing memory preview keeps unknown author and exact original、invalid existing memory records never leave a partial upload、memory content requires explicit mode and rejects vendor guessing 等 5 个场景。 |
| `test_memory_evidence_freshness.py` | `contract` | 新鲜度拆除后:读路径恒为 current;证据修订 / 畸形 proof / segment ref 不再藏记忆或 503。 |
| `test_memory_freshness_integration_review.py` | `integration` | Independent synthetic read-path review; no lifespan, model or user database. |
| `test_memory_governance_api.py` | `integration` | 覆盖 memory governance api preserves evidence and reports cleanup、memory validation does not echo content 场景。 |
| `test_memory_graph_api.py` | `integration` | 覆盖 graph labels where every edge came from、a created memory is on the map with its source links、drawing the map does not stop the app answering 等 12 个场景。 |
| `test_memory_notes.py` | `integration` | 覆盖 empty library can save a note without a matter or configured model、note requires authentication explicit scope and nonempty content、saved note evidence returns raw and consent reference from same record 等 6 个场景。 |
| `test_memory_qa.py` | `contract` | 覆盖 search terms keep meaningful single chinese character、streamed answer emits recall then incremental grounded text、governance change during generation stops the old answer 等 14 个场景。 |
| `test_memory_qa_api.py` | `integration` | 覆盖 ask stream endpoint emits recall deltas and final、ask stream endpoint requires the local token 场景。 |
@@ -213,16 +208,12 @@
| `test_probe_payment_required.py` | `contract` | 覆盖 http 402 without a useful body is still understood as unpaid、a wrong address is still a missing model not an unpaid one 场景。 |
| `test_processing_requests.py` | `contract` | 覆盖 concurrent claim only one snapshot wins、concurrent create same grant has one id and profile、concurrent different profile never reuses another scope 等 14 个场景。 |
| `test_quality_acceptance.py` | `contract` | 覆盖 acceptance pack is deterministic and scores personal boundary、acceptance score fails missing personal memory 场景。 |
| `test_raw_search_without_model.py` | `integration` | Search imported conversations with no generative model configured at all. |
| `test_raw_vector_scale_benchmark.py` | `contract` | 覆盖 vector scale probe uses real 512 dimension filtered search 场景。 |
| `test_read_path_freshness_batch.py` | `contract` | The row-by-row read check must not scale its query count with the page. |
| `test_reader_app.py` | `integration` | The Reader process stays read-only and never blocks a read on its side effects. |
| `test_reader_conversation_hit.py` | `contract` | A search hit carries who wrote the row it matched. |
| `test_real_load_metrics.py` | `contract` | Run the real access layer under a production-like combined load. |
| `test_recall.py` | `contract` | 覆盖 recall is confirmed only grounded and scope bound、wildcard scope means every scope just like it does in qa、recall returns empty instead of unrelated fallback 等 10 个场景。 |
| `test_recall_during_import_probe_contract.py` | `contract` | Guard the source-level recall-during-import probe's sampling semantics. |
| `test_recall_excluded_sources.py` | `contract` | D94: an empty result names the sources this host may not read. |
| `test_recall_fusion_live.py` | `integration` | RCL-04 against a real HY instance instead of a stand-in. |
| `test_recall_library_scale.py` | `contract` | 覆盖 recall reaches a memory that is not among the newest 500、semantic recall reaches a memory outside the newest 500、recall still works just below the boundary 等 5 个场景。 |
| `test_recall_log_probe_exclusion.py` | `contract` | A-21:R-10 一旦引入召回日志,「探针不污染」必须自动变红而不是静默失效。 |
| `test_recall_probe.py` | `contract` | The self-check query, and what it must not touch on its way through. |
@@ -254,6 +245,7 @@
| `test_sdk_data_plane_migration.py` | `contract` | SDK-authority migration ledger, aliases and atomic cutover marker. |
| `test_sdk_memory_qa.py` | `contract` | 覆盖 sdk ask propagates sdk search failure without legacy fallback、sdk ask revalidates memory before returning extract、sdk ask uses native sdk memory without governance fields 场景。 |
| `test_sdk_migration_bundle.py` | `contract` | SDK-authority cutover requires a verified management + SDK data-plane bundle. |
| `test_sdk_migration_log.py` | `integration` | `sdk-migration.log` explains an unfinished upgrade, and a legacy home is never served. |
| `test_sdk_verbatim_fallback.py` | `contract` | D98: the SDK's verbatim fallback must stay off. |
| `test_search_without_profile.py` | `contract` | What 'semantic_search' should do once the profile that filled the store is gone. |
| `test_secondary_source_adapters.py` | `contract` | 覆盖 pi and qwenwork keep tool and sidechain content out of the persons words、qwen code reads only the transcript beside its bookkeeping、qwen native parts preserve visible text not reasoning or tools 等 22 个场景。 |
@@ -263,9 +255,6 @@
| `test_sedimentation_switch.py` | `contract` | Stopping model-driven sedimentation without losing the configuration. |
| `test_service_level_pressure_acceptance.py` | `contract` | 覆盖 workbuddy recall remains available during 100k import 场景。 |
| `test_service_levels.py` | `contract` | 覆盖 background never consumes the realtime reservation、realtime is next for the same source after background yields、import history gets one slice after nine realtime admissions 等 8 个场景。 |
| `test_session_inbox.py` | `contract` | A session inbox is not a shared mailbox and a pull is not adoption. |
| `test_session_lease.py` | `contract` | A session file on disk is not proof that the session is online. |
| `test_session_registry.py` | `contract` | Trusted session identity is not a workspace path. |
| `test_snapshot_adapter.py` | `contract` | Selection-only binding reopens checksum-verified frozen copies. |
| `test_snapshot_pool.py` | `contract` | A whole-directory snapshot must not hold a thread everything else is waiting for. |
| `test_soak_probe_effectiveness.py` | `integration` | The soak probe must observe the load it reports on. |
@@ -305,13 +294,13 @@
| `test_ui_session_api.py` | `integration` | 覆盖 ui session exchange sets httponly strict cookie、ui session requires bearer for exchange、ui session reports missing web assets 场景。 |
| `test_unanswered_turn.py` | `contract` | BUG-06: which conversations end on a question nobody answered. |
| `test_unified_write_entry_wiring.py` | `integration` | The composition root routes every Evidence-first entry through one queue. |
| `test_v2_authority_and_call_events.py` | `integration` | v2 is the SDK data plane: one authority gate, and host calls are history. |
| `test_vendor_sqlite_audit.py` | `contract` | A1-08: the audit that says which SQLite is MemHub's and which is a vendor's. |
| `test_visibility_background.py` | `integration` | The visibility background task: when it runs, and that it says so. |
| `test_whole_turn_hy_input.py` | `contract` | D74: a turn reaches HY whole; only what HY will read counts toward the ceiling. |
| `test_windows_process_control.py` | `contract` | R-29:Windows 上 'os.kill' 不是发信号,是无条件杀进程。 |
| `test_workbuddy_boundary.py` | `contract` | WorkBuddy only injects on SessionStart and UserPromptSubmit. |
| `test_workbuddy_parsing.py` | `contract` | 覆盖 automation preserves query without claiming fresh human authorship、automation context attributes do not depend on order or quote style、quoted full automation envelope is not scheduled provenance 等 17 个场景。 |
| `test_workspace_recall.py` | `contract` | Workspace memories were unreachable from every host; these pin the way in. |
| `test_write_queue_lease_renewal.py` | `contract` | Rule 8: a lease must outlive the work it covers, and be renewed while it runs. |
共 309 个测试文件。
共 298 个测试文件。
@@ -14,8 +14,8 @@ from pathlib import Path
import pytest
PRIMARY_MARKERS = ("unit", "contract", "integration", "performance", "live")
EXPECTED_TEST_FILE_COUNT = 309
EXPECTED_TEST_FILES_SHA256 = "ef6165c249971f5a2ab71a0c49f6d859c8c0f3612e3f749fba0acce0eab513ea"
EXPECTED_TEST_FILE_COUNT = 298
EXPECTED_TEST_FILES_SHA256 = "3b2bece1e20fc1db0e8a82db65bb2cd189317ca34922c97ae8849d38283761e8"
# Pure transformations, validation models, parity checks and state machines.
# This list is intentionally small: uncertain files default to contract.
@@ -51,23 +51,21 @@ INTEGRATION_FILES = frozenset(
{
"test_auth.py",
"test_backup_endpoint.py",
"test_core_ready_before_recovery.py",
"test_backup_restore_api.py",
"test_c1_import_via_api_path.py",
"test_capture_deadline.py",
"test_chat_exports.py",
"test_cli_lifecycle.py",
"test_connection_lifetime.py",
"test_context_search_both_paths.py",
"test_conversation_search.py",
"test_credential_responsiveness.py",
"test_d93_same_process_load.py",
"test_diagnostic_trace.py",
"test_diagnostic_trace_api.py",
"test_external_installation_permissions.py",
"test_fault_matrix.py",
"test_file_import_operations.py",
"test_graph_width_endpoint.py",
"test_home_summary.py",
"test_import_api.py",
"test_live_capture.py",
"test_live_watch_grants.py",
@@ -78,8 +76,6 @@ INTEGRATION_FILES = frozenset(
"test_measure_c1_loop_blocking_contract.py",
"test_memory_content_import.py",
"test_memory_freshness_integration_review.py",
"test_memory_governance_api.py",
"test_memory_graph_api.py",
"test_memory_notes.py",
"test_memory_qa_api.py",
"test_memory_server_durable_queue.py",
@@ -94,13 +90,11 @@ INTEGRATION_FILES = frozenset(
"test_onboarding_profile.py",
"test_parent_watchdog.py",
"test_policy_api_wiring.py",
"test_raw_search_without_model.py",
"test_reader_app.py",
"test_recall_fusion_live.py",
"test_recall_scope_control.py",
"test_retrieval_lifespan.py",
"test_runtime.py",
"test_secret_store_error_code_api.py",
"test_sdk_migration_log.py",
"test_secret_store_error_codes.py",
"test_secure_storage.py",
"test_soak_probe_effectiveness.py",
@@ -114,6 +108,7 @@ INTEGRATION_FILES = frozenset(
"test_source_sync_permissions.py",
"test_ui_session_api.py",
"test_unified_write_entry_wiring.py",
"test_v2_authority_and_call_events.py",
"test_visibility_background.py",
}
)
@@ -1,194 +0,0 @@
"""D93 test 3: inventory every database write a Reader-bound read route makes.
Plan `读写隔离方案-20261009.html` section 5 (B) and section 8 row 3. The Reader
opens the library `mode=ro` + `query_only`, so any write a read route makes
today will fail there. Instead of a manual audit, run the existing route tests
with this plugin loaded; it records, per route, every INSERT / UPDATE / DELETE /
DDL that happens while that route is being served, with the one_memory frames
that issued it. Recording only, nothing is blocked, so one run lists them all.
bash /private/tmp/wb-d76-pytest.sh -p tests.d93_side_effects_plugin \
tests/test_home_summary.py tests/test_memory_library_api.py ...
The report goes to $ONE_MEMORY_D93_SIDE_EFFECTS (default
`d93-side-effects.json` in the pytest root) and a summary is printed at the end.
Writes outside a request (fixture setup, background tasks) are not counted.
"""
from __future__ import annotations
import contextlib
import contextvars
import json
import os
import sqlite3
import traceback
from collections import defaultdict
from pathlib import Path
#: The routes the Reader takes over (plan section 5, B, reader_app.py row).
READER_ROUTES = {
("GET", "/api/v1/home-summary"),
("GET", "/api/v1/memory-library"),
("GET", "/api/v1/memories"),
("GET", "/api/v1/memories/{memory_id}"),
("GET", "/api/v1/memories/graph"),
("POST", "/api/v1/context/search"),
("GET", "/api/v1/context/search"),
("POST", "/api/v1/recall"),
("GET", "/api/v1/conversations/search"),
("POST", "/api/v1/conversations/search"),
("GET", "/api/v1/health"),
("GET", "/health"),
}
_WRITES = {
sqlite3.SQLITE_INSERT: "INSERT",
sqlite3.SQLITE_UPDATE: "UPDATE",
sqlite3.SQLITE_DELETE: "DELETE",
sqlite3.SQLITE_CREATE_TABLE: "CREATE TABLE",
sqlite3.SQLITE_CREATE_INDEX: "CREATE INDEX",
sqlite3.SQLITE_CREATE_TRIGGER: "CREATE TRIGGER",
sqlite3.SQLITE_DROP_TABLE: "DROP TABLE",
sqlite3.SQLITE_ALTER_TABLE: "ALTER TABLE",
}
_current: contextvars.ContextVar[tuple[str, str] | None] = contextvars.ContextVar(
"d93_route", default=None
)
_served: dict[tuple[str, str], dict[int, int]] = defaultdict(lambda: defaultdict(int))
_found: dict[tuple, int] = defaultdict(int)
def _frames() -> tuple[str, ...]:
"""Last product frames; a leading `[test]` marks writes a test's stand-in made."""
kept = []
from_test = False
for frame in traceback.extract_stack()[:-2]:
name = frame.filename.replace(os.sep, "/")
if "/apps/core/tests/" in name and not name.endswith("d93_side_effects_plugin.py"):
from_test = True
if "/one_memory/" in name and "/tests/" not in name:
kept.append(f"{name.split('/one_memory/', 1)[1]}:{frame.lineno} {frame.name}")
return (("[test]",) if from_test else ()) + tuple(kept[-4:])
#: Self-check: record every route, so a known write route (POST /api/v1/memories)
#: proves the recorder sees writes at all.
_ALL = os.environ.get("ONE_MEMORY_D93_ALL_ROUTES") == "1"
def _authorizer(action, arg1, arg2, db_name, _trigger):
route = _current.get()
if route is not None and (_ALL or route in READER_ROUTES) and action in _WRITES:
table = arg1 if action != sqlite3.SQLITE_ALTER_TABLE else arg2
if db_name != "temp" and not str(table).startswith("sqlite_"):
_found[(route, _WRITES[action], str(table), _frames())] += 1
return sqlite3.SQLITE_OK
_real_connect = sqlite3.connect
def _connect(*args, **kwargs):
connection = _real_connect(*args, **kwargs)
with contextlib.suppress(Exception):
connection.set_authorizer(_authorizer)
return connection
class _RouteTag:
def __init__(self, app):
self.app = app
async def __call__(self, scope, receive, send):
if scope.get("type") != "http":
return await self.app(scope, receive, send)
status = {"code": 0}
async def sending(message):
if message.get("type") == "http.response.start":
status["code"] = message.get("status", 0)
await send(message)
holder: dict[str, tuple[str, str]] = {}
token = _current.set(None)
# The route template is only known after routing; resolve it up front
# from the app's routes so writes made during the handler carry it.
for route in getattr(self._root(scope), "routes", ()):
match = getattr(route, "matches", None)
if match is None:
continue
kind, _ = match(scope)
if kind.name == "FULL":
holder["route"] = (scope["method"], getattr(route, "path", scope["path"]))
break
key = holder.get("route", (scope["method"], scope["path"]))
_current.set(key)
try:
await self.app(scope, receive, sending)
finally:
_current.reset(token)
if key in READER_ROUTES:
_served[key][status["code"]] += 1
def _root(self, scope):
return scope.get("app")
def pytest_configure(config):
sqlite3.connect = _connect
# Pooled connections are handed back through `_reset`, which clears any
# authorizer; put ours back so the next checkout is still recorded.
from one_memory import database_access
real_reset = database_access._reset
def _reset(connection, *, reading):
clean = real_reset(connection, reading=reading)
with contextlib.suppress(Exception):
connection.set_authorizer(_authorizer)
return clean
database_access._reset = _reset
from one_memory import api
original = api.create_app
def create_app(*args, **kwargs):
application = original(*args, **kwargs)
application.add_middleware(_RouteTag)
return application
api.create_app = create_app
def pytest_terminal_summary(terminalreporter):
report = {
"served": {
f"{method} {path}": dict(codes) for (method, path), codes in sorted(_served.items())
},
"writes": [
{
"route": f"{route[0]} {route[1]}",
"op": op,
"table": table,
"frames": list(frames),
"count": count,
}
for (route, op, table, frames), count in sorted(_found.items(), key=str)
],
}
target = Path(
os.environ.get("ONE_MEMORY_D93_SIDE_EFFECTS", str(Path.cwd() / "d93-side-effects.json"))
)
target.write_text(json.dumps(report, ensure_ascii=False, indent=2), encoding="utf-8")
write = terminalreporter.write_line
write(f"D93 side effects: {len(report['writes'])} write sites -> {target}")
for name, codes in report["served"].items():
write(f" served {name}: {codes}")
for item in report["writes"]:
write(f" {item['route']}: {item['op']} {item['table']} x{item['count']}")
for frame in item["frames"]:
write(f" {frame}")
-34
View File
@@ -55,37 +55,3 @@ def test_browser_origin_is_checked_in_addition_to_token() -> None:
)
== 200
)
async def internal_request(
path: str,
*,
token: str | None,
method: str = "GET",
) -> int:
settings = Settings(api_token=SecretStr("test-token"))
app = create_app(settings)
headers = {}
if token:
headers["Authorization"] = f"Bearer {token}"
async with AsyncClient(
transport=ASGITransport(app=app),
base_url="http://test",
) as client:
response = await client.request(method, path, headers=headers, json={})
return response.status_code
def test_reader_internal_endpoints_require_local_token() -> None:
for path in (
"/internal/semantic-ids?query=anything",
"/internal/graph-snapshot",
):
assert asyncio.run(internal_request(path, token=None)) == 401
assert asyncio.run(internal_request(path, token="test-token")) == 200
assert asyncio.run(internal_request("/internal/call-event", token=None, method="POST")) == 401
assert (
asyncio.run(internal_request("/internal/call-event", token="test-token", method="POST"))
== 200
)
@@ -1,6 +1,7 @@
import asyncio
import sqlite3
import pytest
from httpx import ASGITransport, AsyncClient
from pydantic import SecretStr
@@ -44,6 +45,8 @@ def _database(path, *, value: str) -> None:
asyncio.run(SqliteMigrationStore(path).record(Migration(34, "synthetic", lambda _: None)))
# A serving Core refuses a legacy home outright, so this one starts on SDK.
@pytest.mark.fresh_sdk
def test_restore_endpoint_refuses_when_runtime_owns_the_core(tmp_path) -> None:
monkeypatch_app = create_app(_settings(tmp_path), manage_runtime=True)
@@ -1,104 +1,10 @@
from datetime import UTC, datetime
from one_memory.api import ConversationSearchHit, ConversationSearchResponse
from one_memory.call_event_recorder import (
ask_call_event,
context_call_event,
conversation_call_event,
)
from one_memory.context_models import ContextPath, ContextPaths, ContextSearchResponse
from one_memory.external_memory_access import ExternalMemoryAccess
from one_memory.call_event_recorder import ask_call_event
from one_memory.memory_governance import GovernedMemory, MemoryKind, MemoryOrigin, MemoryStatus
from one_memory.memory_qa import MemoryAnswer
def _paths(memory: str, raw: str) -> ContextPaths:
lane = dict(elapsed_ms=1.0, returned_count=0, truncated=False)
return ContextPaths(
raw=ContextPath(status=raw, **lane),
memory=ContextPath(status=memory, **lane),
)
def _context(memory: str, raw: str) -> ContextSearchResponse:
return ContextSearchResponse(
request_id="11111111-1111-4111-8111-111111111111",
state="partial",
paths=_paths(memory, raw),
items=(),
used_characters=0,
truncated=False,
total_ms=3.5,
)
def test_denied_memory_lane_is_attributed_to_conversations():
event = context_call_event(_context("not_authorized", "ok"), None)
assert event.tool == "om_conversations"
def test_requested_memory_lane_stays_recall():
event = context_call_event(_context("empty", "not_authorized"), None)
assert event.tool == "om_recall"
def test_zero_result_timeout_is_recorded_separately_from_unavailable() -> None:
lane = dict(elapsed_ms=750.0, returned_count=0)
result = ContextSearchResponse(
request_id="11111111-1111-4111-8111-111111111111",
state="partial",
paths=ContextPaths(
raw=ContextPath(
status="degraded",
reason="timeout_budget",
truncated=True,
truncated_by=("timeout_budget",),
**lane,
),
memory=ContextPath(status="empty", truncated=False, **lane),
),
items=(),
used_characters=0,
truncated=True,
total_ms=1500,
)
assert context_call_event(result, None).state == "timed_out"
def test_conversation_search_records_counts_without_query_or_excerpt():
result = ConversationSearchResponse(
hits=(
ConversationSearchHit(
evidence_id="e1",
conversation_id="c1",
source_id="cursor",
excerpt="SECRET_EXCERPT",
),
ConversationSearchHit(
evidence_id="e2",
conversation_id="c1",
source_id="cursor",
excerpt="SECRET_EXCERPT_2",
),
)
)
access = ExternalMemoryAccess(
installation_id="00000000-0000-4000-8000-000000000001",
revision=1,
scopes=frozenset({"all-apps", "source:cursor"}),
tools=frozenset({"om_conversations"}),
host_id="codex",
)
event = conversation_call_event(result, access, 12.0)
encoded = event.model_dump_json()
assert event.tool == "om_conversations"
assert event.state == "grounded"
assert event.raw_count == 1 and event.memory_count == 0 and not event.references
assert event.host_id == "codex"
assert "SECRET" not in encoded
def test_ask_records_memory_refs_without_answer():
memory = GovernedMemory(
memory_id="m1",
@@ -8,7 +8,7 @@ from one_memory.external_memory_access import require_external_endpoint
def request(
tools, scopes=("all-apps", "source:cursor"), method="POST", path="/api/v1/context/search"
tools, scopes=("all-apps", "source:cursor"), method="POST", path="/api/v2/context/search"
):
headers = {
"x-one-memory-read-context": "external",
@@ -44,9 +44,13 @@ def test_other_capabilities_do_not_enable_composite_read(tools):
@pytest.mark.parametrize(
"method,path",
[
("GET", "/api/v1/context/search"),
("GET", "/api/v2/context/search"),
("POST", "/api/v1/context/search"),
("POST", "/api/v1/memory-library"),
("GET", "/api/v1/conversations/search"),
("GET", "/api/v1/conversations/evidence-id"),
("POST", "/api/v1/memories/ask"),
("POST", "/api/v1/boundary/peek"),
],
)
def test_composite_read_does_not_open_other_routes(method, path):
@@ -59,15 +63,3 @@ def test_composite_read_does_not_open_other_routes(method, path):
def test_legacy_recall_still_requires_its_own_tool():
with pytest.raises(HTTPException):
require_external_endpoint(request(["om_conversations"], path="/api/v1/recall"))
def test_legacy_raw_route_still_requires_raw_source_authority():
with pytest.raises(HTTPException):
require_external_endpoint(
request(
["om_conversations"],
scopes=("all-apps",),
method="GET",
path="/api/v1/conversations/search",
)
)
@@ -1,547 +0,0 @@
import asyncio
import time
from datetime import UTC, datetime
import pytest
from pydantic import ValidationError
from one_memory.context_models import (
ContextLaneResult,
ContextMessage,
ContextSearchRequest,
ContextSearchResponse,
MemoryContextItem,
RawContextItem,
)
from one_memory.context_search import search_context
def memory(identity="m", text="memory", evidence=("e",)):
return MemoryContextItem(
memory_id=identity,
revision=1,
content=text,
memory_kind="chat",
updated_at=datetime.now(UTC),
evidence_ids=evidence,
)
def raw(identity="e", text="raw"):
return RawContextItem(
source_id="cursor",
conversation_id="c",
anchor_evidence_id=identity,
messages=(
ContextMessage(
evidence_id=identity,
revision=2,
role="user",
written_by="person",
created_at=None,
text=text,
),
),
)
def reader(*items, status="ok", reason=None, truncated_by=()):
async def read(deadline):
assert deadline > time.monotonic()
return ContextLaneResult(
items=items, status=status, reason=reason, truncated_by=truncated_by
)
return read
async def finalize(kind, items, deadline):
assert deadline > time.monotonic()
assert all(item.kind == kind for item in items)
return items
def run(request=None, raw_reader=None, memory_reader=None, finalizer=finalize):
return asyncio.run(
search_context(
request or ContextSearchRequest(query="q"),
raw_reader=raw_reader,
memory_reader=memory_reader,
finalizer=finalizer,
)
)
def test_authorization_and_requested_paths_never_invoke_other_lane():
async def forbidden(deadline):
pytest.fail("unrequested lane invoked")
result = run(
ContextSearchRequest(query="q", kinds=("memory",)),
raw_reader=forbidden,
memory_reader=reader(memory()),
)
assert result.paths.raw.status == "not_requested"
assert result.state == "grounded"
denied = run(memory_reader=reader(status="empty"))
assert denied.paths.raw.status == "not_authorized"
assert denied.paths.raw.reason == "permission_required"
assert denied.state == "partial" and denied.items == ()
def test_empty_partial_zero_unavailable_and_safe_failures():
assert (
run(raw_reader=reader(status="empty"), memory_reader=reader(status="empty")).state
== "empty"
)
async def fail(deadline):
raise RuntimeError("PRIVATE_SECRET_SQL_PATH")
result = run(raw_reader=fail, memory_reader=fail)
assert result.state == "unavailable"
assert "PRIVATE" not in result.model_dump_json()
assert result.paths.raw.reason == "read_unavailable"
partial = run(raw_reader=fail, memory_reader=reader(status="empty"))
assert partial.state == "partial" and partial.used_characters == 0
lexical_only = run(
raw_reader=reader(status="degraded", reason="lexical_only"),
memory_reader=reader(status="empty"),
)
assert lexical_only.state == "empty"
assert lexical_only.paths.raw.status == "empty"
def test_parallel_common_deadline_and_finalizers_wait_for_both_reads():
async def scenario():
entered = set()
deadlines = []
finished = set()
async def read(kind, deadline):
entered.add(kind)
deadlines.append(deadline)
while len(entered) != 2:
await asyncio.sleep(0)
await asyncio.sleep(0.005 if kind == "raw" else 0.015)
finished.add(kind)
return ContextLaneResult(status="ok", items=(raw() if kind == "raw" else memory(),))
async def final(kind, items, deadline):
assert finished == {"raw", "memory"}
deadlines.append(deadline)
return () if kind == "raw" else items
result = await search_context(
ContextSearchRequest(query="q"),
raw_reader=lambda d: read("raw", d),
memory_reader=lambda d: read("memory", d),
finalizer=final,
)
assert len(set(deadlines)) == 1
assert [i.kind for i in result.items] == ["memory"]
assert result.paths.raw.reason == "evidence_changed"
assert result.state == "partial"
asyncio.run(scenario())
def test_stage_timings_separate_lanes_revalidation_fusion_and_total():
async def scenario():
observed = []
async def delayed(item, delay):
await asyncio.sleep(delay)
return ContextLaneResult(status="ok", items=(item,))
async def delayed_final(kind, items, deadline):
await asyncio.sleep(0.004 if kind == "raw" else 0.008)
return items
result = await search_context(
ContextSearchRequest(query="q"),
raw_reader=lambda _deadline: delayed(raw(), 0.006),
memory_reader=lambda _deadline: delayed(memory(), 0.012),
finalizer=delayed_final,
timing_observer=observed.append,
)
assert result.state == "grounded" and len(observed) == 1
timing = observed[0]
assert set(timing) == {
"raw_ms",
"memory_ms",
"raw_revalidation_ms",
"memory_revalidation_ms",
"fusion_ms",
"total_ms",
}
assert 4 <= timing["raw_ms"] < timing["memory_ms"]
assert 2 <= timing["raw_revalidation_ms"] < timing["memory_revalidation_ms"]
assert timing["fusion_ms"] >= 0
assert timing["total_ms"] >= timing["memory_ms"] + timing["memory_revalidation_ms"]
asyncio.run(scenario())
def test_finalizer_failure_never_releases_old_items():
async def fail(kind, items, deadline):
raise ValueError("PRIVATE")
result = run(raw_reader=reader(raw()), memory_reader=reader(memory()), finalizer=fail)
assert result.items == () and result.state == "unavailable"
assert "PRIVATE" not in result.model_dump_json()
def test_shared_count_text_fairness_and_anchor_metadata():
result = run(
ContextSearchRequest(query="q", character_budget=200, limit=2),
raw_reader=reader(raw(text="R" * 1000), raw("e2")),
memory_reader=reader(memory(text="M" * 1000), memory("m2")),
)
assert [i.kind for i in result.items] == ["raw", "memory"]
assert result.used_characters == 200
assert len(result.items[0].messages[0].text) == 100
assert len(result.items[1].content) == 100
assert result.items[0].anchor_evidence_id == "e"
assert result.items[0].messages[0].revision == 2
assert result.items[0].messages[0].written_by == "person"
for path in (result.paths.raw, result.paths.memory):
assert path.returned_count == 1
assert set(path.truncated_by) == {"count_budget", "character_budget"}
assert all(i.truncated for i in result.items)
assert result.state == "grounded" and result.adoption == "unknown"
assert result.paths.raw.status == "ok"
assert result.paths.memory.status == "ok"
ContextSearchResponse.model_validate_json(result.model_dump_json())
def test_dedup_preserves_cross_kind_evidence_relations():
item = raw()
result = run(
raw_reader=reader(item, item),
memory_reader=reader(memory(), memory(), memory("other", evidence=("e", "e"))),
)
assert len(result.items) == 3
assert result.paths.raw.returned_count == 1
assert result.paths.memory.returned_count == 2
assert result.items[-1].evidence_ids == ("e",)
def test_timeout_cancels_tasks_and_late_results_are_not_released():
async def scenario():
cancelled = []
async def slow(deadline):
try:
await asyncio.sleep(10)
except asyncio.CancelledError:
cancelled.append(True)
return ContextLaneResult(status="ok", items=(raw(text="late"),))
result = await search_context(
ContextSearchRequest(query="q", timeout_ms=50),
raw_reader=slow,
memory_reader=reader(memory()),
finalizer=finalize,
)
assert cancelled == [True]
assert result.items == ()
assert result.paths.raw.reason == "timeout_budget"
assert result.paths.memory.reason == "timeout_budget"
assert not [t for t in asyncio.all_tasks() if t is not asyncio.current_task()]
asyncio.run(scenario())
def test_caller_cancellation_propagates_and_cleans_readers():
async def scenario():
started = asyncio.Event()
exits = []
async def slow(deadline):
started.set()
try:
await asyncio.sleep(10)
finally:
exits.append(1)
task = asyncio.create_task(
search_context(
ContextSearchRequest(query="q"),
raw_reader=slow,
memory_reader=slow,
finalizer=finalize,
)
)
await started.wait()
task.cancel()
with pytest.raises(asyncio.CancelledError):
await task
assert len(exits) == 2
assert not [t for t in asyncio.all_tasks() if t is not asyncio.current_task()]
asyncio.run(scenario())
def test_invalid_lane_shape_and_wrong_kind_fail_closed():
async def invalid(deadline):
return {"status": "ok", "items": [], "private_diagnostic": "secret"}
assert run(raw_reader=invalid, memory_reader=reader(memory())).paths.raw.status == "unavailable"
assert (
run(raw_reader=reader(memory()), memory_reader=reader(memory())).paths.raw.status
== "unavailable"
)
with pytest.raises(ValidationError):
ContextSearchRequest(query="q", limit=21)
with pytest.raises(ValidationError):
ContextSearchRequest(query="q", character_budget=20001)
def test_expired_entry_deadline_never_reads_or_releases_content():
async def forbidden(*args):
pytest.fail("expired deadline must not invoke work")
result = asyncio.run(
search_context(
ContextSearchRequest(query="q"),
raw_reader=forbidden,
memory_reader=forbidden,
finalizer=forbidden,
deadline=time.monotonic() - 1,
)
)
assert result.state == "partial" and result.items == ()
assert result.paths.raw.reason == result.paths.memory.reason == "timeout_budget"
def test_entry_deadline_is_not_reset_and_request_timeout_caps_it():
async def scenario():
observed = []
async def read(deadline):
observed.append(deadline)
return ContextLaneResult(status="ok", items=(memory(),))
entry = time.monotonic() + 0.1
await search_context(
ContextSearchRequest(query="q"),
raw_reader=None,
memory_reader=read,
finalizer=finalize,
deadline=entry,
)
assert observed == [entry]
started = time.monotonic()
await search_context(
ContextSearchRequest(query="q", timeout_ms=50),
raw_reader=None,
memory_reader=read,
finalizer=finalize,
deadline=started + 10,
)
assert observed[-1] < started + 0.1
asyncio.run(scenario())
def test_finalizer_timeout_cannot_release_previously_read_items():
async def scenario():
cancelled = []
async def slow(kind, items, deadline):
try:
await asyncio.sleep(10)
except asyncio.CancelledError:
cancelled.append(kind)
return items
result = await search_context(
ContextSearchRequest(query="q", timeout_ms=50),
raw_reader=reader(raw()),
memory_reader=reader(memory()),
finalizer=slow,
)
assert set(cancelled) == {"raw", "memory"}
assert result.items == () and result.state == "unavailable"
assert result.paths.memory.truncated_by == ("timeout_budget",)
assert not [t for t in asyncio.all_tasks() if t is not asyncio.current_task()]
asyncio.run(scenario())
def test_overlapping_raw_windows_keep_each_anchor_and_cross_kind_reference():
first = raw("e1")
second = raw("e2")
first = first.model_copy(update={"messages": first.messages + second.messages})
result = run(
raw_reader=reader(first, second), memory_reader=reader(memory(evidence=("e1", "e2")))
)
windows = [item for item in result.items if item.kind == "raw"]
assert [item.anchor_evidence_id for item in windows] == ["e1", "e2"]
assert [message.evidence_id for item in windows for message in item.messages] == ["e1", "e2"]
assert next(item for item in result.items if item.kind == "memory").evidence_ids == ("e1", "e2")
def test_twenty_item_and_twenty_thousand_character_caps():
result = run(
ContextSearchRequest(query="q", limit=20, character_budget=20000),
raw_reader=reader(*(raw(f"e{i}", "r" * 20000) for i in range(21))),
memory_reader=reader(*(memory(f"m{i}", "m" * 20000) for i in range(21))),
)
assert len(result.items) == 20 and result.used_characters == 20000
assert result.paths.raw.returned_count == result.paths.memory.returned_count == 10
assert (
sum(
len(i.content) if i.kind == "memory" else sum(len(m.text) for m in i.messages)
for i in result.items
)
== 20000
)
def test_caller_cancel_during_finalization_cleans_all_tasks():
async def scenario():
started = asyncio.Event()
finished = []
async def final(kind, items, deadline):
started.set()
try:
await asyncio.sleep(10)
finally:
finished.append(kind)
task = asyncio.create_task(
search_context(
ContextSearchRequest(query="q"),
raw_reader=reader(raw()),
memory_reader=reader(memory()),
finalizer=final,
)
)
await started.wait()
task.cancel()
with pytest.raises(asyncio.CancelledError):
await task
assert set(finished) == {"raw", "memory"}
assert not [t for t in asyncio.all_tasks() if t is not asyncio.current_task()]
asyncio.run(scenario())
def test_invalid_finalizer_output_and_constructed_models_fail_closed():
async def wrong(kind, items, deadline):
return (memory("unselected"),)
assert run(memory_reader=reader(memory()), finalizer=wrong).items == ()
invalid = memory().model_copy(update={"content": "x" * 20001})
assert run(memory_reader=reader(invalid)).state == "unavailable"
with pytest.raises(ValidationError):
run(ContextSearchRequest.model_construct(query="q", limit=100))
def test_duplicate_memory_identity_keeps_combined_evidence_relations():
result = run(memory_reader=reader(memory(evidence=("a",)), memory(evidence=("b",))))
assert result.items[0].evidence_ids == ("a", "b")
def test_one_slow_finalizer_exhausts_release_deadline_for_both_lanes():
async def final(kind, items, deadline):
if kind == "raw":
await asyncio.sleep(10)
return items
result = run(
ContextSearchRequest(query="q", timeout_ms=50),
raw_reader=reader(raw()),
memory_reader=reader(memory()),
finalizer=final,
)
assert result.items == ()
assert result.state == "partial"
def test_multiple_raw_messages_clipped_without_losing_anchor_or_authorship():
item = raw("anchor", "A" * 20000)
item = item.model_copy(
update={
"messages": (
raw("before", "B" * 20000).messages[0],
item.messages[0],
raw("after", "C" * 20000).messages[0],
)
}
)
result = run(
ContextSearchRequest(query="q", character_budget=200),
raw_reader=reader(item),
memory_reader=reader(memory(text="M" * 20000)),
)
emitted = result.items[0]
assert emitted.anchor_evidence_id == "anchor"
assert [m.evidence_id for m in emitted.messages] == ["before", "anchor", "after"]
assert all(
m.truncated and m.written_by == "person" and m.revision == 2 for m in emitted.messages
)
assert result.used_characters == 200
def test_metadata_union_stays_bounded_and_reports_truncation():
result = run(
memory_reader=reader(
memory(evidence=tuple(f"a{i}" for i in range(128))), memory(evidence=("extra",))
)
)
assert len(result.items[0].evidence_ids) == 128
assert result.items[0].truncated
assert "metadata_budget" in result.paths.memory.truncated_by
def test_finalizer_removed_context_message_reports_partial_not_complete():
item = raw()
item = item.model_copy(update={"messages": item.messages + raw("other").messages})
async def final(kind, items, deadline):
return tuple(i.model_copy(update={"messages": i.messages[:1]}) for i in items)
result = run(
ContextSearchRequest(query="q", kinds=("raw",)), raw_reader=reader(item), finalizer=final
)
assert result.state == "partial"
assert result.paths.raw.reason == "evidence_changed"
def test_noncooperative_synchronous_reader_return_after_deadline_is_discarded():
async def late(deadline):
time.sleep(0.06)
return ContextLaneResult(status="ok", items=(raw(),))
result = run(ContextSearchRequest(query="q", kinds=("raw",), timeout_ms=50), raw_reader=late)
assert result.items == () and result.paths.raw.reason == "timeout_budget"
def test_timeout_does_not_wait_for_a_reader_that_suppresses_cancellation():
async def scenario():
release = asyncio.Event()
async def stuck(_deadline):
try:
await asyncio.Event().wait()
except asyncio.CancelledError:
await release.wait()
return ContextLaneResult(status="empty")
started = time.monotonic()
result = await search_context(
ContextSearchRequest(query="q", kinds=("memory",), timeout_ms=50),
raw_reader=None,
memory_reader=stuck,
finalizer=finalize,
)
assert time.monotonic() - started < 0.2
assert result.paths.memory.reason == "timeout_budget"
release.set()
await asyncio.sleep(0)
asyncio.run(scenario())
@@ -1,161 +0,0 @@
"""The objective names `recall` and requires BOTH paths: raw and memory.
Verified against a real Core with seeded evidence and one governed memory:
asking for `raw` returns the stored verbatim and leaves `memory` as
`not_requested`; asking for `memory` returns the governed memory and leaves
`raw` as `not_requested`. That the unrequested side is marked, rather than
silently empty, is what proves the filter is actually applied.
"""
from __future__ import annotations
import sqlite3
import uuid
import pytest
from httpx import ASGITransport, AsyncClient
from pydantic import SecretStr
from one_memory.api import create_app
from one_memory.config import Settings
def _seed(data_dir) -> tuple[str, str]:
"""One evidence row and one governed memory; return their verbatim/content."""
verbatim = "帮我总结一下 SQLite 的写锁调优方法"
memory = "用户偏好:数据库调优时优先看写入批处理和读池容量,不要一上来就换引擎。"
connection = sqlite3.connect(data_dir / "one-memory.sqlite3")
try:
cursor = connection.cursor()
columns = [row[1] for row in cursor.execute("PRAGMA table_info(evidence)")]
row: dict[str, object] = {}
for column in columns:
if column == "evidence_id" or column == "idempotency_key":
row[column] = str(uuid.uuid4())
elif column in ("user_id", "source_id"):
row[column] = "local" if column == "user_id" else "codex"
elif column in ("conversation_id", "message_id"):
row[column] = str(uuid.uuid4())
elif column == "role":
row[column] = "user"
elif column == "written_by":
row[column] = "person"
elif column == "verbatim":
row[column] = verbatim
elif column in ("created_at", "updated_at", "imported_at", "recorded_at"):
row[column] = "2026-09-26T00:00:00Z"
elif column == "revision" or column == "source_ordinal":
row[column] = 1
elif column.endswith("_hash") or column == "fingerprint":
row[column] = str(uuid.uuid4())[:32]
names = ",".join(row)
markers = ",".join(["?"] * len(row))
cursor.execute(f"INSERT INTO evidence ({names}) VALUES ({markers})", list(row.values()))
memory_columns = [r[1] for r in cursor.execute("PRAGMA table_info(governed_memories)")]
memory_row: dict[str, object] = {}
for column in memory_columns:
if column == "memory_id":
memory_row[column] = str(uuid.uuid4())
elif column == "user_id":
memory_row[column] = "local"
elif column == "kind":
memory_row[column] = "working"
elif column == "status":
memory_row[column] = "confirmed"
elif column == "content":
memory_row[column] = memory
elif column in ("scopes_json", "tags_json"):
memory_row[column] = "[]"
elif column == "origin":
memory_row[column] = "manual"
elif column == "revision":
memory_row[column] = 1
elif column in ("created_at", "updated_at"):
memory_row[column] = "2026-09-26T00:00:00Z"
names = ",".join(memory_row)
markers = ",".join(["?"] * len(memory_row))
cursor.execute(
f"INSERT INTO governed_memories ({names}) VALUES ({markers})",
list(memory_row.values()),
)
connection.commit()
finally:
connection.close()
return verbatim, memory
def _client(tmp_path):
data_dir = tmp_path / "data"
data_dir.mkdir(parents=True, exist_ok=True)
app = create_app(
Settings(
_env_file=None,
data_dir=data_dir,
source_home=tmp_path,
api_token=SecretStr("test-token"),
)
)
return (
app,
data_dir,
AsyncClient(
transport=ASGITransport(app=app),
base_url="http://test",
headers={
"Authorization": "Bearer test-token",
"Origin": "http://127.0.0.1:5173",
},
),
)
@pytest.mark.anyio
async def test_raw_path_returns_verbatim_and_leaves_memory_unrequested(tmp_path) -> None:
app, data_dir, client = _client(tmp_path)
async with app.router.lifespan_context(app), client:
# Seed inside the lifespan: the schema is created on startup, so seeding
# before it yields no columns and an empty INSERT.
verbatim, _ = _seed(data_dir)
response = await client.post(
"/api/v1/context/search",
json={
"query": verbatim,
"kinds": ["raw"],
"limit": 5,
"character_budget": 2000,
"timeout_ms": 5000,
},
)
assert response.status_code == 200, response.text
document = response.json()
assert document["paths"]["raw"]["status"] == "ok"
assert document["paths"]["memory"]["status"] == "not_requested", "memory must not run"
texts = [message["text"] for item in document["items"] for message in item.get("messages", [])]
assert verbatim in texts, "raw path must return the stored verbatim"
@pytest.mark.anyio
async def test_memory_path_returns_governed_memory_and_leaves_raw_unrequested(tmp_path) -> None:
app, data_dir, client = _client(tmp_path)
async with app.router.lifespan_context(app), client:
_, memory = _seed(data_dir)
response = await client.post(
"/api/v1/context/search",
json={
"query": "数据库调优偏好",
"kinds": ["memory"],
"limit": 5,
"character_budget": 2000,
"timeout_ms": 5000,
},
)
assert response.status_code == 200, response.text
document = response.json()
assert document["paths"]["memory"]["status"] == "ok"
assert document["paths"]["raw"]["status"] == "not_requested", "raw must not run"
contents = [item["content"] for item in document["items"]]
assert memory in contents, "memory path must return the governed memory"
@@ -10,11 +10,7 @@ from contextlib import contextmanager
from datetime import UTC, datetime
import pytest
from httpx import ASGITransport, AsyncClient
from pydantic import SecretStr
from one_memory.api import create_app
from one_memory.config import Settings
from one_memory.imports.models import (
EvidenceInput,
EvidenceOrigin,
@@ -22,7 +18,6 @@ from one_memory.imports.models import (
WrittenBy,
)
from one_memory.imports.storage import ConversationSearchUnavailableError, EvidenceStore
from one_memory.memory_governance import MemoryDraft, MemoryKind
def message(
@@ -479,268 +474,3 @@ def test_a_missing_index_is_raised_rather_than_answered_as_no_matches(tmp_path)
with pytest.raises(ConversationSearchUnavailableError):
store.search_conversations("抽取终于", user_id="local", limit=10)
def _client(tmp_path):
app = create_app(
Settings(
data_dir=tmp_path / "data",
source_home=tmp_path,
api_token=SecretStr("local-token"),
)
)
return app, AsyncClient(
transport=ASGITransport(app=app),
base_url="http://test",
headers={
"Authorization": "Bearer local-token",
"Origin": "http://127.0.0.1:5173",
},
)
@pytest.mark.anyio
async def test_recent_list_ignores_a_missing_search_index(tmp_path) -> None:
app, transport = _client(tmp_path)
async with app.router.lifespan_context(app), transport as client:
app.state.evidence_store.put(
message(verbatim="刚从 Cursor 同步进来的原话", message_id="live")
)
with sqlite3.connect(app.state.evidence_store.database_path) as connection:
connection.execute("DROP TABLE IF EXISTS evidence_search")
response = await client.get("/api/v1/conversations", params={"limit": 5})
assert response.status_code == 200
body = response.json()
assert body["total"] == 1
assert body["items"][0]["excerpt"] == "刚从 Cursor 同步进来的原话"
assert body["items"][0]["truncated"] is False
assert body["items"][0]["source_id"] == "cursor"
@pytest.mark.anyio
async def test_search_answers_with_no_model_profile_configured(tmp_path) -> None:
"""The whole point of C-02: no generation model, and still an answer."""
app, transport = _client(tmp_path)
async with app.router.lifespan_context(app), transport as client:
app.state.evidence_store.put(message(verbatim="昨天晚上重新打了一个包,抽取终于跑通了"))
assert app.state.model_profile_store.active(app.state.secret_store) is None
response = await client.get("/api/v1/conversations/search", params={"query": "抽取终于"})
assert response.status_code == 200
body = response.json()
assert body["empty_reason"] is None
assert len(body["hits"]) == 1
assert "抽取终于跑通了" in body["hits"][0]["excerpt"]
@pytest.mark.anyio
async def test_search_groups_hits_and_keeps_title_separate_from_match(tmp_path) -> None:
app, transport = _client(tmp_path)
async with app.router.lifespan_context(app), transport as client:
app.state.evidence_store.put(message(verbatim="发布准备", message_id="title"))
app.state.evidence_store.put(
message(
verbatim="第一次提到公证",
message_id="match-1",
created_at=datetime(2026, 9, 3, 1, 18, tzinfo=UTC),
)
)
app.state.evidence_store.put(
message(
verbatim="第二次提到公证",
message_id="match-2",
created_at=datetime(2026, 9, 3, 1, 19, tzinfo=UTC),
)
)
response = await client.get("/api/v1/conversations/search", params={"query": "公证"})
assert response.status_code == 200
assert len(response.json()["hits"]) == 1
assert response.json()["hits"][0]["title"] == "发布准备"
assert response.json()["hits"][0]["excerpt"] == "第二次提到公证"
assert response.json()["hits"][0]["turn_count"] == 3
assert response.json()["hits"][0]["turn_offset"] == 0
@pytest.mark.anyio
async def test_nothing_imported_is_told_apart_from_nothing_matching(tmp_path) -> None:
app, transport = _client(tmp_path)
async with app.router.lifespan_context(app), transport as client:
empty = await client.get("/api/v1/conversations/search", params={"query": "抽取终于"})
assert empty.status_code == 200
assert empty.json()["empty_reason"] == "nothing_imported"
app.state.evidence_store.put(message(verbatim="周末想去爬山"))
unmatched = await client.get("/api/v1/conversations/search", params={"query": "量子纠缠"})
assert unmatched.status_code == 200
assert unmatched.json()["empty_reason"] == "no_matches"
@pytest.mark.anyio
async def test_a_broken_index_is_reported_rather_than_shown_as_no_matches(tmp_path) -> None:
app, transport = _client(tmp_path)
async with app.router.lifespan_context(app), transport as client:
app.state.evidence_store.put(message(verbatim="抽取终于跑通了"))
with sqlite3.connect(app.state.evidence_store.database_path) as connection:
connection.execute("DROP TABLE IF EXISTS evidence_search")
response = await client.get("/api/v1/conversations/search", params={"query": "抽取终于"})
assert response.status_code == 503
assert response.json()["detail"] == "conversation_search_unavailable"
@pytest.mark.anyio
async def test_a_hit_opens_the_conversation_it_came_from(tmp_path) -> None:
"""Unverifiable facts read like invented ones, so the original has to open."""
app, transport = _client(tmp_path)
async with app.router.lifespan_context(app), transport as client:
app.state.evidence_store.put(
message(
verbatim="抽取终于跑通了",
message_id="m-1",
created_at=datetime(2026, 9, 3, 1, 17, tzinfo=UTC),
)
)
app.state.evidence_store.put(
message(
verbatim="那就先把包发出去",
message_id="m-2",
created_at=datetime(2026, 9, 3, 1, 19, tzinfo=UTC),
)
)
found = await client.get("/api/v1/conversations/search", params={"query": "抽取终于"})
evidence_id = found.json()["hits"][0]["evidence_id"]
transcript = await client.get(f"/api/v1/conversations/{evidence_id}")
assert transcript.status_code == 200
assert [item["verbatim"] for item in transcript.json()["messages"]] == [
"抽取终于跑通了",
"那就先把包发出去",
]
@pytest.mark.anyio
async def test_assistant_only_conversation_opens_in_time_order(tmp_path) -> None:
app, transport = _client(tmp_path)
async with app.router.lifespan_context(app), transport as client:
first = app.state.evidence_store.put(
message(
verbatim="第一条助手回复",
message_id="assistant-only-1",
created_at=datetime(2026, 9, 3, 1, 17, tzinfo=UTC),
).model_copy(update={"role": "assistant", "written_by": "agent"})
).record
app.state.evidence_store.put(
message(
verbatim="第二条助手回复",
message_id="assistant-only-2",
created_at=datetime(2026, 9, 3, 1, 18, tzinfo=UTC),
).model_copy(update={"role": "assistant", "written_by": "agent"})
)
transcript = await client.get(f"/api/v1/conversations/{first.evidence_id}")
assert transcript.status_code == 200
assert transcript.json()["total_turns"] == 1
assert [item["verbatim"] for item in transcript.json()["messages"]] == [
"第一条助手回复",
"第二条助手回复",
]
@pytest.mark.anyio
async def test_conversation_browser_and_turn_pages_are_bounded(tmp_path) -> None:
app, transport = _client(tmp_path)
async with app.router.lifespan_context(app), transport as client:
first_id = ""
for turn in range(25):
stored = app.state.evidence_store.put(
message(
verbatim=f"用户问题 {turn}",
message_id=f"user-{turn:02d}",
created_at=datetime(2026, 9, 3, 1, turn, tzinfo=UTC),
)
)
if turn == 0:
first_id = stored.record.evidence_id
assistant = message(
verbatim=f"助手回答 {turn}",
message_id=f"assistant-{turn:02d}",
created_at=datetime(2026, 9, 3, 1, turn, 30, tzinfo=UTC),
)
app.state.evidence_store.put(
assistant.model_copy(update={"role": "assistant", "written_by": "agent"})
)
app.state.evidence_store.put(
message(
verbatim="Codex 的同名会话",
source_id="codex",
conversation_id="conversation-1",
message_id="codex-1",
)
)
browser = await client.get("/api/v1/conversation-browser")
recent = await client.get(f"/api/v1/conversations/{first_id}", params={"limit": 20})
older = await client.get(
f"/api/v1/conversations/{first_id}", params={"offset": 20, "limit": 20}
)
assert browser.status_code == 200
assert browser.json()["total"] == 2
assert {
(item["source_id"], item["conversation_id"]) for item in browser.json()["items"]
} == {("cursor", "conversation-1"), ("codex", "conversation-1")}
assert recent.json()["total_turns"] == 25
assert recent.json()["has_more"] is True
assert len(recent.json()["messages"]) == 40
assert older.json()["has_more"] is False
assert len(older.json()["messages"]) == 10
@pytest.mark.anyio
async def test_conversation_memories_returns_only_linked_current_memories(tmp_path) -> None:
app, transport = _client(tmp_path)
async with app.router.lifespan_context(app), transport as client:
linked = app.state.evidence_store.put(
message(verbatim="发布前确认签名", message_id="linked")
).record
other = app.state.evidence_store.put(
message(
verbatim="另一段会话",
conversation_id="other-conversation",
message_id="other",
)
).record
app.state.person_writes.create(
MemoryDraft(
user_id="local",
kind=MemoryKind.WORKING,
content="发布前需要确认签名。",
scopes=("all-apps",),
evidence_ids=(linked.evidence_id,),
),
request_id="linked-memory",
)
app.state.person_writes.create(
MemoryDraft(
user_id="local",
kind=MemoryKind.WORKING,
content="不属于这段对话。",
scopes=("all-apps",),
evidence_ids=(other.evidence_id,),
),
request_id="other-memory",
)
response = await client.get(f"/api/v1/conversations/{linked.evidence_id}/memories")
assert response.status_code == 200
assert [item["content"] for item in response.json()["items"]] == ["发布前需要确认签名。"]
@@ -0,0 +1,144 @@
"""Readiness never waits for picking up work an earlier run left unfinished.
A 1.0.4 home with pending imports and write jobs used to resume all of it
before the lifespan yielded, so /health did not answer within the Runtime's
60s budget and the Runtime killed a healthy Core, which then began the same
recovery again. Recovery now runs after readiness.
"""
import asyncio
import threading
import time
import pytest
from httpx import ASGITransport, AsyncClient
from pydantic import SecretStr
from one_memory import hy_sdk_migration
from one_memory.api import create_app
from one_memory.config import Settings
from one_memory.hy_sdk_migration import MetadataRepairResult
from one_memory.secrets import InMemorySecretStore
pytestmark = pytest.mark.fresh_sdk
def application(tmp_path):
return create_app(
Settings(
data_dir=tmp_path / "data",
source_home=tmp_path,
api_token=SecretStr("ready-test"),
imports_enabled=False,
automatic_memory_processing=False,
live_sync_enabled=False,
),
secret_store=InMemorySecretStore(),
)
def client_for(app):
return AsyncClient(
transport=ASGITransport(app=app),
base_url="http://test",
headers={"Authorization": "Bearer ready-test"},
)
async def health(client):
response = await client.get("/api/v1/health")
assert response.status_code == 200
return response.json()
async def wait_for_recovery(client, *, seconds=10.0):
deadline = time.monotonic() + seconds
while time.monotonic() < deadline:
body = await health(client)
if body["recovery"] in {"done", "failed"}:
return body
await asyncio.sleep(0.01)
raise AssertionError("recovery did not finish")
def test_core_is_ready_while_recovery_is_still_running(tmp_path, monkeypatch):
release = threading.Event()
entered = threading.Event()
def slow_repair(**_):
entered.set()
release.wait(10)
return MetadataRepairResult(None, 0, 0, 0)
monkeypatch.setattr(hy_sdk_migration, "repair_sdk_metadata_from_bundle", slow_repair)
async def scenario():
app = application(tmp_path)
try:
started = time.monotonic()
async with app.router.lifespan_context(app), client_for(app) as client:
# Startup finished although the first recovery step cannot.
assert time.monotonic() - started < 5
assert await asyncio.to_thread(entered.wait, 5)
body = await health(client)
assert body["status"] == "ok"
assert body["recovery"] == "running"
# Nothing after the blocked step has been started yet: the
# write drain in particular would hold the single writer.
assert app.state.memory_write_drain_task is None
# Reads answer during recovery.
assert (await client.get("/api/v1/sources")).status_code == 200
release.set()
body = await wait_for_recovery(client)
assert body["recovery"] == "done"
assert body["recovery_failed_steps"] == []
assert app.state.memory_write_drain_task is not None
finally:
release.set()
asyncio.run(scenario())
def test_a_failed_recovery_step_does_not_stop_readiness_or_later_steps(tmp_path, monkeypatch):
def broken_repair(**_):
raise OSError("bundle unreadable")
monkeypatch.setattr(hy_sdk_migration, "repair_sdk_metadata_from_bundle", broken_repair)
async def scenario():
app = application(tmp_path)
async with app.router.lifespan_context(app), client_for(app) as client:
body = await wait_for_recovery(client)
assert body["status"] == "ok"
assert body["recovery"] == "failed"
assert body["recovery_failed_steps"] == ["sdk_metadata_repair"]
# The steps after the failed one still ran.
assert app.state.memory_write_drain_task is not None
asyncio.run(scenario())
def test_shutdown_during_recovery_stops_it_without_starting_more(tmp_path, monkeypatch):
release = threading.Event()
entered = threading.Event()
def slow_repair(**_):
entered.set()
release.wait(10)
return MetadataRepairResult(None, 0, 0, 0)
monkeypatch.setattr(hy_sdk_migration, "repair_sdk_metadata_from_bundle", slow_repair)
async def scenario():
app = application(tmp_path)
try:
async with app.router.lifespan_context(app):
assert await asyncio.to_thread(entered.wait, 5)
recovery = app.state.recovery_task
# Shutdown cancelled recovery before it could start the drain.
assert recovery.done()
assert app.state.memory_write_drain_task is None
finally:
release.set()
asyncio.run(scenario())
@@ -8,7 +8,6 @@ from test_onboarding_profile import _app, _client
from one_memory.memory_qa import MemoryAnswerer
from one_memory.model_probe import ModelProber
from one_memory.secrets import SecretStoreUnavailableError
def test_disabled_automatic_processing_does_not_digest_onboarding(tmp_path):
@@ -82,21 +81,3 @@ def test_model_probe_credential_lookup_is_off_loop(tmp_path, monkeypatch):
transport=httpx.MockTransport(lambda request: httpx.Response(503)),
)
asyncio.run(prober.probe(profile.profile_id))
def test_unavailable_credentials_keep_governed_graph_readable(tmp_path, monkeypatch):
async def scenario():
app = _app(tmp_path)
async with app.router.lifespan_context(app), _client(app) as client:
def denied(*args):
raise SecretStoreUnavailableError("private-system-error")
monkeypatch.setattr(app.state.model_profile_store, "active", denied)
response = await client.get("/api/v1/memories/graph")
assert response.status_code == 200
assert response.json()["hy_graph_available"] is False
assert "private-system-error" not in response.text
assert (await client.get("/api/v1/health")).status_code == 200
asyncio.run(scenario())
@@ -1,314 +0,0 @@
"""D93 test 2: read latency while pure-Python work holds the GIL.
Plan `读写隔离方案-20261009.html` section 8 row 2, fixing the D92 experiment
as a test. Three pure-Python busy threads stand in for Core's background work
(scan, extraction bookkeeping, recompute). Meanwhile the home summary, the
memory library page and the map are requested in a loop; every non-200 is an
error and latencies give p95.
Two targets:
* `core` (default): the routes are served by `api.create_app` in this same
process, next to the busy threads. This is today's architecture and gives
the baseline. It is expected to be slow; it is measured, not gated, unless
`ONE_MEMORY_D93_GATE=1`.
* `reader`: B's Reader is started as a separate process
(`one-memory serve --reader`) on the same library, the busy threads stay in
this process. This is the claim the plan makes: work in Core no longer
touches reads. Gated: 0 errors, home and library p95 below 1 second.
Knobs: ONE_MEMORY_D93_LOAD_SECONDS (gate 600, default 20),
ONE_MEMORY_D93_MEMORIES (default 600), ONE_MEMORY_D93_TARGET,
ONE_MEMORY_D93_REPORT (JSON path for the numbers).
Recall ("at least one hit on a known question") needs the HY supervisor and is
not exercised here; it stays with acceptance item 7b.
"""
from __future__ import annotations
import asyncio
import json
import os
import socket
import sqlite3
import subprocess
import sys
import threading
import time
from pathlib import Path
import httpx
import pytest
from pydantic import SecretStr
from one_memory import api
from one_memory.config import Settings
from one_memory.imports.grants import GRANT_SCHEMA
from one_memory.imports.models import EvidenceInput, EvidenceOrigin, OriginMetadata, WrittenBy
from one_memory.imports.storage import EvidenceStore
from one_memory.memory_governance import MemoryDraft, MemoryGovernanceStore, MemoryKind
from one_memory.memory_tasks import MemoryTaskOutput, MemoryTaskStore, evidence_ref
from tests.person_door import as_person
SECONDS = float(os.environ.get("ONE_MEMORY_D93_LOAD_SECONDS", "20"))
MEMORIES = int(os.environ.get("ONE_MEMORY_D93_MEMORIES", "600"))
TARGET = os.environ.get("ONE_MEMORY_D93_TARGET", "core")
GATE = os.environ.get("ONE_MEMORY_D93_GATE", "1" if TARGET == "reader" else "") == "1"
TOKEN = "d93-load-token"
AUTH = {"Authorization": f"Bearer {TOKEN}"}
ROUTES = (
("home", "/api/v1/home-summary", {}),
("library", "/api/v1/memory-library", {"limit": 100, "active_only": "true"}),
("map", "/api/v1/memories/graph", {"limit": 160}),
)
def _build_library(database: Path) -> None:
now = "2026-10-09T00:00:00+00:00"
with sqlite3.connect(database) as setup:
setup.executescript(GRANT_SCHEMA)
for source in ("cursor", "codex"):
setup.execute(
"INSERT INTO source_grants "
"(user_id,source_id,root_path,requested_path,root_identity,"
"candidate_kind,granted_at,revoked_at,revision) "
"VALUES ('local',?,?,?,'fixture','default',?,NULL,1)",
(source, f"/fixture/{source}", f"/fixture/{source}", now),
)
evidence = EvidenceStore(database)
tasks = MemoryTaskStore(database)
person = as_person(MemoryGovernanceStore(database))
for index in range(MEMORIES):
record = evidence.put(
EvidenceInput(
source_id=("cursor", "codex")[index % 2],
conversation_id=f"conversation-{index // 10}",
message_id=f"m-{index}",
correlation_id=f"m-{index}",
role="user",
written_by=WrittenBy.PERSON,
verbatim=f"第 {index} 条对话原文,讨论读写隔离和记忆页的加载速度。" * 4,
origin_meta={EvidenceOrigin.FILE: OriginMetadata()},
)
).record
memory = person.create(
MemoryDraft(
user_id="local",
kind=MemoryKind.CHAT,
content=f"记忆 {index}:读写隔离要让记忆页在后台忙时也能打开。",
scopes=("all-apps",),
evidence_ids=(record.evidence_id,),
),
request_id=f"create-{index}",
)
tasks.save_output(
MemoryTaskOutput(
task_id=f"task-{index}",
memory_id=memory.memory_id,
memory_revision=memory.revision,
hy_memory_id=f"hy-{index}",
owner="user",
layer="L2_FACT",
engine_version="fixture",
support_refs=(evidence_ref(record),),
)
)
class _Busy:
"""Three pure-Python threads: no I/O, no sleep, they only want the GIL."""
def __init__(self, count: int = 3) -> None:
self.stop = threading.Event()
self.loops = [0] * count
self.threads = [
threading.Thread(target=self._spin, args=(index,), daemon=True)
for index in range(count)
]
def _spin(self, index: int) -> None:
while not self.stop.is_set():
total = 0
for value in range(20_000):
total += value * value % 7
self.loops[index] += 1
def __enter__(self):
for thread in self.threads:
thread.start()
return self
def __exit__(self, *_):
self.stop.set()
for thread in self.threads:
thread.join(timeout=5)
_SPIN = """
import threading
def spin():
while True:
total = 0
for value in range(20_000):
total += value * value % 7
for _ in range(3):
threading.Thread(target=spin, daemon=True).start()
threading.Event().wait()
"""
def _p95(values: list[float]) -> float | None:
if not values:
return None
ordered = sorted(values)
return ordered[min(len(ordered) - 1, int(round(0.95 * (len(ordered) - 1))))]
async def _drive(client: httpx.AsyncClient, deadline: float) -> dict:
stats = {name: {"ok": 0, "errors": {}, "latency": []} for name, _, _ in ROUTES}
while time.monotonic() < deadline:
for name, url, params in ROUTES:
started = time.monotonic()
try:
response = await client.get(url, params=params, headers=AUTH, timeout=30)
code = str(response.status_code)
except httpx.HTTPError as error:
code = type(error).__name__
elapsed = time.monotonic() - started
entry = stats[name]
entry["latency"].append(elapsed)
if code == "200":
entry["ok"] += 1
else:
entry["errors"][code] = entry["errors"].get(code, 0) + 1
return {
name: {
"requests": len(entry["latency"]),
"ok": entry["ok"],
"errors": entry["errors"],
"p95_seconds": _p95(entry["latency"]),
"max_seconds": max(entry["latency"], default=None),
}
for name, entry in stats.items()
}
def _core_app(data_dir: Path, root: Path):
"""Core as the app runs it (lifespan included), background processing off."""
return api.create_app(
Settings(
data_dir=data_dir,
source_home=root / "home",
api_token=SecretStr(TOKEN),
automatic_memory_processing=False,
live_sync_enabled=False,
),
source_catalog={},
)
def _free_port() -> int:
with socket.socket() as probe:
probe.bind(("127.0.0.1", 0))
return probe.getsockname()[1]
def _start_reader(data_dir: Path) -> tuple[subprocess.Popen, str]:
pytest.importorskip("one_memory.reader_app", reason="B (Reader) not in this build")
port = _free_port()
env = {
**os.environ,
"ONE_MEMORY_DATA_DIR": str(data_dir),
"ONE_MEMORY_READER_PORT": str(port),
"ONE_MEMORY_API_TOKEN": TOKEN,
"ONE_MEMORY_CORE_PORT": str(_free_port()), # nothing there: Core-only parts degrade
}
process = subprocess.Popen(
[sys.executable, "-m", "one_memory.cli", "serve", "--reader"],
env=env,
stdout=subprocess.DEVNULL,
stderr=subprocess.PIPE,
)
base = f"http://127.0.0.1:{port}"
for _ in range(100):
try:
if httpx.get(f"{base}/health", timeout=0.5).status_code < 500:
return process, base
except httpx.HTTPError:
pass
if process.poll() is not None:
raise AssertionError(f"Reader exited: {process.stderr.read()[-2000:]!r}")
time.sleep(0.1)
process.kill()
raise AssertionError("Reader did not answer /health within 10 seconds")
def test_reads_stay_up_while_python_threads_hold_the_gil(tmp_path):
data_dir = tmp_path / "data"
data_dir.mkdir()
database = data_dir / "one-memory.sqlite3"
_build_library(database)
async def run(client):
# Warm once outside the measurement so lazy imports do not count.
for _, url, params in ROUTES:
await client.get(url, params=params, headers=AUTH, timeout=30)
idle = await _drive(client, time.monotonic() + min(5.0, SECONDS))
if TARGET == "reader":
# The busy threads belong to Core, not to this client: put them in
# a process of their own so the client does not queue for the GIL.
spinner = subprocess.Popen([sys.executable, "-c", _SPIN])
try:
loaded = await _drive(client, time.monotonic() + SECONDS)
finally:
spinner.terminate()
spinner.wait(timeout=10)
return idle, loaded, [1, 1, 1]
with _Busy() as busy:
loaded = await _drive(client, time.monotonic() + SECONDS)
return idle, loaded, busy.loops
reader = None
try:
if TARGET == "reader":
reader, base = _start_reader(data_dir)
transport = httpx.AsyncHTTPTransport()
application = None
else:
base = "http://test"
application = _core_app(data_dir, tmp_path)
transport = httpx.ASGITransport(app=application)
async def main():
async with httpx.AsyncClient(transport=transport, base_url=base) as client:
if application is None:
return await run(client)
async with application.router.lifespan_context(application):
return await run(client)
idle, loaded, loops = asyncio.run(main())
finally:
if reader is not None:
reader.terminate()
reader.wait(timeout=10)
report = {
"target": TARGET,
"memories": MEMORIES,
"seconds": SECONDS,
"busy_loops": loops,
"idle": idle,
"loaded": loaded,
}
path = os.environ.get("ONE_MEMORY_D93_REPORT")
if path:
Path(path).write_text(json.dumps(report, ensure_ascii=False, indent=2), encoding="utf-8")
print(json.dumps(report, ensure_ascii=False))
assert all(count > 0 for count in loops), "busy threads never ran"
assert all(entry["requests"] > 0 for entry in loaded.values())
if GATE:
for name, entry in loaded.items():
assert entry["errors"] == {}, f"{name}: {entry['errors']}"
for name in ("home", "library"):
assert loaded[name]["p95_seconds"] < 1.0, f"{name} p95 {loaded[name]['p95_seconds']}"
@@ -1,86 +1,5 @@
"""A host that saves our injection back must not mint a new user decision."""
import sqlite3
from contextlib import closing
from uuid import uuid4
from one_memory.change_outbox import ChangeOutbox, evidence_event_id
from one_memory.derived_echo import render_reference
from one_memory.imports.models import EvidenceInput, EvidenceOrigin, OriginMetadata, WrittenBy
from one_memory.imports.storage import EvidenceStore
from one_memory.session_inbox import SessionInbox
from one_memory.session_registry import SessionRegistry
def _evidence(message_id: str, verbatim: str) -> EvidenceInput:
return EvidenceInput(
source_id="workbuddy",
conversation_id="session-b",
message_id=message_id,
role="assistant",
written_by=WrittenBy.PERSON,
verbatim=verbatim,
origin_meta={EvidenceOrigin.FILE: OriginMetadata(raw_meta={"root": "proj"})},
)
def test_a_restatement_is_not_broadcast_and_a_correction_still_is(tmp_path) -> None:
path = tmp_path / "one-memory.sqlite3"
store = EvidenceStore(path)
registry = SessionRegistry(path)
registry.initialize()
inbox = SessionInbox(path)
inbox.initialize()
opened = registry.open(
installation_id=str(uuid4()),
host_id="workbuddy",
native_session_id="session-b",
runtime_instance="runtime-1",
workspace="/work/same-project",
)
inbox.subscribe(opened.session_key, ("workbuddy",), permission_epoch=1)
with closing(sqlite3.connect(path)) as db:
db.row_factory = sqlite3.Row
db.execute("BEGIN IMMEDIATE")
decision = store.put(_evidence("decision", "周五前交一版"), connection=db)
ChangeOutbox(path).record(
db,
event_id=evidence_event_id(decision.record.evidence_id, decision.record.revision),
cause="import",
evidence_id=decision.record.evidence_id,
evidence_revision=decision.record.revision,
permission_epoch=0,
)
db.commit()
page = inbox.pull(opened.session_key, cursor=None, limit=8)
injected = render_reference(page.events)
with closing(sqlite3.connect(path)) as db:
db.row_factory = sqlite3.Row
db.execute("BEGIN IMMEDIATE")
store.put(_evidence("echo", injected), connection=db)
db.commit()
with closing(sqlite3.connect(path)) as db:
assert db.execute("SELECT COUNT(*) FROM change_outbox").fetchone()[0] == 1
echoed = inbox.pull(opened.session_key, cursor=page.cursor, limit=8)
assert echoed.events == ()
with closing(sqlite3.connect(path)) as db:
db.row_factory = sqlite3.Row
db.execute("BEGIN IMMEDIATE")
correction = store.put(
_evidence("correction", injected + "\n不对,改成下周三"), connection=db
)
ChangeOutbox(path).record(
db,
event_id=evidence_event_id(correction.record.evidence_id, correction.record.revision),
cause="import",
evidence_id=correction.record.evidence_id,
evidence_revision=correction.record.revision,
permission_epoch=0,
)
db.commit()
corrected = inbox.pull(opened.session_key, cursor=page.cursor, limit=8)
assert len(corrected.events) == 1
def test_stripping_removes_the_notice_block_as_well_as_the_memory_block() -> None:
from one_memory.derived_echo import strip_injected_memory
@@ -561,11 +561,11 @@ def test_database_error_records_safe_route_and_sqlite_code(tmp_path, monkeypatch
error.sqlite_errorcode = sqlite3.SQLITE_BUSY
error.sqlite_errorname = "SQLITE_BUSY"
try:
record_database_error("core.api", error, route_path="/api/v1/context/search")
record_database_error("core.api", error, route_path="/api/v2/context/search")
finally:
reset_diagnostics()
payload = json.loads(receipt.path.read_text().splitlines()[-1])
assert payload["details"]["route_path"] == "/api/v1/context/search"
assert payload["details"]["route_path"] == "/api/v2/context/search"
assert payload["details"]["sqlite_errorcode"] == sqlite3.SQLITE_BUSY
assert payload["details"]["sqlite_errorname"] == "SQLITE_BUSY"
@@ -7,7 +7,6 @@ from pydantic import SecretStr
from one_memory.api import create_app
from one_memory.config import Settings
from one_memory.imports.models import EvidenceInput, EvidenceOrigin, OriginMetadata, WrittenBy
def headers(*, scopes=("all-apps",), tools=("om_recall",)):
@@ -51,12 +50,14 @@ def test_external_marker_is_not_an_installation_grant(tmp_path):
)
assert approved.status_code == 200
denied_tool = await client.post(
"/api/v1/memories/ask", json={"question": "history"}, headers=headers()
"/api/v2/memories/ask", json={"question": "history"}, headers=headers()
)
assert denied_tool.status_code == 403
no_raw = await client.get(
"/api/v1/conversations/search",
params={"query": "history"},
# om_conversations reaches Core only as the v2 raw lane, and only
# with a source grant: all-apps alone opens no raw source.
no_raw = await client.post(
"/api/v2/context/search",
json={"query": "history", "kinds": ["raw"]},
headers=headers(tools=("om_conversations",)),
)
assert no_raw.status_code == 403
@@ -99,48 +100,3 @@ def test_external_diagnostic_trace_uses_its_own_read_only_tool(tmp_path):
assert denied.status_code == 403
asyncio.run(scenario())
def test_raw_source_grants_filter_before_limit_and_private_body_loading(tmp_path):
async def scenario():
app = create_app(
Settings(
data_dir=tmp_path,
source_home=tmp_path,
api_token=SecretStr("test"),
automatic_memory_processing=False,
live_sync_enabled=False,
),
source_catalog={},
)
async with (
app.router.lifespan_context(app),
AsyncClient(
transport=ASGITransport(app=app),
base_url="http://test",
headers={"Authorization": "Bearer test"},
) as client,
):
for source in ("allowed", "denied"):
app.state.evidence_store.put(
EvidenceInput(
source_id=source,
conversation_id="one",
message_id="one",
role="user",
written_by=WrittenBy.PERSON,
verbatim="needle " + source,
origin_meta={EvidenceOrigin.FILE: OriginMetadata()},
)
)
result = await client.get(
"/api/v1/conversations/search",
params={"query": "needle", "limit": 1},
headers=headers(scopes=("all-apps", "source:allowed"), tools=("om_conversations",)),
)
assert result.status_code == 200
assert result.json()["imported_messages"] == 1
assert [hit["source_id"] for hit in result.json()["hits"]] == ["allowed"]
assert "needle denied" not in result.text
asyncio.run(scenario())
+7
View File
@@ -45,6 +45,9 @@ def test_health_reports_the_single_core() -> None:
"memory_backfill_error": None,
"content_candidate_backfill": "idle",
"content_candidate_backfill_failure": None,
# No lifespan ran, so recovery has not begun.
"recovery": "pending",
"recovery_failed_steps": [],
}
@@ -210,6 +213,8 @@ def test_lifespan_drains_committed_pending_dispatch_through_enqueue(tmp_path, mo
async def scenario() -> None:
async with app.router.lifespan_context(app):
# Replayed by the recovery that runs after readiness.
await app.state.recovery_task
assert record.evidence_id in seen
assert evidence.take_all_stable_records() == ()
@@ -434,6 +439,7 @@ def test_a_backfill_that_fails_says_so_rather_than_disappearing(tmp_path, monkey
app.router.lifespan_context(app),
AsyncClient(transport=ASGITransport(app=app), base_url="http://test") as client,
):
await app.state.recovery_task
await app.state.memory_backfill
response = await client.get("/api/v1/health")
@@ -497,6 +503,7 @@ def test_a_failed_catch_up_leaves_a_traceback_behind(tmp_path, monkeypatch, capl
app.router.lifespan_context(app),
AsyncClient(transport=ASGITransport(app=app), base_url="http://test") as client,
):
await app.state.recovery_task
with suppress(BaseException):
await app.state.memory_backfill
return (await client.get("/api/v1/health")).json()
-164
View File
@@ -1,164 +0,0 @@
import sqlite3
from one_memory.home_summary import read_home_summary
def database(tmp_path):
path = tmp_path / "summary.sqlite3"
with sqlite3.connect(path) as db:
db.executescript("""
CREATE TABLE governed_memories(memory_id TEXT, user_id TEXT, status TEXT,
deleted_at TEXT, content TEXT, revision INTEGER, updated_at TEXT);
CREATE TABLE governed_memory_evidence(memory_id TEXT, evidence_id TEXT, active INTEGER);
CREATE TABLE governed_memory_versions(memory_id TEXT, revision INTEGER, action TEXT);
CREATE TABLE evidence(evidence_id TEXT, user_id TEXT, source_id TEXT);
CREATE TABLE source_grants(user_id TEXT, source_id TEXT, revoked_at TEXT);
""")
db.execute("INSERT INTO evidence VALUES ('e1','local','cursor')")
db.execute("INSERT INTO evidence VALUES ('e2','local','workbuddy')")
return path
def put(
path,
key,
*,
status="confirmed",
user="local",
content="Memory",
time="2026-09-19T01:00:00+00:00",
deleted=None,
):
with sqlite3.connect(path) as db:
db.execute(
"INSERT INTO governed_memories VALUES (?,?,?,?,?,?,?)",
(key, user, status, deleted, content, 2, time),
)
db.execute("INSERT INTO governed_memory_evidence VALUES (?, 'e1', 1)", (key,))
db.execute("INSERT INTO governed_memory_versions VALUES (?, 2, 'correct')", (key,))
def test_summary_counts_beyond_page_limit_and_excludes_ineligible_rows(tmp_path):
path = database(tmp_path)
for i in range(503):
put(path, str(i))
put(path, "superseded", status="superseded")
put(path, "deleted", deleted="2026-09-19T00:00:00+00:00")
put(path, "other", user="other")
result = read_home_summary(path)
assert result.memory_count == 503
assert result.latest is not None
assert result.latest.source_ids == ("cursor",)
assert result.latest.revision == 2
assert result.latest.action == "correct"
def test_summary_revocation_excludes_whole_mixed_memory_not_only_citation(tmp_path):
path = database(tmp_path)
put(path, "mixed", time="2026-09-19T03:00:00+00:00")
put(path, "retained")
with sqlite3.connect(path) as db:
db.execute("INSERT INTO governed_memory_evidence VALUES ('mixed','e2',1)")
db.execute("INSERT INTO source_grants VALUES ('local','workbuddy','revoked')")
result = read_home_summary(path)
assert result.memory_count == 1
assert result.latest.memory_id == "retained"
with sqlite3.connect(path) as db:
db.execute("UPDATE governed_memory_evidence SET active=0 WHERE evidence_id='e2'")
assert read_home_summary(path).memory_count == 2
def test_latest_is_bounded_and_other_users_revocations_do_not_hide_local_data(tmp_path):
path = database(tmp_path)
put(path, "first")
put(path, "latest", time="2026-09-19T02:00:00+00:00", content="x" * 2000)
with sqlite3.connect(path) as db:
db.execute("INSERT INTO source_grants VALUES ('other','cursor','revoked')")
result = read_home_summary(path)
assert result.memory_count == 2
assert result.latest.memory_id == "latest"
assert result.latest.truncated is True
assert len(result.latest.content) == 320
def test_empty_summary_is_distinct_from_failed_read(tmp_path):
path = database(tmp_path)
result = read_home_summary(path)
assert result.memory_count == 0
assert result.latest is None
def test_a_read_past_its_deadline_returns_stale_projection(tmp_path):
"""A timed-out homepage read must not escape as SQLITE_INTERRUPT.
Real trigger: `/api/v1/home-summary` returned a bare OperationalError,
sqlite_errorcode 9, because the progress handler installed to bound the
read is itself what interrupts it and nothing caught the result.
2026-10-12: instead of raising HomeSummaryUnavailable (which still 500ed
on builds before fab1f6b6), the read answers a `stale` projection so the
UI keeps its last good summary on screen.
"""
import time as time_module
import one_memory.home_summary as home_summary
path = database(tmp_path)
# Enough rows that the progress handler actually gets to tick.
for index in range(20000):
put(path, str(index))
# Expire the deadline up front: the progress handler installed to bound
# the read then interrupts on its first tick, which is exactly the
# slow-machine path.
real = time_module.monotonic
home_summary.time = type("ExpiredClock", (), {"monotonic": staticmethod(lambda: real() + 100)})
try:
result = read_home_summary(path)
except Exception as error: # noqa: BLE001 - nothing may escape
raise AssertionError(
f"a bare {type(error).__name__} "
f"(sqlite_errorcode={getattr(error, 'sqlite_errorcode', '-')}) escaped"
) from error
finally:
home_summary.time = time_module
assert result.stale is True
assert result.memory_count == 0
assert result.latest is None
def test_home_summary_api_is_authenticated_and_has_typed_empty_state(tmp_path):
import asyncio
from httpx import ASGITransport, AsyncClient
from pydantic import SecretStr
from one_memory.api import create_app
from one_memory.config import Settings
async def scenario():
home = tmp_path / "home"
home.mkdir()
app = create_app(
Settings(
data_dir=tmp_path / "data",
source_home=home,
api_token=SecretStr("synthetic-token"),
automatic_memory_processing=False,
live_sync_enabled=False,
),
source_catalog={},
)
async with (
app.router.lifespan_context(app),
AsyncClient(transport=ASGITransport(app=app), base_url="http://test") as client,
):
denied = await client.get("/api/v1/home-summary")
assert denied.status_code in (401, 403)
result = await client.get(
"/api/v1/home-summary", headers={"Authorization": "Bearer synthetic-token"}
)
assert result.status_code == 200
assert result.json()["memory_count"] == 0
assert result.json()["latest"] is None
assert result.json()["observed_at"]
asyncio.run(scenario())
+4 -9
View File
@@ -97,15 +97,10 @@ def test_importing_and_searching_leaves_no_original_query_or_token_behind(tmp_pa
started.raise_for_status()
job = await app.state.import_supervisor.wait(started.json()["job_id"])
assert job.status == "completed" and job.stored == 1
found = await client.post("/api/v1/context/search", json={"query": SECRET_QUERY})
found.raise_for_status()
# The turn is reachable, so there was something to leak.
assert any(
SECRET_TURN in message["text"]
for item in found.json()["items"]
if item["kind"] == "raw"
for message in item["messages"]
)
# The search is logged as a request whatever the SDK answers; the
# query it carried must not be part of that record.
searched = await client.post("/api/v2/context/search", json={"query": SECRET_QUERY})
assert searched.status_code in {200, 503}
try:
asyncio.run(scenario())
@@ -79,6 +79,11 @@ ALLOWED = {
"normal WAL-aware read-only view",
calls=2,
),
("sdk_migration_log.py", "migration_rows"): Allowed(
"a read-only snapshot of migration attempts for the migration log, taken "
"before the access layer exists (Core refusing a legacy home, sdk-migrate)",
opened_with="?mode=ro",
),
("library_upgrade.py", "_backup_problem"): Allowed(
"the finished upgrade backup, checked read-only before migration relies on it",
opened_with="?mode=ro&immutable=1",
@@ -92,6 +97,15 @@ ALLOWED = {
"the memory ids a migrated home must still read back",
opened_with="?mode=ro",
),
("hy_sdk_migration.py", "_legacy_pools"): Allowed(
"each pre-cutover bundle's copy of the legacy database, read-only, to "
"recover the source and times historical cutovers dropped",
opened_with="?mode=ro",
),
("hy_sdk_migration.py", "_repair_coding"): Allowed(
"the SDK's own Coding/Working stores, not the main database: the one-time "
"metadata repair writes them at startup, before the SDK client opens"
),
("sdk_vector_reattach.py", "_stored_names"): Allowed(
"Chroma's own chroma.sqlite3, read-only, to list collection names before "
"the offline migration opens Chroma",
@@ -120,10 +120,10 @@ def test_manual_import_reports_segment_then_follows_normal_authorize_and_dedup(t
assert connected.status_code == 200 and connected.json()["failed_sources"] == {}
job = await app.state.import_supervisor.wait(connected.json()["jobs"][0]["job_id"])
assert job.stored == 2 and job.errors == 0
found = (
await client.get("/api/v1/conversations/search", params={"query": "保留来源"})
).json()
assert found["hits"][0]["created_at"] is None
found = app.state.evidence_store.search_conversations(
"保留来源", user_id="local", limit=5
)
assert found[0].created_at is None
again = await client.post(
"/api/v1/import-jobs", json={"source_id": receipt["source_id"]}
)
@@ -6,7 +6,7 @@ from pathlib import Path
import pytest
from one_memory import context_memory, home_summary, memory_library, source_read_access
from one_memory import context_memory, source_read_access
from one_memory.imports.grants import GRANT_SCHEMA
from one_memory.imports.storage import EvidenceStore
from one_memory.memory_evidence_freshness import register_memory_freshness
@@ -18,15 +18,10 @@ def _source(module) -> str:
def _predicates():
home = re.search(r'readable = """(.*?)"""', _source(home_summary), re.S).group(1)
library = _source(memory_library)
library_clauses = re.findall(r'clauses\.append\("""(NOT EXISTS.*?)"""\)', library, re.S)
access = re.search(
r"_MEMORY_READABLE = \"\"\"(.*?)\"\"\"", _source(source_read_access), re.S
).group(1)
return {
"home": (home, ("local",)),
"library": ("m.user_id = ? AND " + " AND ".join(library_clauses), ("local",)),
"context": (context_memory._READABLE, ()),
"source_read_access": (access, ()),
}
@@ -45,7 +40,7 @@ def db(tmp_path):
conn.close()
@pytest.mark.parametrize("name", ["home", "library", "context", "source_read_access"])
@pytest.mark.parametrize("name", ["context", "source_read_access"])
def test_citation_subqueries_use_the_memory_primary_key(db, name):
predicate, args = _predicates()[name]
statement = f"SELECT COUNT(*) FROM governed_memories m WHERE {predicate}"
@@ -134,10 +134,10 @@ def test_memory_content_normal_api_import_preserves_kind_and_dedup_revoke(tmp_pa
if s["source_id"] == receipt["source_id"]
)
assert source["sync_completeness"] == "one_time_snapshot"
found = (
await client.get("/api/v1/conversations/search", params={"query": "原始聊天"})
).json()
evidence = app.state.evidence_store.get(found["hits"][0]["evidence_id"])
found = app.state.evidence_store.search_conversations(
"原始聊天", user_id="local", limit=5
)
evidence = app.state.evidence_store.get(found[0].evidence_id)
assert evidence.content_kind is EvidenceContentKind.MEMORY_CONTENT
assert evidence.written_by is WrittenBy.UNKNOWN and evidence.created_at is None
assert evidence.verbatim == value["memories"][1]["content"]
@@ -145,7 +145,7 @@ def test_memory_content_normal_api_import_preserves_kind_and_dedup_revoke(tmp_pa
"/api/v1/import-jobs", json={"source_id": receipt["source_id"]}
)
assert (await app.state.import_supervisor.wait(again.json()["job_id"])).stored == 0
assert (await client.get("/api/v1/memories")).json() == []
assert app.state.memory_governance_store.list(user_id="local") == []
await client.post(f"/api/v1/sources/{receipt['source_id']}/revoke")
assert (
await client.get(f"/api/v1/sources/{receipt['source_id']}/import-receipt")
@@ -1,6 +1,5 @@
"""Independent synthetic read-path review; no lifespan, model or user database."""
import asyncio
import json
import sqlite3
import time
@@ -8,19 +7,15 @@ from contextlib import closing
from types import SimpleNamespace
import pytest
from httpx import ASGITransport, AsyncClient
from one_memory import api, source_read_access
from one_memory.config import Settings
from one_memory import source_read_access
from one_memory.context_memory import read_memory_context
from one_memory.context_models import ContextSearchRequest
from one_memory.home_summary import read_home_summary
from one_memory.imports.grants import GRANT_SCHEMA
from one_memory.imports.models import EvidenceInput, EvidenceOrigin, OriginMetadata, WrittenBy
from one_memory.imports.storage import EvidenceStore
from one_memory.memory_evidence_freshness import register_memory_freshness
from one_memory.memory_governance import MemoryDraft, MemoryGovernanceStore, MemoryKind
from one_memory.memory_library import read_memory_library
from one_memory.memory_tasks import MemoryTaskOutput, MemoryTaskStore, evidence_ref
from one_memory.source_read_access import SourceReadAccess
from tests.person_door import as_person
@@ -89,8 +84,6 @@ def context(case, *, deadline=None):
def visible(case, memory=None):
return (
read_memory_library(case.path).total,
read_home_summary(case.path).memory_count,
len(context(case).items),
len(SourceReadAccess(case.path).memories([memory or case.memory])),
)
@@ -104,9 +97,9 @@ def revise(case):
def test_revised_evidence_remains_readable_after_freshness_retirement(case):
"""Evidence revision no longer hides library / context / source reads."""
assert visible(case) == (1, 1, 1, 1)
assert visible(case) == (1, 1)
revise(case)
assert visible(case) == (1, 1, 1, 1)
assert visible(case) == (1, 1)
assert case.governance.get(case.memory.memory_id, user_id="local") == case.memory
assert case.tasks.outputs(case.output.task_id) == (case.output,)
@@ -117,7 +110,7 @@ def test_revised_evidence_remains_readable_after_freshness_retirement(case):
def test_scope_only_revision_keeps_memory_readable_after_evidence_change(case, revise_first):
if revise_first:
revise(case)
assert visible(case) == (1, 1, 1, 1)
assert visible(case) == (1, 1)
changed = as_person(case.governance).set_scopes(
case.memory.memory_id,
user_id="local",
@@ -127,9 +120,9 @@ def test_scope_only_revision_keeps_memory_readable_after_evidence_change(case, r
)
assert changed.content == case.memory.content
if not revise_first:
assert visible(case, changed) == (1, 1, 1, 1)
assert visible(case, changed) == (1, 1)
revise(case)
assert visible(case, changed) == (1, 1, 1, 1)
assert visible(case, changed) == (1, 1)
def test_explicit_correct_remains_readable_but_never_overrides_revocation(case):
@@ -141,7 +134,7 @@ def test_explicit_correct_remains_readable_but_never_overrides_revocation(case):
expected_revision=case.memory.revision,
content="alpha beta explicit correction",
)
assert visible(case, corrected) == (1, 1, 1, 1)
assert visible(case, corrected) == (1, 1)
with closing(sqlite3.connect(case.path)) as db, db:
db.execute(
"""INSERT INTO source_grants
@@ -150,14 +143,14 @@ def test_explicit_correct_remains_readable_but_never_overrides_revocation(case):
VALUES ('local','cursor','/fixture','/fixture','fixture','default',?,?,1)""",
("2026-09-20T00:00:00Z", "2026-09-20T00:00:01Z"),
)
assert visible(case, corrected) == (0, 0, 0, 0)
assert visible(case, corrected) == (0, 0)
def test_missing_proof_is_legacy_compatibility_not_a_new_authorization(case):
with closing(sqlite3.connect(case.path)) as db, db:
db.execute("DELETE FROM memory_task_outputs")
revise(case)
assert visible(case) == (1, 1, 1, 1)
assert visible(case) == (1, 1)
result = read_memory_context(
case.path,
ContextSearchRequest(query="alpha beta"),
@@ -188,7 +181,6 @@ def test_revocation_before_freshness_snapshot_cannot_release_memory(case, monkey
monkeypatch.setattr(source_read_access, "register_memory_freshness", revoke_then_register)
returned = SourceReadAccess(case.path).memories([case.memory])
assert changes == [True]
assert read_memory_library(case.path).total == 0
assert returned == []
@@ -219,8 +211,7 @@ def _oversize(case, bad):
def test_material_too_large_to_verify_reads_as_current(case, bad):
"""D76: a budget overrun is not stale and must not fail the whole read."""
_oversize(case, bad)
assert read_home_summary(case.path).memory_count == 1
read_memory_library(case.path)
assert len(SourceReadAccess(case.path).memories([case.memory])) == 1
assert context(case).status != "unavailable"
@@ -231,40 +222,12 @@ def test_bad_proof_material_no_longer_makes_reads_unavailable(case, bad):
"UPDATE memory_task_outputs SET body=?",
('{"memory_revision":2,"memory_revision":2}',),
)
assert visible(case) == (1, 1, 1, 1)
assert visible(case) == (1, 1)
result = context(case)
assert result.status != "unavailable"
assert len(result.items) == 1
def test_route_serves_memories_whose_proof_is_too_large(case, tmp_path):
app = api.create_app(
Settings(
data_dir=tmp_path / "unused",
api_token="fixture-token",
automatic_memory_processing=False,
live_sync_enabled=False,
),
source_catalog={},
)
app.state.evidence_store = SimpleNamespace(database_path=case.path)
with closing(sqlite3.connect(case.path)) as db, db:
db.execute("UPDATE memory_task_outputs SET body=?", (" " * 65537,))
async def run():
async with AsyncClient(
transport=ASGITransport(app=app, raise_app_exceptions=False),
base_url="http://fixture",
headers={"Authorization": "Bearer fixture-token"},
) as client:
library = await client.get("/api/v1/memory-library")
home = await client.get("/api/v1/home-summary")
assert library.status_code == 200
assert home.status_code == 200
asyncio.run(run())
def test_expired_context_budget_never_reads_proofs(case):
result = context(case, deadline=time.monotonic() - 1)
assert result.status == "unavailable" and result.reason == "timeout_budget"
@@ -285,7 +248,7 @@ def test_indexed_unrelated_proofs_keep_small_read_bounded(case):
).fetchall()
assert any("memory_task_outputs_memory" in row[3] for row in plan)
started = time.monotonic()
assert visible(case) == (1, 1, 1, 1)
assert visible(case) == (1, 1)
elapsed = time.monotonic() - started
print(f"indexed 4000 unrelated proofs, four reads: {elapsed:.6f}s")
assert elapsed < 2
@@ -1,151 +0,0 @@
import asyncio
from httpx import ASGITransport, AsyncClient
from pydantic import SecretStr
from one_memory.api import create_app
from one_memory.config import Settings
from one_memory.imports.models import (
EvidenceInput,
EvidenceOrigin,
OriginMetadata,
WrittenBy,
)
def test_memory_governance_api_preserves_evidence_and_reports_cleanup(tmp_path) -> None:
async def scenario() -> None:
app = create_app(
Settings(
data_dir=tmp_path / "data",
source_home=tmp_path,
api_token=SecretStr("local-token"),
)
)
async with (
app.router.lifespan_context(app),
AsyncClient(
transport=ASGITransport(app=app),
base_url="http://test",
headers={
"Authorization": "Bearer local-token",
"Origin": "http://127.0.0.1:5173",
},
) as client,
):
evidence = app.state.evidence_store.put(
EvidenceInput(
source_id="cursor",
conversation_id="conversation",
message_id="message",
correlation_id="message",
role="user",
written_by=WrittenBy.PERSON,
verbatim="Original private words; token=local-secret-value",
origin_meta={
EvidenceOrigin.FILE: OriginMetadata(
raw_meta={"absolute_path": "/must/not/leave/api"}
)
},
)
).record
created = await client.post(
"/api/v1/memories",
json={
"request_id": "create-1",
"draft": {
"user_id": "local",
"kind": "chat",
"content": "Initial memory",
"scopes": ["all-apps"],
"evidence_ids": [evidence.evidence_id],
},
},
)
assert created.status_code == 200
memory_id = created.json()["memory_id"]
assert created.json()["status"] == "confirmed"
recalled = await client.post(
"/api/v1/recall",
json={"query": "Initial memory", "scope": "all-apps"},
)
assert recalled.status_code == 200
assert recalled.json()["state"] == "grounded"
assert recalled.json()["hits"][0]["memory_id"] == memory_id
corrected = await client.post(
f"/api/v1/memories/{memory_id}/correct",
json={
"request_id": "correct-1",
"expected_revision": 1,
"content": "Corrected memory",
"scopes": ["cursor"],
},
)
assert corrected.status_code == 200
detail = await client.get(f"/api/v1/memories/{memory_id}")
assert evidence.verbatim == "Original private words; token=local-secret-value"
assert detail.json()["evidence"][0]["verbatim"] == (
"Original private words; token=<redacted-secret>"
)
assert "absolute_path" not in detail.text
assert detail.json()["history"][0]["action"] == "correct"
graph = await client.get("/api/v1/memories/graph")
assert graph.status_code == 200
assert graph.json()["projected_memories"] == 1
assert {node["node_type"] for node in graph.json()["nodes"]} == {"memory", "source"}
assert graph.json()["edges"][0]["relation"] == "evidence_source"
deleted = await client.post(
f"/api/v1/memories/{memory_id}/actions/delete",
json={"request_id": "delete-1", "expected_revision": 2},
)
assert deleted.json()["status"] == "deleted"
cleanup = await client.get(f"/api/v1/memories/{memory_id}/deletion")
assert cleanup.json()["status"] == "failed"
assert cleanup.json()["last_error_code"] == "model_profile_not_active"
asyncio.run(scenario())
def test_memory_validation_does_not_echo_content(tmp_path) -> None:
async def scenario() -> None:
app = create_app(
Settings(
data_dir=tmp_path / "data",
source_home=tmp_path,
api_token=SecretStr("local-token"),
)
)
async with (
app.router.lifespan_context(app),
AsyncClient(
transport=ASGITransport(app=app),
base_url="http://test",
headers={
"Authorization": "Bearer local-token",
"Origin": "http://127.0.0.1:5173",
},
) as client,
):
response = await client.post(
"/api/v1/memories",
json={
"request_id": "create-1",
"draft": {
"user_id": "local",
"kind": "chat",
"content": "secret body",
"scopes": [],
"evidence_ids": [],
},
},
)
assert response.status_code == 422
assert "secret body" not in response.text
assert "input" not in response.text
asyncio.run(scenario())
@@ -1,895 +0,0 @@
import asyncio
import time
from httpx import ASGITransport, AsyncClient
from pydantic import SecretStr
from one_memory import api as api_module
from one_memory.api import create_app
from one_memory.config import Settings
from one_memory.imports.candidates import RootCandidateKind
from one_memory.imports.models import (
EvidenceInput,
EvidenceOrigin,
OriginMetadata,
WrittenBy,
)
def _confirm(app, client, content: str, evidence_id: str, index: int):
async def run() -> None:
created = await client.post(
"/api/v1/memories",
json={
"request_id": f"create-{index}",
"draft": {
"user_id": "local",
"kind": "chat",
"content": content,
"scopes": ["all-apps"],
"evidence_ids": [evidence_id],
},
},
)
assert created.status_code == 200
return run()
def test_graph_labels_where_every_edge_came_from(tmp_path) -> None:
async def scenario() -> None:
app = create_app(
Settings(
data_dir=tmp_path / "data",
source_home=tmp_path,
api_token=SecretStr("local-token"),
)
)
async with (
app.router.lifespan_context(app),
AsyncClient(
transport=ASGITransport(app=app),
base_url="http://test",
headers={
"Authorization": "Bearer local-token",
"Origin": "http://127.0.0.1:5173",
},
) as client,
):
for index, (message_id, content) in enumerate(
(
("message-1", "用户要求看板只用真实数据。"),
("message-2", "用户希望问答给出引用。"),
)
):
evidence = app.state.evidence_store.put(
EvidenceInput(
source_id="cursor",
conversation_id="conversation-a",
message_id=message_id,
correlation_id=message_id,
role="user",
written_by=WrittenBy.PERSON,
verbatim=content,
origin_meta={EvidenceOrigin.FILE: OriginMetadata()},
)
).record
await _confirm(app, client, content, evidence.evidence_id, index)
other = app.state.evidence_store.put(
EvidenceInput(
source_id="cursor",
conversation_id="conversation-b",
message_id="message-3",
correlation_id="message-3",
role="user",
written_by=WrittenBy.PERSON,
verbatim="用户在另一次对话里提到偏好。",
origin_meta={EvidenceOrigin.FILE: OriginMetadata()},
)
).record
await _confirm(app, client, "用户在另一次对话里提到偏好。", other.evidence_id, 2)
graph = await client.get("/api/v1/memories/graph")
assert graph.status_code == 200
body = graph.json()
assert body["projected_memories"] == 3
origins = {edge["origin"] for edge in body["edges"]}
assert origins == {"provenance", "same_conversation"}
# Only the two memories from one conversation are linked to each
# other; the memory from another conversation is not.
same = [edge for edge in body["edges"] if edge["origin"] == "same_conversation"]
assert len(same) == 1
# HY was never reachable in this test, and the response says so
# instead of letting the map imply the relations are all there is.
assert body["hy_graph_available"] is False
asyncio.run(scenario())
def test_a_created_memory_is_on_the_map_with_its_source_links(tmp_path):
async def scenario():
app = create_app(
Settings(
data_dir=tmp_path / "data", source_home=tmp_path, api_token=SecretStr("local-token")
)
)
async with (
app.router.lifespan_context(app),
AsyncClient(
transport=ASGITransport(app=app),
base_url="http://test",
headers={"Authorization": "Bearer local-token"},
) as client,
):
evidence = app.state.evidence_store.put(
EvidenceInput(
source_id="cursor",
conversation_id="candidate",
message_id="c1",
role="assistant",
written_by=WrittenBy.AGENT,
verbatim="Agent建议蓝色按钮。",
origin_meta={EvidenceOrigin.FILE: OriginMetadata()},
)
).record
created = await client.post(
"/api/v1/memories",
json={
"request_id": "candidate-create",
"draft": {
"user_id": "local",
"kind": "chat",
"content": "Agent建议蓝色按钮。",
"origin": "agent",
"scopes": ["all-apps"],
"evidence_ids": [evidence.evidence_id],
},
},
)
assert created.status_code == 200
identifier = created.json()["memory_id"]
graph = (await client.get("/api/v1/memories/graph")).json()
assert graph["projected_memories"] == 1
assert any(
node["memory_id"] == identifier and node["status"] == "confirmed"
for node in graph["nodes"]
)
assert any(edge["origin"] == "provenance" for edge in graph["edges"])
asyncio.run(scenario())
def test_drawing_the_map_does_not_stop_the_app_answering(tmp_path) -> None:
"""The map is what the memory view opens with, and it used to block the
process while it was read.
Every store call in the read is synchronous and the evidence is fetched an
id at a time, so a full map is hundreds of blocking calls. Made on the
loop, they were made instead of answering anyone: the watchdog caught this
endpoint holding the loop for 2.1s, and /health — which does no work at
all — was measured at 17.9s on the real library while a map was drawn.
"""
async def scenario() -> None:
app = create_app(
Settings(
data_dir=tmp_path / "data",
source_home=tmp_path,
api_token=SecretStr("local-token"),
)
)
async with (
app.router.lifespan_context(app),
AsyncClient(
transport=ASGITransport(app=app),
base_url="http://test",
headers={
"Authorization": "Bearer local-token",
"Origin": "http://127.0.0.1:5173",
},
) as client,
):
for index in range(4):
evidence = app.state.evidence_store.put(
EvidenceInput(
source_id="cursor",
conversation_id="conversation-a",
message_id=f"message-{index}",
correlation_id=f"message-{index}",
role="user",
written_by=WrittenBy.PERSON,
verbatim=f"用户说了第 {index} 件事。",
origin_meta={EvidenceOrigin.FILE: OriginMetadata()},
)
).record
said = f"用户说了第 {index} 件事。"
await _confirm(app, client, said, evidence.evidence_id, index)
# Each evidence read takes a moment, the way a contended one does.
inner = app.state.evidence_store
delay = 0.1
class _Sluggish:
def __getattr__(self, name: str):
attribute = getattr(inner, name)
if not callable(attribute):
return attribute
def slowly(*args, **kwargs):
time.sleep(delay)
return attribute(*args, **kwargs)
return slowly
app.state.evidence_store = _Sluggish()
longest = 0.0
answered = 0
async def keep_asking() -> None:
# The gap between turns, not the duration of one request: a
# task cannot time a stall it was not running for, and timing
# its own request measures nothing while the loop is blocked.
nonlocal longest, answered
last = time.perf_counter()
while True:
reply = await client.get("/api/v1/health")
assert reply.status_code == 200
now = time.perf_counter()
longest = max(longest, now - last)
last = now
answered += 1
await asyncio.sleep(0.005)
asking = asyncio.create_task(keep_asking())
graph = await client.get("/api/v1/memories/graph")
asking.cancel()
assert graph.status_code == 200
# The map really was read, so this cannot pass by drawing nothing.
assert graph.json()["projected_memories"] == 4
# Read on the loop, not one of these got an answer until the map
# was finished, which is the whole defect.
assert answered > 1, "画图期间没有任何请求被应答:循环被堵住了"
# Several tenth-of-a-second reads happen while the map is drawn;
# on the loop none of these could have been answered until it
# finished.
assert longest < delay * 3, f"画图期间循环被堵住 {longest:.3f}s"
asyncio.run(scenario())
def _confirm_tagged(app, client, content: str, evidence_id: str, index: int, tags):
async def run() -> None:
created = await client.post(
"/api/v1/memories",
json={
"request_id": f"create-tagged-{index}",
"draft": {
"user_id": "local",
"kind": "chat",
"content": content,
"scopes": ["all-apps"],
"evidence_ids": [evidence_id],
"tags": list(tags),
},
},
)
assert created.status_code == 200, created.text
return run()
def test_a_shared_tag_becomes_a_topic_on_the_map(tmp_path) -> None:
"""标签要在图上长成一个点,并把带它的记忆连起来。
图上原先只有三种点:记忆、来源 App、以及 HY 自己 TOPIC 层的归纳。我们从 HY
接回来的 tags 只在左栏当文字展示,没有进图 —— 于是一张记忆还很少的库打开来
满屏都是来源 App,看起来像是按 App 聚类,而不是按事项聚类。
只连「被两条以上记忆共用」的标签:只出现一次的标签连出去的是一根挂着单个
记忆的线,那不是聚类,只是把同一条记忆画了两遍。
"""
async def scenario() -> None:
app = create_app(
Settings(
data_dir=tmp_path / "data",
source_home=tmp_path,
api_token=SecretStr("local-token"),
)
)
async with (
app.router.lifespan_context(app),
AsyncClient(
transport=ASGITransport(app=app),
base_url="http://test",
headers={
"Authorization": "Bearer local-token",
"Origin": "http://127.0.0.1:5173",
},
) as client,
):
plans = (
("m-1", "用户要求前端统一走 axios 封装。", ("axios", "frontend")),
("m-2", "用户要求测试先行。", ("axios", "testing")),
("m-3", "用户在做记忆产品。", ("memory-product",)),
)
for index, (message_id, content, tags) in enumerate(plans):
evidence = app.state.evidence_store.put(
EvidenceInput(
source_id="cursor",
conversation_id=f"conversation-{index}",
message_id=message_id,
correlation_id=message_id,
role="user",
written_by=WrittenBy.PERSON,
verbatim=content,
origin_meta={EvidenceOrigin.FILE: OriginMetadata()},
)
).record
await _confirm_tagged(app, client, content, evidence.evidence_id, index, tags)
body = (await client.get("/api/v1/memories/graph")).json()
topics = {
node["label"]: node["node_id"]
for node in body["nodes"]
if node["node_type"] == "topic"
}
# axios 被两条记忆共用,所以它是一个主题。
assert "axios" in topics
# frontend / testing / memory-product 各只出现一次 —— 不成聚类。
assert set(topics) == {"axios"}
tagged = [edge for edge in body["edges"] if edge["origin"] == "shared_tag"]
assert len(tagged) == 2
assert {edge["source_node_id"] for edge in tagged} == {topics["axios"]}
asyncio.run(scenario())
def test_a_tag_used_once_does_not_become_a_topic(tmp_path) -> None:
"""正向对照:全库只有一条带标签的记忆时,图上不许出现主题点。
没有这条,上一条可以被「每个标签都建一个点」骗过 —— 那样 axios 照样在,
只是同时多出三个各挂一条记忆的孤点。
"""
async def scenario() -> None:
app = create_app(
Settings(
data_dir=tmp_path / "data",
source_home=tmp_path,
api_token=SecretStr("local-token"),
)
)
async with (
app.router.lifespan_context(app),
AsyncClient(
transport=ASGITransport(app=app),
base_url="http://test",
headers={
"Authorization": "Bearer local-token",
"Origin": "http://127.0.0.1:5173",
},
) as client,
):
evidence = app.state.evidence_store.put(
EvidenceInput(
source_id="cursor",
conversation_id="conversation-solo",
message_id="m-solo",
correlation_id="m-solo",
role="user",
written_by=WrittenBy.PERSON,
verbatim="用户要求前端统一走 axios 封装。",
origin_meta={EvidenceOrigin.FILE: OriginMetadata()},
)
).record
await _confirm_tagged(
app, client, "用户要求前端统一走 axios 封装。", evidence.evidence_id, 9, ("axios",)
)
body = (await client.get("/api/v1/memories/graph")).json()
assert [node for node in body["nodes"] if node["node_type"] == "topic"] == []
assert [edge for edge in body["edges"] if edge["origin"] == "shared_tag"] == []
asyncio.run(scenario())
def test_the_map_only_verifies_the_memories_it_draws(tmp_path) -> None:
"""A full library must not be verified row by row just to count it.
The map reads 500 memories but draws at most `limit`. Verifying all 500
meant one freshness check plus one citation check per memory, and on a
1,226-memory library it passed the 5 s interrupt, so /memories/graph
answered 503 every time while /memories, which verifies one page, stayed
at 0.1 s.
"""
calls: list[int] = []
def counting(memories):
calls.append(len(memories))
return list(memories)
async def scenario() -> None:
app = create_app(
Settings(
data_dir=tmp_path / "data",
source_home=tmp_path,
api_token=SecretStr("local-token"),
)
)
async with (
app.router.lifespan_context(app),
AsyncClient(
transport=ASGITransport(app=app),
base_url="http://test",
headers={
"Authorization": "Bearer local-token",
"Origin": "http://127.0.0.1:5173",
},
) as client,
):
for index in range(12):
evidence = app.state.evidence_store.put(
EvidenceInput(
source_id="cursor",
conversation_id=f"conversation-{index}",
message_id=f"m-{index}",
correlation_id=f"m-{index}",
role="user",
written_by=WrittenBy.PERSON,
verbatim=f"第 {index} 条真实对话内容。",
origin_meta={EvidenceOrigin.FILE: OriginMetadata()},
)
).record
await _confirm(
app, client, f"第 {index} 条真实对话内容。", evidence.evidence_id, index
)
app.state.source_read_access.memories = counting
body = (await client.get("/api/v1/memories/graph?limit=5")).json()
assert body["total_memories"] == 12
assert body["projected_memories"] == 5
# The count is not row-verified, and neither is the tail: only the
# five memories the map draws are checked.
assert calls == [5]
asyncio.run(scenario())
def test_the_map_resolves_the_source_catalog_once(tmp_path) -> None:
"""The catalog is resolved once per request, not once per drawn source.
`_source_catalog` re-reads and re-resolves every registered source. The
map called it inside the per-memory loop, once per distinct source: on a
160-memory library that is over a thousand resolutions, and it is where
the map's five seconds went.
"""
from one_memory.imports.candidates import SourceCandidateStore
calls: list[int] = []
original = SourceCandidateStore.resolve_catalog
def counting(self, catalog, *, user_id="local"):
calls.append(1)
return original(self, catalog, user_id=user_id)
SourceCandidateStore.resolve_catalog = counting
try:
async def scenario() -> None:
app = create_app(
Settings(
data_dir=tmp_path / "data",
source_home=tmp_path,
api_token=SecretStr("local-token"),
)
)
async with (
app.router.lifespan_context(app),
AsyncClient(
transport=ASGITransport(app=app),
base_url="http://test",
headers={
"Authorization": "Bearer local-token",
"Origin": "http://127.0.0.1:5173",
},
) as client,
):
for index in range(6):
evidence = app.state.evidence_store.put(
EvidenceInput(
source_id="cursor",
conversation_id=f"conversation-{index}",
message_id=f"m-{index}",
correlation_id=f"m-{index}",
role="user",
written_by=WrittenBy.PERSON,
verbatim=f"第 {index} 条真实对话内容。",
origin_meta={EvidenceOrigin.FILE: OriginMetadata()},
)
).record
await _confirm(
app, client, f"第 {index} 条真实对话内容。", evidence.evidence_id, index
)
calls.clear()
body = (await client.get("/api/v1/memories/graph")).json()
assert body["projected_memories"] == 6
# Six memories share one source. One resolution, not six.
assert len(calls) == 1
asyncio.run(scenario())
finally:
SourceCandidateStore.resolve_catalog = original
def test_the_map_count_leaves_a_revoked_source_out(tmp_path) -> None:
"""The count is the cheap one, but not so cheap it counts revoked work.
D22 settled that a total skips the per-row freshness check. It still has
to leave out memories a revoked source supplied, or the number the map
reports includes memories it refuses to draw.
"""
async def scenario() -> None:
app = create_app(
Settings(
data_dir=tmp_path / "data",
source_home=tmp_path,
api_token=SecretStr("local-token"),
)
)
async with (
app.router.lifespan_context(app),
AsyncClient(
transport=ASGITransport(app=app),
base_url="http://test",
headers={
"Authorization": "Bearer local-token",
"Origin": "http://127.0.0.1:5173",
},
) as client,
):
for index, source_id in enumerate(("cursor", "claude-code")):
folder = tmp_path / source_id
folder.mkdir(exist_ok=True)
app.state.source_grant_store.grant(
user_id="local",
source_id=source_id,
root=folder,
candidate_kind=RootCandidateKind.DEFAULT,
)
evidence = app.state.evidence_store.put(
EvidenceInput(
source_id=source_id,
conversation_id=f"conversation-{source_id}",
message_id=f"m-{source_id}",
correlation_id=f"m-{source_id}",
role="user",
written_by=WrittenBy.PERSON,
verbatim=f"{source_id} 的真实对话内容。",
origin_meta={EvidenceOrigin.FILE: OriginMetadata()},
)
).record
await _confirm(
app, client, f"{source_id} 的真实对话内容。", evidence.evidence_id, index
)
before = (await client.get("/api/v1/memories/graph")).json()
assert before["total_memories"] == 2
app.state.source_grant_store.revoke(user_id="local", source_id="claude-code")
after = (await client.get("/api/v1/memories/graph")).json()
assert after["total_memories"] == 1
assert after["projected_memories"] == 1
asyncio.run(scenario())
def test_the_map_reads_its_evidence_in_one_pass(tmp_path) -> None:
"""D89: one query for the map's evidence, not one per citation.
Each drawn memory looked up every one of its citations separately: 1,274
round trips for a 160-memory map, each taking a turn through the pool.
That is what put the map over its five second budget on a library being
written to at the same time — the acceptance run saw 4.2-14.8 s and a 503
on three calls out of ten.
"""
from one_memory.imports.storage import EvidenceStore
calls: list[int] = []
original = EvidenceStore.get
def counting(self, evidence_id):
calls.append(1)
return original(self, evidence_id)
EvidenceStore.get = counting
try:
async def scenario() -> None:
app = create_app(
Settings(
data_dir=tmp_path / "data",
source_home=tmp_path,
api_token=SecretStr("local-token"),
)
)
async with (
app.router.lifespan_context(app),
AsyncClient(
transport=ASGITransport(app=app),
base_url="http://test",
headers={
"Authorization": "Bearer local-token",
"Origin": "http://127.0.0.1:5173",
},
) as client,
):
for index in range(6):
evidence = app.state.evidence_store.put(
EvidenceInput(
source_id="cursor",
conversation_id=f"conversation-{index % 2}",
message_id=f"m-{index}",
correlation_id=f"m-{index}",
role="user",
written_by=WrittenBy.PERSON,
verbatim=f"第 {index} 条真实对话内容。",
origin_meta={EvidenceOrigin.FILE: OriginMetadata()},
)
).record
await _confirm(
app, client, f"第 {index} 条真实对话内容。", evidence.evidence_id, index
)
calls.clear()
body = (await client.get("/api/v1/memories/graph")).json()
assert body["projected_memories"] == 6
# Six memories with one citation each, read together.
assert len(calls) <= 1
asyncio.run(scenario())
finally:
EvidenceStore.get = original
def test_a_timed_out_map_falls_back_to_the_last_good_one(tmp_path) -> None:
"""D91: a slow map must not answer 503 while a good one is known.
While a library is being written to, the map read can cross its five
second interrupt. The memory view then has nothing to show, even though
the very same map was drawn successfully a moment ago and the library has
not changed since.
"""
import sqlite3
calls: list[int] = []
real = api_module._governed_graph_part
def counting(app, limit):
calls.append(limit)
if len(calls) > 1:
raise sqlite3.OperationalError("interrupted")
return real(app, limit)
async def scenario() -> None:
app = create_app(
Settings(
data_dir=tmp_path / "data",
source_home=tmp_path,
api_token=SecretStr("local-token"),
)
)
async with (
app.router.lifespan_context(app),
AsyncClient(
transport=ASGITransport(app=app),
base_url="http://test",
headers={
"Authorization": "Bearer local-token",
"Origin": "http://127.0.0.1:5173",
},
) as client,
):
for index in range(3):
evidence = app.state.evidence_store.put(
EvidenceInput(
source_id="cursor",
conversation_id=f"conversation-{index}",
message_id=f"m-{index}",
correlation_id=f"m-{index}",
role="user",
written_by=WrittenBy.PERSON,
verbatim=f"第 {index} 条真实对话内容。",
origin_meta={EvidenceOrigin.FILE: OriginMetadata()},
)
).record
await _confirm(
app, client, f"第 {index} 条真实对话内容。", evidence.evidence_id, index
)
api_module._governed_graph_part = counting
try:
first = await client.get("/api/v1/memories/graph")
assert first.status_code == 200
assert first.json()["projected_memories"] == 3
# The next read crosses the interrupt. Nothing has changed
# since the one that succeeded, so the known map is the better
# answer than a 503.
second = await client.get("/api/v1/memories/graph")
finally:
api_module._governed_graph_part = real
assert second.status_code == 200
assert second.json()["projected_memories"] == 3
asyncio.run(scenario())
def test_a_changed_library_is_drawn_again(tmp_path) -> None:
"""The cache is keyed on the library, so a new memory redraws the map."""
calls: list[int] = []
real = api_module._governed_graph_part
def counting(app, limit):
calls.append(limit)
return real(app, limit)
async def scenario() -> None:
app = create_app(
Settings(
data_dir=tmp_path / "data",
source_home=tmp_path,
api_token=SecretStr("local-token"),
)
)
async with (
app.router.lifespan_context(app),
AsyncClient(
transport=ASGITransport(app=app),
base_url="http://test",
headers={
"Authorization": "Bearer local-token",
"Origin": "http://127.0.0.1:5173",
},
) as client,
):
evidence = app.state.evidence_store.put(
EvidenceInput(
source_id="cursor",
conversation_id="conversation-0",
message_id="m-0",
correlation_id="m-0",
role="user",
written_by=WrittenBy.PERSON,
verbatim="第 0 条真实对话内容。",
origin_meta={EvidenceOrigin.FILE: OriginMetadata()},
)
).record
await _confirm(app, client, "第 0 条真实对话内容。", evidence.evidence_id, 0)
api_module._governed_graph_part = counting
try:
assert (await client.get("/api/v1/memories/graph")).status_code == 200
# Same library: served from the last good map.
assert (await client.get("/api/v1/memories/graph")).status_code == 200
assert len(calls) == 1
second = app.state.evidence_store.put(
EvidenceInput(
source_id="cursor",
conversation_id="conversation-1",
message_id="m-1",
correlation_id="m-1",
role="user",
written_by=WrittenBy.PERSON,
verbatim="第 1 条真实对话内容。",
origin_meta={EvidenceOrigin.FILE: OriginMetadata()},
)
).record
await _confirm(app, client, "第 1 条真实对话内容。", second.evidence_id, 1)
body = await client.get("/api/v1/memories/graph")
finally:
api_module._governed_graph_part = real
assert body.status_code == 200
assert body.json()["projected_memories"] == 2
assert len(calls) == 2
asyncio.run(scenario())
def test_withdrawing_a_second_source_redraws_the_map(tmp_path) -> None:
"""D95: the cache must notice a second revocation, not just the first.
Keying on whether *any* source is revoked keeps serving the map drawn
before the second withdrawal, so that source's memories stay on a map the
reader is no longer allowed to see them on.
"""
from one_memory.imports.candidates import RootCandidateKind
calls: list[int] = []
real = api_module._governed_graph_part
def counting(app, limit):
calls.append(limit)
return real(app, limit)
async def scenario() -> None:
app = create_app(
Settings(
data_dir=tmp_path / "data",
source_home=tmp_path,
api_token=SecretStr("local-token"),
)
)
async with (
app.router.lifespan_context(app),
AsyncClient(
transport=ASGITransport(app=app),
base_url="http://test",
headers={
"Authorization": "Bearer local-token",
"Origin": "http://127.0.0.1:5173",
},
) as client,
):
for source_id in ("cursor", "claude-code"):
folder = tmp_path / source_id
folder.mkdir(exist_ok=True)
app.state.source_grant_store.grant(
user_id="local",
source_id=source_id,
root=folder,
candidate_kind=RootCandidateKind.DEFAULT,
)
evidence = app.state.evidence_store.put(
EvidenceInput(
source_id=source_id,
conversation_id=f"conversation-{source_id}",
message_id=f"m-{source_id}",
correlation_id=f"m-{source_id}",
role="user",
written_by=WrittenBy.PERSON,
verbatim=f"{source_id} 的真实对话内容。",
origin_meta={EvidenceOrigin.FILE: OriginMetadata()},
)
).record
await _confirm(
app, client, f"{source_id} 的真实对话内容。", evidence.evidence_id, source_id
)
api_module._governed_graph_part = counting
try:
assert (await client.get("/api/v1/memories/graph")).status_code == 200
app.state.source_grant_store.revoke(user_id="local", source_id="cursor")
assert (await client.get("/api/v1/memories/graph")).status_code == 200
assert len(calls) == 2
# A second withdrawal with one already revoked. The library
# itself has not changed, so only the grant fingerprint can
# tell this apart from the previous call.
app.state.source_grant_store.revoke(user_id="local", source_id="claude-code")
body = await client.get("/api/v1/memories/graph")
finally:
api_module._governed_graph_part = real
assert body.status_code == 200
assert len(calls) == 3
asyncio.run(scenario())
+213 -34
View File
@@ -78,9 +78,6 @@ def test_empty_library_can_save_a_note_without_a_matter_or_configured_model(tmp_
assert (
app.state.memory_note_store.require(saved["evidence_id"]).verbatim == note["text"]
)
retired = await client.get(f"/api/v1/memory-notes/{saved['evidence_id']}/evidence")
assert retired.status_code == 409
assert retired.json()["detail"] == "memory_note_evidence_retired"
assert legacy_tables_present(app.state.settings.data_dir / "one-memory.sqlite3") == ()
restarted = application(tmp_path)
async with restarted.router.lifespan_context(restarted), client_for(restarted) as client:
@@ -118,37 +115,6 @@ def test_note_requires_authentication_explicit_scope_and_nonempty_content(tmp_pa
asyncio.run(scenario())
def test_saved_note_evidence_returns_raw_and_consent_reference_from_same_record(tmp_path):
async def scenario():
app = legacy_application(tmp_path)
async with app.router.lifespan_context(app), client_for(app) as client:
text = "Original spacing\n预算<5,不能改为>5。"
saved = (
await client.post(
"/api/v1/memory-notes",
json={
"request_id": str(uuid4()),
"text": text,
"scope": "all-apps",
},
)
).json()
endpoint = f"/api/v1/memory-notes/{saved['evidence_id']}/evidence"
response = await client.get(endpoint)
assert response.status_code == 200, response.text
value = response.json()
assert value["verbatim"] == text
assert value["evidence_id"] == saved["evidence_id"] and value["revision"] == 1
assert value["role"] == "user" and value["written_by"] == "person"
assert value["available"] is True and value["source_id"] == "one-memory-manual"
assert app.state.memory_task_store.list() == []
assert (await client.get("/api/v1/memory-notes/missing/evidence")).status_code == 404
client.headers.pop("Authorization")
assert (await client.get(endpoint)).status_code == 401
asyncio.run(scenario())
def test_note_and_source_grant_are_atomic(tmp_path):
from one_memory.memory_notes import MemoryNoteInput
@@ -278,3 +244,216 @@ def test_sdk_user_profile_http_returns_native_assembled_profile(tmp_path):
}
asyncio.run(scenario())
@pytest.mark.fresh_sdk
def test_sdk_memory_library_uses_sdk_search_and_skips_empty_rows_without_mutation(
tmp_path,
caplog,
):
async def scenario():
app = application(tmp_path)
list_calls = []
search_calls = []
deleted = []
records = {
"release": {
"memory_id": "release",
"content": "Release plan",
"layer": "l2_fact",
"status": "active",
"user_id": "local",
"agent_id": "all-apps",
"session_id": "s1",
"source": "cursor",
"tags": ["shipping"],
},
"retired": {
"memory_id": "retired",
"content": "Release plan, first draft",
"layer": "l2_fact",
"status": "superseded",
"user_id": "local",
"agent_id": "all-apps",
"session_id": "s1",
"source": "cursor",
"tags": [],
},
"default": {
"memory_id": "default",
"content": "Unknown source",
"layer": "l2_fact",
"status": "active",
"user_id": "local",
"agent_id": "all-apps",
"session_id": "s3",
"source": "default",
"tags": [],
},
}
class Client:
def list_memories(self, **kwargs):
list_calls.append(kwargs)
rows = [
records["release"],
records["default"],
{
"memory_id": "empty",
"content": "",
"layer": "l2_fact",
"status": "active",
"user_id": "local",
"agent_id": "all-apps",
"session_id": "s2",
"source": "workbuddy",
"tags": [],
},
]
include_sources = kwargs.get("include_sources") or []
if include_sources:
rows = [row for row in rows if row["source"] in include_sources]
return {
"vdb": {
"memories": rows,
"total": len(rows),
"limit": kwargs["limit"],
"offset": kwargs["offset"],
}
}
def search(self, query, **kwargs):
search_calls.append((query, kwargs))
return {
"memories": {
"schema": [],
"proactive": [],
"normal": [
{
"memory_id": "release",
"content": "Release plan",
"score": 0.9,
},
{
"memory_id": "retired",
"content": "Release plan, first draft",
"score": 0.8,
},
],
}
}
def get(self, memory_id):
return records.get(memory_id)
def delete(self, memory_id):
deleted.append(memory_id)
return {"success": True, "deleted_count": 1}
sdk = Client()
class Handle:
async def client(self):
return sdk
async def close(self):
return None
async with app.router.lifespan_context(app):
app.state.sdk_client_handle = Handle()
async with client_for(app) as client:
response = await client.get(
"/api/v2/memory-library",
params={"source": "cursor", "query": "release"},
)
assert response.status_code == 200
body = response.json()
# `total` counts the window's keyword matches; the superseded
# hit matches but is not listed as an active memory.
assert body["total"] == 2
assert body["search_window"] is None
assert [item["memory_id"] for item in body["memories"]] == ["release"]
assert search_calls[0][0] == "release"
assert search_calls[0][1]["include_sources"] == ["cursor"]
assert search_calls[0][1]["limit"] == 20
assert search_calls[0][1]["intention_limit"] == 0
assert search_calls[0][1]["min_score"] == 0.0
assert list_calls == []
# The SDK still returns its nearest rows for a word that is in
# none of them; the library reports no match instead.
unmatched = await client.get(
"/api/v2/memory-library", params={"query": "zzz nowhere"}
)
assert unmatched.json()["total"] == 0
assert unmatched.json()["memories"] == []
listed = await client.get(
"/api/v2/memory-library",
params={"source": "workbuddy", "limit": 100},
)
assert listed.status_code == 200
assert listed.json()["memories"] == []
assert listed.json()["total"] == 1
assert deleted == []
assert list_calls[0]["limit"] == 100
assert "sdk_memory_invalid_rows_skipped count=1" in caplog.messages
graph = await client.get("/api/v2/memories/graph")
assert graph.status_code == 200
node_ids = {node["node_id"] for node in graph.json()["nodes"]}
assert "source:cursor" in node_ids
assert "source:default" not in node_ids
asyncio.run(scenario())
@pytest.mark.fresh_sdk
def test_sdk_context_search_sets_each_recall_lane_apart(tmp_path):
"""Normal, schema and intention lanes default to 10 / 3 / 1 and pass through."""
async def scenario():
app = application(tmp_path)
calls = []
class Client:
def search(self, query, **kwargs):
calls.append(kwargs)
return {"memories": {"schema": [], "proactive": [], "normal": []}}
def get(self, memory_id):
return None
sdk = Client()
class Handle:
async def client(self):
return sdk
async def close(self):
return None
async with app.router.lifespan_context(app):
app.state.sdk_client_handle = Handle()
async with client_for(app) as client:
defaulted = await client.post(
"/api/v2/context/search",
json={"query": "release", "kinds": ["memory"]},
)
chosen = await client.post(
"/api/v2/context/search",
json={
"query": "release",
"kinds": ["memory"],
"limit": 5,
"schema_limit": 0,
"intention_limit": 2,
},
)
assert defaulted.status_code == 200
assert chosen.status_code == 200
assert [
(call["limit"], call["schema_limit"], call["intention_limit"]) for call in calls
] == [(10, 3, 1), (5, 0, 2)]
asyncio.run(scenario())
+2 -124
View File
@@ -7,35 +7,9 @@ from pydantic import SecretStr
from one_memory import api
from one_memory.api import create_app
from one_memory.config import Settings
from one_memory.imports.models import (
EvidenceInput,
EvidenceOrigin,
OriginMetadata,
WrittenBy,
)
from one_memory.sdk_data_plane_migration import MemoryAuthority
from one_memory.sdk_memory_qa import SdkAskMemory
class _StubAnswerer:
def __init__(self, events: tuple[dict, ...]) -> None:
self.events = events
self.calls: list[tuple[str, str]] = []
self.last_workspace_path: str | None = None
async def answer_stream(
self,
question: str,
*,
scope: str = "all-apps",
workspace_path: str | None = None,
):
self.calls.append((question, scope))
self.last_workspace_path = workspace_path
for event in self.events:
yield event
def _sse_events(text: str) -> list[dict]:
return [
json.loads(frame.strip().removeprefix("data:").strip())
@@ -44,96 +18,6 @@ def _sse_events(text: str) -> list[dict]:
]
def test_ask_stream_endpoint_emits_recall_deltas_and_final(tmp_path) -> None:
async def scenario() -> None:
app = create_app(
Settings(
data_dir=tmp_path / "data",
source_home=tmp_path,
api_token=SecretStr("local-token"),
)
)
async with (
app.router.lifespan_context(app),
AsyncClient(
transport=ASGITransport(app=app),
base_url="http://test",
headers={
"Authorization": "Bearer local-token",
"Origin": "http://127.0.0.1:5173",
},
) as client,
):
evidence = app.state.evidence_store.put(
EvidenceInput(
source_id="cursor",
conversation_id="conversation",
message_id="message",
correlation_id="message",
role="user",
written_by=WrittenBy.PERSON,
verbatim="我要求看板只用真实数据。",
origin_meta={EvidenceOrigin.FILE: OriginMetadata()},
)
).record
created = await client.post(
"/api/v1/memories",
json={
"request_id": "create-1",
"draft": {
"user_id": "local",
"kind": "chat",
"content": "用户要求看板只用真实数据。",
"scopes": ["all-apps"],
"evidence_ids": [evidence.evidence_id],
},
},
)
memory = app.state.memory_governance_store.get(
created.json()["memory_id"],
user_id="local",
)
stub = _StubAnswerer(
(
{"type": "recall", "memories": (memory,)},
{"type": "delta", "text": "你要求"},
{"type": "delta", "text": "真实数据。"},
{
"type": "final",
"answer": "你要求真实数据。",
"memories": (memory,),
"mode": "generated",
"retrieval": {"semantic_state": "failed"},
},
)
)
app.state.memory_answerer = stub
response = await client.post(
"/api/v1/memories/ask/stream",
json={"question": "我对数据有什么要求?", "scope": "all-apps"},
)
assert response.status_code == 200
assert response.headers["content-type"].startswith("text/event-stream")
events = _sse_events(response.text)
assert [event["type"] for event in events] == [
"recall",
"delta",
"delta",
"final",
]
assert events[0]["memories"][0]["memory_id"] == memory.memory_id
assert (
"".join(event["text"] for event in events if event["type"] == "delta")
== events[-1]["answer"]
)
assert events[-1]["retrieval"] == {"semantic_state": "failed"}
assert stub.calls == [("我对数据有什么要求?", "all-apps")]
asyncio.run(scenario())
def test_ask_stream_endpoint_requires_the_local_token(tmp_path) -> None:
async def scenario() -> None:
app = create_app(
@@ -152,7 +36,7 @@ def test_ask_stream_endpoint_requires_the_local_token(tmp_path) -> None:
) as client,
):
response = await client.post(
"/api/v1/memories/ask/stream",
"/api/v2/memories/ask/stream",
json={"question": "我是谁?", "scope": "all-apps"},
)
@@ -161,7 +45,7 @@ def test_ask_stream_endpoint_requires_the_local_token(tmp_path) -> None:
asyncio.run(scenario())
def test_sdk_ask_v2_stream_is_native_and_v1_fails_after_cutover(tmp_path, monkeypatch) -> None:
def test_sdk_ask_v2_stream_is_native(tmp_path, monkeypatch) -> None:
memory = SdkAskMemory(
memory_id="memory-1",
content="用户要求真实数据。",
@@ -204,12 +88,6 @@ def test_sdk_ask_v2_stream_is_native_and_v1_fails_after_cutover(tmp_path, monkey
},
) as client,
):
app.state.memory_authority = MemoryAuthority.SDK
legacy = await client.post(
"/api/v1/memories/ask/stream",
json={"question": "要求是什么?"},
)
assert legacy.status_code == 409
response = await client.post(
"/api/v2/memories/ask/stream",
json={"question": "要求是什么?"},
@@ -742,49 +742,6 @@ def test_change_outbox_ddl_mysql_and_sqlite_agree_on_columns() -> None:
assert "event_id" in columns(mysql)
@pytest.mark.parametrize(
("getter", "table"),
[
("session_leases_ddl", "session_leases"),
("trusted_sessions_ddl", "trusted_sessions"),
],
)
def test_migrated_ddl_has_one_source(getter: str, table: str) -> None:
"""Each migrated module must use the migration's definition, not a copy.
Modules that ran on one engine held a single `_SCHEMA` constant. Migrated
ones hold both renderings and select at use time, so the check reads
whichever the module actually uses — the point is still that the text comes
from the migration and is not retyped inline.
"""
import importlib
migrations = importlib.import_module("one_memory.memory_server.migrations")
render = getattr(migrations, getter)
module_name = {
"session_leases_ddl": "one_memory.session_lease",
"trusted_sessions_ddl": "one_memory.session_registry",
}[getter]
module = importlib.import_module(module_name)
if hasattr(module, "schema_for"):
sqlite_text = module.schema_for(False)
mysql_text = module.schema_for(True)
else:
sqlite_text = module._SCHEMA
mysql_text = None
assert sqlite_text == render("sqlite"), "module copies the DDL instead of using the migration"
assert table in sqlite_text
assert "ENGINE=InnoDB" not in sqlite_text
# A migrated module must also be able to render MySQL, or it will send
# SQLite DDL to MySQL when the engine switches.
if mysql_text is not None:
assert mysql_text == render("mysql")
assert table in mysql_text
def test_import_jobs_ddl_is_split_before_execution() -> None:
"""MySQL takes one DDL per call, so a script must be split.
+1 -127
View File
@@ -10,11 +10,7 @@ import time
from contextlib import closing
import pytest
from httpx import ASGITransport, AsyncClient
from pydantic import SecretStr
from one_memory.api import create_app
from one_memory.config import Settings
from one_memory.context_memory import read_memory_context
from one_memory.context_models import ContextSearchRequest
from one_memory.imports.grants import SourceGrantStore
@@ -30,8 +26,6 @@ from one_memory.memory_governance import (
MemoryMutationRequestConflictError,
MemoryStatus,
)
from one_memory.memory_library import read_memory_library
from one_memory.memory_mutation_receipts import read_memory_mutation_receipt
from one_memory.recall import GovernedRecallTarget, RecallRequest, RecallState
from tests.person_door import as_person
@@ -267,7 +261,6 @@ def test_a_superseded_memory_leaves_every_current_read_path(tmp_path) -> None:
)
return {
"listed": {m.memory_id for m in store.list(user_id="local")},
"library": {m.memory_id for m in read_memory_library(database).items},
"recall": {hit.memory_id for hit in recall.hits},
"semantic": {hit.memory_id for hit in semantic.hits},
"matching": {
@@ -286,7 +279,7 @@ def test_a_superseded_memory_leaves_every_current_read_path(tmp_path) -> None:
after = current_ids()
assert all(old.memory_id not in ids for ids in after.values()), after
assert all(new.memory_id in after[path] for path in ("listed", "library", "recall")), after
assert all(new.memory_id in after[path] for path in ("listed", "recall")), after
assert GovernedRecallTarget(store).recall(RecallRequest(query="权限评审")).state is (
RecallState.GROUNDED
)
@@ -295,9 +288,6 @@ def test_a_superseded_memory_leaves_every_current_read_path(tmp_path) -> None:
assert [m.memory_id for m in store.list(user_id="local", status=MemoryStatus.SUPERSEDED)] == [
old.memory_id
]
library = read_memory_library(database, status=MemoryStatus.SUPERSEDED)
assert [m.memory_id for m in library.items] == [old.memory_id]
assert library.items[0].superseded_by == (new.memory_id,)
async def _async(value):
@@ -413,122 +403,6 @@ def test_the_link_is_stored_additively_without_rewriting_existing_rows(tmp_path)
).fetchone()
def test_an_earlier_receipt_stays_readable_after_its_memory_is_superseded(tmp_path) -> None:
database, store, old, new = _pair(tmp_path)
old = as_person(store).set_scopes(
old.memory_id,
user_id="local",
request_id="scope-old",
expected_revision=old.revision,
scopes=("all-apps", "cursor"),
)
before = read_memory_mutation_receipt(database, request_id="scope-old")
assert before.state == "committed" and before.result_status == "confirmed"
_supersede(store, old, new.memory_id)
after = read_memory_mutation_receipt(database, request_id="scope-old")
assert after == before
assert set(type(after).model_fields) == {
"schema_version",
"request_id",
"state",
"operation",
"memory_id",
"result_revision",
"result_status",
"memory_state",
"migrated_from",
}
def test_the_api_shows_the_chain_refuses_edits_and_still_deletes_through_hy(tmp_path) -> None:
async def scenario() -> None:
app = create_app(
Settings(
data_dir=tmp_path / "data",
source_home=tmp_path,
api_token=SecretStr("local-token"),
)
)
async with (
app.router.lifespan_context(app),
AsyncClient(
transport=ASGITransport(app=app),
base_url="http://test",
headers={
"Authorization": "Bearer local-token",
"Origin": "http://127.0.0.1:5173",
},
) as client,
):
store = app.state.memory_governance_store
evidence_id = _evidence(app.state.evidence_store.database_path)
door = app.state.person_writes
old = _confirmed(door, evidence_id, OLD, "old")
new = _confirmed(door, evidence_id, NEW, "new")
listed = await client.get("/api/v1/memories")
assert {m["memory_id"] for m in listed.json()} == {"old", "new"}
replaced = _supersede(store, old, new.memory_id)
listed = await client.get("/api/v1/memories")
assert [m["memory_id"] for m in listed.json()] == ["new"]
library = await client.get("/api/v1/memory-library")
assert [m["memory_id"] for m in library.json()["items"]] == ["new"]
only_old = await client.get("/api/v1/memory-library", params={"status": "superseded"})
assert [m["memory_id"] for m in only_old.json()["items"]] == ["old"]
detail = (await client.get("/api/v1/memories/old")).json()["memory"]
assert detail["status"] == "superseded"
assert detail["superseded_by"] == ["new"]
assert detail["supersedes"] == []
head = (await client.get("/api/v1/memories/new")).json()["memory"]
assert head["supersedes"] == ["old"] and head["superseded_by"] == []
recalled = await client.post(
"/api/v1/recall", json={"query": "权限评审", "scope": "all-apps"}
)
assert [hit["memory_id"] for hit in recalled.json()["hits"]] == ["new"]
for action in ("undo",):
refused = await client.post(
f"/api/v1/memories/old/actions/{action}",
json={"request_id": f"{action}-x", "expected_revision": replaced.revision},
)
assert (refused.status_code, refused.json()["detail"]) == (
409,
"memory_superseded",
)
refused = await client.post(
"/api/v1/memories/old/correct",
json={
"request_id": "correct-x",
"expected_revision": replaced.revision,
"content": "改一下",
},
)
assert (refused.status_code, refused.json()["detail"]) == (409, "memory_superseded")
cascaded: list[str] = []
async def finish_deletion(memory_id: str) -> None:
cascaded.append(memory_id)
app.state.sdk_memory_runtime.finish_deletion = finish_deletion
deleted = await client.post(
"/api/v1/memories/old/actions/delete",
json={"request_id": "delete-x", "expected_revision": replaced.revision},
)
assert deleted.status_code == 200 and deleted.json()["status"] == "deleted"
assert cascaded == ["old"]
cleanup = await client.get("/api/v1/memories/old/deletion")
assert cleanup.json()["status"] == "completed"
spec = app.openapi()["components"]["schemas"]
assert "superseded" in spec["MemoryStatus"]["enum"]
assert {"supersedes", "superseded_by"} <= set(spec["GovernedMemory"]["properties"])
asyncio.run(scenario())
def test_supersession_integrity_reports_a_cycle_and_stays_quiet_when_healthy(tmp_path) -> None:
"""The read-only audit must flag a real cycle and not cry wolf otherwise.
@@ -2,310 +2,123 @@
Mounted on a minimal app rather than the production one, so the test proves the
router's own contract (status codes, replay, conflict) instead of depending on
the whole server's auth and state wiring.
the whole server's auth and state wiring. Core serves only the SDK write door,
`/api/v2/memory-writes`.
"""
from __future__ import annotations
import sqlite3
from datetime import UTC, datetime, timedelta
import pytest
from fastapi import FastAPI
from fastapi.testclient import TestClient
from one_memory.memory_server.durable_queue import DurableWriteRegistry
from one_memory.memory_server.registry import WriteRegistry
from one_memory.memory_server.routes import router
from one_memory.memory_server.contracts import BatchIdentity, WriteRequest
from one_memory.memory_server.durable_queue import DurableWriteDrain, DurableWriteRegistry
from one_memory.memory_server.routes import sdk_router
@pytest.fixture()
def client() -> TestClient:
app = FastAPI()
app.state.memory_write_registry = WriteRegistry()
app.include_router(router())
return TestClient(app)
ROUTE = "/api/v2/memory-writes"
def _payload(**overrides: object) -> dict:
body = {
"internal_session_id": "sess-1",
"first_round": 1,
"last_round": 5,
"evidence_ids": ["e1"],
body: dict[str, object] = {
"request_id": "request-1",
"session_id": "session-1",
"messages": [
{"role": "user", "content": "remember the deploy window"},
{"role": "assistant", "content": "noted"},
],
}
body.update(overrides)
return body
def test_submit_returns_202_with_a_request_id(client: TestClient) -> None:
response = client.post("/api/v1/memory-writes", json=_payload())
assert response.status_code == 202
assert response.json()["request_id"]
assert response.json()["state"] == "accepted"
def test_a_replay_is_not_a_second_write(client: TestClient) -> None:
"""Duplicating a memory is worse than admitting it already exists."""
first = client.post("/api/v1/memory-writes", json=_payload())
second = client.post("/api/v1/memory-writes", json=_payload())
assert second.status_code == 202
assert second.json()["request_id"] == first.json()["request_id"]
assert "already accepted" in second.json()["detail"]
def test_same_key_different_payload_is_409(client: TestClient) -> None:
"""Overwriting would discard the earlier intent."""
client.post(
"/api/v1/memory-writes",
json=_payload(idempotency_key="fixed", evidence_ids=["e1"]),
)
conflict = client.post(
"/api/v1/memory-writes",
json=_payload(idempotency_key="fixed", evidence_ids=["different"]),
)
assert conflict.status_code == 409
def test_status_can_be_queried_after_submit(client: TestClient) -> None:
created = client.post("/api/v1/memory-writes", json=_payload())
request_id = created.json()["request_id"]
status = client.get(f"/api/v1/memory-writes/{request_id}")
assert status.status_code == 200
assert status.json()["request_id"] == request_id
def test_status_of_an_unknown_id_is_404(client: TestClient) -> None:
"""'We never accepted this' must not read as 'accepted and failed'."""
assert client.get("/api/v1/memory-writes/absent").status_code == 404
def test_missing_evidence_is_rejected(client: TestClient) -> None:
response = client.post("/api/v1/memory-writes", json=_payload(evidence_ids=[]))
assert response.status_code == 422
def test_unknown_fields_are_rejected(client: TestClient) -> None:
"""extra=forbid: a typo'd field must not be silently ignored."""
response = client.post("/api/v1/memory-writes", json=_payload(evidences=["e1"]))
assert response.status_code == 422
def test_no_registry_means_unavailable_not_a_crash(client: TestClient) -> None:
app = FastAPI()
app.include_router(router())
bare = TestClient(app)
assert bare.post("/api/v1/memory-writes", json=_payload()).status_code == 503
def test_durable_queue_can_back_the_same_route_contract(tmp_path) -> None:
"""The route shape must survive a process restart when backed by SQLite.
This is the missing D52 bridge: the HTTP contract already exists, and the
durable queue already records acceptance. The adapter must make the same
POST/GET contract durable without changing the route or inventing a second
idempotency rule.
"""
path = tmp_path / "one-memory.sqlite3"
app = FastAPI()
app.state.memory_write_registry = DurableWriteRegistry(path)
app.include_router(router())
client = TestClient(app)
created = client.post("/api/v1/memory-writes", json=_payload())
assert created.status_code == 202
request_id = created.json()["request_id"]
assert created.json()["state"] == "accepted"
reopened = FastAPI()
reopened.state.memory_write_registry = DurableWriteRegistry(path)
reopened.include_router(router())
status = TestClient(reopened).get(f"/api/v1/memory-writes/{request_id}")
assert status.status_code == 200
assert status.json()["request_id"] == request_id
assert status.json()["state"] == "accepted"
replay = TestClient(reopened).post("/api/v1/memory-writes", json=_payload())
assert replay.status_code == 202
assert replay.json()["request_id"] == request_id
assert "already accepted" in replay.json()["detail"]
def test_completed_write_status_returns_sdk_raw_and_memory_ids(tmp_path) -> None:
path = tmp_path / "one-memory.sqlite3"
app = FastAPI()
registry = DurableWriteRegistry(path)
app.state.memory_write_registry = registry
app.include_router(router())
client = TestClient(app)
created = client.post("/api/v1/memory-writes", json=_payload())
request_id = created.json()["request_id"]
with sqlite3.connect(path) as database:
job_id = str(
database.execute(
"SELECT job_id FROM memory_write_jobs WHERE idempotency_key=?",
(request_id,),
).fetchone()[0]
)
registry.queue.settle(
job_id,
state="completed",
sdk_raw_ids=("raw-1",),
sdk_memory_ids=("memory-1",),
)
status = client.get(f"/api/v1/memory-writes/{request_id}")
assert status.status_code == 200
assert status.json()["raw_ids"] == ["raw-1"]
assert status.json()["memory_ids"] == ["memory-1"]
def test_route_selects_durable_registry_when_a_path_is_configured(tmp_path) -> None:
"""C0 only has to hand the route a database path.
The route keeps its legacy in-memory fallback for isolated tests, but when
the production app provides `memory_write_queue_path`, the first request
must create the durable adapter and write a real row.
"""
path = tmp_path / "one-memory.sqlite3"
def _durable_client(path) -> tuple[FastAPI, TestClient]:
app = FastAPI()
app.state.memory_write_queue_path = path
app.include_router(router())
client = TestClient(app)
app.state.memory_authority = "sdk"
app.include_router(sdk_router())
return app, TestClient(app)
created = client.post("/api/v1/memory-writes", json=_payload())
assert created.status_code == 202
def test_submit_returns_202_and_replay_is_not_a_second_write(tmp_path) -> None:
path = tmp_path / "one-memory.sqlite3"
app, client = _durable_client(path)
first = client.post(ROUTE, json=_payload())
second = client.post(ROUTE, json=_payload())
assert first.status_code == 202
assert second.status_code == 202
assert first.json()["request_id"] == second.json()["request_id"]
assert "already accepted" in second.json()["detail"]
assert isinstance(app.state.memory_write_registry, DurableWriteRegistry)
with sqlite3.connect(path) as db:
assert db.execute("SELECT count(*) FROM memory_write_jobs").fetchone()[0] == 1
def test_durable_route_reports_unknown_without_collapsing_to_failed(tmp_path) -> None:
"""The HTTP state vocabulary must preserve the no-blind-retry outcome."""
path = tmp_path / "one-memory.sqlite3"
app = FastAPI()
registry = DurableWriteRegistry(path)
app.state.memory_write_registry = registry
app.include_router(router())
client = TestClient(app)
def test_a_write_without_a_user_turn_is_rejected(tmp_path) -> None:
_app, client = _durable_client(tmp_path / "one-memory.sqlite3")
created = client.post("/api/v1/memory-writes", json=_payload())
request_id = created.json()["request_id"]
assert registry.set_state(request_id, "unknown", "outcome not proven")
response = client.get(f"/api/v1/memory-writes/{request_id}")
response = client.post(
ROUTE,
json=_payload(messages=[{"role": "assistant", "content": "only me"}]),
)
assert response.status_code == 200
assert response.json()["state"] == "unknown"
assert response.status_code == 422
assert response.json()["detail"] == "messages_require_user_turn"
def test_unknown_fields_are_rejected(tmp_path) -> None:
_app, client = _durable_client(tmp_path / "one-memory.sqlite3")
assert client.post(ROUTE, json=_payload(evidence_ids=["e1"])).status_code == 422
def test_status_of_an_unknown_id_is_404(tmp_path) -> None:
_app, client = _durable_client(tmp_path / "one-memory.sqlite3")
client.post(ROUTE, json=_payload())
assert client.get(f"{ROUTE}/absent").status_code == 404
def test_durable_route_maps_a_dead_letter_to_a_valid_failed_state(tmp_path) -> None:
"""`dead` is a queue concept; the HTTP contract has no such state.
Retries exhausted must still return a terminal receipt the response model
accepts, without hiding that it was a dead letter rather than a normal
failure.
"""
app = FastAPI()
"""`dead` is a queue concept; the HTTP contract has no such state."""
database_path = tmp_path / "one-memory.sqlite3"
app.state.memory_write_queue_path = database_path
app.include_router(router())
client = TestClient(app)
created = client.post("/api/v1/memory-writes", json=_payload())
request_id = created.json()["request_id"]
_app, client = _durable_client(database_path)
request_id = client.post(ROUTE, json=_payload()).json()["request_id"]
registry = DurableWriteRegistry(database_path)
registry.queue.settle( # noqa: SLF001 - test needs the durable job id
registry = DurableWriteRegistry(database_path, sdk_authority=True)
registry.queue.settle(
registry.queue.get(request_id).job_id,
state="dead",
error_code="attempts exhausted",
)
response = client.get(f"/api/v1/memory-writes/{request_id}")
response = client.get(f"{ROUTE}/{request_id}")
assert response.status_code == 200
assert response.json()["state"] == "failed"
assert "dead letter" in response.json()["detail"].lower()
def test_durable_configuration_overrides_an_in_memory_registry(tmp_path) -> None:
"""`api.py` always installs an in-memory registry; durable must still win.
The production composition root sets `memory_write_registry` eagerly, so a
durable router that preferred that object would silently accept writes
into memory and lose them on restart. Setting the durable queue path must
take precedence.
"""
path = tmp_path / "one-memory.sqlite3"
app = FastAPI()
app.state.memory_write_registry = WriteRegistry()
app.state.memory_write_queue_path = path
app.include_router(router())
client = TestClient(app)
created = client.post("/api/v1/memory-writes", json=_payload())
assert created.status_code == 202
request_id = created.json()["request_id"]
# The receipt is durable: it survives a brand-new registry built from the
# same file, which the in-memory registry could not do.
assert DurableWriteRegistry(path).get(request_id) is not None
with sqlite3.connect(path) as db:
assert db.execute("SELECT count(*) FROM memory_write_jobs").fetchone()[0] == 1
def test_route_passes_the_evidence_store_into_the_durable_registry(tmp_path, monkeypatch) -> None:
"""A configured Evidence store must enforce Evidence-first at the route.
Without this wiring the queue accepts a write pointing at evidence the
business database does not hold, and the failure only surfaces later as a
`failed` drain job. When the composition root provides the store, the POST
must be refused up front and nothing may be enqueued.
"""
from one_memory.imports.storage import EvidenceStore
path = tmp_path / "one-memory.sqlite3"
app = FastAPI()
app.state.memory_write_queue_path = path
app.state.memory_write_evidence_store = EvidenceStore(path)
app.include_router(router())
client = TestClient(app)
response = client.post(
"/api/v1/memory-writes",
json=_payload(evidence_ids=["does-not-exist"]),
)
assert response.status_code >= 400
with sqlite3.connect(path) as db:
assert db.execute("SELECT count(*) FROM memory_write_jobs").fetchone()[0] == 0
def test_http_get_reports_a_claimed_running_job_as_accepted(tmp_path) -> None:
"""A live drain turns a submitted row `running`; GET must not 500."""
from datetime import timedelta
from one_memory.memory_server.durable_queue import DurableWriteDrain
async def never_execute(_job_id: str) -> None:
return None
path = tmp_path / "one-memory.sqlite3"
app = FastAPI()
registry = DurableWriteRegistry(path)
app.state.memory_write_registry = registry
app.include_router(router())
client = TestClient(app)
created = client.post("/api/v1/memory-writes", json=_payload())
request_id = created.json()["request_id"]
app, client = _durable_client(path)
request_id = client.post(ROUTE, json=_payload()).json()["request_id"]
DurableWriteDrain(
registry.queue,
app.state.memory_write_registry.queue,
execute=never_execute,
owner="owner-a",
lease=timedelta(seconds=30),
).claim(limit=1)
response = client.get(f"/api/v1/memory-writes/{request_id}")
response = client.get(f"{ROUTE}/{request_id}")
assert response.status_code == 200
assert response.json()["state"] == "accepted"
@@ -319,10 +132,6 @@ def test_read_get_does_not_reclaim_an_expired_lease(tmp_path) -> None:
row to `unknown`/`pending` — a read that mutates the queue and can collide
with the drain. The read path must open without initializing.
"""
from datetime import UTC, datetime, timedelta
from one_memory.memory_server.contracts import BatchIdentity, WriteRequest
path = tmp_path / "one-memory.sqlite3"
queue = DurableWriteRegistry(path).queue
created = queue.submit(
@@ -342,10 +151,7 @@ def test_read_get_does_not_reclaim_an_expired_lease(tmp_path) -> None:
((datetime.now(UTC) - timedelta(hours=1)).isoformat(), created.job_id),
)
app = FastAPI()
app.state.memory_write_queue_path = path
app.include_router(router())
client = TestClient(app)
_app, client = _durable_client(path)
def row():
with sqlite3.connect(path) as db:
@@ -358,7 +164,7 @@ def test_read_get_does_not_reclaim_an_expired_lease(tmp_path) -> None:
)
before = row()
response = client.get(f"/api/v1/memory-writes/{created.request_id}")
response = client.get(f"{ROUTE}/{created.request_id}")
after = row()
assert response.status_code == 200
@@ -373,11 +179,6 @@ def test_read_get_on_an_unmigrated_database_is_503_not_500(tmp_path) -> None:
with sqlite3.connect(path) as db:
db.execute("CREATE TABLE unrelated (id INTEGER)")
app = FastAPI()
app.state.memory_write_queue_path = path
app.include_router(router())
client = TestClient(app)
_app, client = _durable_client(path)
response = client.get("/api/v1/memory-writes/does-not-exist")
assert response.status_code == 503
assert client.get(f"{ROUTE}/does-not-exist").status_code == 503
@@ -136,6 +136,64 @@ def test_cursor_store_timestamp_envelope_keeps_only_the_user_query_text(tmp_path
assert meta["turn_event_at"] == "2026-09-22T23:34:00+00:00"
def test_cursor_wal_change_token_does_not_become_a_future_conversation_time(
tmp_path,
) -> None:
root = tmp_path / ".cursor"
database = root / "chats" / "workspace" / "wal-time" / "store.db"
database.parent.mkdir(parents=True)
connection = sqlite3.connect(database)
connection.execute("PRAGMA journal_mode=WAL")
connection.execute("PRAGMA wal_autocheckpoint=0")
connection.execute("CREATE TABLE blobs (id TEXT, data TEXT)")
connection.execute(
"INSERT INTO blobs VALUES (?, ?)",
("message", json.dumps({"role": "assistant", "content": "answer"})),
)
connection.commit()
stamp = 1_788_220_800
os.utime(database, (stamp, stamp))
try:
records = evidence(
asyncio.run(scan(CursorUnifiedSourceAdapter(grant=grant("cursor", root))))
)
finally:
connection.close()
assert len(records) == 1
assert int(records[0].evidence.created_at.timestamp()) == stamp
def test_cursor_wal_store_resumes_from_its_rowid_after_new_frames(tmp_path) -> None:
"""The WAL change token marks a store changed; it must not reset its checkpoint."""
root = tmp_path / ".cursor"
database = root / "chats" / "workspace" / "wal-resume" / "store.db"
database.parent.mkdir(parents=True)
connection = sqlite3.connect(database)
connection.execute("PRAGMA journal_mode=WAL")
connection.execute("PRAGMA wal_autocheckpoint=0")
connection.execute("CREATE TABLE blobs (id TEXT, data TEXT)")
connection.execute(
"INSERT INTO blobs VALUES (?, ?)",
("old", json.dumps({"id": "old", "role": "user", "content": "old row"})),
)
connection.commit()
adapter = CursorUnifiedSourceAdapter(grant=grant("cursor", root))
try:
baseline = asyncio.run(adapter.live_baseline())
assert evidence(asyncio.run(scan(adapter, cursors=baseline))) == []
connection.execute(
"INSERT INTO blobs VALUES (?, ?)",
("new", json.dumps({"id": "new", "role": "user", "content": "new row"})),
)
connection.commit()
appended = asyncio.run(scan(adapter, cursors=baseline))
finally:
connection.close()
assert [item.evidence.verbatim for item in evidence(appended)] == ["new row"]
def test_cursor_transcript_timestamp_envelope_keeps_only_the_user_query_text(tmp_path) -> None:
root = tmp_path / ".cursor"
transcript = (
@@ -1,125 +0,0 @@
"""Search imported conversations with no generative model configured at all.
This answers the question a user would ask: I imported my history and I
never set up a model, can One Memory find things I said? This goes through the
product's own entry points only: authorize the source, run an import job,
search `/api/v1/context/search`.
"""
import asyncio
import json
from datetime import UTC, datetime
import pytest
from httpx import ASGITransport, AsyncClient
from pydantic import SecretStr
from one_memory.api import create_app
from one_memory.config import Settings
from one_memory.imports.sources.registry import build_source_catalog
from one_memory.secrets import InMemorySecretStore
from tests.source_consent import authorized_request, thaw_source
TOKEN = "raw-search-no-model"
AUTH = {"Authorization": f"Bearer {TOKEN}"}
LICENCE = "我们最后决定 One Memory 用 MIT 协议发布,不再考虑 GPL。"
BACKUP = "备份先写到本地磁盘,再同步到对象存储。"
SYNONYM = "开源软件的授权条款最后怎么挑的"
pytestmark = [pytest.mark.integration, pytest.mark.serial]
def transcript(root, name, *texts):
lines = [
json.dumps(
{
"type": "response_item",
"timestamp": datetime(2026, 9, 1, tzinfo=UTC).isoformat(),
"payload": {
"type": "message",
"role": "user",
"content": [{"type": "text", "text": text}],
},
}
)
for text in texts
]
(root / f"rollout-{name}.jsonl").write_text("\n".join(lines) + "\n")
def application(tmp_path):
home = tmp_path / "home"
root = home / ".codex/sessions"
root.mkdir(parents=True)
transcript(root, "licence", LICENCE)
transcript(root, "backup", BACKUP)
data = tmp_path / "data"
data.mkdir()
return create_app(
Settings(
_env_file=None,
data_dir=data,
source_home=home,
api_token=SecretStr(TOKEN),
automatic_memory_processing=False,
live_sync_enabled=False,
),
source_catalog={"codex": build_source_catalog(home=home, environment={})["codex"]},
secret_store=InMemorySecretStore(),
)
async def imported(app, client):
assert (await client.get("/api/v1/model-profiles")).json() == [], "no model may be configured"
authorized = await client.post(
"/api/v1/sources/codex/authorize",
json=await authorized_request(client, "codex"),
)
authorized.raise_for_status()
await thaw_source(client, "codex", authorized.json())
started = await client.post("/api/v1/import-jobs", json={"source_id": "codex"})
started.raise_for_status()
job = await app.state.import_supervisor.wait(started.json()["job_id"])
assert job.status == "completed" and job.stored == 2
async def search(client, query, *, attempts=1):
for remaining in range(attempts, 0, -1):
response = await client.post("/api/v1/context/search", json={"query": query}, headers=AUTH)
response.raise_for_status()
body = response.json()
if body["items"] or remaining == 1:
return body
await asyncio.sleep(0.5)
raise AssertionError("unreachable")
def texts(body):
return [
message["text"]
for item in body["items"]
if item["kind"] == "raw"
for message in item["messages"]
]
def test_shared_words_are_found_without_any_model_or_index(tmp_path):
async def scenario():
app = application(tmp_path)
async with (
app.router.lifespan_context(app),
AsyncClient(
transport=ASGITransport(app=app), base_url="http://test", headers=AUTH
) as client,
):
await imported(app, client)
body = await search(client, "MIT 协议")
assert body["state"] == "grounded"
assert any(LICENCE in text for text in texts(body))
missed = await search(client, SYNONYM)
# Different words are not a lexical match, and there is no private
# vector index to pretend otherwise.
assert not texts(missed)
asyncio.run(scenario())
@@ -102,7 +102,7 @@ def test_the_read_does_not_scale_its_query_count_with_the_page(library):
the citation subquery 0.08 s. The UDF runs several queries per memory —
proofs, versions, active citations, evidence sizes — so the count grows
with the page, and the cost grows again when the library is being written
to. That is what put both `/api/v1/memories` and the map over their five
to. That is what put both the legacy memory list and the map over their five
second interrupt.
The per-memory lookups can be read once for the whole page. Quadrupling
-255
View File
@@ -1,255 +0,0 @@
"""The Reader process stays read-only and never blocks a read on its side effects."""
import asyncio
from sqlite3 import OperationalError
from pydantic import SecretStr
from one_memory.imports.models import (
EvidenceInput,
EvidenceOrigin,
OriginMetadata,
WrittenBy,
)
from one_memory.imports.storage import EvidenceStore
from one_memory.memory_governance import MemoryDraft, MemoryGovernanceStore, MemoryKind
from one_memory.reader_app import ReaderSettings, SideEffectForwarder, _ReaderGovernance
from one_memory.recall import GovernedRecallTarget, RecallRequest, RecallState
from tests.person_door import as_person
def _settings(tmp_path) -> ReaderSettings:
return ReaderSettings(
data_dir=tmp_path,
api_token=SecretStr("reader-token"),
core_port=8765,
core_api_token="core-token",
)
def test_side_effects_are_queued_not_written(tmp_path) -> None:
settings = _settings(tmp_path)
forwarder = SideEffectForwarder(settings)
for index in range(5):
forwarder.enqueue({"tool": "om_recall", "index": index})
# The queue is in memory only: nothing was written to the data directory.
assert not any(tmp_path.iterdir())
assert forwarder.queue.qsize() == 5
def test_a_full_queue_drops_instead_of_blocking(tmp_path) -> None:
settings = _settings(tmp_path)
forwarder = SideEffectForwarder(settings)
for index in range(300):
forwarder.enqueue({"index": index})
assert forwarder.queue.qsize() == 256
assert forwarder.dropped == 44
def test_an_unreachable_core_counts_the_drop(tmp_path) -> None:
settings = _settings(tmp_path).model_copy(update={"core_port": 1})
async def scenario() -> None:
forwarder = SideEffectForwarder(settings)
forwarder.enqueue({"tool": "om_recall"})
task = asyncio.ensure_future(forwarder._drain())
# The read answered already; the forward never waits on it.
deadline = asyncio.get_running_loop().time() + 2.0
while forwarder.dropped < 1 and asyncio.get_running_loop().time() < deadline:
await asyncio.sleep(0.01)
task.cancel()
await asyncio.gather(task, return_exceptions=True)
assert forwarder.dropped >= 1
asyncio.run(scenario())
def test_the_reader_connection_refuses_writes(tmp_path) -> None:
"""query_only is what keeps a read from becoming a write.
Without it, a store that opens DDL would silently take the write lock in a
process whose whole reason for existing is to stay out of the writer's way.
"""
from one_memory.database_access import main_database
database_path = tmp_path / "one-memory.sqlite3"
with main_database(database_path).write() as connection:
connection.execute("CREATE TABLE t (a INTEGER)")
with main_database(database_path).read() as connection:
try:
connection.execute("INSERT INTO t VALUES (1)")
except OperationalError as error:
assert "readonly" in str(error).lower() or "read-only" in str(error).lower()
else:
raise AssertionError("reader connection accepted a write")
def test_reader_governance_supports_non_probe_recall(tmp_path) -> None:
database = tmp_path / "one-memory.sqlite3"
evidence = (
EvidenceStore(database)
.put(
EvidenceInput(
source_id="cursor",
conversation_id="conversation",
message_id="message",
role="user",
written_by=WrittenBy.PERSON,
verbatim="用户偏好本地优先架构。",
origin_meta={EvidenceOrigin.FILE: OriginMetadata()},
)
)
.record
)
governance = MemoryGovernanceStore(database)
memory = as_person(governance).create(
MemoryDraft(
user_id="local",
kind=MemoryKind.CHAT,
content="用户偏好本地优先架构",
scopes=("all-apps",),
evidence_ids=(evidence.evidence_id,),
),
request_id="reader-recall",
)
result = GovernedRecallTarget(_ReaderGovernance(database)).recall(
RecallRequest(query="本地优先架构")
)
assert result.state is RecallState.GROUNDED
assert [hit.memory_id for hit in result.hits] == [memory.memory_id]
def test_importing_the_reader_never_loads_the_write_side() -> None:
"""The Reader must not load keyring, HY, Chroma or ONNX.
Checked in a child process because `sys.modules` in the test process is
already polluted by whatever else the suite imported; only a fresh
interpreter can say what importing `reader_app` drags in. This is the
check the Reader's own startup guard performs, so a regression here is
also a Reader that exits immediately on launch.
"""
import subprocess
import sys
from pathlib import Path
source_root = Path(__file__).resolve().parents[1] / "src"
script = (
"import sys\n"
"import one_memory.reader_app\n"
"loaded = sorted(\n"
" name.split('.')[0]\n"
" for name in sys.modules\n"
" if name.split('.')[0] in ('keyring', 'chromadb', 'onnxruntime', 'hy_memory')\n"
")\n"
"sys.stdout.write(repr(loaded))\n"
)
result = subprocess.run(
[sys.executable, "-c", script],
capture_output=True,
text=True,
env={"PYTHONPATH": str(source_root), "PATH": "/usr/bin:/bin"},
timeout=120,
)
assert result.returncode == 0, result.stderr
assert result.stdout.strip() == "[]", result.stdout
def test_reader_reports_the_port_it_serves_on(tmp_path) -> None:
"""D103: the supervisor can only learn the Reader port from stderr.
It asks for an ephemeral port (`ONE_MEMORY_READER_PORT=0`), so the Reader
has to claim a real port first, print `reader_port:<n>` and then serve on
that same port. Letting uvicorn pick its own port, or printing a different
one, leaves the supervisor waiting out its 10s timeout and then connecting
to a port nobody listens on.
Run in a child process for the same reason as the import guard above:
`create_reader_app` refuses to start when the suite has already imported
`hy_memory`, so only a fresh interpreter exercises this path.
"""
import json
import subprocess
import sys
from pathlib import Path
from one_memory.database_access import main_database
source_root = Path(__file__).resolve().parents[1] / "src"
script = """
import io
import json
import re
import sys
from pathlib import Path
import uvicorn
from pydantic import SecretStr
from one_memory import cli
from one_memory.config import Settings
served_on = []
served_with = []
def fake_run(application, *, host=None, port=None, **_kwargs):
served_on.append(port)
served_with.append(_kwargs)
uvicorn.run = fake_run
cli.load_settings = lambda: Settings(
_env_file=None,
data_dir=Path(sys.argv[1]),
api_token=SecretStr("token"),
port=0,
)
stderr = io.StringIO()
previous = sys.stderr
sys.stderr = stderr
try:
cli.serve(reader=True)
finally:
sys.stderr = previous
printed = stderr.getvalue()
match = re.search(r"reader_port:([0-9]+)", printed)
json.dump(
{
"printed": printed,
"served_on": served_on,
"served_with": served_with,
"port": int(match.group(1)) if match else None,
},
sys.stdout,
)
"""
database = tmp_path / "one-memory.sqlite3"
with main_database(database).write() as connection:
connection.execute("CREATE TABLE t (a INTEGER)")
result = subprocess.run(
[sys.executable, "-c", script, str(tmp_path)],
capture_output=True,
text=True,
env={
"PYTHONPATH": str(source_root),
"PATH": "/usr/bin:/bin",
"HOME": str(tmp_path),
"ONE_MEMORY_DATA_DIR": str(tmp_path),
"ONE_MEMORY_READER_PORT": "0",
"ONE_MEMORY_CORE_PORT": "8765",
"ONE_MEMORY_CORE_API_TOKEN": "core-token",
},
timeout=120,
)
assert result.returncode == 0, result.stderr
payload = json.loads(result.stdout)
assert payload["port"], payload["printed"]
# Port 0 means "any free port", so the reported value must be a real one.
assert payload["port"] > 0
assert payload["served_on"] == [payload["port"]]
assert payload["served_with"] == [{"reload": False, "access_log": False}]
@@ -1,41 +0,0 @@
"""A search hit carries who wrote the row it matched.
The Reader used to send back everything about a matched row except its author,
and the one client that lists hits as dialogue filled the gap in by hand, as
"the user". These pin the author to the row so the gap cannot reopen.
"""
from one_memory.reader_app import _conversation_hit
def _row(**overrides: object) -> dict[str, object]:
row: dict[str, object] = {
"evidence_id": "evidence-1",
"conversation_id": "conversation-1",
"source_id": "codex",
"workspace": "/Users/k/one-memory",
"project": "one-memory",
"created_at": "2026-10-04T10:00:00Z",
"verbatim": "红眼航班要靠过道",
"role": "assistant",
"written_by": "agent",
}
row.update(overrides)
return row
def test_an_assistant_row_stays_attributed_to_the_assistant() -> None:
hit = _conversation_hit(_row())
assert hit["role"] == "assistant"
assert hit["written_by"] == "agent"
def test_a_row_the_person_wrote_is_not_relabelled_on_the_way_out() -> None:
hit = _conversation_hit(_row(role="user", written_by="person"))
assert hit["role"] == "user"
assert hit["written_by"] == "person"
def test_a_long_row_is_excerpted_and_an_empty_one_is_not_a_crash() -> None:
assert _conversation_hit(_row(verbatim="航" * 400))["excerpt"] == "航" * 200
assert _conversation_hit(_row(verbatim=None))["excerpt"] == ""
@@ -1,196 +0,0 @@
"""RCL-04 against a real HY instance instead of a stand-in.
Enable it with a configured, redacted model-profile database:
ONE_MEMORY_MODEL_TEST_DB=/path/to/profile.sqlite3 \
uv run --directory apps/core pytest -q tests/test_recall_fusion_live.py
The fusion tests elsewhere drive a fake client, so they prove the aggregator
merges two lanes but not that the semantic lane is really HY. This one puts a
memory in through the product's own import path, lets real HY store it, and
asks two questions of the same live instance:
a natural question -- overlaps the stored wording on one word, so only
HY's semantics can reach the answer
the stored wording -- which the lexical lane can answer on its own
The second is not decoration. The natural question comes back with an empty
raw lane, and empty alone cannot tell an unauthorized lane from an unwired one
from a merely strict one. Asking both in the same run settles which.
Seeding goes through `add_extracted` on purpose. This test isolates semantic
storage and retrieval from generative extraction quality, which has separate
deterministic and live acceptance coverage.
"""
import asyncio
import os
import runpy
from pathlib import Path
import pytest
from httpx import ASGITransport, AsyncClient
from pydantic import SecretStr
from one_memory.api import create_app
from one_memory.config import Settings
from one_memory.recall import _terms
from one_memory.secrets import KeyringSecretStore
from tests.source_consent import connection_request
PROFILE_DB = os.environ.get("ONE_MEMORY_MODEL_TEST_DB")
pytestmark = [
pytest.mark.live,
pytest.mark.serial,
pytest.mark.skipif(not PROFILE_DB, reason="explicit configured provider required"),
]
STORED = "项目周会固定避开周五,星期五不安排项目周会。"
# Shares no term with the stored wording at all, so the lexical half of the
# memory lane cannot reach it and a hit can only have come from HY. The test
# checks that disjointness rather than trusting this comment.
QUESTION = "团队碰头会应该躲开一周里的哪一天"
# The same fact in the stored wording: the control for the lexical lane.
QUOTED = "项目周会固定避开周五"
def test_both_lanes_answer_through_one_real_hy_instance(tmp_path, monkeypatch):
from hy_memory.client import HyMemoryClient
# This run is fast because `add_extracted` needs no generation, which makes
# "real HY" easy to assert and impossible to see. Counting the calls on the
# real client is the difference between the two.
stored: list[list[str]] = []
searched: list[str] = []
real_add_extracted = HyMemoryClient.add_extracted
real_search = HyMemoryClient.search
def counted_add_extracted(self, contents, **kwargs):
stored.append(list(contents))
return real_add_extracted(self, contents, **kwargs)
def counted_search(self, query=None, **kwargs):
searched.append(query)
return real_search(self, query, **kwargs)
def forbidden(*args, **kwargs):
pytest.fail("Already-extracted content must not be replayed as raw chat")
monkeypatch.setattr(HyMemoryClient, "add_extracted", counted_add_extracted)
monkeypatch.setattr(HyMemoryClient, "search", counted_search)
monkeypatch.setattr(HyMemoryClient, "add", forbidden)
async def scenario():
helpers = runpy.run_path(
str(Path(__file__).resolve().parents[3] / "scripts/run_matter_first_import.py")
)
directory = tmp_path / "data"
directory.mkdir(parents=True)
helpers["copy_model_configuration"](Path(PROFILE_DB), directory / "one-memory.sqlite3")
app = create_app(
Settings(
data_dir=directory,
source_home=tmp_path / "home",
api_token=SecretStr("live-fusion"),
live_sync_enabled=False,
),
secret_store=KeyringSecretStore(),
)
async with (
app.router.lifespan_context(app),
AsyncClient(
transport=ASGITransport(app=app),
base_url="http://test",
headers={"Authorization": "Bearer live-fusion"},
) as client,
):
upload = await client.post(
"/api/v1/import-files?filename=weekly.txt&input_kind=memory_content",
content=STORED.encode(),
)
assert upload.status_code == 201, upload.text
receipt = upload.json()
(
await client.post(
f"/api/v1/import-files/{receipt['upload_id']}/commit",
json={"confirmed": True},
)
).raise_for_status()
linked = await client.post(
"/api/v1/sources/connect",
json=await connection_request(
client,
[receipt["source_id"]],
enable_sync=True,
),
)
linked.raise_for_status()
job = await app.state.import_supervisor.wait(linked.json()["jobs"][0]["job_id"])
assert job.stored == 1 and job.errors == 0, job
deadline = asyncio.get_running_loop().time() + 300
while True:
counts = app.state.sdk_memory_runtime.status_counts()
if not any(
counts.get(state, 0)
for state in ("pending", "running", "waiting_for_model")
):
break
if asyncio.get_running_loop().time() >= deadline:
raise AssertionError(f"SDK write queue did not drain: {counts}")
await asyncio.sleep(0.05)
# Imported content is not shared with every app by default. Sharing
# it here is what the memory lane is later asked to find.
memories = (await client.get("/api/v1/memories")).json()
assert memories, "real HY stored nothing to search for"
for item in memories:
(
await client.post(
f"/api/v1/memories/{item['memory_id']}/correct",
json={
"request_id": f"share-{item['memory_id']}",
"expected_revision": item["revision"],
"content": item["content"],
"scopes": ["all-apps"],
},
)
).raise_for_status()
async def search(query: str) -> dict:
response = await client.post(
"/api/v1/context/search",
json={"query": query, "kinds": ["raw", "memory"]},
)
assert response.status_code == 200, response.text
return response.json()
return await search(QUESTION), await search(QUOTED)
asked, quoted = asyncio.run(scenario())
assert stored == [[STORED]], stored
# Both questions reached the real client, so the memory lane below is HY
# answering and not a cache, a fixture or a lexical result mislabelled.
assert searched == [QUESTION, QUOTED], searched
# The memory lane matches terms as well as taking HY's ids, so a shared
# word would let it answer without HY and make the next assertion empty.
assert not (set(_terms(STORED)) & set(_terms(QUESTION))), "the question is not disjoint"
memory = asked["paths"]["memory"]
assert memory["status"] == "ok" and memory["returned_count"] > 0, asked["paths"]
remembered = [item for item in asked["items"] if item["kind"] == "memory"]
assert any("周五" in item["content"] or "星期五" in item["content"] for item in remembered), (
remembered
)
# Same instance, same authorization, wording the lexical lane can match.
# Without this the empty raw lane above would be unreadable.
assert quoted["paths"]["raw"]["status"] == "ok", quoted["paths"]
assert quoted["paths"]["raw"]["returned_count"] > 0, quoted["paths"]
assert {item["kind"] for item in quoted["items"]} == {"raw", "memory"}, quoted["items"]
# So the finding is about reach, not plumbing: the lexical lane wants most
# of the query's words present and drops a question overlapping on one.
# The synthetic benchmark measured the same shape (lexical-only hit 0.429);
# this is that result on a memory the product actually stored.
assert asked["paths"]["raw"]["status"] == "empty", asked["paths"]
@@ -8,8 +8,6 @@ from pydantic import SecretStr
from one_memory.api import create_app
from one_memory.config import Settings
from one_memory.imports.access import opaque_workspace_id
from one_memory.imports.models import EvidenceInput, EvidenceOrigin, OriginMetadata, WrittenBy
from one_memory.memory_governance import MemoryDraft, MemoryKind
from one_memory.secrets import InMemorySecretStore
@@ -32,47 +30,22 @@ def test_external_scope_cannot_override_shared_or_host_workspace_boundary(tmp_pa
headers={"Authorization": "Bearer scope-test"},
) as client,
):
source = app.state.evidence_store.put(
EvidenceInput(
source_id="manual",
conversation_id="scope",
message_id="1",
role="user",
written_by=WrittenBy.PERSON,
verbatim="Project meeting avoids Friday",
origin_meta={EvidenceOrigin.FILE: OriginMetadata()},
)
).record
scopes = ["private", "all-apps", opaque_workspace_id("claude-code", "/test/project")]
ids = []
for scope in scopes:
item = app.state.person_writes.create(
MemoryDraft(
user_id="local",
kind=MemoryKind.CHAT,
content="Project meeting avoids Friday",
scopes=(scope,),
evidence_ids=(source.evidence_id,),
),
request_id=f"create-{scope}",
)
ids.append(item.memory_id)
workspace = opaque_workspace_id("claude-code", "/test/project")
semantic = AsyncMock(return_value=())
app.state.recall_target.semantic_search = semantic
app.state.memory_answerer.semantic_search = semantic
external = {
"X-One-Memory-Read-Context": "external",
"X-One-Memory-Installation-Id": "24361b44-5a15-4f87-ad09-05571dc281c3",
"X-One-Memory-Installation-Revision": "1",
"X-One-Memory-Allowed-Tools": json.dumps(["om_recall", "om_ask"]),
"X-One-Memory-Allowed-Scopes": json.dumps(["all-apps", scopes[2]]),
"X-One-Memory-Allowed-Scopes": json.dumps(["all-apps", workspace]),
}
for endpoint, key in [
("/api/v1/recall", "query"),
("/api/v1/memories/ask", "question"),
("/api/v1/memories/ask/stream", "question"),
("/api/v2/memories/ask", "question"),
("/api/v2/memories/ask/stream", "question"),
]:
for scope in ["*", "private", scopes[2], "unrelated-scope"]:
for scope in ["*", "private", workspace, "unrelated-scope"]:
response = await client.post(
endpoint, json={key: "Project meeting", "scope": scope}, headers=external
)
@@ -81,28 +54,8 @@ def test_external_scope_cannot_override_shared_or_host_workspace_boundary(tmp_pa
normal = await client.post(
"/api/v1/recall", json={"query": "Project meeting"}, headers=external
)
assert [hit["memory_id"] for hit in normal.json()["hits"]] == [ids[1]]
workspace = await client.post(
"/api/v1/recall",
json={
"query": "Project meeting",
"scope": scopes[2],
"workspace_path": "/test/project",
},
headers=external,
)
assert [hit["memory_id"] for hit in workspace.json()["hits"]] == [ids[2]]
denied_expansion = await client.post(
"/api/v1/recall",
json={"query": "Project meeting", "workspace_path": "/test/project"},
headers={**external, "X-One-Memory-Allowed-Scopes": '["all-apps"]'},
)
assert denied_expansion.status_code == 200
assert [hit["memory_id"] for hit in denied_expansion.json()["hits"]] == [ids[1]]
internal = await client.post(
"/api/v1/recall", json={"query": "Project meeting", "scope": "*"}
)
assert {hit["memory_id"] for hit in internal.json()["hits"]} == set(ids)
assert normal.status_code == 200
semantic.assert_awaited()
asyncio.run(scenario())
@@ -16,7 +16,6 @@ from fastapi import FastAPI
from starlette.requests import Request
from one_memory.api import _require_external_recall_scope
from one_memory.workspace_resolution import resolve_workspace_scopes
class _Access:
@@ -61,73 +60,3 @@ def test_workspace_lookup_for_an_external_recall_runs_off_the_loop(monkeypatch)
)
asyncio.run(scenario())
def test_the_ask_path_also_reads_the_workspace_lookup_off_the_loop(monkeypatch) -> None:
"""Same read, other door: /ask reaches it through MemoryAnswerer.retrieve.
That file is not ours to edit, so the read is done in `_request_answerer`
— where both ask paths meet — and the answerer is handed a callable that
only resolves against the list already fetched. Two things have to hold:
the read is off the loop, and the scopes are exactly what the inline
version would have returned. The second is the one that matters: this
callable decides which memories a host may be answered from.
"""
known = [_Known("/repo", "p")]
seen: list[int] = []
class Store:
def known_workspaces(self, *, user_id: str) -> list[Any]:
seen.append(threading.get_ident())
return known
class Access:
scopes = frozenset({"all-apps"})
def require(self, tool: str, scopes: frozenset[str]) -> None:
return None
import one_memory.api as api
monkeypatch.setattr(api, "external_access", lambda headers: Access())
monkeypatch.setattr(api, "_evidence_store", lambda app: Store())
answerer = _BareAnswerer()
answerer.resolve_workspace = lambda path: ("replaced-by-the-fix",)
monkeypatch.setattr(api, "_memory_answerer", lambda app: answerer)
async def scenario() -> None:
loop_thread = threading.get_ident()
request = Request(
{
"type": "http",
"method": "POST",
"path": "/api/v1/memories/ask",
"headers": [],
"query_string": b"",
}
)
resolved = await api._request_answerer(FastAPI(), request, workspace_path="/repo")
assert seen, "the workspace lookup was never called"
assert seen[0] != loop_thread, (
"known_workspaces ran on the event loop thread; it must run in a worker"
)
# The boundary must be unchanged, not merely non-empty.
assert resolved.resolve_workspace("/repo") == resolve_workspace_scopes("/repo", known)
asyncio.run(scenario())
class _Known:
def __init__(self, workspace: str, project: str) -> None:
self.source_id = "cursor"
self.workspace = workspace
self.project = project
class _BareAnswerer:
"""Just enough of MemoryAnswerer for the per-request copy to work."""
filter_memories = staticmethod(tuple)
resolve_workspace = None
@@ -4,6 +4,8 @@ import asyncio
import hashlib
import io
import json
import sqlite3
from contextlib import closing
from datetime import UTC, datetime, timedelta
from pathlib import Path
@@ -15,6 +17,7 @@ from one_memory import cli
from one_memory.api import create_app
from one_memory.config import Settings
from one_memory.database_access import close_main_databases, main_database
from one_memory.hy_sdk_migration import repair_sdk_metadata_from_bundle
from one_memory.hy_sdk_types import HySdkRawPage
from one_memory.imports.storage import EvidenceStore
from one_memory.memory_server.contracts import BatchIdentity, WriteRequest
@@ -67,6 +70,260 @@ def _start(store, migration_id="m1"):
)
def test_post_cutover_metadata_repair_restores_source_and_original_times(store):
data_dir = store.database_path.parent
migration_id = "sdk-authority-v2-source-repair"
with main_database(store.database_path).write() as database, database:
database.execute(
"UPDATE memory_authority_state SET authority='sdk',migration_id=? WHERE singleton=1",
(migration_id,),
)
legacy = (
data_dir
/ "backups"
/ f"{migration_id}-pre-cutover.sdk-bundle"
/ "management"
/ "one-memory.sqlite3"
)
legacy.parent.mkdir(parents=True)
with sqlite3.connect(legacy) as database:
database.executescript(
"""
CREATE TABLE memory_task_outputs (body TEXT);
CREATE TABLE governed_memories (
memory_id TEXT, created_at TEXT, updated_at TEXT
);
CREATE TABLE evidence (
evidence_id TEXT, source_id TEXT, created_at TEXT, imported_at TEXT
);
"""
)
database.execute(
"INSERT INTO governed_memories VALUES (?,?,?)",
("legacy-1", "2026-10-18T00:00:00+00:00", "2026-10-18T01:00:00+00:00"),
)
database.execute(
"INSERT INTO evidence VALUES (?,?,?,?)",
("e1", "workbuddy", "2254-01-01T00:00:00+00:00", "2026-10-17T00:00:00+00:00"),
)
database.execute(
"INSERT INTO memory_task_outputs VALUES (?)",
(
json.dumps(
{
"memory_id": "legacy-1",
"hy_memory_id": "sdk-1",
"support_refs": [{"evidence_id": "e1"}],
}
),
),
)
import chromadb
from chromadb.config import Settings as ChromaSettings
vector = data_dir / "hy-memory" / "governed-v2" / "vector"
collection = chromadb.PersistentClient(
path=str(vector),
settings=ChromaSettings(anonymized_telemetry=False),
).get_or_create_collection("memory")
collection.add(
ids=["sdk-1"],
documents=["remember"],
embeddings=[[0.1, 0.2]],
metadatas=[
{
"source": "default",
"gmt_created": 1,
"memory_at": 8_962_051_200,
}
],
)
coding = data_dir / "hy-memory" / "governed-v2" / "coding_cv.sqlite3"
with sqlite3.connect(coding) as database:
database.execute(
"CREATE TABLE coding_memory_meta ("
"experience_id TEXT,user_id TEXT,agent_id TEXT,solution TEXT,"
"source TEXT,created_at TEXT,updated_at TEXT)"
)
database.execute(
"CREATE TABLE working_fact ("
"fact_id TEXT,user_id TEXT,agent_id TEXT,statement TEXT,"
"source TEXT,created_at TEXT)"
)
database.execute(
"INSERT INTO coding_memory_meta VALUES "
"('sdk-1','local','scope','remember','default','now','now')"
)
result = repair_sdk_metadata_from_bundle(
data_dir=data_dir,
database_path=store.database_path,
now=datetime(2026, 10, 25, tzinfo=UTC),
)
metadata = collection.get(ids=["sdk-1"], include=["metadatas"])["metadatas"][0]
assert result.vector_rows == 1
assert result.coding_rows == 1
assert metadata["source"] == "workbuddy"
assert datetime.fromtimestamp(metadata["memory_at"], tz=UTC) == datetime(
2026, 10, 17, tzinfo=UTC
)
assert datetime.fromtimestamp(metadata["gmt_created"], tz=UTC) == datetime(
2026, 10, 18, tzinfo=UTC
)
with sqlite3.connect(coding) as database:
source, created_at = database.execute(
"SELECT source,created_at FROM coding_memory_meta WHERE experience_id='sdk-1'"
).fetchone()
assert source == "workbuddy"
assert created_at.startswith("2026-10-18T00:00:00")
repeated = repair_sdk_metadata_from_bundle(
data_dir=data_dir,
database_path=store.database_path,
now=datetime(2026, 10, 25, tzinfo=UTC),
)
assert repeated == result
def test_metadata_repair_reaches_memories_the_cutover_backfilled_under_legacy_ids(store):
"""The backfill wrote governed memories with no source; their evidence links them."""
data_dir = store.database_path.parent
migration_id = "sdk-authority-v2-backfill"
with main_database(store.database_path).write() as database, database:
database.execute(
"UPDATE memory_authority_state SET authority='sdk',migration_id=? WHERE singleton=1",
(migration_id,),
)
# A v1 receipt from the earlier, narrower repair must not block this one.
(data_dir / f".sdk-metadata-repair-v1-{migration_id}.json").write_text("{}\n")
# The later bundle was taken after legacy tables were retired; the earlier
# cutover's bundle still carries them.
for bundle, populated in (("sdk-authority-v1-old", True), (migration_id, False)):
legacy = data_dir / "backups" / f"{bundle}-pre-cutover.sdk-bundle" / "management"
legacy.mkdir(parents=True)
with closing(sqlite3.connect(legacy / "one-memory.sqlite3")) as database, database:
database.executescript(
"""
CREATE TABLE evidence (
evidence_id TEXT, source_id TEXT, created_at TEXT, imported_at TEXT
);
"""
)
if not populated:
continue
database.executescript(
"""
CREATE TABLE governed_memories (
memory_id TEXT, created_at TEXT, updated_at TEXT
);
CREATE TABLE governed_memory_evidence (
memory_id TEXT, evidence_id TEXT, active INTEGER
);
"""
)
database.execute(
"INSERT INTO governed_memories VALUES ('legacy-7','2026-09-02T00:00:00+00:00',NULL)"
)
database.execute("INSERT INTO governed_memory_evidence VALUES ('legacy-7','e7',1)")
database.execute(
"INSERT INTO evidence VALUES ('e7','workbuddy','2026-09-01T00:00:00+00:00',NULL)"
)
import chromadb
from chromadb.config import Settings as ChromaSettings
vector = data_dir / "hy-memory" / "governed-v2" / "vector"
client = chromadb.PersistentClient(
path=str(vector), settings=ChromaSettings(anonymized_telemetry=False)
)
collection = client.get_or_create_collection("memory")
imported = int(datetime(2026, 10, 20, tzinfo=UTC).timestamp())
collection.add(
ids=["legacy-7", "unlinked"],
documents=["backfilled", "hash-dated"],
embeddings=[[0.1, 0.2], [0.2, 0.1]],
metadatas=[
{"source": "default", "gmt_created": imported, "memory_at": imported},
{"source": "cursor", "gmt_created": imported, "memory_at": 8_962_051_200},
],
)
client.close()
result = repair_sdk_metadata_from_bundle(
data_dir=data_dir,
database_path=store.database_path,
now=datetime(2026, 10, 25, tzinfo=UTC),
)
client = chromadb.PersistentClient(
path=str(vector), settings=ChromaSettings(anonymized_telemetry=False)
)
rows = client.get_collection("memory").get(ids=["legacy-7", "unlinked"], include=["metadatas"])
client.close()
metadata = dict(zip(rows["ids"], rows["metadatas"], strict=True))
assert result.vector_rows == 2
assert metadata["legacy-7"]["source"] == "workbuddy"
assert metadata["legacy-7"]["memory_at"] == int(datetime(2026, 9, 1, tzinfo=UTC).timestamp())
assert metadata["legacy-7"]["gmt_created"] == int(datetime(2026, 9, 2, tzinfo=UTC).timestamp())
assert metadata["unlinked"]["source"] == "cursor"
assert metadata["unlinked"]["memory_at"] == imported
def test_metadata_repair_keeps_source_unknown_when_outputs_disagree(tmp_path):
"""Two legacy memories folded into one SDK memory pool their evidence."""
from one_memory.hy_sdk_migration import _legacy_metadata
legacy = tmp_path / "one-memory.sqlite3"
with closing(sqlite3.connect(legacy)) as database, database:
database.executescript(
"""
CREATE TABLE memory_task_outputs (body TEXT);
CREATE TABLE governed_memories (
memory_id TEXT, created_at TEXT, updated_at TEXT
);
CREATE TABLE evidence (
evidence_id TEXT, source_id TEXT, created_at TEXT, imported_at TEXT
);
"""
)
database.executemany(
"INSERT INTO governed_memories VALUES (?,?,?)",
[
("legacy-1", "2026-10-18T00:00:00+00:00", None),
("legacy-2", "2026-10-12T00:00:00+00:00", None),
],
)
database.executemany(
"INSERT INTO evidence VALUES (?,?,?,NULL)",
[
("e1", "workbuddy", "2026-10-17T00:00:00+00:00"),
("e2", "cursor", "2026-10-11T00:00:00+00:00"),
],
)
database.executemany(
"INSERT INTO memory_task_outputs VALUES (?)",
[
(
json.dumps(
{
"memory_id": legacy_id,
"hy_memory_id": "sdk-1",
"support_refs": [{"evidence_id": evidence_id}],
}
),
)
for legacy_id, evidence_id in (("legacy-1", "e1"), ("legacy-2", "e2"))
],
)
repaired = _legacy_metadata(legacy, now=datetime(2026, 10, 25, tzinfo=UTC))["sdk-1"]
assert repaired.source is None
assert repaired.memory_at == int(datetime(2026, 10, 17, tzinfo=UTC).timestamp())
assert repaired.created_at == int(datetime(2026, 10, 12, tzinfo=UTC).timestamp())
def _insert_evidence(store, evidence_id, *, conversation="session-1", role="user", seq=1):
verbatim = f"text of {evidence_id}"
with main_database(store.database_path).write() as database, database:
@@ -813,18 +1070,14 @@ def _app_settings(tmp_path) -> Settings:
async def _authority(tmp_path) -> tuple[dict[str, object], str]:
"""What a starting Core reads as authority, and the gate it then holds."""
app = create_app(_app_settings(tmp_path), secret_store=InMemorySecretStore())
async with (
app.router.lifespan_context(app),
AsyncClient(
transport=ASGITransport(app=app),
base_url="http://test",
headers={"Authorization": "Bearer authority-test"},
) as client,
):
response = await client.get("/api/v2/memory-authority")
assert response.status_code == 200
return response.json(), app.state.memory_authority.value
async with app.router.lifespan_context(app):
authority, migration_id = app.state.sdk_data_plane_migrations.authority_state()
return (
{"authority": authority.value, "migration_id": migration_id},
app.state.memory_authority.value,
)
@pytest.mark.integration
@@ -887,10 +1140,24 @@ def test_the_sdk_memory_map_draws_sdk_memories_without_legacy_tables(tmp_path) -
"source": "cursor",
}
for memory_id in ("a", "b")
] + [
{
"memory_id": memory_id,
"content": f"backfilled {memory_id}",
"layer": "l2_fact",
"status": "active",
"user_id": "local",
"agent_id": "one-memory",
"session_id": "default_session",
"source": "default",
}
for memory_id in ("c", "d")
]
listed = []
class Client:
def list_memories(self, **_kwargs):
def list_memories(self, **kwargs):
listed.append(kwargs)
return {"vdb": {"memories": rows, "total": 7}}
class Handle:
@@ -925,20 +1192,32 @@ def test_the_sdk_memory_map_draws_sdk_memories_without_legacy_tables(tmp_path) -
)
app.state.sdk_memory_runtime.graph_snapshot = graph_snapshot
assert (await client.get("/api/v1/memories/graph")).status_code == 409
response = await client.get("/api/v2/memories/graph")
response = await client.get(
"/api/v2/memories/graph",
params={"source": "cursor", "offset": 100, "limit": 100},
)
app.state.sdk_client_handle = original
assert response.status_code == 200
return response.json()
graph = asyncio.run(scenario())
assert (graph["total_memories"], graph["projected_memories"]) == (7, 2)
assert listed[0]["include_sources"] == ["cursor"]
assert (listed[0]["offset"], listed[0]["limit"]) == (100, 100)
assert (graph["total_memories"], graph["projected_memories"]) == (7, 4)
assert {node["node_id"] for node in graph["nodes"]} == {
"memory:a",
"memory:b",
"memory:c",
"memory:d",
"source:cursor",
"hy:t1",
}
# "default_session" is the SDK's filler, not a conversation.
assert [
(edge["source_node_id"], edge["target_node_id"])
for edge in graph["edges"]
if edge["relation"] == "same_conversation"
] == [("memory:a", "memory:b")]
assert {edge["relation"] for edge in graph["edges"]} == {
"evidence_source",
"same_conversation",
@@ -0,0 +1,361 @@
"""`sdk-migration.log` explains an unfinished upgrade, and a legacy home is never served."""
import asyncio
import json
import os
import sqlite3
from io import StringIO
from pathlib import Path
import pytest
from pydantic import SecretStr
from one_memory import cli
from one_memory.api import SdkMigrationRequired, create_app
from one_memory.config import Settings
from one_memory.imports.storage import EvidenceStore
from one_memory.runtime_client import RuntimeStatus
from one_memory.sdk_data_plane_migration import DataPlaneMigrationError
from one_memory.sdk_migration_log import describe_error, log_path, write_migration_log
from one_memory.secrets import InMemorySecretStore
from one_memory.target_schema import bootstrap_fresh_management_schema
SECRET = "我的私人记忆 password=hunter2"
MIGRATE = ["sdk-migrate", "--runtime-build", "22", "--yes", "--json"]
def lines(data_dir: Path) -> list[dict[str, object]]:
return [json.loads(line) for line in log_path(data_dir).read_text().splitlines()]
@pytest.fixture
def home(tmp_path, monkeypatch) -> Path:
data_dir = tmp_path / "data"
data_dir.mkdir()
monkeypatch.setenv("ONE_MEMORY_DATA_DIR", str(data_dir))
monkeypatch.setenv("ONE_MEMORY_MIGRATION_CALLER", "desktop")
monkeypatch.setenv("ONE_MEMORY_MIGRATION_RUNTIME_BUILD", "22")
monkeypatch.setattr(
cli,
"inspect_runtime",
lambda _data_dir: RuntimeStatus(state="stopped", code="runtime_not_running"),
)
return data_dir
def legacy_home(data_dir: Path) -> Path:
database = data_dir / "one-memory.sqlite3"
EvidenceStore(database)
return database
def test_a_line_never_carries_free_text_or_an_exception_message(tmp_path):
try:
try:
raise ValueError(SECRET)
except ValueError as cause:
raise DataPlaneMigrationError("sdk_data_plane_write_job_input_missing") from cause
except DataPlaneMigrationError as error:
write_migration_log(
tmp_path,
writer="core-sdk-migrate",
event="migrate.failed",
outcome="failed",
code="sdk_migration_preflight_failed",
error=error,
details={"note": SECRET, "count": 3, "nested": {"text": SECRET}},
)
raw = log_path(tmp_path).read_text()
assert "hunter2" not in raw and "私人" not in raw
[line] = lines(tmp_path)
assert line["details"] == {"note": "<redacted>", "count": 3, "nested": {"text": "<redacted>"}}
chain = line["error"]["chain"]
assert [item["type"] for item in chain] == ["DataPlaneMigrationError", "ValueError"]
assert chain[0]["code"] == "sdk_data_plane_write_job_input_missing"
assert "code" not in chain[1]
assert any(
frame.endswith(":test_a_line_never_carries_free_text_or_an_exception_message")
for frame in chain[0]["frames"]
)
def test_logging_never_raises(tmp_path):
blocker = tmp_path / "logs"
blocker.write_text("a file where the directory should be")
write_migration_log(tmp_path, writer="core-startup", event="x", outcome="succeeded")
def test_a_failed_migration_records_the_phase_and_the_real_cause(home, monkeypatch):
legacy_home(home)
def run_startup_migration(**options):
options["progress"]("bundle")
options["progress"]("prepare")
raise RuntimeError(SECRET)
import one_memory.sdk_migration_runner as runner
monkeypatch.setattr(runner, "run_startup_migration", run_startup_migration)
out = StringIO()
assert cli.run(MIGRATE, output=out) == 1
assert json.loads(out.getvalue().splitlines()[-1])["code"] == "sdk_migration_preflight_failed"
written = lines(home)
assert "hunter2" not in log_path(home).read_text()
assert [line["event"] for line in written] == [
"migrate.invoked",
"phase.finished",
"phase.started",
"phase.finished",
"phase.started",
"migrate.failed",
]
assert all(line["writer"] == "core-sdk-migrate" for line in written)
assert all(line["caller"] == "desktop" and line["runtime_build"] == "22" for line in written)
assert written[0]["details"]["authority"] == "legacy"
assert written[1]["details"]["phase"] == "preflight"
failed = written[-1]
assert failed["code"] == "sdk_migration_preflight_failed"
# The generic code says nothing; the phase and the exception chain do.
assert failed["details"]["phase"] == "prepare"
assert failed["details"]["code_is_fallback"] is True
assert failed["error"]["chain"][0]["type"] == "RuntimeError"
assert any("run_startup_migration" in frame for frame in failed["error"]["chain"][0]["frames"])
def test_a_named_failure_keeps_its_code(home, monkeypatch):
legacy_home(home)
def run_startup_migration(**_options):
raise DataPlaneMigrationError("sdk_migration_model_profile_required")
import one_memory.sdk_migration_runner as runner
monkeypatch.setattr(runner, "run_startup_migration", run_startup_migration)
assert cli.run(MIGRATE, output=StringIO()) == 1
failed = lines(home)[-1]
assert failed["code"] == "sdk_migration_model_profile_required"
assert failed["details"]["phase"] == "preflight"
assert failed["details"]["code_is_fallback"] is False
def test_a_finished_migration_records_the_result(home, monkeypatch):
legacy_home(home)
from one_memory.sdk_migration_runner import StartupMigrationResult
def run_startup_migration(**options):
options["progress"]("cutover")
return StartupMigrationResult(state="cutover", migration_id="sdk-authority-v2-abc")
import one_memory.sdk_migration_runner as runner
monkeypatch.setattr(runner, "run_startup_migration", run_startup_migration)
assert cli.run(MIGRATE, output=StringIO()) == 0
finished = lines(home)[-1]
assert finished["event"] == "migrate.finished"
assert finished["migration_id"] == "sdk-authority-v2-abc"
assert finished["details"]["state"] == "cutover"
assert finished["details"]["last_phase"] == "cutover"
def test_a_refused_migration_is_logged(home, monkeypatch):
legacy_home(home)
monkeypatch.setattr(
cli,
"inspect_runtime",
lambda _data_dir: RuntimeStatus(state="running", code="runtime_running"),
)
assert cli.run(MIGRATE, output=StringIO()) == 4
assert lines(home)[-1]["code"] == "sdk_migration_runtime_running"
def test_the_status_probe_is_logged(home):
legacy_home(home)
assert cli.run(["sdk-migration-status", "--json"], output=StringIO()) == 0
probed = lines(home)[-1]
assert probed["writer"] == "core-sdk-migration-status"
assert probed["details"] == {"authority": "legacy", "state": "required"}
def test_serving_core_exits_on_an_unmigrated_home(home):
legacy_home(home)
with pytest.raises(SystemExit) as exit_:
cli._refuse_unmigrated_home(home)
assert exit_.value.code == cli.SDK_MIGRATION_REQUIRED_EXIT == 78
refused = lines(home)[-1]
assert refused["writer"] == "core-startup"
assert refused["event"] == "core.refused_legacy"
assert refused["code"] == "sdk_migration_required"
assert refused["details"]["authority"] == "legacy"
def test_serving_core_starts_on_a_new_or_migrated_home(home):
cli._refuse_unmigrated_home(home)
bootstrap_fresh_management_schema(home / "one-memory.sqlite3")
cli._refuse_unmigrated_home(home)
assert not log_path(home).exists()
def _settings(data_dir: Path) -> Settings:
return Settings(
data_dir=data_dir,
source_home=data_dir.parent,
api_token=SecretStr("migration-log-test"),
imports_enabled=False,
automatic_memory_processing=False,
live_sync_enabled=False,
)
def test_a_serving_app_refuses_a_legacy_home_at_startup(tmp_path):
data_dir = tmp_path / "data"
data_dir.mkdir()
legacy_home(data_dir)
app = create_app(_settings(data_dir), manage_runtime=True, secret_store=InMemorySecretStore())
async def start() -> None:
async with app.router.lifespan_context(app):
pass
with pytest.raises(SdkMigrationRequired):
asyncio.run(start())
refused = [line for line in lines(data_dir) if line["event"] == "core.refused_legacy"]
assert refused and refused[-1]["details"]["serving"] is True
@pytest.mark.fresh_sdk
def test_startup_records_the_authority_it_read(tmp_path):
data_dir = tmp_path / "data"
app = create_app(_settings(data_dir), secret_store=InMemorySecretStore())
async def start() -> None:
async with app.router.lifespan_context(app):
pass
asyncio.run(start())
read = [line for line in lines(data_dir) if line["event"] == "core.authority_read"]
assert read[-1]["details"]["authority"] == "sdk"
assert read[-1]["writer"] == "core-startup"
def test_describe_error_keeps_only_the_file_name_of_a_path(tmp_path):
error = OSError(28, SECRET, str(tmp_path / "private-name" / "one-memory.sqlite3"))
described = describe_error(error)
[item] = described["chain"]
assert item["errno"] == 28
assert item["file"] == "one-memory.sqlite3"
assert "private-name" not in json.dumps(described)
def test_the_log_is_private(tmp_path):
write_migration_log(tmp_path, writer="core-startup", event="x", outcome="succeeded")
mode = os.stat(log_path(tmp_path)).st_mode & 0o777
assert mode == 0o600
assert sqlite3 # the module under test reads migration rows with sqlite3
def _migrated_schema_home(data_dir: Path) -> Path:
from one_memory.database_access import close_main_databases
from one_memory.memory_server.migration_runner import SqliteMigrationStore, migrate
from one_memory.memory_server.migrations import MIGRATIONS
database = data_dir / "one-memory.sqlite3"
asyncio.run(migrate(SqliteMigrationStore(database), MIGRATIONS))
close_main_databases()
return database
def test_the_runner_names_the_step_and_sub_step_that_failed(tmp_path, monkeypatch):
"""`sdk_migration_preflight_failed` alone said nothing; the step lines do."""
import one_memory.sdk_migration_runner as runner
_migrated_schema_home(tmp_path)
def broken_queue(_database_path):
raise ValueError(SECRET)
monkeypatch.setattr(runner, "prepare_legacy_write_queue_for_cutover", broken_queue)
with pytest.raises(ValueError):
runner.run_startup_migration(
data_dir=tmp_path,
runtime_build="22",
sdk_version="2.0.5",
namespace="governed-v2",
)
assert cli.migration_failure_code(ValueError(SECRET)) == "sdk_migration_preflight_failed"
raw = log_path(tmp_path).read_text()
assert "hunter2" not in raw and "私人" not in raw
written = lines(tmp_path)
steps = [
(line["event"], line["details"]["step"])
for line in written
if "step" in line.get("details", {})
]
assert steps[:6] == [
("step.finished", "preflight.authority_read"),
("step.finished", "preflight.profile_read"),
("step.finished", "preflight.resumable_identity"),
("step.finished", "preflight.legacy_ids_scan"),
("step.finished", "preflight.model_profile_check"),
("step.finished", "bundle.ensure"),
]
assert steps[-1] == ("step.failed", "prepare.legacy_work")
failed = written[-1]
assert failed["migration_id"].startswith("sdk-authority-v2-")
assert failed["details"]["sub_step"] == "write_queue"
assert failed["error"]["chain"][0]["type"] == "ValueError"
assert any("broken_queue" in frame for frame in failed["error"]["chain"][0]["frames"])
# The ledger transition before it is there too, with the phase it left.
started = next(line for line in written if line["event"] == "ledger.started")
assert started["details"] == {"created": True, "state": "running", "phase": "preflight"}
def test_a_coded_failure_with_a_row_id_keeps_both_name_and_id(tmp_path):
error = DataPlaneMigrationError("sdk_data_plane_write_job_input_missing:job-0193")
[item] = describe_error(error)["chain"]
assert item["code"] == "sdk_data_plane_write_job_input_missing"
assert item["subject"] == "job-0193"
# The dialog gets the specific name instead of the generic fallback.
assert cli.migration_failure_code(error) == "sdk_data_plane_write_job_input_missing"
def test_a_second_migrator_waits_for_the_first(home, monkeypatch):
"""The Runtime service and Desktop can both migrate at login; never at once."""
import threading
import one_memory.sdk_migration_runner as runner
from one_memory.sdk_migration_runner import StartupMigrationResult
legacy_home(home)
monkeypatch.setattr(
runner,
"run_startup_migration",
lambda **_options: StartupMigrationResult(state="not_required"),
)
held = threading.Event()
release = threading.Event()
def first() -> None:
with cli._migration_lock(home):
held.set()
release.wait(5)
holder = threading.Thread(target=first)
holder.start()
held.wait(5)
results: list[int] = []
second = threading.Thread(target=lambda: results.append(cli.run(MIGRATE, output=StringIO())))
second.start()
for _ in range(100):
if log_path(home).exists() and any(
line["event"] == "migrate.waiting_for_lock" for line in lines(home)
):
break
threading.Event().wait(0.02)
assert results == []
release.set()
holder.join(5)
second.join(5)
assert results == [0]
events = [line["event"] for line in lines(home)]
assert events.index("migrate.waiting_for_lock") < events.index("migrate.lock_acquired")
@@ -1,166 +0,0 @@
"""A session inbox is not a shared mailbox and a pull is not adoption."""
import sqlite3
from contextlib import closing
from uuid import uuid4
import pytest
from one_memory.imports.models import EvidenceInput, EvidenceOrigin, OriginMetadata, WrittenBy
from one_memory.imports.storage import EvidenceStore
from one_memory.session_inbox import CursorRejected, InboxPosition, SessionInbox
from one_memory.session_registry import ForgedSession, SessionRegistry
def _evidence(message_id: str, verbatim: str) -> EvidenceInput:
return EvidenceInput(
source_id="workbuddy",
conversation_id="session-a",
message_id=message_id,
role="user",
written_by=WrittenBy.PERSON,
verbatim=verbatim,
origin_meta={EvidenceOrigin.FILE: OriginMetadata(raw_meta={"root": "proj"})},
)
def _world(tmp_path):
path = tmp_path / "one-memory.sqlite3"
store = EvidenceStore(path)
registry = SessionRegistry(path)
registry.initialize()
inbox = SessionInbox(path)
inbox.initialize()
installation = str(uuid4())
shared = dict(
installation_id=installation,
host_id="workbuddy",
runtime_instance="runtime-1",
workspace="/work/same-project",
)
first = registry.open(**shared, native_session_id="session-a")
second = registry.open(**shared, native_session_id="session-b")
inbox.subscribe(first.session_key, ("workbuddy",), permission_epoch=1)
inbox.subscribe(second.session_key, ("workbuddy",), permission_epoch=1)
with closing(sqlite3.connect(path)) as db:
db.row_factory = sqlite3.Row
db.execute("BEGIN IMMEDIATE")
store.put(_evidence("m1", "SECRET_DECISION"), connection=db)
store.put(_evidence("m2", "SECOND_UPDATE"), connection=db)
db.commit()
return inbox, first, second
def test_another_session_cannot_use_this_cursor(tmp_path) -> None:
inbox, first, second = _world(tmp_path)
page = inbox.pull(first.session_key, cursor=None, limit=10)
with pytest.raises(CursorRejected):
inbox.pull(second.session_key, cursor=page.cursor, limit=10)
def test_forged_session_cannot_pull(tmp_path) -> None:
inbox, _first, _second = _world(tmp_path)
with pytest.raises(ForgedSession):
inbox.pull("0" * 64, cursor=None, limit=10)
def test_a_full_page_does_not_skip_the_rest(tmp_path) -> None:
inbox, first, _second = _world(tmp_path)
page = inbox.pull(first.session_key, cursor=None, limit=1)
assert len(page.events) == 1
rest = inbox.pull(first.session_key, cursor=page.cursor, limit=10)
assert len(rest.events) == 1
assert {item.evidence_id for item in (*page.events, *rest.events)} == {
page.events[0].evidence_id,
rest.events[0].evidence_id,
}
def test_a_lost_signal_still_leaves_the_updates_pullable(tmp_path) -> None:
inbox, first, _second = _world(tmp_path)
signal = inbox.signal(first.session_key)
dumped = repr(signal)
assert "SECRET_DECISION" not in dumped
assert "SECOND_UPDATE" not in dumped
page = inbox.pull(first.session_key, cursor=None, limit=10)
assert len(page.events) == 2
def test_an_event_arriving_during_snapshot_is_not_lost(tmp_path) -> None:
path = tmp_path / "one-memory.sqlite3"
inbox, first, _second = _world(tmp_path)
store = EvidenceStore(path)
token = inbox.begin_snapshot(first.session_key)
with closing(sqlite3.connect(path)) as db:
db.row_factory = sqlite3.Row
db.execute("BEGIN IMMEDIATE")
arrived = store.put(_evidence("m3", "DURING_SNAPSHOT"), connection=db)
db.commit()
closed = inbox.finish_snapshot(first.session_key, token)
rest = inbox.pull(first.session_key, cursor=closed.cursor, limit=10)
found = {event.evidence_id for event in (*closed.events, *rest.events)}
assert arrived.record.evidence_id in found
def test_revoking_a_source_hides_a_queued_update(tmp_path) -> None:
inbox, first, _second = _world(tmp_path)
inbox.subscribe(first.session_key, (), permission_epoch=2)
page = inbox.pull(first.session_key, cursor=None, limit=10)
assert page.events == ()
def test_a_position_whose_target_is_no_longer_granted_does_not_confirm(tmp_path) -> None:
inbox, first, _second = _world(tmp_path)
peeked = inbox.peek(first.session_key, 8)
assert peeked.events
inbox.subscribe(first.session_key, (), permission_epoch=2)
with pytest.raises(CursorRejected):
inbox.advance_to(first.session_key, peeked.position)
def test_a_position_that_does_not_receipt_the_whole_range_does_not_confirm(tmp_path) -> None:
inbox, first, _second = _world(tmp_path)
peeked = inbox.peek(first.session_key, 8)
assert len(peeked.events) == 2
partial = InboxPosition(
peeked.position.event_id,
peeked.position.permission_epoch,
frozenset({peeked.events[-1].event_id}),
peeked.position.receipt_id,
)
with pytest.raises(CursorRejected):
inbox.advance_to(first.session_key, partial)
def test_a_cursor_whose_event_is_gone_resyncs_instead_of_an_empty_page(tmp_path) -> None:
inbox, first, _second = _world(tmp_path)
page = inbox.pull(first.session_key, cursor=None, limit=10)
assert page.state == "delivered"
assert len(page.events) == 2
with closing(sqlite3.connect(tmp_path / "one-memory.sqlite3")) as db, db:
db.execute("DELETE FROM evidence WHERE evidence_id = ?", (page.events[1].evidence_id,))
again = inbox.pull(first.session_key, cursor=page.cursor, limit=10)
assert again.state == "resync-required"
assert again.adoption == "unknown"
assert [event.event_id for event in again.events] == [page.events[0].event_id]
quiet = inbox.pull(first.session_key, cursor=again.cursor, limit=10)
assert quiet.state == "delivered"
assert quiet.events == ()
def test_nothing_new_after_a_live_cursor_is_not_a_resync(tmp_path) -> None:
inbox, first, _second = _world(tmp_path)
page = inbox.pull(first.session_key, cursor=None, limit=10)
again = inbox.pull(first.session_key, cursor=page.cursor, limit=10)
assert again.state == "delivered"
assert again.events == ()
def test_pull_and_ack_are_not_adoption(tmp_path) -> None:
inbox, first, _second = _world(tmp_path)
page = inbox.pull(first.session_key, cursor=None, limit=10)
assert page.adoption == "unknown"
assert page.state == "delivered"
acked = inbox.ack(first.session_key, page.batch_id)
assert acked.state == "acknowledged"
assert acked.adoption == "unknown"
@@ -1,66 +0,0 @@
"""A session file on disk is not proof that the session is online."""
from datetime import UTC, datetime, timedelta
from uuid import uuid4
from one_memory.session_lease import SessionLease
from one_memory.session_registry import SessionRegistry
def _session(tmp_path):
path = tmp_path / "one-memory.sqlite3"
path.write_bytes(b"")
registry = SessionRegistry(path)
registry.initialize()
lease = SessionLease(path)
lease.initialize()
opened = registry.open(
installation_id=str(uuid4()),
host_id="workbuddy",
native_session_id="session-a",
runtime_instance="runtime-1",
workspace="/work/same-project",
)
return lease, opened.session_key
def test_a_present_file_does_not_keep_an_expired_session_online(tmp_path) -> None:
lease, session_key = _session(tmp_path)
marker = tmp_path / "session.jsonl"
marker.write_text("still on disk")
started = datetime(2026, 9, 22, 12, 0, tzinfo=UTC)
lease.beat(session_key, pid=42, generation=1, now=started, ttl_seconds=10)
assert lease.online(session_key, now=started, generation=1, source_file=marker)
assert (
lease.online(
session_key,
now=started + timedelta(seconds=11),
generation=1,
source_file=marker,
)
is False
)
def test_a_reused_pid_does_not_revive_the_previous_generation(tmp_path) -> None:
lease, session_key = _session(tmp_path)
started = datetime(2026, 9, 22, 12, 0, tzinfo=UTC)
lease.beat(session_key, pid=42, generation=1, now=started, ttl_seconds=60)
lease.beat(session_key, pid=42, generation=2, now=started, ttl_seconds=60)
assert lease.online(session_key, now=started, generation=1) is False
assert lease.online(session_key, now=started, generation=2) is True
def test_a_file_without_a_lease_is_offline(tmp_path) -> None:
lease, session_key = _session(tmp_path)
marker = tmp_path / "session.jsonl"
marker.write_text("still on disk")
assert (
lease.online(
session_key,
now=datetime(2026, 9, 22, 12, 0, tzinfo=UTC),
generation=1,
source_file=marker,
)
is False
)
@@ -1,60 +0,0 @@
"""Trusted session identity is not a workspace path."""
import sqlite3
from contextlib import closing
from uuid import uuid4
import pytest
from one_memory.session_registry import (
AmbiguousWorkspace,
ForgedSession,
SessionRegistry,
)
def database(tmp_path):
path = tmp_path / "one-memory.sqlite3"
with closing(sqlite3.connect(path)):
pass
registry = SessionRegistry(path)
registry.initialize()
return registry
def test_same_directory_does_not_collapse_two_main_sessions(tmp_path) -> None:
registry = database(tmp_path)
installation = str(uuid4())
shared = dict(
installation_id=installation,
host_id="workbuddy",
runtime_instance="runtime-1",
workspace="/work/same-project",
)
first = registry.open(**shared, native_session_id="session-a")
second = registry.open(**shared, native_session_id="session-b")
assert first.session_key != second.session_key
with pytest.raises(AmbiguousWorkspace):
registry.sole_session("/work/same-project")
def test_forged_session_key_is_rejected(tmp_path) -> None:
registry = database(tmp_path)
with pytest.raises(ForgedSession):
registry.require("0" * 64)
def test_resume_keeps_the_same_trusted_session(tmp_path) -> None:
registry = database(tmp_path)
shared = dict(
installation_id=str(uuid4()),
host_id="workbuddy",
native_session_id="session-a",
runtime_instance="runtime-1",
workspace="/work/same-project",
)
first = registry.open(**shared)
again = registry.open(**shared)
assert again.session_key == first.session_key
assert len(registry.in_workspace("/work/same-project")) == 1
@@ -96,7 +96,7 @@ def test_receipt_counts_saved_records_without_inventing_dates_or_other_sources(t
assert excerpt["truncated"] and len(excerpt["excerpt"]) == 600
assert excerpt["created_at"] is None
assert "<5" in excerpt["excerpt"] and ">5" in excerpt["excerpt"]
assert (await client.get("/api/v1/memories")).json() == []
assert app.state.memory_governance_store.list(user_id="local") == []
assert (await client.get("/api/v1/model-profiles")).json() == []
(await client.post("/api/v1/sources/claude-code/revoke")).raise_for_status()
denied = await client.get(endpoint)
@@ -295,6 +295,8 @@ def test_startup_reconciles_enabled_source_from_config(tmp_path, monkeypatch):
secret_store=InMemorySecretStore(),
)
async with restored.router.lifespan_context(restored):
# The reconcile runs after readiness, in recovery.
await restored.state.recovery_task
history = [
job
for job in restored.state.import_job_store.list_jobs()
@@ -111,6 +111,7 @@ def test_large_import_status_does_not_scan_source_trees(tmp_path, monkeypatch):
app, catalog = build_app(tmp_path, monkeypatch)
seed_layouts(catalog)
async with app.router.lifespan_context(app), client_for(app) as client:
await app.state.recovery_task
await app.state.memory_backfill
await asyncio.to_thread(seed_evidence, app)
probes = []
@@ -217,6 +217,6 @@ def test_one_processing_switch_starts_history_and_future_scans_then_freezes(tmp_
assert app.state.source_grant_store.authorized_root(
user_id="local", source_id="claude-code"
)
assert (await client.get("/api/v1/memory-tasks")).json() == []
assert app.state.memory_task_store.list() == []
asyncio.run(scenario())
@@ -0,0 +1,197 @@
"""v2 is the only memory data plane, and host calls through it are history."""
import asyncio
import json
import pytest
from httpx import ASGITransport, AsyncClient
from pydantic import SecretStr
from one_memory import api
from one_memory.api import create_app
from one_memory.config import Settings
from one_memory.sdk_data_plane_migration import MemoryAuthority
from one_memory.sdk_memory_qa import SdkAskMemory, SdkMemoryAnswer
from one_memory.secrets import InMemorySecretStore
EXTERNAL = {
"x-one-memory-read-context": "external",
"x-one-memory-installation-id": "00000000-0000-0000-0000-000000000001",
"x-one-memory-installation-revision": "1",
"x-one-memory-allowed-scopes": json.dumps(["all-apps", "source:cursor"]),
"x-one-memory-allowed-tools": json.dumps(["om_recall", "om_ask", "om_conversations"]),
"x-one-memory-host-id": "cursor",
}
MEMORY = SdkAskMemory(
memory_id="memory-1",
content="用户要求真实数据。",
layer="l2_fact",
status="active",
source="cursor",
)
def application(tmp_path):
return create_app(
Settings(
data_dir=tmp_path / "data",
source_home=tmp_path,
api_token=SecretStr("v2-test"),
imports_enabled=False,
automatic_memory_processing=False,
live_sync_enabled=False,
call_event_retention_days=30,
call_event_max_events=1000,
),
secret_store=InMemorySecretStore(),
)
def client_for(app, headers=None):
return AsyncClient(
transport=ASGITransport(app=app),
base_url="http://test",
headers={"Authorization": "Bearer v2-test", **(headers or {})},
)
class Answerer:
async def answer(self, question):
return SdkMemoryAnswer(answer="真实数据。", memories=(MEMORY,), mode="extractive")
async def answer_stream(self, question):
yield {"type": "recall", "memories": (MEMORY,)}
yield {
"type": "final",
"schema_version": 2,
"answer": "真实数据。",
"memories": (MEMORY,),
"mode": "extractive",
}
class Sdk:
def search(self, query, **kwargs):
if kwargs.get("layers") == ["l1_raw"]:
return {"memories": {"schema": [], "proactive": [], "normal": []}}
return {
"memories": {
"schema": [],
"proactive": [],
"normal": [{"memory_id": "memory-1", "content": "real data", "score": 0.9}],
}
}
def get(self, memory_id):
return {
"memory_id": memory_id,
"content": "real data",
"layer": "l2_fact",
"status": "active",
"user_id": "local",
"agent_id": "all-apps",
"is_latest": True,
}
class Handle:
async def client(self):
return Sdk()
async def close(self):
return None
RETIRED_V1_DATA_PLANE = (
("GET", "/api/v2/memory-authority", None),
("GET", "/api/v1/home-summary", None),
("GET", "/api/v1/memory-library", None),
("GET", "/api/v1/memories", None),
("GET", "/api/v1/memories/graph", None),
("GET", "/api/v1/memories/memory-1", None),
("POST", "/api/v1/memories/ask", {"question": "release?"}),
("GET", "/api/v1/conversations/search?query=release", None),
("GET", "/api/v1/conversation-browser", None),
("POST", "/api/v1/context/search", {"query": "release"}),
("POST", "/api/v1/boundary/peek", {}),
("GET", "/api/v1/memory-tasks", None),
("POST", "/api/v1/memory-writes", {}),
)
@pytest.mark.fresh_sdk
def test_sdk_authority_keeps_v2_open(tmp_path, monkeypatch):
monkeypatch.setattr(api, "_sdk_request_answerer", lambda _app: Answerer())
app = application(tmp_path)
async def scenario():
async with app.router.lifespan_context(app), client_for(app) as client:
assert app.state.memory_authority is MemoryAuthority.SDK
app.state.sdk_client_handle = Handle()
ask = await client.post("/api/v2/memories/ask", json={"question": "release?"})
assert ask.status_code == 200, ask.text
context = await client.post(
"/api/v2/context/search", json={"query": "release", "kinds": ["memory"]}
)
assert context.status_code == 200, context.text
for method, path, body in RETIRED_V1_DATA_PLANE:
response = await client.request(method, path, json=body)
assert response.status_code in {404, 405}, (method, path, response.text)
asyncio.run(scenario())
@pytest.mark.fresh_sdk
def test_v2_host_calls_are_recorded_and_renderer_calls_are_not(tmp_path, monkeypatch):
monkeypatch.setattr(api, "_sdk_request_answerer", lambda _app: Answerer())
app = application(tmp_path)
async def scenario():
async with app.router.lifespan_context(app):
app.state.sdk_client_handle = Handle()
async with client_for(app) as renderer:
await renderer.post("/api/v2/memories/ask", json={"question": "release?"})
await renderer.post("/api/v2/memories/ask/stream", json={"question": "release?"})
await renderer.post(
"/api/v2/context/search", json={"query": "release", "kinds": ["memory"]}
)
assert app.state.call_event_store.list().total == 0
async with client_for(app, EXTERNAL) as host:
ask = await host.post("/api/v2/memories/ask", json={"question": "release?"})
assert ask.status_code == 200, ask.text
assert ask.headers["X-One-Memory-Call-Recorded"] == "true"
stream = await host.post(
"/api/v2/memories/ask/stream", json={"question": "release?"}
)
assert stream.status_code == 200, stream.text
context = await host.post(
"/api/v2/context/search", json={"query": "release", "kinds": ["memory"]}
)
assert context.status_code == 200, context.text
assert context.headers["X-One-Memory-Call-Recorded"] == "true"
probe = await host.post("/api/v2/memories/ask", json={"question": "__om_probe__"})
assert probe.status_code == 200
page = app.state.call_event_store.list()
assert page.total == 3
events = sorted(page.items, key=lambda event: event.tool)
assert [event.tool for event in events] == ["om_ask", "om_ask", "om_recall"]
for event in events:
assert event.host_id == "cursor"
assert event.state == "grounded"
# SDK items attest no revision; history keeps counts only.
assert event.references == ()
assert [event.memory_count for event in events] == [1, 1, 1]
assert str(events[2].event_id) == context.json()["request_id"]
history = await client_for(app).__aenter__()
try:
listed = await history.get("/api/v1/call-events")
finally:
await history.__aexit__(None, None, None)
assert listed.status_code == 200, listed.text
assert listed.json()["total"] == 3
asyncio.run(scenario())
@@ -62,6 +62,8 @@ def test_the_switch_on_starts_it(library, monkeypatch):
async def scenario() -> None:
app = _app(library)
async with app.router.lifespan_context(app):
# Started by the recovery that runs after readiness.
await app.state.recovery_task
assert app.state.memory_visibility_task is not None
asyncio.run(scenario())
Loaded 100 of 237 files, more files were not shown because too many files have changed in this diff. Show more