Compare commits

...
Author SHA1 Message Date
alvinfeiandClaude Opus 5.5 2b75adc718 release: hy-memory 2.0.11
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-11-01 21:55:18 +08:00
feitianxiang 75ac1b01f2 Merge pull request #261 from Tencent-Hunyuan/feat/sdk-multilingual-prompts
feat(sdk): prompts in 11 languages and an explicit language option
2026-11-01 21:53:38 +08:00
feitianxiang b2c91d9751 Merge pull request #260 from Tencent-Hunyuan/feat/memhub-embedding-switch
feat(memhub): switch the embedding model safely — space registry, paused re-embed, startup merge of stray spaces
2026-11-01 21:34:53 +08:00
alvinfeiandClaude Opus 5.5 16da11f788 feat(sdk): prompts in 11 languages and an explicit language option
Move every LLM prompt that had a zh variant into one registry,
hy_memory/prompts/, with one module per language (en, zh-Hans, zh-Hant,
ja, es, de, pt, fr, ko, ru, id). Call sites look prompts up with
get_prompt(name, language), so a new language needs only a new module.
The en and zh-Hans strings are moved verbatim, a test pins their sha256,
and the old names (prompts_zh.*_ZH, extractor.EXTRACT_SYSTEM_PROMPT, ...)
still import.

Add an optional language: MemoryConfig.language (env MEMORY_LANGUAGE)
as the client default, plus a per-call language= on add / async_add,
add_extracted, finalize_session, evolve and sweep. It goes through
extract and basic profile, reconcile and search query, the coding
extractor and both coding reconcilers with the chain summary, the
episode and topic summaries, and System 2. Codes are validated, with
aliases (zh, zh-CN, zh-TW, zh-HK, pt-BR); an unknown code raises
ValueError. With an explicit language, memories are written in that
language. Without one, the zh/en auto-detection is unchanged. The
server accepts language on /add and /add_extracted.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-11-01 21:29:59 +08:00
alvinfeiandClaude Opus 5.5 7df371bb10 fix(memhub): make the embedding switch work on real graphs and SDK services
Reviewing the switch against the real SDK, Kuzu and EmbedService found what
the in-memory backend could not:

- Verifying the graph ran beside live clients, and the SDK refuses a graph
  one holds open, so an ultra switch failed whenever a read landed after the
  copy. Clients are closed for the verify too.
- The graph was only re-embedded in ultra, so a pro home's graph from its
  ultra days went with the old space. It moves in every mode now.
- The re-embed built a full client config, which needs the generation key;
  an Ollama embedding has none resolved. It uses the retrieval view.
- The scan opened Chroma outside the first-open lock, the race that once lost
  a clean install's first write. It takes the client-construction lock.
- A live space whose profile was deleted raised KeyError on every read. It
  now answers `embedding_space_profile_missing`.

Also tests the legacy-collection blocker and the real startup migration on a
home mid-switch.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-11-01 21:19:01 +08:00
alvinfeiandClaude Opus 5.5 f38c370fbd feat(memhub-cli): embedding switch commands and the App contract
`memhub models activate` asks Core for the switch plan, shows it, and needs
`--yes` (a prompt on a terminal, exit 2 otherwise); with `--yes` it starts
the switch and follows it (`--wait`, JSON lines with `--json`). Ctrl-C only
stops watching. `models add --activate` now probes before activating and
takes the same path. New `models switch status|cancel` and `models spaces
[--cleanup <id> --yes]`; `memhub status` shows the switch, merge or scan
line; `embedding_unavailable` reads as "the embedding service is
unavailable".

docs/contracts/embedding-switch-v1.md is the App's contract: endpoints,
states, progress fields, error codes, and zh/en/es copy for the
loading-screen merge, the confirmation dialog, the progress banner, the
failure states and the `embedding_unavailable` toast.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-11-01 20:49:23 +08:00
alvinfeiandClaude Opus 5.5 e87d6305be feat(memhub): switch the embedding model safely in Core
The embedding space is explicit now. An `embedding_spaces` registry in the
main Core SQLite pins the live space; existing homes register the
collection they already use under its four-attribute name, byte for byte
(an "auto" width keeps its unsuffixed name and only records the measured
width). The identity of a space is the embedding model and width: a new
endpoint for the same model keeps it. `build_sdk_config(profile, space,
mode, features=)` takes the space and refuses a profile whose identity is
not the space's (`embedding_space_mismatch`); mode and features never
change a name. The read handle and the write coordinator resolve the live
space through the registry and key their handles by it.

B2: a probe on the profile that fills the live space no longer records a
width that would move writes to new collections; the same stored width is
a relabel only.

An identity change becomes a switch job (`embedding_switch_jobs`):
preparing, reembedding, verifying, committing, done / failed (rollback) /
rolled_back, with a heartbeat and owner pid. Model writes pause (claim and
admit gates, direct edit/delete/graph-width answer 409
`embedding_switch_in_progress`); intake keeps queueing. Every vector of
every user is re-embedded into a new space through the SDK re-embed
module (`hy_sdk_migration.SdkReembedBackend`), the graph too in ultra,
verified per role and per user, then one SQLite transaction makes the new
space live, retires the old one and activates the profile. Reads serve the
old space until that commit.

Every Core start scans the vector store before model writes resume. A
non-empty stray space (an earlier build wrote one collection and read
another) is merged into the live space with the live embedder; live
records win, merged strays retire. Unfinished jobs resume at startup,
including after an App update: `sdk-migration-status` reports an open job
without requiring or blocking a migration, and `sdk-migrate` leaves the
switch tables alone. Retired spaces are dropped after 14 days or on
request.

Activation answers 409 `embedding_switch_required` with the plan unless
the identity is unchanged; an empty home activates directly. A search,
recall or context call whose remote embedding does not answer fails with
503 `embedding_unavailable` (no lexical fallback).

API: GET/DELETE /api/v1/embedding-spaces, POST
/api/v1/embedding-switches/plan, POST /api/v1/embedding-switches, GET
/api/v1/embedding-switches/current and /{id}, POST
/api/v1/embedding-switches/{id}/cancel; `embedding_switch` on health and
memory-processing.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-11-01 20:49:12 +08:00
alvinfeiandClaude Opus 5.5 7a78fca807 build(memhub): pin hy-memory 2.0.10
Core pins and locks the published hy-memory 2.0.10 from PyPI, which adds
the re-embed module the embedding switch needs. The version constant,
README, architecture notes, SBOM, notices and their tests follow.
CHANGELOG records 2.0.10 as published.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-11-01 20:48:55 +08:00
feitianxiang 1520b6de1f Merge pull request #259 from Tencent-Hunyuan/feat/memhub-sdk-feature-settings
feat(memhub): configurable SDK retrieval and extraction features (graph rerank, episode recall, …) via CLI and API
2026-11-01 20:34:32 +08:00
alvinfeiandClaude Opus 5.5 3742b3e453 feat(memhub): configurable SDK retrieval and extraction features
MemHub hard-codes none of the SDK's optional features any more. A person
can turn them on from the CLI or the API; with nothing set the SDK config
is byte for byte what it was, so the App behaves as before.

- Features (one global `memory_features` document in config.json, beside
  `memory_mode`; strict booleans, unknown keys refused, absent = SDK
  default):
  - graph_rerank: graph_store.graph_rerank_enabled, usage A, ultra only.
  - same_series_timeline: recall.same_series_timeline_enabled, needs
    graph_rerank.
  - episode_recall: recall.episode_recall_enabled, usage B, pro or ultra.
  - session_episodes: recall.session_episode_enabled, the episode write
    side.
  - episode_summary: recall.episode_summary_enabled, one LLM call per
    closed episode.
  - intention: llm.enable_intention, on by default.
  - few_shot: llm.few_shot_enabled.
- How they reach the SDK: build_sdk_config sets the MemoryConfig fields
  per client. Every exposed flag is a config field read at call time, never
  os.environ; the SDK's own env var still sets the default of an unset
  feature. A feature never touches the vector store, embedder or store
  paths, so collection identity is unchanged.
- A change rebuilds the handles like a mode change: write handles are keyed
  by (profile, revision, mode, features) and retired with the #218 use
  count; the read handle (ActiveProfileReadHandle) is keyed by (profile,
  mode, features) and builds a new client on the next read, closing the old
  one once no request uses it.
- API: GET/PUT /api/v1/settings/memory-features. Each feature reports its
  effective value, configured value, default, whether it is active in the
  current mode (e.g. graph_rerank in pro: requires_ultra), cost, SDK field,
  env var and a description. PUT: a value sets, null resets, an absent key
  is kept, anything else is 422.
- Status: pipeline.features (the non-default ones) in
  /api/v1/memory-processing, memhub status --json and progress --json.
- CLI: memhub settings features [set <name>=<bool> ... [--yes] |
  reset [<name> ...]]; set previews without --yes.
- Contract: docs/contracts/memory-features-v1.md. It has the inventory,
  what is left out and why, and zh/en/es copy.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-11-01 19:54:44 +08:00
feitianxiang 98bc52b9f4 Merge pull request #253 from Tencent-Hunyuan/feat/memhub-mode-evolve
feat(memhub): memory mode (lite/pro/ultra), evolve and nightly schedule (B, stacked on #252)
2026-11-01 19:53:15 +08:00
feitianxiang 30fea9394f Merge pull request #252 from Tencent-Hunyuan/feat/memhub-sdk-surface
feat(memhub): SDK search params, memory layers, raw, v2 source aliases, profile versions (A)
2026-11-01 19:53:02 +08:00
feitianxiang e966e913b6 Merge pull request #258 from Tencent-Hunyuan/release/hy-memory-2.0.10
release: hy-memory 2.0.10
2026-11-01 19:32:15 +08:00
alvinfeiandClaude Opus 5.5 ba6248b90b docs: capitalise the 2.0.10 changelog entry
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-11-01 19:26:34 +08:00
alvinfeiandClaude Opus 5.5 6cad02bc82 release: hy-memory 2.0.10
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-11-01 19:26:07 +08:00
feitianxiang fdc57822b5 Merge pull request #254 from Tencent-Hunyuan/feat/sdk-trimmed-raw
feat(sdk): store raw conversations already trimmed; history concatenates without re-trimming
2026-11-01 19:24:43 +08:00
alvinfeiandClaude Opus 5.5 4e1748f65a feat(memhub): memory mode (lite/pro/ultra), evolve and its nightly schedule
- Mode: a saved setting (config.json memory_mode, default pro);
  ONE_MEMORY_GRAPH_STORE still forces ultra for development.
  - API: GET/PUT /api/v1/settings/memory-mode, which reports the switch's
    effects and the graph backfill.
  - CLI: memhub settings mode [lite|pro|ultra] [--yes], which prints the plan
    first.
- The mode drives every SDK client.
  - acquire_client passes it to HyMemoryClient. The read handle rebuilds on a
    change.
  - Write handles are keyed by (profile, revision, mode) and retired with the
    #218 use count, so a batch in flight finishes in the old mode.
  - Jobs read the mode when they run: a job completed in lite is never
    reprocessed; pending jobs take the new mode.
  - Mode never touches the vector collection identity.
- lite needs no generation model: the built-in embedder writes raw, and the
  claim gate admits lite without an active profile. lite never calls
  add_extracted (a note is one raw user turn through add). Recall serves the
  raw layer.
- → ultra: build_memory_graph once per known user, in the background, with a
  status (graph_build).
- Raw retention removed in every mode: MemHub deletes no raw (owner
  decision); the retention loop, its startup step and its test are gone.
- P1-3 / R7: POST /api/v2/evolve answers 202 with a run receipt;
  GET /api/v2/evolve[/{run_id}].
  - Ultra only: 409 memory_mode_not_ultra otherwise.
  - One run per user. Never beside that user's write: it waits for the write
    in flight, and holds that user's jobs (released pre-SDK) until it ends.
  - Loops while should_continue, up to 20 rounds.
  - Per-user last_run_at, last_result and last_error in evolve-state.json
    (counts and codes only).
  - CLI: memhub evolve [--user-id] [--agent-id] [--max-batches] [--wait].
- Nightly scheduler: 03:00 local by default, ultra only, with catch-up when
  the last run is older than the slot or 24h.
  - Users come from Core's records (local, grant holders, write-queue users)
    that the SDK holds memories for.
  - GET/PUT /api/v1/settings/evolve-schedule.
- R10: pipeline {mode, configured_mode, forced_by_env, evolve, last_evolve_at,
  graph_build} in /api/v1/memory-processing, memhub status --json and
  memhub progress.
- Contract for the App: docs/contracts/memory-mode-evolve-v1.md, including
  what SDK ultra additionally enables and zh/en/es copy.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-11-01 19:09:19 +08:00
alvinfeiandClaude Opus 5.5 b20331aa39 Merge main into feat/sdk-trimmed-raw
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-11-01 18:56:23 +08:00
feitianxiang 104fbcce09 Merge pull request #256 from Tencent-Hunyuan/feat/sdk-reembed
feat(sdk): re-embed vector stores and the graph into a new embedding space
2026-11-01 18:54:40 +08:00
alvinfeiandClaude Opus 5.5 302fd30af1 feat(sdk): re-embed vector stores and the graph into a new embedding space
Add hy_memory.reembed for switching the embedding model while writes are
paused. Source and target are named explicitly; a space's identity is the
model name plus the width.

- vector_space_names, plan_reembed, reembed_vector_space (resumable,
  cancellable at batch boundaries, widths validated before any upsert, target
  wins on existing ids, source never written), verify_reembed,
  drop_vector_space.
- scan_vector_spaces and diff_spaces for MemHub's startup reconcile of stray
  spaces.
- reembed_graph builds a new Kuzu DB at the target width, copies every node
  and edge, recomputes Memory/beh/Topic/HierarchyTopic embeddings and rebuilds
  the HNSW index; the source graph is opened read-only. verify_graph_reembed
  and drop_graph.

EmbedService.embed_batch now chunks to embedder.batch_size (default 64),
honours Retry-After on 429, takes a per-call max_retries, and rejects vectors
whose width differs from embedding_dims.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-11-01 18:33:52 +08:00
alvinfeiandClaude Opus 5.5 8d0a069950 feat(memhub): expose SDK search, layer, raw and profile-version surface
- P2-1 / R8: POST /api/v2/context/search and /api/v1/recall pass the public
  client.search knobs through, validated: min_score, schema_min_score,
  working_fact_min_score, working_experience_min_score, schema_limit,
  intention_limit, rerank, reader (legacy|hybrid), scene
  (chat|working|coding|all), search_mode (default|agentic),
  include_sources, exclude_sources, created_after, session_ids, agent_ids,
  workspace_id, branch (needs workspace_id), layers. The v2 ask routes take
  the floors and filters. Absent, every route makes the call it made before
  (relevance floor 0.35, scene normal, hybrid reader). A caller's floor is
  also enforced on what Core returns, per lane (the fused scene=all score is
  a rank, so the lane score is used): min_score=0.9 answers only >= 0.9.
  user_ids, embed_override, query_embedding and request_id are not exposed.
- P2-2 / R9: memory-library takes layers (or layer), statuses,
  exclude_sources, session_ids and agent_ids, repeated or comma separated;
  a layer filter lists the chat store and refuses scene=working|coding.
  New GET /api/v2/memory-library/layers counts memories per SDK layer.
  The memory graph takes layers, session_ids and exclude_sources.
- P2-4: /api/v2/raw lists layers=["l1_raw"] with statuses active+shadow (pro
  shadows a raw once extracted), filters by source, session_id, statuses
  and order, cuts after/before in Core, and returns each turn's messages
  with roles, a sha256 content_hash, status and created.
- P2-3: GET /api/v2/sources, POST /api/v2/sources/{id}/sync,
  POST /api/v2/import-jobs, GET /api/v2/import-jobs[/{id}] reuse the v1
  handlers on one user's lane (Hermes only for another user; another
  user's job answers 404).
- Profile: GET /api/v2/user-profile/versions (get_user_basic_info) returns
  the L0 version chains; memhub profile --versions prints them.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-11-01 18:28:54 +08:00
feitianxiang cdb44658cf Merge pull request #255 from Tencent-Hunyuan/fix/upgrade-migration-issues
fix(memhub): upgrade migration without a model profile, tolerant readback, grant triggers after cutover, reads use the active embedding
2026-11-01 18:26:00 +08:00
alvinfeiandClaude Opus 5.5 2ab04291a9 fix(memhub): the embedding endpoint no longer follows the generation base_url
A remote embedding with no `embedding_base_url` resolved to the generation
`base_url` at runtime (`build_sdk_config`, `vector_collection_bases`, the
probe). The collection name hashes that endpoint, so a proxy or region edit
of the generation endpoint moved writes to a new, empty collection, and
providers without embeddings were sent embedding requests.

- `resolve_embedding_endpoint` no longer inherits; creating or updating a
  remote (non-local_onnx) embedding without an endpoint is refused with
  422 `embedding_base_url_required`. The runtime and probe read only
  `embedding_base_url`. The API key stays shared.
- Opening the profile store pins, once and idempotently, every stored
  remote embedding left without an endpoint to the one it resolved to
  (the generation base_url, or the provider default), so the collection
  name is byte-identical. `read_active_profile` applies the same rule for
  `sdk-migrate`, which may run before Core has opened the store.
- Web: settings write the prefilled endpoint instead of relying on the
  fallback; onboarding writes an empty custom field as the endpoint it
  means. CLI headless configs now get the 422 code.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-11-01 18:11:20 +08:00
alvinfeiandClaude Opus 5.5 3772a69c6b feat(sdk): store raw conversations already trimmed; history concatenates without re-trimming
Raw is kept forever (MemHub dropped its retention; lite stores nothing else),
so trim it once when it is written instead of on every history read.

- hy_memory/utils/raw_trim.py: one idempotent rule per speaker label.
  user/system kept; assistant and other speakers cut to 500 chars + "...";
  tool head/tail with coding.tool_result_max_bytes (default 2048).
- writer: L1_RAW (lite/pro/ultra) is flattened from trimmed messages and
  flagged custom.raw_trimmed; request.messages (extractor input) untouched.
- _get_recent_history concatenates flagged raws as stored; legacy unflagged
  raws are trimmed with the same rule on read.
- client: memory_history ADD record uses the same trimmed messages.
- coding preproc: tool/assistant truncation made idempotent so re-trimming
  coding_trace history no longer rewrites omitted-count markers.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-11-01 18:02:40 +08:00
alvinfeiandClaude Opus 5.5 80d3b55723 fix(memhub): reads use the active profile's embedding space
The read handle was always built from `builtin_retrieval_snapshot()`, while
writes follow the active profile, so a custom embedding (2560 dims, say)
wrote `memhub_<sha>_2560` and every read opened the empty built-in 512-dim
collection.

`_sdk_client_handle` now returns an `ActiveProfileReadHandle`. It opens a
`retrieval_view` of the active profile: the profile's embedding fields with
the built-in generation placeholder, so a read never needs the generation
model; a remote OpenAI-compatible embedding gets the profile's shared key.
With no active profile it stays on the built-in space.

A profile change builds a new client and retires the old one, closed once no
task still uses it, as `SdkWriteCoordinator` does (#218). A use is the task
that asked for the client, so every caller (library, search, recall and its
raw fallback, graph, profile, raw, home summary, ask, context, multi-user
routes) keeps its shape; each raw-retention pass runs in its own task.

Refs #242

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-11-01 17:50:21 +08:00
alvinfeiandClaude Opus 5.5 b5ec0861f0 fix(memhub): migrate without a model profile and check readback against the pre-migration store
#234: with no active model profile the migration aborted in preflight with
`sdk_migration_model_profile_required`. No profile means no generation
model, not no embedding: the vectors are the built-in model's, the space the
Runtime reads with. The runner, and `sdk-migration-status`, now use
`builtin_retrieval_snapshot()` for reattach, the collection bases and the
readback client; `runtime_sdk_client` resolves no secret for a key-less
profile. When legacy memories have vectors only under another embedding
space (a profile since deleted), the preflight stops with
`sdk_migration_vector_profile_missing` before anything changes.

#229/#240: the readback required every confirmed governed chat id to read
back, but a 1.0.4 library can hold confirmed rows whose SDK node was merged
or deleted before the upgrade (658 confirmed, 657 readable, upgrade failed).
The ids in the vector store are now read before reattach; chat is held to
the confirmed ids present there, and the rest is only counted as
`chat_missing_before_migration`. A memory present before the migration but
unreadable after it still fails, and if none of the confirmed chats is in
the store none is excused.

Fixes #234, Fixes #229, Fixes #240

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-11-01 17:15:51 +08:00
alvinfeiandClaude Opus 5.5 47e938da1b fix(memhub): drop grant triggers that read a table the cutover dropped
The two `source_grants` triggers of the visibility cache select from
`governed_memory_evidence`, which the SDK cutover drops. SQLite resolves a
trigger body only when it fires, so every grant insert or update on a
migrated home failed with "no such table": revoke, connect and the
multi-user Hermes connect all answered 503.

`install_triggers` now installs a trigger only when every table its body
names exists, and drops an old copy otherwise. Startup on SDK authority
(which does not install the cache) and the cutover cleanup drop the stale
copies an earlier build left behind.

Fixes #233

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-11-01 16:20:34 +08:00
feitianxiang f5ea0af75d Merge pull request #251 from Tencent-Hunyuan/fix/hermes-injected-content
feat(memhub): land #250 on main — multi-user Runtime, hy-memory 2.0.9
2026-11-01 14:39:43 +08:00
alvinfeiandClaude Opus 5.5 1c2f48579f Merge main into fix/hermes-injected-content to land #250
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-11-01 14:33:00 +08:00
feitianxiang d29615abeb Merge pull request #250 from Tencent-Hunyuan/feat/memhub-multi-user
feat(memhub): multi-user Runtime — user_id on memory APIs and CLI, per-user Hermes profiles, hy-memory 2.0.9
2026-11-01 14:25:05 +08:00
feitianxiang 0965519e16 Merge pull request #249 from Tencent-Hunyuan/fix/hermes-injected-content
fix(memhub): drop Hermes gateway notices and unwrap relayed messages on import; no false 'damaged log' warning
2026-11-01 14:24:05 +08:00
alvinfeiandClaude Opus 5.5 7e9f0d98d8 fix(memhub): another user's memory id answers like a missing one
DELETE of another user's id answered 404 while a missing id answered 204,
so comparing the two told that the id existed. It now answers 204 and
deletes nothing; GET and PUT answer the same 404 as a missing id, and
receipt and addition reads already matched. A test compares every by-id
route's response for another user's id with a nonexistent id's.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-11-01 10:23:10 +08:00
alvinfeiandClaude Opus 5.5 0ff4538f36 feat(memhub-cli): --user-id for memory commands, connect a Hermes profile
- Global --user-id <id> (default local) for memories list/show/add/
  add-status/update/delete, recall, profile and write submit/status; sent
  only when it is not the default, so default requests are unchanged.
- memhub sources connect hermes --user-id <id> --root <path> [--import]
  grants one Hermes profile to its own user; refused for other sources.
- memhub sources list shows the user of every grant (from
  GET /api/v1/source-grants, skipped on an older Core).
- Help catalog, contract tests and docs/cli/EXPERIENCE.md updated.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-11-01 10:12:31 +08:00
alvinfeiandClaude Opus 5.5 a9709788d8 feat(memhub): multi-user Core — user_id on memory APIs, per-user Hermes grants
One Runtime now serves many user_ids. A user is the root that is read,
never a session or platform field; the App never sends user_id and keeps
reading and writing `local`, and only Hermes may hold another user's grant.

- R1: every memory read/write route takes an optional user_id (query for
  GET/DELETE, body for POST/PUT), default `local`, pattern
  ^[A-Za-z0-9._-]{1,64}$ (no `:`).
- R2: user_id reaches the SDK: Evidence, durable payload and client.add
  carry it; reads filter by user. get/update/delete/history and
  get_write_outcome take no user in hy-memory 2.0.9, so Core reads the
  memory first and answers 404 for another user's id; memory-write keys
  are namespaced per user and addition receipts are keyed by the user's
  Evidence id.
- R3: grants, cursors, jobs, live sync and the claim gate work per
  (user_id, source_id) through lane keys (`hermes` for local,
  `hermes@wife` otherwise). POST /api/v1/sources/connect takes user_id and
  root (Hermes only, 422 otherwise); revoke takes ?user_id; new
  GET /api/v1/source-grants lists every user's grants. Another user's
  grant syncs while it stands; the App's per-source switch stays local's.
- R4: a Hermes root scan never enters profiles/** (discovery, history and
  live sync share the same walk).
- R5: session keys, assembling rows, FIFO source keys and request ids are
  per user, so two users' turns never share a batch.
- R6: memory routes refuse undeclared query parameters and body fields
  with 422.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-11-01 10:12:31 +08:00
alvinfeiandClaude Opus 5.5 c1e5916b51 build(memhub): pin hy-memory 2.0.9
Core pins and locks the published hy-memory 2.0.9 from PyPI; the
version constant, README, architecture notes, SBOM, notices and their
tests follow. CHANGELOG records 2.0.9 as published.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-11-01 09:12:21 +08:00
alvinfeiandClaude Opus 5.5 97de5480bd Merge github/main (hy-memory 2.0.9 release) into feat/memhub-multi-user
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-11-01 09:11:25 +08:00
feitianxiang 97d9309ca8 Merge pull request #248 from Tencent-Hunyuan/feat/sdk-2.0.9
fix(sdk): placeholder values never overwrite the basic profile (2.0.9)
2026-11-01 08:26:49 +08:00
alvinfeiandClaude Opus 5.5 b1b3e0a291 fix(sdk): drop the user-turn filter for basic_info
Owner decision on #248: the writer no longer filters basic_info against
the user turns. The placeholder guard (with healing on read) and the
prompt hard rules stay; the guard alone keeps an assistant quoting
"name is 用户" out of the profile.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-11-01 08:18:32 +08:00
alvinfeiandClaude Opus 5.5 53507c7abe fix(memhub): only call non-JSON diagnostic lines damaged in memhub logs
After an upgrade the diagnostic files still hold an older build's records.
They are valid JSON that the reader leaves out, but were counted as damaged,
so `memhub logs` warned "跳过 N 行损坏的诊断记录" and a support export
failed with diagnostic_export_invalid_lines. Count them apart as
`unrecognized_lines` (reported only in the JSON empty-state record) and
keep the warning for lines that are not JSON.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-11-01 05:10:19 +08:00
alvinfeiandClaude Opus 5.5 5a1d252787 fix(memhub): drop Hermes gateway notices and unwrap relayed messages on import
Hermes stores its gateway's own posts in `user` rows: finished background
process notices (often with the gateway's "not written by a human" note),
voice placeholders that only name an audio file, and "Reply with exactly:
PONG" probes. Relayed Telegram messages arrive wrapped in out-of-band
markers and a gateway-origin JSON block. All of it reached extraction as
the person's words.

The Hermes adapter now cleans `user` rows (state.db and session
transcripts) before evidence is built, so the content hash and dedup see
the cleaned text:
- skipped with new reason codes `injected_machine_notice`,
  `voice_placeholder` and `probe` (media-only rows as `no_text`), counted
  in `import.parse.issue_summary` like other parse skips;
- out-of-band markers, the gateway-origin block and the interrupted
  response label are removed, keeping only the person's message.

Assistant and tool rows are unchanged. Tests use real rows with the
Telegram ids and home paths replaced by fake values.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-11-01 05:10:19 +08:00
alvinfeiandClaude Opus 5.5 8895fea775 release: hy-memory 2.0.9
Placeholder values no longer overwrite the user's basic profile (L0). Not
yet published to PyPI.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-11-01 04:56:01 +08:00
alvinfeiandClaude Opus 5.5 78882f3300 fix(sdk): placeholder values never overwrite the basic profile
A weak extraction model filled basic_info with "用户", "The user" or "N/A"
when the user stated nothing, and each placeholder superseded the real L0
value. upsert_basic_profile now drops placeholder values and writes no new
version when nothing is left; profile assembly skips stored placeholders.
The writer keeps a basic_info field only when its value appears in a user
message of the request. The zh and en extraction prompts add a hard rule
and a small-talk counter-example.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-11-01 04:56:01 +08:00
feitianxiang 988667cd87 Merge pull request #224 from Tencent-Hunyuan/chore/build-36-docs
chore(release): bump MemHub build number to 36 for v1.0.5
2026-10-28 22:07:22 +08:00
alvinfeiandClaude Opus 5.5 72968f831f chore(release): bump MemHub build number to 36
The macOS v1.0.5 artifacts were built from release/memhub-build-36-main
(dbba403): main's code at a621262 with build 36. Main moves to the same
build so the Linux Runtimes built from the v1.0.5 tag match them. The
release docs name build 36.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 22:02:27 +08:00
feitianxiang af25f51f9e Merge pull request #223 from Tencent-Hunyuan/docs/release-1.0.5
docs: prepare MemHub v1.0.5 (news, build 35, macOS arm64 only)
2026-10-28 21:59:59 +08:00
alvinfeiandClaude Opus 5.5 b3f13e130f docs: MemHub 1.0.5 news, build 35, macOS arm64 only
The README news line announces v1.0.5 briefly; details live in the release
notes. The release facts said build 14 and promised a macOS Intel build that
this version does not ship. CHANGELOG's Unreleased section becomes MemHub
1.0.5.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 21:58:28 +08:00
kristinhe1111 a621262fac Merge pull request #222 from Tencent-Hunyuan/fix/memory-detail-structure-and-stability
fix(memhub): stabilize memory detail navigation
2026-10-28 20:44:13 +08:00
kristinhe e027635d3f Merge remote-tracking branch 'origin/main' into fix/memory-detail-structure-and-stability
Co-authored-by: Cursor <cursoragent@cursor.com>

# Conflicts:
#	memhub/apps/web/src/MemoryMap.test.tsx
2026-10-28 20:36:06 +08:00
feitianxiang 4cb33b7108 Merge pull request #217 from Tencent-Hunyuan/fix/memory-source-filter-sync
fix(memhub): 同步记忆筛选范围并优化首次启动状态
2026-10-28 20:32:24 +08:00
alvinfeiandClaude Opus 5.5 b50c9f33eb fix(memhub): give the map its own graph-loading state
The map hid the previous scope's graph by passing stale into the library
view, which also blanked the list status, pagination and empty states.
A separate graphLoading flag now hides the old graph and shows
"正在读取记忆地图…" while stale keeps its original meaning and text.
The client-side revoked-source graph filter is dropped to match Core.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 20:24:39 +08:00
alvinfeiandClaude Opus 5.5 545b7d38bf fix(memhub): keep revoked-source memories on the memory map
Memories already distilled from a revoked source stay visible, so the
graph endpoint no longer checks revocation before or after the SDK read,
no longer filters memories by revoked source and reports the SDK total
again. The map and the list now agree. The scene="all" and
statuses=["active"] scope stays.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 20:24:39 +08:00
feitianxiang 5f78d67363 Merge pull request #221 from Tencent-Hunyuan/fix/linux-lifecycle-queue-check
fix(memhub): Linux lifecycle checks the write queue, not recall without a model
2026-10-28 20:17:45 +08:00
alvinfei 33d7713d4e Merge remote-tracking branch 'github/main' into fix/memory-source-filter-sync 2026-10-28 20:14:52 +08:00
kristinheandCursor 68a4b546f9 fix(memhub): stabilize memory detail navigation
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-28 20:11:12 +08:00
kristinhe cbcc50f35e Merge remote-tracking branch 'origin/main' into fix/memory-source-filter-sync
Co-authored-by: Cursor <cursoragent@cursor.com>

# Conflicts:
#	memhub/apps/web/src/MemorySpace.tsx
2026-10-28 20:09:35 +08:00
alvinfeiandClaude Opus 5.5 528d2c9fd7 fix(memhub): Linux lifecycle checks the write queue, not recall without a model
The lifecycle verification imported a conversation on a runner with no
generation model and then expected recall to find it. Under SDK authority
nothing reaches the SDK, not even its raw layer, until a model is active,
so the check could never pass and blocked the Linux release archive.

It now asserts what the runner can prove: the imported and appended turns
reach Core's durable write queue, and survive an idempotent upgrade.
docs/TODO-raw-recall-without-model.md records the product gap.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 20:05:44 +08:00
kristinhe1111 0eb9a2844e Merge pull request #216 from Tencent-Hunyuan/fix/onboarding-language-menu-clipping
fix(memhub): keep option menus inside viewport
2026-10-28 20:01:37 +08:00
feitianxiang 8e56bf7a37 Merge pull request #218 from Tencent-Hunyuan/fix/sdk-client-race-and-unknown-jobs
fix(memhub): serialize SDK client startup, log failure frames, settle pre-upgrade unknown writes
2026-10-28 19:49:24 +08:00
alvinfeiandClaude Opus 5.5 352fffb3da fix(memhub): serialize SDK client startup, log failure frames, settle pre-upgrade unknown writes
- hy_sdk_runtime: construct one HyMemoryClient at a time per home. Two
  first-time Chroma opens on a fresh vector dir race Chroma's unlocked
  System cache; on a clean install the drain's and the UI's clients both
  failed and the first write job was lost as unknown.
- sdk_write_processor: a client that fails to construct raises
  hy_memory_client_unavailable before HY is invoked, which parks the job
  (waiting_for_model) instead of leaving it unknown.
- sdk_write_coordinator: a retired profile's handle closes only after its
  last in-flight use (defensive; a profile revision change used to close
  the client under a running write).
- durable_queue: durable_write_job_failed/parked log the exception chain as
  file:line:function frames, never the message.
- sdk_migration_runner: the cutover settles pre-cutover unknown (and
  invoked running) write jobs as cancelled/abandoned_at_cutover and counts
  them as abandoned.unknown_write_jobs; never retried.
- durable_queue/api: retry-unknown and the reconcile helpers no longer
  assume the dropped legacy task tables; the endpoint uses the startup
  queue so it does not recreate memory_write_job_evidence.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 19:39:13 +08:00
kristinheandCursor 5103492762 fix(memhub): harden memory overview recovery
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-28 19:26:01 +08:00
kristinheandCursor 082aeee1f2 fix(memhub): align graph scope and startup states
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-28 19:16:48 +08:00
kristinheandCursor 201901d35e fix(memhub): keep option menus inside viewport
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-28 19:04:11 +08:00
feitianxiang 42c0ca1d14 Merge pull request #215 from Tencent-Hunyuan/feat/memhub-sdk-logs-quiet
feat(memhub): HY pipeline log off, SDK text logs only in hy_memory.log (hy-memory 2.0.8)
2026-10-28 15:46:40 +08:00
alvinfeiandClaude Opus 5.5 0ca2e56ccc chore(release): bump MemHub build number to 35
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 15:38:05 +08:00
alvinfeiandClaude Opus 5.5 f328cdbf8e docs: record hy-memory 2.0.8 as published
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 15:38:05 +08:00
alvinfeiandClaude Opus 5.5 3f57a3c978 build(memhub): lock hy-memory 2.0.8 from PyPI
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 15:37:51 +08:00
alvinfeiandClaude Opus 5.5 19a7af0e71 feat(memhub): remove the HY pipeline section from the Debug panel
MemHub no longer writes the HY pipeline JSONL, so the Desktop diagnostics
reader stops scanning hy-memory/governed-v2/logs and the renderer drops the
"hy-memory" service and "pipeline" source. A test covers leftover pipeline
files and hy_memory.log staying out of the panel.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 15:36:50 +08:00
alvinfeiandClaude Opus 5.5 5791c0d1e7 feat(memhub): HY pipeline log off, SDK text logs only in hy_memory.log
- MEMORY_PIPELINE_LOG_ENABLED and MEMORY_PIPELINE_TRACE_ENABLED are false
  at every diagnostic level, both at startup and on a live level change.
  MEMORY_LOG_LLM_CONTENT keeps its debug-only setting: it still decides
  whether HY's DEBUG lines in hy_memory.log carry full prompts/responses.
- After each HyMemoryClient construction (success or failure), Core drops
  the SDK's stdout/stderr StreamHandlers, which under launchd duplicated
  every SDK line into service.log/service-error.log, and forwards SDK
  WARNING+ to Core's core.log handler so the Debug panel still shows SDK
  problems.
- A live level change also sets the level of HY's hy_memory.log handler,
  which the SDK otherwise fixes at the level of the first client.
- log_file.py no longer claims core.log covers the HY SDK at every level.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 15:36:50 +08:00
alvinfeiandClaude Opus 5.5 bf92105210 build(memhub): pin hy-memory 2.0.8
2.0.8 adds MEMORY_PIPELINE_LOG_ENABLED and keeps conversation text out of
INFO logs. apps/core/uv.lock is not updated: 2.0.8 is not on PyPI yet, so
the lock is regenerated (uv lock --directory apps/core) once it is
published.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 15:36:50 +08:00
feitianxiang 3387247842 Merge pull request #214 from Tencent-Hunyuan/feat/sdk-pipeline-log-switch
feat(sdk): switch to turn the pipeline log off, keep conversation text out of INFO logs (2.0.8)
2026-10-28 15:34:41 +08:00
alvinfeiandClaude Opus 5.5 791285f4c9 fix(sdk): keep model output out of the reconcile parse-failure warning
The final RECONCILE JSON parse failure warning embedded the raw response
whenever MEMORY_LOG_LLM_CONTENT was on. It now reports the length; the raw
output is already logged per attempt at DEBUG.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 15:06:09 +08:00
alvinfeiandClaude Opus 5.5 dc762df78b feat(sdk): switch to turn the pipeline log off, keep conversation text out of INFO logs (2.0.8)
MEMORY_PIPELINE_LOG_ENABLED (default true) turns off the pipeline JSONL
log. With the DB trace also off, store_pipeline_log becomes a no-op that
does no timing, redaction, background work or database write. The init line
states the real on/off state.

Lines that carried conversation or memory text at INFO/WARNING now log ids,
counts and lengths; the text goes to DEBUG: [add] input, search queries and
rewrites, coding tasks and fact statements/subjects, reconcile op detail,
extracted info, embedding inputs and unparsable model output.

Bump hy-memory to 2.0.8 (not yet published).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 15:03:21 +08:00
feitianxiang 8277a72aea Merge pull request #206 from rahul05ranjan/fix/runtime-stale-lease-boot-clean
fix(runtime): 清理早于本次开机的服务锁,异常关机后不再崩溃循环
2026-10-28 14:42:46 +08:00
feitianxiang 5eed0f81fb Merge pull request #200 from u7-u7/fix/literal-special-token-counting
fix(tokens): count literal special-token spellings as ordinary text
2026-10-28 14:36:16 +08:00
feitianxiang 59ca5afca6 Merge pull request #199 from u7-u7/fix/guarded-thread-context
fix(async): preserve context in guarded thread calls
2026-10-28 14:36:11 +08:00
feitianxiang 2648012ad5 Merge pull request #203 from u7-u7/fix/migration-log-write-order
fix(memhub): preserve migration log write order
2026-10-28 14:36:05 +08:00
alvinfeiandClaude Opus 5.5 63a80092c2 fix(runtime): identify stale leases by boot id and publish them atomically
Comparing the lock directory's mtime with Date.now() - os.uptime() breaks
when the wall clock jumps forward after a lease is taken (no RTC, VM
resume, NTP or manual correction): a live lease looks older than the boot
and is cleared, so a second Runtime can open the same data dir.

owner.json now records the kernel boot identity (kern.bootsessionuuid on
macOS, /proc/sys/kernel/random/boot_id on Linux, null elsewhere), read
once per process. removeStaleLease clears a lease whose boot_id differs
from the current one, which still fixes the recycled-PID crash loop of
#184 without consulting the clock; when either id is unknown the PID
probe decides as before. bootTime, BOOT_SLACK_MS and defaultBootTime are
gone; the boot id source is injectable via RuntimeServiceOptions.bootId.

The lease is published in one step: owner.json is written into a
runtime-service.lock.tmp-<uuid> staging dir that is then renamed onto the
lock, so a power cut can no longer leave a lock without an owner. Because
rename() replaces an existing empty directory, an existing lock of any
kind is reported as taken first. releaseLease also removes the lock with
a single rename, so this version never shows an owner-less lock; staging
and cleared-lease debris is swept once the lease is held.

An owner-less lock can therefore only come from an older version. It is
cleared when its mtime predates the kernel-reported boot time
(kern.boottime, /proc/stat btime) and otherwise keeps failing as
runtime_service_lock_invalid, as before.

Refs #184

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 14:33:33 +08:00
feitianxiang 6ba49bf112 Merge pull request #209 from Tencent-Hunyuan/fix/memory-map-node-navigation
fix(memhub): open source memories from map nodes
2026-10-28 14:10:28 +08:00
feitianxiang cf410aa9cf Merge pull request #210 from Tencent-Hunyuan/fix/harness-refresh-on-upgrade
fix(memhub): refresh harness skills and MCP entry after an upgrade
2026-10-28 14:08:01 +08:00
kristinheandCursor 81ed1d816f fix(memhub): open source memories from map nodes
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-28 12:00:26 +08:00
Rahul Ranjanandrahul05ranjan a17230dadb fix(runtime): clear leases that predate the current boot
A forced power-off leaves runtime-service.lock behind with no chance to
release it (#184). After the reboot the liveness probe cannot tell the
dead owner from the process that reused its PID, so launchd's KeepAlive
restarts the wrapper into runtime_service_already_running without end;
and a power cut inside the mkdir -> privateWrite window leaves the lock
without owner.json at all, which fails as runtime_service_lock_invalid
forever.

A lease created before the current boot can never have a living owner:
every process from that boot is gone. removeStaleLease now compares the
lock directory's mtime with the boot time (os.uptime(), injectable via
bootTime for tests) and clears such leases before it ever trusts owner
or PID. A two-minute slack absorbs forward clock steps taken since
boot, so a lease taken right after boot is not mistaken for stale.
Same-boot leases keep the old behavior exactly: live owner still
refused, dead owner still reclaimed, corrupt same-boot lease still
refused.

Fixes #184

Co-authored-by: rahul05ranjan <11793455+rahul05ranjan@users.noreply.github.com>
2026-10-28 01:00:05 +05:30
u7-u7 d76bfa6474 fix(memhub): preserve migration log write order 2026-10-28 00:49:54 +08:00
u7-u7 e9698a293c fix(tokens): count literal special-token spellings as ordinary text 2026-10-28 00:28:14 +08:00
u7-u7 6718ba8bdf fix(async): preserve context in guarded thread calls 2026-10-27 10:41:13 +08:00
242 changed files with 54741 additions and 6238 deletions

No files matched your search

+2
View File
@@ -40,6 +40,7 @@ MEMORY_EMBEDDING_DIMS=1024
# MEMORY_EMBEDDER_TIMEOUT=15
# MEMORY_EMBEDDER_MAX_RETRIES=2
# MEMORY_EMBEDDER_RETRY_DELAY=0.3
# MEMORY_EMBEDDER_BATCH_SIZE=64
# MEMORY_EMBEDDER_EXTRA_HEADERS={}
# MEMORY_EMBEDDER_EXTRA_BODY={}
@@ -77,6 +78,7 @@ MEMORY_VECTOR_STORE=chroma
# ── 处理模式 ──
# MEMORY_MODE=pro # lite | pro | ultra
# MEMORY_LANGUAGE= # 提示词/记忆语言:en | zh-Hans | zh-Hant | ja | es | de | pt | fr | ko | ru | id;空=zh/en 自动检测
# ── Coding Memory(生产力 / 编码场景独立链路,默认开启)──
# 独立于 chat 记忆的一套链路:写入端对 messages 判类,coding 场景抽「任务→方案」存 sqlite;
+161
View File
@@ -4,6 +4,124 @@ Hy Memory and MemHub are versioned and released independently.
## Unreleased
## Hy Memory 2.0.11 — 2026-11-01
- LLM prompts now ship in 11 languages: `en`, `zh-Hans`,
`zh-Hant`, `ja`, `es`, `de`, `pt`, `fr`, `ko`, `ru` and `id`. They live in
one registry, `hy_memory/prompts/`, with one module per language. Call
sites use `get_prompt(name, language)`, so a new language only needs a new
module. The registry covers chat extraction (system prompt, basic_info,
intentions, few-shot including Example 11), the no-intention variant,
reconcile and its search-query prompt, the working/coding extractor, both
coding reconcilers and the chain summary, the episode and topic summaries,
System 2 evolve, and the user name line. The English and Simplified Chinese
strings are unchanged byte for byte, and a test pins them. The old names
(`prompts_zh.*_ZH`, `extractor.EXTRACT_SYSTEM_PROMPT`, ...) still import.
The nine new languages are machine translations, and native-speaker review
is recommended.
- New optional `language` to choose the prompt language
explicitly. Set it on `MemoryConfig.language` (env `MEMORY_LANGUAGE`) for
a client default, or per call on `add` / `async_add`, `add_extracted`,
`finalize_session`, `evolve` and `sweep`. A per-call value overrides the
config. The value goes through every LLM call of the request: extract and
basic profile, reconcile, the coding path, the summaries and System 2.
Aliases: `zh`, `zh-CN` → `zh-Hans`; `zh-TW`, `zh-HK` → `zh-Hant`; `pt-BR` →
`pt`. An unknown code raises `ValueError` listing the supported codes, and
the HTTP server returns 400. With an explicit language, memories are
written in that language even when the conversation uses another one.
Without a language, the existing zh/en auto-detection runs unchanged and
memories keep the source language. `search` has no `language`: its judge
and rewrite prompts are English-only and keep the query's language.
- The HTTP server accepts `language` on `/api/v1/add` and
`/api/v1/add_extracted`.
## Hy Memory 2.0.10 — 2026-11-01
- Published
[`hy-memory==2.0.10`](https://pypi.org/project/hy-memory/2.0.10/).
- New `hy_memory.reembed` module for switching the embedding
model. Writers must be paused for the whole run. Nothing is derived from
the provider or endpoint: source and target are named explicitly, and the
identity of a space is the model name plus the width (`space_identity`).
Only Chroma and Kuzu are supported.
- `vector_space_names(config, base_name=None)` returns every physical
collection a client opens for the six vector roles: memory, entities,
coding keys and working facts, each in the working and coding variants.
- `reembed_vector_space(source, target, embedding=...)` pages every raw
source collection, covering all users and all layers including L1_RAW. It
embeds the text the write path embedded (`reembed_text`: the canonical
state's `state_retrieval_text`, a schema's `retrieval_text`, otherwise the
document). It then writes each record into the target with the same id,
document and metadata. Each batch's widths are checked before anything is
written. A cancel stops at a batch boundary (`ReembedCancelled`). A rerun
skips the ids the target already holds, so it resumes, and a merge never
overwrites the target. The source is never written.
- `plan_reembed`, `verify_reembed` and `drop_vector_space` complete the
vector side. `verify_reembed` checks ids, metadata, per-user counts and
the width, and runs a self-recall sample. `drop_vector_space` refuses
collections that a live client in this process holds open.
- `scan_vector_spaces(persist_directory)` is read-only. It lists every
space on disk with its roles, counts, stored width, per-user counts and
the models recorded in metadata. `diff_spaces(source, target)` returns the
ids that the source holds and the target lacks, whatever their status.
- `reembed_graph(source_path, target_path, embedding=..., target_dims=...)`
builds a new Kuzu graph at the new width. It copies every node and edge
and recomputes `Memory.embedding`, `Memory.beh_embedding`, and the Topic
and HierarchyTopic embeddings, then rebuilds the HNSW index. The source
graph is opened read-only and kept for rollback. A run that fails or is
cancelled leaves no target behind, and a rerun starts over.
`verify_graph_reembed` and `drop_graph` complete the graph side.
- `EmbedService.embed_batch` now splits large batches into
requests of `embedder.batch_size` texts each (`MEMORY_EMBEDDER_BATCH_SIZE`,
default 64); before, it sent them all in one request. On a 429 it waits for
the `Retry-After` time plus jitter, capped at 60 s. It takes a per-call
`max_retries`. A returned vector whose width differs from the configured
`embedding_dims` now raises `EmbeddingDimensionMismatchError` and is not
retried.
- Raw conversations are stored already trimmed. MemHub now keeps every raw
forever and lite mode stores nothing else, so tool output and long pasted
logs no longer stay in full. One rule (`hy_memory/utils/raw_trim.py`) is
applied to each message when the raw is written:
- `user` and `system` turns are kept in full;
- `assistant` turns and other speakers (`[John]:`) longer than 500
characters, after stripping, are cut to the first 500 plus `...`. This is
the limit history injection used before;
- `tool` turns keep their head and tail, and the middle becomes
`[...N bytes omitted by truncate...]`, with the coding limit
`coding.tool_result_max_bytes` (`MEMORY_CODING_TOOL_RESULT_MAX_BYTES`,
default 2048).
The rule is idempotent. It covers chat `L1_RAW` in lite, pro and ultra (lite
keeps tool turns, because it has no coding route) and the `memory_history`
ADD record. The coding/working `coding_trace` was already trimmed when it
was written. Its tool and assistant truncations are now idempotent, so
reading the history back no longer rewrites the omitted-count markers. The
text embedded for `L1_RAW` is the trimmed text. New raws carry
`custom.raw_trimmed=true`.
- Extractor history ("last k messages") now joins the stored raws directly
and does not trim them again. Raws written before this change have no
`raw_trimmed` flag, so they are still trimmed with the same rule when they
are read. The extractor still receives the current batch in full (tool
turns are still removed on the chat route).
## MemHub 1.0.5 — 2026-10-28
Build 36, on `hy-memory==2.0.8`. Linux x64/arm64 Runtime and macOS arm64
App and Runtime. Upgrading from 1.0.4 migrates the existing library to SDK
authority on first launch.
- MemHub turns off the HY pipeline log and trace at every diagnostic level
(hy-memory 2.0.8 `MEMORY_PIPELINE_LOG_ENABLED=false`,
`MEMORY_PIPELINE_TRACE_ENABLED=false`), so the per-step JSONL files under
`hy-memory/governed-v2/logs/` and the `pipeline_logs` rows stop growing.
The Debug panel no longer shows the HY pipeline section. Existing files are
left in place.
- HY SDK text logs are written once, to `hy_memory.log`. They are no longer
copied to `service.log` and `service-error.log` through the SDK's
stdout/stderr streams. SDK warnings and errors also go to `core.log`, so the
Debug panel still shows them. A live switch of the diagnostic level now also
changes the level of `hy_memory.log`.
- MemHub shows when working and coding experiences were talked about. After
the SDK upgrade every one of them read "时间未知" (time unknown) because the
SDK never stored their conversation time; hy-memory 2.0.7 stores it and
@@ -76,6 +194,49 @@ Hy Memory and MemHub are versioned and released independently.
`hy_node_permissions` proof table. Evidence, governed memories, model
profiles, SDK stores, and source/host grants are preserved.
## Hy Memory 2.0.9 — 2026-11-01
- Published
[`hy-memory==2.0.9`](https://pypi.org/project/hy-memory/2.0.9/).
- Placeholder values no longer overwrite the user's basic profile (L0). A
small extraction model filled `basic_info` with `{"name": "用户"}`,
`The user` or `N/A` when the user had said nothing about themselves, and
each placeholder superseded the real value, so the L0 chain read
费天翔 → Fei → 用户 → The user. Two defences:
- `upsert_basic_profile` drops placeholder values: empty, `用户`,
`the user`, `user`, `n/a`, `na`, `none`, `null`, `unknown`, `未提供`,
`未知`, `不详`, `-`, `?` and the field's own name, compared
case-insensitively with whitespace and punctuation stripped. When nothing
is left it writes no new version and no new L0 head. Placeholders already
stored in the chain no longer override earlier real values when the
profile is assembled, so `get_user_profile` of an affected user shows the
last real value again without any migration. This also stops an
assistant quoting an old memory ("The user's name is 用户.") from being
written back.
- The extraction prompts (zh and en) state the rule: only what the user
said in a `user` turn, never the field name or a placeholder, `{}`
otherwise, with a small-talk counter-example.
## Hy Memory 2.0.8 — 2026-10-28
- Published
[`hy-memory==2.0.8`](https://pypi.org/project/hy-memory/2.0.8/).
- New `MEMORY_PIPELINE_LOG_ENABLED` (default `true`) turns the pipeline JSONL
log (`{MEMORY_LOG_DIR}/<subdir>/<date>.log`) off. With `false` no JSONL
file or directory is written. When `MEMORY_PIPELINE_TRACE_ENABLED` is also
`false`, `store_pipeline_log` becomes a no-op that returns `True` at once:
no timing, redaction, background task or database write. The init log line
now states whether the JSONL log is on or off.
- Conversation and memory text no longer appears in INFO or WARNING logs. The
`[add]` input, search queries and rewrites, coding experience tasks, fact
statements and subjects, reconcile op details, extracted info, embedding
inputs and unparsable model output moved to DEBUG. INFO lines keep ids,
counts and character lengths; parse-failure warnings remain but report only
the length of the output, e.g. `failed to parse output (812 chars)`. The
`[embed]` INFO line now carries `request.input_chars` instead of
`request.input`. The `JSON_PARSE_FAILED` error message from extraction no
longer embeds the model output either.
## Hy Memory 2.0.7 — 2026-10-28
- Published
+7
View File
@@ -33,6 +33,7 @@ English | [中文](README.zh.md)
## Latest News
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-10]**: 🚀 **Open-sourced** quantized **FP8** (**[Hy-Memory-30B-A3B-FP8](https://huggingface.co/tencent/Hy-Memory-30B-A3B-FP8)**) and **GGUF** (**[Hy-Memory-30B-A3B-GGUF](https://huggingface.co/tencent/Hy-Memory-30B-A3B-GGUF)**) versions of **Hy-Memory-30B-A3B** on Hugging Face!
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-10]**: 📦 **Released [MemHub v1.0.5](https://github.com/Tencent-Hunyuan/Hy-Memory/releases/tag/v1.0.5)** with Chinese, English and Spanish interfaces, one-time pairing codes for more custom Harnesses, and smoother browsing, search, startup and recovery, on **`hy-memory==2.0.8`**. See the release notes for details.
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-10]**: 📦 **Released [MemHub v1.0.4](https://github.com/Tencent-Hunyuan/Hy-Memory/releases/tag/v1.0.4)** and **[memhub-cli 1.0.5](https://www.npmjs.com/package/memhub-cli/v/1.0.5)**, with a signed macOS desktop app and verified macOS/Linux Runtimes!
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-10]**: 🚀 **Released** **Hy Memory 2.0 (`hy-memory==2.0.4`)** on [PyPI](https://pypi.org/project/hy-memory/2.0.4/) and open-sourced the **Hy-Memory** monorepo together with **[MemHub](memhub/README.md)**!
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-10]**: 🚀 **Open-sourced** **[Hy-Memory-30B-A3B](https://huggingface.co/tencent/Hy-Memory-30B-A3B)** on Hugging Face along with its [technical report](https://arxiv.org/abs/2610.22196) on attributed reinforcement learning for long-term memory writing!
@@ -97,6 +98,12 @@ export OPENAI_API_KEY="sk-your-key"
See the [Environment Variable Reference](docs/env_reference.md) for the full list.
Prompts ship in 11 languages (`en`, `zh-Hans`, `zh-Hant`, `ja`, `es`, `de`, `pt`,
`fr`, `ko`, `ru`, `id`). By default the SDK picks Chinese or English from the
input. Set `MEMORY_LANGUAGE` (or `language=` on `add()`) to choose one
explicitly; memories are then written in that language. See
[Prompt language](docs/usage.md#提示词语言language).
### 3. Chat Memory
```python
+6
View File
@@ -33,6 +33,7 @@
## 最新动态
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-10]**: 🚀 **模型开源** 在 Hugging Face 发布 **Hy-Memory-30B-A3B** 的 **FP8**(**[Hy-Memory-30B-A3B-FP8](https://huggingface.co/tencent/Hy-Memory-30B-A3B-FP8)**)与 **GGUF**(**[Hy-Memory-30B-A3B-GGUF](https://huggingface.co/tencent/Hy-Memory-30B-A3B-GGUF)**)量化版本模型!
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-10]**: 📦 **正式发布 [MemHub v1.0.5](https://github.com/Tencent-Hunyuan/Hy-Memory/releases/tag/v1.0.5)**:支持中文、英文和西班牙语界面,可通过一次性配对码连接更多自定义 Harness,并优化了记忆浏览、搜索、启动和异常恢复;底层使用 **`hy-memory==2.0.8`**。详见发布说明。
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-10]**: 📦 **正式发布 [MemHub v1.0.4](https://github.com/Tencent-Hunyuan/Hy-Memory/releases/tag/v1.0.4)** 与 **[memhub-cli 1.0.5](https://www.npmjs.com/package/memhub-cli/v/1.0.5)**,提供已签名的 macOS 桌面应用及经过验证的 macOS/Linux Runtime!
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-10]**: 🚀 **正式发布** **Hy Memory 2.0 (`hy-memory==2.0.4`)**(已上线 [PyPI](https://pypi.org/project/hy-memory/2.0.4/)),并开源 **Hy-Memory** monorepo 与 **[MemHub](memhub/README.md)**!
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-10]**: 🚀 **模型开源** 在 Hugging Face 发布 **[Hy-Memory-30B-A3B](https://huggingface.co/tencent/Hy-Memory-30B-A3B)** 长期记忆写入模型及[技术报告](https://arxiv.org/abs/2610.22196)!
@@ -97,6 +98,11 @@ export OPENAI_API_KEY="sk-your-key"
完整配置见 [环境变量参考](docs/env_reference.md)。
提示词提供 11 种语言(`en`、`zh-Hans`、`zh-Hant`、`ja`、`es`、`de`、`pt`、`fr`、
`ko`、`ru`、`id`)。默认按输入自动在中文与英文之间选择;设置 `MEMORY_LANGUAGE`(或在
`add()` 上传 `language=`)可显式指定,记忆随之用该语言书写。详见
[提示词语言](docs/usage.md#提示词语言language)。
### 3. Chat 记忆
```python
+3
View File
@@ -9,6 +9,7 @@
|---|---|---|
| `MEMORY_MODE` | `pro` | `lite` / `pro` / `ultra` |
| `MEMORY_DATA_DIR` | `~/.hy_memory` | 本地数据、缓存和日志根目录 |
| `MEMORY_LANGUAGE` | — | LLM 提示词与记忆正文语言:`en` / `zh-Hans` / `zh-Hant` / `ja` / `es` / `de` / `pt` / `fr` / `ko` / `ru` / `id`(别名 `zh`、`zh-CN` → `zh-Hans`,`zh-TW`、`zh-HK` → `zh-Hant`,`pt-BR` → `pt`)。未设置时按输入做 zh/en 自动检测;per-call `language=` 优先 |
- `lite`:纯 Embedding 写入,Graph 强制关闭。
- `pro`:LLM 提取与 reconcile;Graph 由 `MEMORY_ENABLE_GRAPH` 控制。
@@ -48,6 +49,7 @@
| `MEMORY_EMBEDDER_TIMEOUT` | `15` | 单次超时,秒 |
| `MEMORY_EMBEDDER_MAX_RETRIES` | `2` | 最大重试次数 |
| `MEMORY_EMBEDDER_RETRY_DELAY` | `0.3` | 重试间隔,秒 |
| `MEMORY_EMBEDDER_BATCH_SIZE` | `64` | `embed_batch` 单次请求最多发送的文本条数,超出按此切块 |
| `MEMORY_EMBEDDER_EXTRA_HEADERS` | — | JSON object,附加 HTTP headers |
| `MEMORY_EMBEDDER_EXTRA_BODY` | — | JSON object,附加请求 body |
@@ -191,6 +193,7 @@ SDK 不设置 Rerank Search 并发上限。开启后模型在 `HyMemoryClient`
| `MEMORY_LOG_FORMAT` | `text` | `text` / `json` |
| `MEMORY_LOG_DIR` | — | 统一日志根目录 |
| `MEMORY_TRACE_LOG_DIR` | — | JSONL 日志目录的兼容旧名;优先使用 `MEMORY_LOG_DIR` |
| `MEMORY_PIPELINE_LOG_ENABLED` | `true` | 是否写 pipeline JSONL 日志(`{MEMORY_LOG_DIR}/<subdir>/<date>.log`);`false` 完全不写 |
| `MEMORY_PIPELINE_TRACE_ENABLED` | `true` | 是否写入 cache 的 pipeline trace 表 |
| `MEMORY_MEMORY_OPERATIONS_ENABLED` | `false` | 是否写入 memory operation 审计记录 |
| `MEMORY_HISTORY_ENABLE` | `true` | 是否启用历史记录 |
+49
View File
@@ -103,6 +103,7 @@ result = client.add(
metadata=None,
meta=None,
embed_override=None,
language=None,
)
```
@@ -136,6 +137,54 @@ LLM。pro / ultra 仍遵循各自的层级与场景参数,默认不公开原
正交:前者说明“从哪个产品/入口写入”,后者仍是记忆隔离中的 agent 身份。source 会
执行 `strip().lower()` 后存储,并随 chat、working、coding 记忆一起返回。
### 提示词语言(`language`)
SDK 的 LLM 提示词提供 11 种语言,按语言码注册在 `hy_memory/prompts/`
(每种语言一个模块,`get_prompt(name, language)` 统一取用):
| 语言码 | 语言 | 别名 |
|---|---|---|
| `en` | English | `en-US`、`en-GB` |
| `zh-Hans` | 简体中文 | `zh`、`zh-CN`、`zh-SG` |
| `zh-Hant` | 繁體中文 | `zh-TW`、`zh-HK`、`zh-MO` |
| `ja` | 日本語 | |
| `es` | Español | |
| `de` | Deutsch | |
| `pt` | Português | `pt-BR`、`pt-PT` |
| `fr` | Français | |
| `ko` | 한국어 | |
| `ru` | Русский | |
| `id` | Bahasa Indonesia | |
语言码大小写不敏感,`_` 与 `-` 等价;未知语言码抛 `ValueError` 并列出支持的语言码。
```python
# client 默认值(也可用环境变量 MEMORY_LANGUAGE=ja)
client = HyMemoryClient.from_config({"mode": "pro", "language": "ja"})
# per-call 覆盖 client 默认值
client.add(messages, user_id="u1", language="de")
client.add_extracted(["Der Nutzer trinkt keinen Kaffee."], user_id="u1", language="de")
client.finalize_session(user_id="u1", session_id="s1", language="de")
client.evolve(user_id="u1", language="de") # ultra
client.sweep(user_ids=["u1"], language="de") # ultra
```
优先级:per-call `language` > `MemoryConfig.language`(`MEMORY_LANGUAGE`)> 自动检测。
- **未指定语言(默认)**:行为与以前完全一致。SDK 按输入做 zh/en 自动检测,中文输入用
简体中文提示词,其余用英文提示词;自动检测不会选到其他语言。提示词里的「输出语言与
输入语言一致」规则仍然生效,所以记忆正文保持对话的原始语言。
- **显式指定语言**:本次请求的全部 LLM 调用(抽取、基础画像、reconcile、working /
coding 链路的抽取与两个 reconciler 及演变总结、episode / topic 摘要、System 2)都使用
该语言的提示词,并在提示词末尾追加该语言的「输出语言」规则:记忆正文、摘要等自然语言
字段一律用该语言书写,即使对话是另一种语言。JSON 键、字段名、枚举值、代码、路径、
报错信息和专有名词保持原样;tags 仍是英文小写关键词。
- 只有英文版的提示词(session / dynamic episode、cross-domain sweeper)在显式指定语言时
同样追加该输出规则;搜索侧的场景判断 / query rewrite 只有英文版且保持 query 原语言,
因此 `search()` 不接受 `language`。
- 自定义 `extract_system_prompt_zh` / `extract_system_prompt_en` 只覆盖 `zh-Hans` / `en`。
### `search()` / `async_search()`
普通 chat 检索:
+53
View File
@@ -55,6 +55,33 @@ except Exception:
from .client import HyMemoryClient
from .inspector import MemoryInspector
from .runtime import SharedRuntime
from .reembed import (
EmbeddingSpaceIdentity,
GraphReembedPlan,
GraphReembedProgress,
GraphReembedReport,
GraphVerification,
ReembedCancelled,
ReembedPlan,
ReembedProgress,
ReembedReport,
ReembedVerification,
ScannedSpace,
SpaceDiff,
VectorSpace,
diff_spaces,
drop_graph,
drop_vector_space,
plan_reembed,
reembed_graph,
reembed_text,
reembed_vector_space,
scan_vector_spaces,
space_identity,
vector_space_names,
verify_graph_reembed,
verify_reembed,
)
from .snapshot import (
DataPlaneRestore,
DataPlaneSnapshot,
@@ -129,6 +156,32 @@ __all__ = [
"DataPlaneSnapshotRejected",
"restore_local_data_plane",
"snapshot_local_data_plane",
# 换 embedding 模型:把向量库和图重新 embed 进新的 embedding 空间
"EmbeddingSpaceIdentity",
"GraphReembedPlan",
"GraphReembedProgress",
"GraphReembedReport",
"GraphVerification",
"ReembedCancelled",
"ReembedPlan",
"ReembedProgress",
"ReembedReport",
"ReembedVerification",
"ScannedSpace",
"SpaceDiff",
"VectorSpace",
"diff_spaces",
"drop_graph",
"drop_vector_space",
"plan_reembed",
"reembed_graph",
"reembed_text",
"reembed_vector_space",
"scan_vector_spaces",
"space_identity",
"vector_space_names",
"verify_graph_reembed",
"verify_reembed",
# 配置
"MemoryConfig",
# 公共扩展协议 / 精确读取类型
+38 -57
View File
@@ -43,6 +43,12 @@ from datetime import datetime
from typing import Any, Dict, List, Optional
import uuid
from ..prompts import (
get_prompt,
normalize_language,
resolve_language,
with_output_language,
)
from ..utils.source_filter import merge_sources
logger = logging.getLogger(__name__)
@@ -115,56 +121,9 @@ Messages:
"""
EPISODE_SUMMARY_PROMPT = """You are summarizing ONE episode of conversation with the user.
An episode is a single stretch of discussion — one thing that was talked about,
possibly across several sessions of writing.
## Memories captured during this episode (oldest → newest)
{memories}
---
Write a single dense summary of what happened / what was discussed in this episode.
Rules:
* 2–4 sentences, third person, starting with "The user ..." (or the named person
if the memories are about someone else).
* Tie the memories together into one coherent account — an episode usually has a
thread running through it (a plan, a problem, an outing, a decision).
* Preserve concrete details: names, places, numbers, dates.
* Do NOT invent anything that is not in the memories.
* Do NOT enumerate the memories one by one; synthesize them.
* Output language must match the language of the memories.
Return ONLY the summary text, no preamble, no JSON, no quotes."""
EPISODE_SUMMARY_PROMPT_ZH = """你在为**一段讨论**(一个 episode)做总结。
一个 episode 是一段连续的讨论 —— 围绕一件事,可能跨越好几次写入。
## 这段讨论中抽取到的记忆(时间正序)
{memories}
---
写一段关于这次讨论内容的高密度摘要。
要求:
* 2–4 句,第三人称,以"用户…"开头(若记忆讲的是别人,则以那个人的名字开头)。
* 把这些记忆串成一个连贯的叙述 —— 一段讨论通常有一根主线(一个计划、一个问题、
一次出行、一个决定)。
* 保留具体细节:人名、地点、数字、日期。
* 绝不编造记忆里没有的内容。
* 不要逐条罗列记忆,要综合成一段。
* 输出语言与记忆的语言一致。
只返回摘要正文,不要任何前言、不要 JSON、不要引号。"""
# 多语言版本见 hy_memory/prompts/<language>.py;旧名字保留为别名。
from ..prompts.en import EPISODE_SUMMARY_PROMPT # noqa: E402,F401 (re-export)
from ..prompts.zh_hans import EPISODE_SUMMARY_PROMPT as EPISODE_SUMMARY_PROMPT_ZH # noqa: E402,F401 (re-export)
SESSION_EPISODE_PROMPT = """Convert this complete conversation session into one retrieval-friendly episode narrative.
@@ -596,9 +555,13 @@ def _assign_facts_to_segments(facts: List[Any], segments: List[dict]) -> Dict[in
async def finalize_dynamic_episode_capsules(
*, vector_store, embed_service, llm_provider, user_id: str,
agent_id: Optional[str], session_id: str, summary_concurrency: int = 3,
language: Optional[str] = None,
) -> dict:
"""Split a completed session, create compact L3 capsules, and link its facts.
The capsule prompt is English-only; an explicit ``language`` appends that
language's output rule so the capsule text is written in it.
Boundary and summary generation are separate calls. Boundary failure degrades
to one whole-session capsule. A segmented-summary failure also retries the
session as one capsule so the write never loses the complete event context.
@@ -802,6 +765,7 @@ async def finalize_dynamic_episode_capsules(
start = end + 1
semaphore = asyncio.Semaphore(max(1, min(16, int(summary_concurrency or 1))))
explicit_language = normalize_language(language)
async def summarize(spec):
segment_index, segment_start, segment_end, segment = spec
@@ -809,9 +773,13 @@ async def finalize_dynamic_episode_capsules(
try:
async with semaphore:
response = await llm_provider.complete(
prompt=DYNAMIC_CAPSULE_PROMPT.format(
timestamp=timestamp_text,
messages=_render_messages(segment),
prompt=with_output_language(
DYNAMIC_CAPSULE_PROMPT.format(
timestamp=timestamp_text,
messages=_render_messages(segment),
),
explicit_language,
explicit=explicit_language is not None,
),
max_tokens=4096,
)
@@ -1128,9 +1096,13 @@ def _cosine(a: List[float], b: List[float]) -> float:
async def finalize_session_episode(
*, vector_store, embed_service, llm_provider, user_id: str,
agent_id: Optional[str], session_id: str,
language: Optional[str] = None,
) -> dict:
"""Create one first-class narrative from all raw chunks in a caller session.
The prompt is English-only; an explicit ``language`` appends that
language's output rule so the narrative is written in it.
This is deliberately explicit: the SDK cannot infer that the final write of a
session has arrived. Repeated calls upsert the same deterministic node id.
"""
@@ -1213,6 +1185,10 @@ async def finalize_session_episode(
timestamp=timestamp_text,
conversation=conversation,
)
explicit_language = normalize_language(language)
prompt = with_output_language(
prompt, explicit_language, explicit=explicit_language is not None,
)
response = await llm_provider.complete(prompt=prompt, max_tokens=3000)
narrative = (getattr(response, "content", "") or "").strip()
if not narrative:
@@ -1274,9 +1250,12 @@ async def summarize_closed_episode(
max_memories: int = 60,
temperature: Optional[float] = None,
trace=None,
language: Optional[str] = None,
) -> Optional[str]:
"""总结一个**刚刚关闭**的 episode,返回摘要节点 id(跳过则 None)。
language 为显式 prompt 语言码;None 时按记忆内容做 zh/en 自动检测。
由 writer 在检测到 episode 滚动时 fire-and-forget 调用:此时上一个 episode
不会再有新内容进来,所以这一次总结就是最终产物,不需要后续 merge / 重写。
"""
@@ -1317,6 +1296,7 @@ async def summarize_closed_episode(
_t0 = _dt.now()
summary, prompt, tokens = await _generate(
llm_provider=llm_provider, memories=contents, temperature=temperature,
language=language,
)
_elapsed = (_dt.now() - _t0).total_seconds() * 1000
if trace is not None:
@@ -1351,15 +1331,16 @@ async def summarize_closed_episode(
async def _generate(
*, llm_provider, memories: List[str], temperature: Optional[float] = None,
language: Optional[str] = None,
) -> tuple:
"""返回 (摘要文本, 实际 prompt, token 用量) —— 后两个供埋点,线上要能复盘。"""
from ..utils.lang_detect import is_chinese
body = "\n".join(f"- {c}" for c in memories if c)
if not body:
return "", "", {}
tpl = EPISODE_SUMMARY_PROMPT_ZH if is_chinese(body) else EPISODE_SUMMARY_PROMPT
prompt = tpl.format(memories=body)
explicit = normalize_language(language)
code = resolve_language(explicit, body)
prompt = get_prompt("EPISODE_SUMMARY_PROMPT", code).format(memories=body)
prompt = with_output_language(prompt, code, explicit=explicit is not None)
resp = await llm_provider.complete(
prompt=prompt,
max_tokens=400,
+73 -816
View File
@@ -20,6 +20,12 @@ import re
from .llm_provider import LLMProvider, _content_fingerprint, _log_llm_content_enabled
from ..config import LLMConfig as GlobalLLMConfig
from ..prompts import (
get_prompt,
normalize_language,
resolve_language,
with_output_language,
)
logger = logging.getLogger(__name__)
@@ -164,671 +170,13 @@ class V2ExtractResult:
# - 不再输出 time_ref 字段。时间相关信息如果是时间敏感的(schedule / past event),
# 让 LLM 把已解析的时间自然嵌入 content 即可;稳定偏好类 content 保持不含时间。
# System prompt(固定,不参与 format)
EXTRACT_SYSTEM_PROMPT = """# ROLE
You are a Memory Extractor.
Extract durable, high-value memories from conversations and convert them into structured, self-contained statements.
Only extract information that is likely to matter in future conversations.
---
# INPUTS
You will receive:
* Last k Messages
* New Messages
* Memory Date
* Current Date
Use:
* Last k Messages only for coreference resolution
* Memory Date as the ONLY temporal anchor
* Current Date only as metadata, never for resolving relative time
---
# WHAT TO EXTRACT
Extract ONLY genuinely new information introduced in the NEW user messages.
Good memories include:
* preferences
* long-term projects
* habits
* goals
* recurring activities
* opinions
* relationships
* important experiences
* stable traits
* future plans
* decisions
* constraints
* accepted recommendations
Do NOT extract:
* small talk
* acknowledgements
* generic emotions
* temporary conversational context
* assistant reminders of old memories
* information already implied by prior context
* vague personality judgments
---
# SUBJECT
Every memory must be self-contained, written in third person, with an explicit
subject. Which subject to use is decided by **who spoke**:
| Speaker | Subject to write | Example |
| --- | --- | --- |
| `role: "user"` | the user | "The user ..." / "用户..." |
| any speaker with a `name` other than the user (e.g. `name: "John"`) | that person, by name | "John ..." |
| `role: "assistant"` with NO `name` | the AI assistant | "The assistant ..." |
The two input sections express the speaker differently:
* **New Messages** is a JSON array — read `role` and the optional `name` field.
* **Last k Messages** is rendered text — the speaker is the bracket label,
`[user]:` / `[John]:` / `[assistant]:`, optionally preceded by one line
"The user's name is Tim" / "用户名叫Tim".
Rules:
* The user is the person these memories belong to. Even when the user's own name
is known (from `name` on their message, or from the leading name line), still
write "The user ...", NEVER "Tim ...".
* **`role: "assistant"` WITH a `name` is another real person, not the AI.**
Write their memories with their name as the subject: "John ...".
* Only a nameless `role: "assistant"` is the AI assistant.
* Never invent a name for the AI assistant, it has none.
---
# LANGUAGE RULES
* Output language must match input language.
* Preserve original scripts and technical terms exactly.
* Never transliterate names, brands, or equations.
---
# TWO KINDS OF MEMORY
You extract two kinds of memory: **memory** and **intentions**.
## memory
Everything durable and worth remembering about the user — and about any other
named person in the conversation — goes here:
* preferences, opinions, values, attitudes
* events, actions, experiences
* habits, behavioral patterns
* relationships, traits
* objective facts, decisions, constraints
* ongoing activities
Implicit preferences MUST be inferred when strongly suggested
(e.g. resuming an activity → positive interest; avoiding something →
negative preference; repeated voluntary effort → sustained interest).
Do NOT separate "what kind of person the user is" from "what happened".
Both go into `memory`.
## intentions
A forward-looking thing the user **plans or intends to do in the future**,
which can be acted on and which becomes irrelevant once done or past.
Examples:
* "The user has a job interview next Tuesday."
* "The user plans to move apartments at the end of the month."
* "The user intends to finish the report before Friday."
NOT intentions (these belong in `memory`):
* stable preferences / traits ("The user likes basketball")
* completed past events ("The user moved to Beijing last year")
* general goals with no actionable future step
### valid_until
For each intention, provide `valid_until`: the date after which the
intention no longer matters (the deadline / target date), resolved
against **Memory Date** (e.g. "next Tuesday" → that absolute date).
If no deadline can be determined, set `valid_until` to `null`.
---
# OWNER
Every memory and intention MUST carry an `owner` saying which memory store it
belongs to. There are exactly two values:
* `"user"` — anything belonging to the user's own memory store. This includes
facts/preferences/traits about the user themselves, AND facts about other
named real people in the conversation, because those are still things the
user's memory should hold.
* `"agent"` — information the AI assistant (a nameless `role: "assistant"`)
provides (recommendations, confirmations, things it researched), or a future
action the user wants the **assistant** to perform ("remind me in 8 hours").
A named person is NEVER `owner: "agent"` — they are a real person, not the AI,
even though their messages carry `role: "assistant"`. Their memories are
`owner: "user"` with their name as the content subject.
Decide `owner` per item. When unsure, default to `"user"`.
---
# TEMPORAL RULES
Resolve relative time using Memory Date.
Examples:
* "next week"
* "last month"
* "tomorrow"
If time cannot be resolved safely:
* remove the relative wording
* keep the memory atemporal
Do not invent chronology.
**Just-happened events are events — keep them.** When the user reports something as
just completed ("just got back", "just finished", "this morning", "yesterday"), the
completion date is the Memory Date. Emit the **event itself** with that date; do not
fold it into a background statement.
* "I just got back from a trip with my family"
→ "The user returned from a family trip on <Memory Date>."
NOT → "The user takes family trips." (the return date is the answer to "when did
they get back", and dropping it makes the question unanswerable)
Where an event has both a start and an end that the conversation gives separately,
record both rather than collapsing them into one span.
---
# QUALITY RULES
Each memory must be:
* self-contained
* contextually complete
* retrieval-friendly
* information-dense
Retain important details:
* names
* products
* locations
* numbers
* equations
* constraints
Never generalise a specific into a category. Write what was actually said.
* "Banff" → keep "Banff", NOT "the Rocky Mountains"
* "a 3-year-old border collie named Luna" → keep the breed, age and name
**Specifics supplied by the other speaker still count.** A fact is often built across
turns: the other speaker names the thing, the user confirms it. The subject rules
decide *whose* memory it is — they do not license dropping detail.
* [Sam]: "Did you take advantage of the skiing opportunities in Banff?"
[user]: "Yeah, it was great for skiing!"
→ "The user went skiing in Banff and found the snow amazing."
NOT → "The user enjoys winter sports." (both the place and the activity were
established, one by each speaker)
Avoid fragmentary memories lacking context.
Choose the smallest number of memories that still preserves distinct future
retrieval needs.
* Compression means combining, not omitting. Every explicit durable proposition
in the new messages must appear exactly once in the output.
* Combine details when they share the same subject and the same real-world event,
state, preference, plan, or decision. A coherent event may use 1–3 sentences.
* Do not emit both a broad statement and narrower restatements of the same evidence.
* Split only when the subject differs, the events happened independently, or the
memories describe different states.
* Keep a cause, recommendation, or consequence with the fact it explains when
separating them would lose the relationship.
* Never merge separate dated occurrences merely because they share a topic.
* Preserve the source's time precision. Do not turn vague phrases such as "last
weekend" or "last summer" into exact calendar dates unless the date is explicit
or uniquely determined; "the weekend before <Memory Date>" is acceptable.
---
# TAGS
Each memory must include 1–3 concise tags.
- Topical keywords stay lowercase (travel, health, work).
- Proper nouns keep capitalization (Jasper, Canada, Banff, Prius).
Examples:
* work
* travel
* Jasper
* Canada
* health
---
"""
STRUCTURE_METADATA_PROMPT_EN = """
---
# STRUCTURAL ORGANIZATION
For every `memory`, also output:
* `topic_paths`: a backward-compatible list of zero to three broad-to-specific
semantic paths.
* `event_facet`: an object with only `subject`, `participants`, `action`,
`objects`, `locations`, and `state`; state is `planned`, `ongoing`,
`completed`, or `unknown`.
* `topic_assignments`: zero to six subject-specific memberships. Each item is
`{subject, role, path}`, where `path` is a list of `{name, type}` segments and
`type` is one of `topic`, `facet`, `series`, or `episode`. Use `topic` for a
broad domain, `facet` for a concrete activity/semantic aspect, `series` only
for repeated occurrences of the same concrete activity/state line, and
`episode` only for one bounded real-world occurrence. A shared event may have
one assignment per explicit participant while keeping one canonical memory.
Types follow `topic -> facet -> series -> episode`; levels may be skipped but
types may not repeat or move backward.
This extractor sees only the current write and therefore must not emit a
`series` segment. Emit Topic/Facet/Episode here. Reconcile may later establish a
Series after comparing independent stored occurrences.
This is write-time organization. Do not infer Series/Episode from path depth,
manufacture hierarchy from a proper noun or broad tag, or assign a person only
because they co-occur in unrelated context. Stop at the most specific supported
level and use empty values when uncertain. Never encode query forms such as
"when", "how many", "who", plurality, or ordinal wording.
"""
# Few-shot examples (appended to the system prompt only when few_shot_enabled).
# Each example mirrors the real input shape (Last k Messages / New Messages /
# Memory Date) and our output schema (memory / intentions / basic_info with
# owner & tags), and ends with a one-line takeaway.
EXTRACT_FEW_SHOT_EN = """
---
# EXAMPLES
## Example 1 — Boundary: old context is for resolution only, never re-extracted
Last k Messages:
[{"role": "user", "content": "I've been looking at rescue dogs at the shelter downtown."},
{"role": "assistant", "content": "That's lovely — found one you like?"}]
New Messages:
[{"role": "user", "content": "Yeah, picked her up yesterday — a 3-year-old border collie named Luna."}]
Memory Date: 2025-04-10
Output:
```json
{
"memory": [
{"content": "The user adopted a 3-year-old border collie named Luna from a downtown shelter on 2025-04-09.", "tags": ["pets", "family"], "owner": "user"}
],
"intentions": [],
"basic_info": {}
}
```
The old messages only resolve "her" and supply context ("downtown shelter"). Do NOT emit a stale memory like "the user is considering adopting a dog" — only the genuinely new fact (the completed adoption) is extracted, with "yesterday" resolved against Memory Date.
## Example 2 — Condense a long assistant reply
New Messages:
[{"role": "user", "content": "Write me a short essay on why remote work boosts productivity."},
{"role": "assistant", "content": "Sure — opening: 'Remote work has reshaped how teams operate...' [~2000 words of body omitted] ...closing: 'In short, autonomy and fewer interruptions drive higher output.'"}]
Memory Date: 2025-03-22
Output:
```json
{
"memory": [
{"content": "The assistant drafted a short essay for the user arguing that remote work boosts productivity because autonomy and fewer interruptions drive higher output.", "tags": ["writing", "remote-work"], "owner": "agent"}
],
"intentions": [],
"basic_info": {}
}
```
The assistant's reply is long. Never store the verbatim essay — distill it to topic + thesis. owner is "agent" because the assistant produced it. Bloated verbatim content makes memory unusable.
## Example 3 — Composite extraction: basic_info + intention + condensed agent plan
New Messages:
[{"role": "user", "content": "Hi, I'm Kenji. I'm planning a trip to Japan next month — could you put together a 7-day itinerary?"},
{"role": "assistant", "content": "# 7-Day Classic Japan Golden Route (Tokyo → Kyoto → Nara → Osaka)\\nFly into Tokyo, Shinkansen south to Kyoto, day-trip Nara, depart from Osaka (Kansai).\\n\\n## Day 1: Arrival in Tokyo\\n- Land at Haneda/Narita, drop bags at hotel near Tokyo Station.\\n- Late afternoon: Sensoji Temple (Asakusa), wander Nakamise Street, try taiyaki.\\n- Evening: Shibuya Crossing + Shibuya Sky rooftop for sunset; izakaya dinner (yakitori, sake).\\n\\n## Day 2: Tokyo markets & landmarks — [omitted]\\n## Day 3: Shinkansen to Kyoto, Fushimi Inari, Gion — [omitted]\\n## Day 4: Kyoto temples (Kiyomizu-dera, Kinkaku-ji, Ginkaku-ji) — [omitted]\\n## Day 5: Nara day trip (Nara Park deer, Todai-ji) — [omitted]\\n## Day 6: Osaka (Osaka Castle, Dotonbori street food) — [omitted]\\n## Day 7: Osaka sightseeing + depart from Kansai Airport — [omitted]\\n\\nTip: buy a 7-day JR Pass before arriving. Want me to adjust anything?"}]
Memory Date: 2025-09-15
Output:
```json
{
"memory": [
{"content": "The assistant created a 7-day Japan itinerary for the user along the classic golden route: Tokyo (Asakusa, Shibuya), Shinkansen to Kyoto (Fushimi Inari, Kiyomizu-dera, Kinkaku-ji, Gion), a Nara day trip (deer park, Todai-ji), then Osaka (Osaka Castle, Dotonbori), departing from Kansai Airport.", "tags": ["travel", "japan", "itinerary"], "owner": "agent"}
],
"intentions": [
{"content": "The user plans to take a 7-day trip to Japan in October 2025.", "tags": ["travel", "japan"], "valid_until": "2025-10-31", "owner": "user"}
],
"basic_info": {"name": "Kenji"}
}
```
One turn produces three outputs in different buckets: the name goes to basic_info, the trip is a forward-looking intention with a deadline, and the assistant's long multi-section itinerary is condensed into ONE dense agent memory — the route and key stops per city, NOT every day's bullets, tips, or food lists. Storing the whole reply verbatim would bloat memory; storing the skeleton keeps it queryable.
## Example 4 — A later message confirming/adopting an earlier recommendation is itself a fact
New Messages:
[{"role": "user", "content": "My 6-year-old won't eat any vegetables. Any ideas?"},
{"role": "assistant", "content": "Try blending spinach or carrots into pasta sauce — the color and flavor mostly disappear, and many kids accept it without noticing."},
{"role": "user", "content": "Oh nice, we'll try that tonight. By the way, what's a good bedtime for that age?"},
{"role": "assistant", "content": "For a 6-year-old, aim for 7–9 PM with about 10–11 hours of sleep."}]
Memory Date: 2025-06-20
Output:
```json
{
"memory": [
{"content": "The user has a 6-year-old child who refuses to eat vegetables, and accepted the assistant's suggestion to blend spinach or carrots into pasta sauce to hide them.", "tags": ["parenting", "food"], "owner": "user"},
{"content": "The assistant advised the user that a 6-year-old should aim for a 7–9 PM bedtime with about 10–11 hours of sleep.", "tags": ["parenting", "sleep"], "owner": "agent"}
],
"intentions": [],
"basic_info": {}
}
```
In the third message the user said "we'll try that tonight" — explicitly adopting the earlier suggestion. That acceptance is valuable and must be recorded ("accepted the assistant's suggestion to ..."), not discarded as small talk. The bedtime advice was just given and not yet adopted, so it is recorded as advice (owner "agent").
## Example 5 — RAG/injected context: never store the injected text
New Messages:
[{"role": "user", "content": "[Retrieved context: The Eiffel Tower is 330m tall and was completed in 1889.] Given the above, I'm going to propose to my girlfriend at the top of the Eiffel Tower this summer."}]
Memory Date: 2025-05-02
Output:
```json
{
"memory": [],
"intentions": [
{"content": "The user plans to propose to his girlfriend at the top of the Eiffel Tower in summer 2025.", "tags": ["relationship", "travel"], "valid_until": "2025-08-31", "owner": "user"}
],
"basic_info": {}
}
```
The bracketed retrieved context (the tower's height and year) is RAG-injected reference material, NOT something the user told us about themselves — do NOT store it. Only the user's own forward-looking plan is extracted. System/RAG-injected text is context for the model, never a memory.
## Example 6 — Shared reference material: extract the content, not the sharing action
New Messages:
[{"role": "user", "content": "Remember my grandma's dumpling recipe so I don't lose it. Filling: 500g ground pork, 300g napa cabbage, 2 tbsp soy sauce, 1 tbsp sesame oil, a thumb of ginger minced. Cabbage salted 15 min then squeezed dry. Boil dumplings, add cold water twice, ready when they float."},
{"role": "assistant", "content": "Got it, saved!"}]
Memory Date: 2025-07-01
Output:
```json
{
"memory": [
{"content": "The user's grandmother's dumpling recipe: filling of 500g ground pork, 300g napa cabbage (salted 15 minutes then squeezed dry), 2 tbsp soy sauce, 1 tbsp sesame oil, and a thumb of minced ginger; boil the dumplings, add cold water twice, and they are done when they float.", "tags": ["food", "recipe", "family"], "owner": "user"}
],
"intentions": [],
"basic_info": {}
}
```
Extract the actual factual content (ingredients, quantities, the salt-and-squeeze step, the boiling method) — NOT "the user shared a recipe" or "the user asked to remember the recipe". The sharing action is worthless; the content is what's queryable.
## Example 7 — Structured numbers and attributes preserved exactly
New Messages:
[{"role": "user", "content": "Logging my home-gym PRs this cycle: Squat 140kg x 3, Bench 100kg x 5, Deadlift 180kg x 2, Overhead Press 60kg x 6. Bodyweight 78kg, 3 sets per lift, 3 min rest."},
{"role": "assistant", "content": "Logged — strong numbers!"}]
Memory Date: 2026-01-20
Output:
```json
{
"memory": [
{"content": "The user's current lifting PRs are Squat 140kg x 3, Bench 100kg x 5, Deadlift 180kg x 2, and Overhead Press 60kg x 6, at a bodyweight of 78kg.", "tags": ["fitness", "strength"], "owner": "user"},
{"content": "The user trains with 3 sets per lift and 3 minutes of rest between sets.", "tags": ["fitness", "training"], "owner": "user"}
],
"intentions": [],
"basic_info": {}
}
```
Every number and unit (kg, reps, sets, rest time) is preserved exactly — dropping them would destroy the most queryable information. Split into two clean topics (PRs vs. training protocol), but never round or omit the specifics.
## Example 8 — Long multi-topic conversation: do not stop after the first topic
New Messages:
[{"role": "user", "content": "Big week — I finally defended my PhD thesis on Tuesday!"},
{"role": "assistant", "content": "Huge congrats! How do you feel?"},
{"role": "user", "content": "Relieved! I also signed a lease on an apartment in Berlin and move next month. And I've started learning German on Duolingo — 30-day streak. My partner Sofia is staying in Madrid for now, which is hard."}]
Memory Date: 2025-11-08
Output:
```json
{
"memory": [
{"content": "The user defended their PhD thesis on 2025-11-04.", "tags": ["education", "milestone"], "owner": "user"},
{"content": "The user signed a lease on an apartment in Berlin.", "tags": ["housing"], "owner": "user"},
{"content": "The user has been learning German on Duolingo and has a 30-day streak.", "tags": ["language", "learning"], "owner": "user"},
{"content": "The user's partner Sofia is staying in Madrid for now, which the user finds hard.", "tags": ["relationship"], "owner": "user"}
],
"intentions": [
{"content": "The user plans to move to Berlin next month (around December 2025).", "tags": ["housing", "relocation"], "valid_until": "2025-12-31", "owner": "user"}
],
"basic_info": {}
}
```
Four+ topics — thesis defense, the Berlin apartment, learning German, and the partner in Madrid. Do not stop after the first topic. Note one turn split into a completed fact (lease signed) AND an intention (the upcoming move), and the emotional-but-substantive detail about Sofia is kept because it carries lasting relationship context.
## Example 9 — Two real people: `name` on an assistant message means a real person
New Messages:
[{"role": "user", "name": "Tim", "content": "Just got back from the climbing gym, third time this week. Finally sent that overhang route."},
{"role": "assistant", "name": "John", "content": "Nice! I've been swimming instead — my shoulder still hurts from last month."}]
Memory Date: 2025-08-14
Output:
```json
{
"memory": [
{"content": "The user has been climbing three times a week and recently completed a difficult overhang route.", "tags": ["fitness", "climbing"], "owner": "user"},
{"content": "John has been swimming instead of other exercise because his shoulder still hurt after an injury around July 2025.", "tags": ["fitness", "health"], "owner": "user"}
],
"intentions": [],
"basic_info": {"name": "Tim"}
}
```
John carries `role: "assistant"` but he has a `name`, so he is another real person, NOT the AI: his memory uses "John ..." as the subject and is still `owner: "user"` (it belongs to the user's memory store). The user's own memory stays "The user ..." even though we know the name is Tim — never "Tim has been climbing". `basic_info.name` comes from `name: "Tim"` on the user message, which is authoritative even though Tim never said his own name out loud.
## Example 10 — Counter-example: another person's attributes never go into basic_info
New Messages:
[{"role": "user", "name": "Tim", "content": "你最近在忙什么?"},
{"role": "assistant", "name": "John", "content": "我在柏林一家医院当护士,上的是夜班,挺累的。"}]
Memory Date: 2025-09-02
Output:
```json
{
"memory": [
{"content": "John是柏林一家医院的护士,上夜班,觉得很累。", "tags": ["work", "health"], "owner": "user"}
],
"intentions": [],
"basic_info": {"name": "Tim"}
}
```
John's occupation and location are HIS attributes, so they go into `memory` with "John" as the subject. They must NEVER land in `basic_info.occupation` / `basic_info.location` — `basic_info` is exclusively the user's own profile. Only `name` (Tim, from the user message's `name` field) belongs there.
---
"""
EXTRACT_PROMPT = """## Last k Messages
{last_messages}
---
## New Messages
{content}
---
## Memory Date
{memory_at}
---
## Current Date
{current_date}
---
# Output Format
Return ONLY a JSON object inside a ```json fenced block.
```json
{{
"memory": [
{{
"content": "...",
"tags": ["..."],
"owner": "user"
}}
],
"intentions": [
{{
"content": "...",
"tags": ["..."],
"valid_until": "YYYY-MM-DD",
"owner": "user"
}}
],
"basic_info": {{}}
}}
```
Rules:
* `memory` = durable info worth remembering: preferences, opinions, attitudes,
traits, events, plans, experiences, habits, decisions — about the user or
about any other named real person, with the subject written per "# SUBJECT"
* `intentions` = forward-looking things the user (or another named person) plans /
intends to do in the future, with `valid_until` = the deadline/target date
(resolved against Memory Date) or `null` if none
* `owner` (required on every memory & intention) = `"user"` or `"agent"`
(see "# OWNER" above; default `"user"` when unsure)
* `basic_info` = the **user's own profile only**. Include stable profile
fields (e.g. name, age, location, occupation, employer) **explicitly stated**
in the NEW user messages. Never infer, never invent. If no field is explicitly
stated, output `{{}}`.
* Exception for `name`: the `name` field on the user's own message, or a leading
line "The user's name is X" / "用户名叫X", is an authoritative source for
`name` — even though the user never said it out loud.
* NEVER put another real person's attributes here. If John (an assistant-role
message carrying `name`) says "I'm a nurse in Berlin", that is a `memory`
("John is a nurse in Berlin."), NOT `basic_info.occupation` / `.location`.
* No duplicate information across sections
* Tags must contain 1–3 concise lowercase keywords
* Use empty arrays `[]` or `{{}}` when nothing is extracted
* Return valid JSON only
Output the JSON now."""
# User prompt when MEMORY_ENABLE_INTENTION=false(无 intentions 桶)
EXTRACT_PROMPT_NO_INTENTION = """## Last k Messages
{last_messages}
---
## New Messages
{content}
---
## Memory Date
{memory_at}
---
## Current Date
{current_date}
---
# Output Format
Return ONLY a JSON object inside a ```json fenced block.
```json
{{
"memory": [
{{
"content": "...",
"tags": ["..."],
"owner": "user"
}}
],
"basic_info": {{}}
}}
```
Rules:
* `memory` = durable info worth remembering: preferences, opinions, attitudes,
traits, events, experiences, habits, decisions, **and future plans / upcoming
actions** — about the user or about any other named real person, with the
subject written per "# SUBJECT". Do **not** invent a separate intentions bucket.
* `owner` (required on every memory) = `"user"` or `"agent"`
(see "# OWNER" above; default `"user"` when unsure)
* `basic_info` = the **user's own profile only**. Include stable profile
fields (e.g. name, age, location, occupation, employer) **explicitly stated**
in the NEW user messages. Never infer, never invent. If no field is explicitly
stated, output `{{}}`.
* Exception for `name`: the `name` field on the user's own message, or a leading
line "The user's name is X" / "用户名叫X", is an authoritative source for
`name` — even though the user never said it out loud.
* NEVER put another real person's attributes here. If John (an assistant-role
message carrying `name`) says "I'm a nurse in Berlin", that is a `memory`
("John is a nurse in Berlin."), NOT `basic_info.occupation` / `.location`.
* No duplicate information across sections
* Tags: 1–3 concise keywords; topical tags lowercase; proper nouns keep capitalization
* Use empty arrays `[]` or `{{}}` when nothing is extracted
* Return valid JSON only — **never** include an `intentions` key
Output the JSON now."""
# Prompt 文本统一放在 hy_memory/prompts/<language>.py,按语言码经 get_prompt()
# 取用。以下旧名字保留为英文版的别名(向后兼容)。
from ..prompts.en import EXTRACT_SYSTEM_PROMPT # noqa: E402,F401 (re-export)
from ..prompts.en import STRUCTURE_METADATA_PROMPT as STRUCTURE_METADATA_PROMPT_EN # noqa: E402,F401 (re-export)
from ..prompts.en import EXTRACT_FEW_SHOT as EXTRACT_FEW_SHOT_EN # noqa: E402,F401 (re-export)
from ..prompts.en import EXTRACT_PROMPT # noqa: E402,F401 (re-export)
from ..prompts.en import EXTRACT_PROMPT_NO_INTENTION # noqa: E402,F401 (re-export)
# ================================================================
@@ -837,101 +185,26 @@ Output the JSON now."""
def _system_prompt_without_intention(system_prompt: str, *, lang: str) -> str:
"""从 system prompt 去掉 intentions 分桶,改为单一 memory(含未来计划)。"""
"""从 system prompt 去掉 intentions 分桶,改为单一 memory(含未来计划)。
``lang`` 是语言码(``en`` / ``zh-Hans`` / ``ja`` ...;旧值 ``zh`` 视作 zh-Hans)。
每种语言的替换片段在 ``hy_memory/prompts/<lang>.py`` 中定义。
"""
import re
if lang == "zh":
repl = (
"# 记忆\n"
"\n"
"你只提取一类记忆:**memory(记忆)**。\n"
"\n"
"## memory(记忆)\n"
"\n"
"一切值得长期记住的信息都放这里——既包括用户本人的,也包括对话中其他具名真人的:\n"
"\n"
"* 偏好、观点、态度、价值观\n"
"* 事件、行动、经历\n"
"* 习惯、行为模式\n"
"* 关系、稳定特质\n"
"* 客观事实、决策、约束条件\n"
"* 正在进行的活动\n"
"* **未来计划 / 即将发生的安排**(若值得记住,直接写进 `memory`;"
"不要单独开 intentions 桶)\n"
"\n"
"如果用户隐式表达偏好,也应提取。\n"
"\n"
"不要把\"用户是怎样的人\"和\"发生了什么\"分开。两者都放进 `memory`。\n"
"\n"
"---\n"
"\n"
)
system_prompt = re.sub(
r"# 两类记忆.*?(?=\n# OWNER)",
repl,
system_prompt,
count=1,
flags=re.S,
)
system_prompt = system_prompt.replace(
"每条 memory 和 intention 都必须带 `owner`",
"每条 memory 都必须带 `owner`",
)
system_prompt = system_prompt.replace(
"每条 memory 和 intention 都必须带",
"每条 memory 都必须带",
)
# few-shot 里若仍出现 intentions,用硬约束盖住
system_prompt += (
"\n\n---\n\n# INTENTION 关闭\n"
"当前配置关闭了 intention 抽取。即使示例里出现 `intentions`,"
"也**不要**输出该字段;未来计划写入 `memory`。\n"
)
else:
repl = (
"# MEMORY\n"
"\n"
"You extract one kind of memory: **memory**.\n"
"\n"
"## memory\n"
"\n"
"Everything durable and worth remembering about the user — and about any other\n"
"named person in the conversation — goes here:\n"
"\n"
"* preferences, opinions, values, attitudes\n"
"* events, actions, experiences\n"
"* habits, behavioral patterns\n"
"* relationships, traits\n"
"* objective facts, decisions, constraints\n"
"* ongoing activities\n"
"* **future plans / upcoming actions** that are worth remembering\n"
" (put them in `memory`; do NOT invent a separate intentions bucket)\n"
"\n"
"Implicit preferences MUST be inferred when strongly suggested.\n"
"\n"
"Do NOT separate \"what kind of person the user is\" from \"what happened\".\n"
"Both go into `memory`.\n"
"\n"
"---\n"
"\n"
)
system_prompt = re.sub(
r"# TWO KINDS OF MEMORY.*?(?=\n# OWNER\n)",
repl,
system_prompt,
count=1,
flags=re.S,
)
system_prompt = system_prompt.replace(
"Every memory and intention MUST carry an `owner`",
"Every memory MUST carry an `owner`",
)
system_prompt += (
"\n\n---\n\n# INTENTION DISABLED\n"
"Intention extraction is disabled. Even if few-shot examples show "
"`intentions`, do **not** emit that key; put future plans in `memory`.\n"
)
return system_prompt
language = "zh-Hans" if lang == "zh" else lang
repl = get_prompt("NO_INTENTION_MEMORY_SECTION", language)
system_prompt = re.sub(
get_prompt("NO_INTENTION_SECTION_PATTERN", language),
lambda _m: repl,
system_prompt,
count=1,
flags=re.S,
)
for old, new in get_prompt("NO_INTENTION_REPLACEMENTS", language):
system_prompt = system_prompt.replace(old, new)
# few-shot 里若仍出现 intentions,用硬约束盖住
return system_prompt + get_prompt("INTENTION_DISABLED_NOTE", language)
class Extractor:
@@ -968,6 +241,7 @@ class Extractor:
tool_context: Optional[Dict[str, Any]] = None,
history_context: str = "",
basic_profile_fields: Optional[Dict[str, str]] = None,
language: Optional[str] = None,
) -> ExtractResult:
"""
统一提取:identity / facts / basic_info
@@ -989,6 +263,9 @@ class Extractor:
**DEPRECATED**: 曾用于渲染 EXTRACT prompt 的基础画像 schema 段落,
现已移除该段落(basic_info 由内置 few-shot / 指令引导)。保留参数
仅为向后兼容,传入不再生效。
language: 显式 prompt 语言码(见 hy_memory.prompts.SUPPORTED_LANGUAGES,
接受别名)。None 时按 content 做 zh/en 自动检测。显式指定时
追加 OUTPUT_LANGUAGE_DIRECTIVE,记忆正文用该语言书写。
"""
MAX_TOOL_ROUNDS = 5
@@ -1015,73 +292,52 @@ class Extractor:
else:
_memory_at = _current_date
# Select prompt based on input language
from ..utils.lang_detect import is_chinese
# 选 prompt 语言:显式 language(per-call / MemoryConfig.language)优先;
# 未指定时沿用 zh/en 自动检测(行为与以前完全一致)。
_explicit_language = normalize_language(language)
_language = resolve_language(_explicit_language, content)
_enable_intention = bool(
getattr(self._llm_config, "enable_intention", True)
)
if is_chinese(content):
from .prompts_zh import (
EXTRACT_PROMPT_ZH,
EXTRACT_PROMPT_ZH_NO_INTENTION,
EXTRACT_SYSTEM_PROMPT_ZH,
EXTRACT_FEW_SHOT_ZH,
)
# client 自定义 system prompt 仅覆盖对应语言(zh → zh-Hans, en → en);
# 直接用,不追加内置 few-shot
_custom_sys = None
if _language == "zh-Hans":
_custom_sys = getattr(
self._llm_config, "extract_system_prompt_zh", None
)
if _custom_sys:
# client 自定义 system prompt:直接用,不追加内置 few-shot
system_prompt = _custom_sys
else:
system_prompt = EXTRACT_SYSTEM_PROMPT_ZH
if getattr(self._llm_config, "few_shot_enabled", False):
system_prompt = system_prompt + EXTRACT_FEW_SHOT_ZH
if not _enable_intention and not _custom_sys:
system_prompt = _system_prompt_without_intention(
system_prompt, lang="zh"
)
if getattr(self._llm_config, "structure_metadata_enabled", False):
from .prompts_zh import STRUCTURE_METADATA_PROMPT_ZH
system_prompt += STRUCTURE_METADATA_PROMPT_ZH
_user_tmpl = (
EXTRACT_PROMPT_ZH
if _enable_intention
else EXTRACT_PROMPT_ZH_NO_INTENTION
)
prompt = _user_tmpl.format(
content=content,
current_date=_current_date,
memory_at=_memory_at,
last_messages=_last_messages,
)
else:
elif _language == "en":
_custom_sys = getattr(
self._llm_config, "extract_system_prompt_en", None
)
if _custom_sys:
system_prompt = _custom_sys
else:
system_prompt = EXTRACT_SYSTEM_PROMPT
if getattr(self._llm_config, "few_shot_enabled", False):
system_prompt = system_prompt + EXTRACT_FEW_SHOT_EN
if not _enable_intention and not _custom_sys:
system_prompt = _system_prompt_without_intention(
system_prompt, lang="en"
if _custom_sys:
system_prompt = _custom_sys
else:
system_prompt = get_prompt("EXTRACT_SYSTEM_PROMPT", _language)
if getattr(self._llm_config, "few_shot_enabled", False):
system_prompt = system_prompt + get_prompt(
"EXTRACT_FEW_SHOT", _language
)
if getattr(self._llm_config, "structure_metadata_enabled", False):
system_prompt += STRUCTURE_METADATA_PROMPT_EN
_user_tmpl = (
EXTRACT_PROMPT if _enable_intention else EXTRACT_PROMPT_NO_INTENTION
)
prompt = _user_tmpl.format(
content=content,
current_date=_current_date,
memory_at=_memory_at,
last_messages=_last_messages,
if not _enable_intention and not _custom_sys:
system_prompt = _system_prompt_without_intention(
system_prompt, lang=_language
)
if getattr(self._llm_config, "structure_metadata_enabled", False):
system_prompt += get_prompt("STRUCTURE_METADATA_PROMPT", _language)
system_prompt = with_output_language(
system_prompt, _language, explicit=_explicit_language is not None,
)
_user_tmpl = get_prompt(
"EXTRACT_PROMPT" if _enable_intention else "EXTRACT_PROMPT_NO_INTENTION",
_language,
)
prompt = _user_tmpl.format(
content=content,
current_date=_current_date,
memory_at=_memory_at,
last_messages=_last_messages,
)
# 累计 token 统计
if _log_llm_content_enabled():
@@ -1177,7 +433,7 @@ class Extractor:
if not tc_id:
logger.error(
f"[extractor] tool_call_id is empty! "
f"raw tc: {json.dumps(tc_raw, ensure_ascii=False, default=str)[:500]}"
f"raw_keys={list(tc_raw.keys())}"
)
try:
t_res = await tool_registry.dispatch(
@@ -1311,11 +567,12 @@ class Extractor:
if parsed is None:
if not all_tool_calls:
logger.warning(
f"[extractor] JSON parse failed: {raw_text[:200]}"
f"[extractor] JSON parse failed ({len(raw_text)} chars)"
)
logger.debug(f"[extractor] unparsable output: {raw_text[:200]!r}")
return ExtractResult(
success=False,
error=f"JSON parse failed: {raw_text[:200]}",
error=f"JSON parse failed ({len(raw_text)} chars)",
error_code="JSON_PARSE_FAILED",
raw_response=raw_text,
tokens_used=total_tokens,
+1 -2
View File
@@ -129,8 +129,7 @@ def _parse_tool_calls_list(raw_calls: list, _json) -> List[Dict[str, Any]]:
tc_id = tc.get("id") or ""
if not tc_id:
logger.error(
f"[llm] tool_call missing 'id'! raw: "
f"{_json.dumps(tc, ensure_ascii=False, default=str)[:500]}"
f"[llm] tool_call missing 'id'! keys={list(tc.keys())}"
)
# 保留原始 tc 完整结构,只规范化 arguments
+4 -1
View File
@@ -144,6 +144,7 @@ class MemAgent:
tool_context: Optional[Dict[str, Any]] = None,
history_context: str = "",
basic_profile_fields: Optional[Dict[str, str]] = None,
language: Optional[str] = None,
) -> AgentResult:
"""
处理添加记忆请求
@@ -161,6 +162,7 @@ class MemAgent:
basic_profile_fields:
{field_name: description} 字段表,由调用方从 MemoryConfig.basic_profile.fields
透传过来,extractor 会渲染到 prompt 中要求 LLM 在 JSON basic_info 字段返回。
language: 显式 prompt 语言码,透传给 extractor;None 时 zh/en 自动检测。
Returns:
处理结果
@@ -187,6 +189,7 @@ class MemAgent:
tool_context=tool_context,
history_context=history_context,
basic_profile_fields=basic_profile_fields,
language=language,
)
_t_parallel = datetime.now()
@@ -234,7 +237,7 @@ class MemAgent:
f"identity={len(identity)} facts={len(facts)} "
f"basic_info={basic_field_count}fields"
)
logger.info(f"MemAgent: extracted_info={extract_result.info}")
logger.debug(f"MemAgent: extracted_info={extract_result.info}")
# 3. 冲突检测(如果有已有记忆)
if mode == ProcessMode.FULL and existing_memories:
File diff suppressed because it is too large. Load diff
+60 -459
View File
@@ -50,7 +50,7 @@ def is_canonical_state_node(node: Any) -> bool:
def _log_ops_detail(ops: List["ReconcileOp"]) -> None:
"""以单条 INFO(JSON list)输出 reconcile 的逐条结果,便于线上 debug。"""
"""以单条 DEBUG(JSON list)输出 reconcile 的逐条结果;含记忆正文,不进 INFO。"""
if not ops:
return
view = [
@@ -68,7 +68,7 @@ def _log_ops_detail(ops: List["ReconcileOp"]) -> None:
}
for o in ops
]
logger.info(
logger.debug(
f"[reconciler] ops detail: {json.dumps(view, ensure_ascii=False, default=str)}"
)
@@ -160,299 +160,18 @@ class ReconcileResult:
# Prompts
# ================================================================
SEARCH_QUERY_PROMPT = """You are a search query generator. Given a list of newly extracted memories, generate a set of short search queries that can be used to find related existing memories in a vector database.
The goal is to maximize recall — find existing memories that are semantically related to the new memories, even if the wording is very different.
## New memories:
{new_memories}
## Instructions
Generate search queries that cover:
- Key topics, entities, and themes mentioned in the memories
- Rephrased or abstracted versions of the core concepts
- Related concepts that might exist in the user's memory store
Output a JSON array of query strings (5-15 queries, short and focused):
["query1", "query2", "query3", ...]
Output JSON array only, no other text."""
RECONCILE_PROMPT = """You are a memory management system.
Your task is to merge new memories into an existing memory base while keeping it:
* lossless
* non-contradictory
* compact
* highly retrievable
---
# Operations
## ADD
Use when the information is entirely new.
```json
{{
"op": "ADD",
"content": "...",
"owner": "user",
"tags": ["..."]
}}
```
---
## UPDATE
Use when:
* the new memory is compatible with an existing one
* they describe the same topic
* merging improves completeness or retrievability
Rules:
* merge losslessly
* preserve all useful information
* do NOT create duplicates
* if the two disagree on a value, keep the value from the memory with the later
`memory_at`; never let an older memory overwrite a newer value
### Same occurrence vs. another occurrence
Use **UPDATE** only when the new memory adds detail to the same real-world
occurrence or durable fact already recorded.
If the same person does the same kind of activity/event again at another time
(another trip, game, injury, purchase, health scare, etc.), keep the occurrence
as a separate **ADD**. Preserve its own date and details. When an existing memory
is clearly another occurrence in that same concrete series, link it with
`related: [{{"memory_id":"...","relation":"same_series"}}]`.
Do not merge separate dated occurrences into one long count/timeline memory.
The retrieval layer builds timelines from the linked L2 memories.
```json
{{
"op": "UPDATE",
"memory_id": "...",
"content": "...",
"owner": "user",
"tags": ["..."]
}}
```
---
## SUPERSEDE
Use ONLY when:
* the new memory contradicts an existing memory
* both cannot be true at the same time
* the old memory would now give the wrong current-state answer
* the new memory's `memory_at` is the same as or LATER than the existing memory's `memory_at`
Examples:
* city changed
* employer changed
* relationship status changed
Do NOT use for:
* added detail
* compatible preferences
* wording differences
* semantic equivalence
* a new memory whose `memory_at` is EARLIER than the existing memory's — it is an
older state and must never replace a newer one (see "Time order")
Rules:
* content must contain ONLY the new state
* do NOT copy unchanged information
* the system keeps the old node and links it onto the evolution chain automatically; you don't need to handle history
```json
{{
"op": "SUPERSEDE",
"memory_id": "...",
"content": "...",
"supersede_reason": "...",
"owner": "user",
"tags": ["..."]
}}
```
### Folding several existing memories into one chain
If TWO OR MORE existing memories track the SAME dimension and the new memory is
the latest change on that dimension — its `memory_at` is later than every folded
memory's `memory_at` — you may fold them into a single evolution
chain. Use `memory_ids` (an ordered list, OLDEST → NEWEST) instead of a single
`memory_id`:
```json
{{
"op": "SUPERSEDE",
"memory_ids": ["<oldest_id>", "<newer_id>"],
"content": "<only the newest state>",
"supersede_reason": "...",
"owner": "user",
"tags": ["..."]
}}
```
The listed memories are chained oldest → newest and the new memory becomes the
chain head. `content` still describes ONLY the newest state — never copy the
older memories' text into it; they remain queryable in the chain history.
---
# Rules
* Preserve all meaningful information.
* Skip exact duplicates.
* Prefer merging related fragments.
* Separate dated occurrences stay separate ADDs; use `same_series` only for the
same subject and same concrete activity/event type.
* Keep each memory focused on one coherent topic.
* Follow "Time order" below.
* **NEVER merge or supersede across different subjects.** The subject is whoever
the content is about: "The user ..." vs "John ..." vs "Mary ...". Memories about
different people are independent facts even when the sentence is otherwise
identical — "The user likes coffee" and "John likes coffee" are two separate
memories, never an UPDATE or a SUPERSEDE of each other.
* `owner` (`user`/`agent`): copy it from the memory being written. NEVER merge or supersede across different owners — a user's fact and the assistant's fact are distinct even on the same topic.
* A SUPERSEDE either targets ONE memory (`memory_id`) or folds several memories on the same dimension into one chain (`memory_ids`, oldest→newest). An UPDATE targets exactly ONE memory.
* Never touch the same existing memory from more than one op.
* Output language matches input memory language.
---
# Time order
Every memory, new or existing, has `memory_at`: when it happened. Memories are
NOT always written in the order they happened — history can be imported late,
and a memory can be added while older history is still being processed. A new
memory can therefore be OLDER than an existing one.
* Decide which memory is newer ONLY by `memory_at`. "New memories" means "being
written now", not "more recent".
* An older memory must never replace a newer one: no SUPERSEDE of a newer
memory, and no UPDATE that overwrites a newer value with an older one.
* If a new memory is older than an existing memory it conflicts with, keep the
existing memory as the current state. ADD the new one as a past state, worded
in the past with its time (for example "In 2023 the user lived in Shanghai"),
or output nothing if it adds no information.
* `memory_at: null` means unknown time. Unknown is NOT newer: never SUPERSEDE a
memory that has a known `memory_at` with one whose `memory_at` is null.
* Do not invent chronology.
---
# Optional: `related`
On any op you MAY add a `related` array recording how the memory being written
relates to an EXISTING memory. This does not change what gets stored — it only
records a link.
```json
{{
"op": "ADD",
"content": "...",
"owner": "user",
"tags": ["..."],
"related": [{{"memory_id": "<existing_id>", "relation": "causes"}}]
}}
```
Only these four relations are allowed:
* `causes` — the two memories are in a cause/effect relationship
(e.g. "started running daily" → "knee started hurting")
* `elaborates` — adds detail to, or is a later development of, the existing memory
* `contrasts` — they are in tension but BOTH still true (if one makes the other
false, that is a SUPERSEDE, not a `contrasts` link)
* `same_series` — separate dated occurrences of the same concrete activity or
event type for the same subject (for example, two different skiing trips)
Rules:
* Only link to memories that appear in "Existing memories" — never invent an id.
* Only emit a link when the relationship is **explicit and specific**. Two memories
merely being about the same broad topic is NOT a relation — leave `related` out.
* `same_series` requires the same subject and a concrete repeated action/event;
shared broad tags such as health, travel, sport, food, or work are insufficient.
* Omit the field entirely when there is nothing to link. Zero links is the normal case.
* Never use `related` as a substitute for UPDATE / SUPERSEDE.
---
{few_shot_section}
###################
Input Data
###################
Current date: {current_date}
## Existing memories
{existing_memories}
## New memories
{new_memories}
---
# Output
Return ONLY a JSON array inside a fenced ```json code block.
If no changes are needed:
```json
[]
```
Now produce the JSON array."""
RECONCILE_STRUCTURE_METADATA_PROMPT_EN = """
Additional structural output contract:
For every ADD, UPDATE, or SUPERSEDE result, preserve or refine `topic_paths`,
`event_facet`, and `topic_assignments`. `event_facet` contains only `subject`,
`participants`, `action`, `objects`, `locations`, and `state`.
`topic_assignments` contains subject-specific `{subject, role, path}` entries;
each path segment is `{name, type}` and type is `topic|facet|series|episode`.
Types follow `topic -> facet -> series -> episode`; levels may be skipped but
types may not repeat or move backward.
Use Series only for repeated occurrences of one concrete activity/state line and
Episode only for one bounded real-world occurrence. Never infer these types from
depth, a broad shared tag, or unrelated co-occurrence. When an Existing memory
already has a semantically equivalent subject-specific path, reuse its exact node
names and types instead of creating a synonym fork. To request exact reuse, emit
`structure_references: [{"memory_id":"<existing_id>", "subject":"...",
"through":"topic|facet|series|episode"}]`. The referenced ID must appear in
Existing memories. `through` means copy that existing path prefix through the
named structural type; an explicitly authored deeper suffix remains new. Reuse
is forbidden when the subject, concrete activity/state line, or Episode boundary
differs. These fields describe the memory, never a future query form such as
"when", "how many", or "who".
Series is a reconcile-time relation: establish it only when the new memory and
at least one independent Existing memory are distinct occurrences of the same
concrete activity/state line. Multiple propositions from the current write do
not prove repetition.
"""
# Prompt 文本统一放在 hy_memory/prompts/<language>.py(get_prompt 按语言码取);
# 以下旧名字保留为英文版别名(向后兼容)。
from ..prompts import ( # noqa: E402
get_prompt,
normalize_language,
resolve_language,
with_output_language,
)
from ..prompts.en import SEARCH_QUERY_PROMPT # noqa: E402,F401 (re-export)
from ..prompts.en import RECONCILE_PROMPT # noqa: E402,F401 (re-export)
from ..prompts.en import RECONCILE_STRUCTURE_METADATA_PROMPT as RECONCILE_STRUCTURE_METADATA_PROMPT_EN # noqa: E402,F401 (re-export)
from ..prompts.en import RECONCILE_FEW_SHOT as RECONCILE_FEW_SHOT_EN # noqa: E402,F401 (re-export)
def append_reconcile_structure_prompt(
@@ -461,10 +180,8 @@ def append_reconcile_structure_prompt(
"""Append the structure contract only for an explicit experiment config."""
if not enabled:
return prompt
if language == "zh":
from .prompts_zh import RECONCILE_STRUCTURE_METADATA_PROMPT_ZH
return prompt + RECONCILE_STRUCTURE_METADATA_PROMPT_ZH
return prompt + RECONCILE_STRUCTURE_METADATA_PROMPT_EN
code = "zh-Hans" if language == "zh" else language
return prompt + get_prompt("RECONCILE_STRUCTURE_METADATA_PROMPT", code)
def _as_aware(value: Any) -> Optional[datetime]:
@@ -550,147 +267,42 @@ def build_reconcile_prompt(
current_date: str,
few_shot_enabled: bool = False,
structure_metadata_enabled: bool = False,
language: Optional[str] = None,
) -> str:
"""Build the language-matched reconcile prompt used by the write path."""
from ..utils.lang_detect import is_chinese
"""Build the reconcile prompt used by the write path.
if is_chinese("\n".join(new_memories)):
from .prompts_zh import RECONCILE_FEW_SHOT_ZH, RECONCILE_PROMPT_ZH
few_shot = ("\n" + RECONCILE_FEW_SHOT_ZH + "\n") if few_shot_enabled else ""
prompt = RECONCILE_PROMPT_ZH.format(
current_date=current_date,
new_memories=new_memories_block,
existing_memories=existing_memories_block,
few_shot_section=few_shot,
)
return append_reconcile_structure_prompt(
prompt,
enabled=structure_metadata_enabled,
language="zh",
)
few_shot = ("\n" + RECONCILE_FEW_SHOT_EN + "\n") if few_shot_enabled else ""
prompt = RECONCILE_PROMPT.format(
``language`` set → that language's prompt plus the output-language rule;
None → zh/en auto-detection over the new memories (unchanged behaviour).
"""
explicit = normalize_language(language)
code = resolve_language(explicit, "\n".join(new_memories))
few_shot = (
("\n" + get_prompt("RECONCILE_FEW_SHOT", code) + "\n")
if few_shot_enabled else ""
)
prompt = get_prompt("RECONCILE_PROMPT", code).format(
current_date=current_date,
new_memories=new_memories_block,
existing_memories=existing_memories_block,
few_shot_section=few_shot,
)
return append_reconcile_structure_prompt(
prompt = append_reconcile_structure_prompt(
prompt,
enabled=structure_metadata_enabled,
language="en",
language=code,
)
return with_output_language(prompt, code, explicit=explicit is not None)
# Reconcile few-shot examples (injected into {few_shot_section} only when
# few_shot_enabled). Each shows the existing/new input and the expected ops,
# with a one-line takeaway. IDs are shortened for readability.
RECONCILE_FEW_SHOT_EN = """
###################
Examples
###################
## Example 1 — One new memory: part conflicts with an existing chain, part is unrelated
Existing memories:
[
{"memory_id": "m1", "content": "The user works as a product manager at Stripe.", "owner": "user", "history_versions": [{"content": "The user works as a product manager at a fintech startup."}]}
]
New memories:
[
{"content": "The user has left Stripe and is now a product lead at Notion, and has recently taken up pottery on weekends.", "owner": "user"}
]
Output:
```json
[
{"op": "SUPERSEDE", "memory_id": "m1", "content": "The user works as a product lead at Notion.", "supersede_reason": "employer changed from Stripe to Notion", "owner": "user", "tags": ["work"]},
{"op": "ADD", "content": "The user has recently taken up pottery on weekends.", "owner": "user", "tags": ["hobby"]}
]
```
A SINGLE new memory carries two things: the job change (same dimension as the existing employment chain m1, which already has history → SUPERSEDE, becomes the new chain head) and an unrelated detail (pottery → ADD). Split the one memory into the right op per part — the new head describes ONLY the employment state, never the pottery.
## Example 2 — Fold two unlinked existing memories + the new change into one 3-node chain
Existing memories:
[
{"memory_id": "a1", "content": "The user is learning to play the guitar.", "owner": "user", "memory_at": "2024-02-10 09:00"},
{"memory_id": "a2", "content": "The user can now play a few full songs on the guitar.", "owner": "user", "memory_at": "2024-09-05 09:00"}
]
New memories:
[
{"content": "The user now performs guitar at local open-mic nights.", "owner": "user"}
]
Output:
```json
[
{"op": "SUPERSEDE", "memory_ids": ["a1", "a2"], "content": "The user performs guitar at local open-mic nights.", "supersede_reason": "guitar skill progressed: learning → playing songs → performing live", "owner": "user", "tags": ["music", "hobby"]}
]
```
a1 and a2 were never linked, but they track the same dimension (guitar progress) and the new memory is the latest step. Fold them with `memory_ids` ordered oldest→newest; they become a 3-node chain (a1 ← a2 ← new head). The head `content` describes ONLY the current state — it does not restate "learning" or "a few songs".
## Example 3 — New head must NOT copy the old memory's content
Existing memories:
[
{"memory_id": "c1", "content": "The user lives in a rented one-bedroom apartment in Shibuya, Tokyo, pays ¥180,000/month, loves the 5-minute walk to the station, but complains the place is noisy at night and has no parking.", "owner": "user"}
]
New memories:
[
{"content": "The user just bought and moved into a three-bedroom house in the suburbs of Yokohama with a garden and a two-car garage.", "owner": "user"}
]
Output:
```json
[
{"op": "SUPERSEDE", "memory_id": "c1", "content": "The user bought and moved into a three-bedroom house in suburban Yokohama, with a garden and a two-car garage.", "supersede_reason": "residence changed from a rented Shibuya apartment to an owned Yokohama house", "owner": "user", "tags": ["housing"]}
]
```
The new memory SUPERSEDEs c1 and becomes the chain head. The head `content` describes ONLY the new Yokohama house. Do NOT drag the old apartment's distinctive details (Shibuya, ¥180,000 rent, the station walk, the noise/parking complaints) into the new node — those clearly belong to the old residence, not the new one. They stay in chain history and remain retrievable; copying them would make the head self-contradictory (renting in Shibuya AND owning in Yokohama at once).
## Example 4 — Different subjects are never merged, however similar the wording
Existing memories:
[
{"memory_id": "d1", "content": "The user drinks black coffee every morning.", "owner": "user"}
]
New memories:
[
{"content": "John drinks black coffee every morning.", "owner": "user"},
{"content": "The user now drinks black coffee twice a day, morning and after lunch.", "owner": "user"}
]
Output:
```json
[
{"op": "ADD", "content": "John drinks black coffee every morning.", "owner": "user", "tags": ["food"]},
{"op": "UPDATE", "memory_id": "d1", "content": "The user drinks black coffee twice a day, in the morning and after lunch.", "owner": "user", "tags": ["food"]}
]
```
The two new memories are near-identical in wording but have DIFFERENT subjects. "John ..." is about another person, so it is a plain ADD — never an UPDATE or SUPERSEDE of d1, even though d1 says almost the same thing about the user. Only the memory whose subject is also "The user" may merge with d1.
## Example 5 — Recurring event: bump count and keep a timeline (UPDATE, not ADD)
Existing memories:
[
{"memory_id": "i1", "content": "John injured his ankle last season and needed time off plus physical therapy.", "owner": "user", "memory_at": "2023-05-21 19:48"}
]
New memories:
[
{"content": "John twisted his ankle again during a recent game.", "owner": "user", "memory_at": "2023-10-13 13:50"}
]
Output:
```json
[
{"op": "UPDATE", "memory_id": "i1", "content": "John has injured his ankle two times: first last season (around the 2022–2023 season), needing time off and physical therapy; second during a game around 2023-10-13.", "owner": "user", "tags": ["health", "basketball", "injury"]}
]
```
Same event type happening again → UPDATE the old memory. State the **count** and list occurrences with times when known. Do NOT ADD a second "John injured his ankle" fact, and do NOT SUPERSEDE (the first injury did not stop being true).
"""
def build_search_query_prompt(
new_memories: List[str], *, language: Optional[str] = None,
) -> str:
"""Search-query generation prompt (same language rule as reconcile)."""
explicit = normalize_language(language)
code = resolve_language(explicit, "\n".join(new_memories))
mem_lines = "\n".join(f"{i+1}. {m}" for i, m in enumerate(new_memories))
prompt = get_prompt("SEARCH_QUERY_PROMPT", code).format(new_memories=mem_lines)
return with_output_language(prompt, code, explicit=explicit is not None)
# ================================================================
@@ -745,7 +357,9 @@ class MemoryReconciler:
self._llm = LLMProvider(self.config)
return self._llm
async def _generate_search_queries(self, new_memories: List[str]) -> tuple:
async def _generate_search_queries(
self, new_memories: List[str], language: Optional[str] = None,
) -> tuple:
"""
Pre-process: 从全部新 memory 中提取 search queries,
用于补充向量搜索的召回能力。
@@ -753,17 +367,7 @@ class MemoryReconciler:
Returns:
(queries: List[str], llm_response: Optional[LLMResponse])
"""
mem_lines = "\n".join(f"{i+1}. {m}" for i, m in enumerate(new_memories))
# Select prompt based on input language
from ..utils.lang_detect import is_chinese
if is_chinese("\n".join(new_memories)):
from .prompts_zh import SEARCH_QUERY_PROMPT_ZH
prompt = SEARCH_QUERY_PROMPT_ZH.format(new_memories=mem_lines)
else:
prompt = SEARCH_QUERY_PROMPT.format(new_memories=mem_lines)
prompt = build_search_query_prompt(new_memories, language=language)
try:
llm = self._get_llm()
@@ -849,6 +453,7 @@ class MemoryReconciler:
graph_store=None, # Optional GraphStore:Entity MENTIONS boost
new_memory_embeddings: Optional[Dict[str, List[float]]] = None,
source: str = "default",
language: Optional[str] = None,
) -> ReconcileResult:
"""
对 new_memories 做 reconcile。
@@ -856,6 +461,7 @@ class MemoryReconciler:
Args:
new_memories: 新记忆内容列表(content only,用于 embed 搜索)
new_memories_with_meta: 可选,含 tags 的完整 meta
language: 显式 prompt 语言码;None 时按新记忆做 zh/en 自动检测
"""
if not new_memories:
return ReconcileResult(ops=[], success=True)
@@ -876,7 +482,7 @@ class MemoryReconciler:
if self._enable_search_query:
search_queries, sq_response = await self._generate_search_queries(
new_memories
new_memories, language=language,
)
sq_elapsed = (_time.perf_counter() - t_sq) * 1000
@@ -887,18 +493,10 @@ class MemoryReconciler:
# 写 SEARCH_QUERY pipeline log
if cache and request_id:
mem_lines = "\n".join(f"{i+1}. {m}" for i, m in enumerate(new_memories))
if self._enable_search_query:
from ..utils.lang_detect import is_chinese as _is_zh
if _is_zh("\n".join(new_memories)):
from .prompts_zh import SEARCH_QUERY_PROMPT_ZH
sq_prompt = SEARCH_QUERY_PROMPT_ZH.format(
new_memories=mem_lines
)
else:
sq_prompt = SEARCH_QUERY_PROMPT.format(new_memories=mem_lines)
sq_prompt = build_search_query_prompt(
new_memories, language=language,
)
else:
sq_prompt = "[search query disabled — hybrid retrieval per new memory]"
try:
@@ -1447,6 +1045,7 @@ class MemoryReconciler:
getattr(self.config.llm, "few_shot_enabled", False)
),
structure_metadata_enabled=_structure_metadata_on,
language=language,
)
# ── Step 5: LLM 调用 + 重试 ──
@@ -1514,9 +1113,10 @@ class MemoryReconciler:
logger.warning(
f"[reconciler] RECONCILE JSON parse failed "
f"(attempt {_attempt+1}/{_MAX_PARSE_ATTEMPTS}), "
f"retrying: raw="
f"{raw_response[:200] if _log_llm_content_enabled() else '<redacted>'}"
f"retrying: raw({len(raw_response or '')} chars)"
)
if _log_llm_content_enabled():
logger.debug(f"[reconciler] unparsable raw={raw_response[:200]!r}")
recon_elapsed = (_time.perf_counter() - t_recon) * 1000
@@ -1528,8 +1128,8 @@ class MemoryReconciler:
error_code = "LLM_UNAVAILABLE"
elif raw_response.strip():
error_msg = (
f"RECONCILE JSON parse failed after {_n_attempts} attempts: "
f"{raw_response[:200] if _log_llm_content_enabled() else '<redacted>'}"
f"RECONCILE JSON parse failed after {_n_attempts} attempts "
f"({len(raw_response)} chars)"
)
error_code = "JSON_PARSE_FAILED"
else:
@@ -1806,7 +1406,8 @@ class MemoryReconciler:
except Exception as _re:
logger.debug(f"[reconciler] json_repair failed: {_re}")
if data is None:
logger.warning(f"[reconciler] failed to parse ops JSON: {text[:200]}")
logger.warning(f"[reconciler] failed to parse ops JSON ({len(text)} chars)")
logger.debug(f"[reconciler] unparsable ops output: {text[:200]!r}")
return []
if not isinstance(data, list):
@@ -2018,11 +1619,11 @@ class MemoryReconciler:
if clash is not None:
logger.warning(
f"[reconciler] memory_id {clash} already touched by '{touched[clash]}', "
f"current op '{op.op} ({op.reason[:60]})' dropped"
f"current op '{op.op}' dropped"
)
continue
for t in target_ids:
touched[t] = f"{op.op}({op.reason[:40]})"
touched[t] = op.op
validated.append(op)
return validated
+3 -1
View File
@@ -211,7 +211,9 @@ def parse_tool_calls_from_json(raw: Any) -> List[ToolCall]:
try:
args = json.loads(args_raw) if args_raw.strip() else {}
except json.JSONDecodeError:
logger.warning(f"[tools] failed to parse arguments as JSON: {args_raw[:120]}")
logger.warning(
f"[tools] failed to parse arguments as JSON ({len(args_raw)} chars)"
)
args = {}
elif isinstance(args_raw, dict):
args = args_raw
+87
View File
@@ -26,6 +26,8 @@ from __future__ import annotations
import asyncio
import logging
import re
import unicodedata
import uuid
import weakref
from dataclasses import dataclass
@@ -208,6 +210,86 @@ def _sanitize_kv(
return result
# ================================================================
# 占位符守卫(P0-1)
# ================================================================
# 弱模型(如 qwen3.5-2b)在用户什么都没说时,会把 schema 字段说明里的词当值
# 填进 basic_info:{"name": "用户"} / "The user" / "N/A"。每个占位符都会
# supersede 掉真实值,演化链变成 费天翔 → Fei → 用户 → The user。
# 规则:占位符永远不入库;读侧组装 KV 时占位符也不覆盖已有真实值。
_PLACEHOLDER_VALUES = frozenset({
"用户", "该用户", "本用户", "用户本人",
"the user", "the user's", "user", "a user", "this user",
"n/a", "na", "none", "null", "nil", "unknown", "unspecified",
"not provided", "not specified", "not mentioned",
"未提供", "未知", "不详", "未说明", "未提及", "无",
"-", "?",
# 其他提示词语言里的「用户」/「未知」(显式 language 时模型可能照抄)
"使用者", "該使用者", "用戶", "該用戶", "不詳", "無",
"ユーザー", "ユーザ", "不明", "なし",
"사용자", "유저", "알 수 없음", "없음",
"el usuario", "usuario", "la usuaria", "desconocido", "ninguno",
"der nutzer", "nutzer", "der benutzer", "benutzer", "unbekannt", "keine",
"o usuário", "usuário", "o utilizador", "utilizador", "desconhecido", "nenhum",
"l'utilisateur", "utilisateur", "inconnu", "aucun",
"пользователь", "неизвестно", "нет",
"pengguna", "tidak diketahui", "tidak ada",
})
# 内置字段的中文标签:弱模型也会把字段说明的中文词当值。
_FIELD_LABELS: Dict[str, tuple] = {
"name": ("姓名", "名字", "用户名", "用户姓名", "用户名字"),
"age": ("年龄", "用户年龄"),
"location": ("所在地", "地点", "城市", "用户所在地"),
"occupation": ("职业", "工作", "用户职业"),
"employer": ("雇主", "公司", "用户雇主"),
}
# 去掉标点/符号后再比较,让 "The user." / "用户。" / "N/A" 都命中。
# "-" / "?" 本身就是纯标点,剥完为空串,同样判为占位符。
_PUNCT_RE = re.compile(r"[\s\.,;:!?'\"`~()\[\]{}<>/\\|_\-\u3000-\u303f\uff00-\uff0f\uff1a-\uff20]+")
def _placeholder_key(value: str) -> str:
s = unicodedata.normalize("NFKC", str(value or "")).strip().lower()
s = s.replace("\u2019", "'")
s = _PUNCT_RE.sub(" ", s)
return " ".join(s.split())
_PLACEHOLDER_KEYS = frozenset(_placeholder_key(v) for v in _PLACEHOLDER_VALUES)
def is_placeholder_value(field_name: str, value: Any) -> bool:
"""value 是否为占位符(空 / 用户 / The user / N/A / 字段名本身 等)。
大小写不敏感、首尾空白与标点不敏感。数字(如 age=30)永远不是占位符。
"""
if value is None:
return True
if isinstance(value, bool):
return False
if isinstance(value, (int, float)):
return False
key = _placeholder_key(value)
if not key:
return True
if key in _PLACEHOLDER_KEYS:
return True
field_key = _placeholder_key(field_name)
if field_key and key in (field_key, field_key.replace("_", " ")):
return True
for label in _FIELD_LABELS.get(field_name, ()):
if key == _placeholder_key(label):
return True
return False
def _drop_placeholders(kv: Dict[str, Any]) -> Dict[str, Any]:
return {k: v for k, v in kv.items() if not is_placeholder_value(k, v)}
# ================================================================
# 公共入口:upsert_basic_profile
# ================================================================
@@ -287,6 +369,8 @@ async def _upsert_basic_profile_unlocked(
# L0 Basic Info 从输入、唯一性到落盘都只属于 user_id。
new_kv = _sanitize_kv(kv or {}, allowed_fields=allowed_fields)
# 单调性守卫:占位符永远不覆盖已有值,过滤后为空就不写新版本、不产生新链头。
new_kv = _drop_placeholders(new_kv)
if not new_kv:
return BasicProfileUpsertResult(
success=True, changed=False, reason="no valid fields after sanitize"
@@ -513,5 +597,8 @@ def _assemble_full_kv_from_nodes(
for k, v in kv.items():
if allowed_set is not None and k not in allowed_set:
continue
# 历史上已落盘的占位符版本(如 name=用户)不覆盖更早的真实值。
if is_placeholder_value(k, v):
continue
full_kv[k] = v
return full_kv
+96 -19
View File
@@ -744,7 +744,7 @@ class HyMemoryClient:
def _install_pipeline_observability_hooks(self) -> None:
"""
包装 cache.store_pipeline_log:
- Log:JSONL 文件,始终写入(环节级,不依赖 Trace 开关)
- Log:JSONL 文件,由 MEMORY_PIPELINE_LOG_ENABLED 控制(默认开启)
- Trace:SQLite pipeline_logs,由 MEMORY_PIPELINE_TRACE_ENABLED 控制(Inspector)
- 落盘 fire-and-forget,不阻塞 search/write 热路径
"""
@@ -909,6 +909,19 @@ class HyMemoryClient:
"""
return "coding" if str(scene or "").strip().lower() == "coding" else "working"
def _resolve_language(self, language: Optional[str]) -> Optional[str]:
"""per-call language > MemoryConfig.language; None = zh/en 自动检测。
返回 canonical 码;未知码抛 ValueError(列出支持的码与别名)。
"""
from .prompts import normalize_language
code = normalize_language(language)
if code is not None:
return code
config = getattr(self, "_config", None)
return normalize_language(getattr(config, "language", None))
@staticmethod
def _parse_input(data: AddInput):
"""
@@ -958,7 +971,7 @@ class HyMemoryClient:
parsed_tool_calls = json.loads(raw_tool_calls) if raw_tool_calls else []
except Exception:
logger.warning(
"tool_calls 是非法 JSON 字符串,已忽略: %r", raw_tool_calls[:200]
"tool_calls 是非法 JSON 字符串,已忽略 (%d chars)", len(raw_tool_calls)
)
parsed_tool_calls = []
raw_tool_calls = parsed_tool_calls if isinstance(parsed_tool_calls, list) else []
@@ -1723,6 +1736,7 @@ class HyMemoryClient:
request_id: Optional[str] = None,
meta: Optional[Dict[str, Any]] = None,
embed_override: Optional[Dict[str, Any]] = None,
language: Optional[str] = None,
) -> Dict[str, Any]:
"""
写入记忆。
@@ -1773,6 +1787,13 @@ class HyMemoryClient:
meta 独立持久化,只做业务透传。
语义为「该次 write 时刻的上下文」,故存为 list——同一节点被
多次写入合并时会追加而非覆盖,详见 get() 返回的 meta 字段。
language: 本次请求的 LLM prompt 语言码(en / zh-Hans / zh-Hant / ja / es /
de / pt / fr / ko / ru / id;别名 zh、zh-CN → zh-Hans,zh-TW、
zh-HK → zh-Hant,pt-BR → pt)。优先于 MemoryConfig.language
(env MEMORY_LANGUAGE);两者都未设时沿用 zh/en 自动检测。
显式指定时 extract / reconcile / coding 链路 / 基础画像等本次
请求的全部 LLM 调用都用该语言的 prompt,记忆正文也用该语言书写。
未知码抛 ValueError(列出支持的码)。
Returns:
chat 链路:{"success": True, "memory_id": "...", "request_id": "...", "elapsed_ms": ...}
@@ -1788,6 +1809,7 @@ class HyMemoryClient:
workspace_id=workspace_id, branch=branch,
request_id=request_id, meta=meta,
embed_override=embed_override,
language=language,
),
include_runtime_diag=True,
)
@@ -1798,15 +1820,20 @@ class HyMemoryClient:
user_id: str,
agent_id: str = "default_agent",
session_id: str,
language: Optional[str] = None,
) -> Dict[str, Any]:
"""Finalize one caller session into a first-class episode narrative.
The feature is opt-in through MEMORY_SESSION_EPISODE_ENABLED. Calls are
idempotent: the same user/agent/session tuple upserts one deterministic node.
``language`` (or MemoryConfig.language) makes the narrative be written in
that language; see add(). The episode prompts themselves are English-only.
"""
return self._loop_thread.run(
self.async_finalize_session(
user_id=user_id, agent_id=agent_id, session_id=session_id,
language=language,
),
include_runtime_diag=True,
)
@@ -2263,6 +2290,7 @@ class HyMemoryClient:
user_id: str,
agent_id: str = "default_agent",
max_batches: Optional[int] = None,
language: Optional[str] = None,
) -> Dict[str, Any]:
"""
手动触发 System 2 认知加工(ultra 模式专用,同步阻塞直到完成)。
@@ -2285,6 +2313,8 @@ class HyMemoryClient:
Args:
user_id: 用户 ID
agent_id: Agent ID
language: System 2 / sweeper prompt 语言码,语义同 add()(per-call >
MemoryConfig.language > 按 materials 做 zh/en 检测)。
Returns:
{"success": True, "request_id": "...", "tasks_processed": 1, "results": {...},
@@ -2298,6 +2328,7 @@ class HyMemoryClient:
return self._loop_thread.run(
self.async_evolve(
user_id=user_id, agent_id=agent_id, max_batches=max_batches,
language=language,
)
)
@@ -2307,8 +2338,10 @@ class HyMemoryClient:
user_id: str,
agent_id: str = "default_agent",
max_batches: Optional[int] = None,
language: Optional[str] = None,
) -> Dict[str, Any]:
"""手动触发 System 2 认知加工(异步)"""
language = self._resolve_language(language)
t0 = __import__("time").perf_counter()
# 获取 System2Writer
@@ -2331,6 +2364,7 @@ class HyMemoryClient:
result = await writer.evolve(
user_id=user_id, agent_id=agent_id, max_batches=max_batches,
language=language,
)
result["elapsed_ms"] = round((__import__("time").perf_counter() - t0) * 1000, 2)
return result
@@ -2339,6 +2373,7 @@ class HyMemoryClient:
self,
user_ids: List[str],
agent_ids: Optional[List[str]] = None,
language: Optional[str] = None,
) -> Dict[str, Any]:
"""
手动触发 cross-domain sweeper(同步)。
@@ -2352,6 +2387,8 @@ class HyMemoryClient:
Args:
user_ids: SDK user_id 列表
agent_ids: Agent ID 列表(默认 ["default_agent"])
language: 输出语言码,语义同 add()。sweeper prompt 只有英文版,显式
指定时追加该语言的输出规则。
Returns:
{"success": True, "results": [{...}, ...]}
@@ -2361,7 +2398,9 @@ class HyMemoryClient:
f"sweep() requires mode='ultra', but this client is mode='{self._mode}'."
)
return self._loop_thread.run(
self.async_sweep(user_ids=user_ids, agent_ids=agent_ids)
self.async_sweep(
user_ids=user_ids, agent_ids=agent_ids, language=language,
)
)
@_ensure_internal_loop
@@ -2369,11 +2408,13 @@ class HyMemoryClient:
self,
user_ids: List[str],
agent_ids: Optional[List[str]] = None,
language: Optional[str] = None,
) -> Dict[str, Any]:
"""手动触发 cross-domain sweeper(异步)
对每个 user_id × agent_id 组合执行一次 sweeper。
"""
language = self._resolve_language(language)
t0 = __import__("time").perf_counter()
try:
@@ -2385,7 +2426,9 @@ class HyMemoryClient:
results = []
for uid in user_ids:
for aid in effective_agents:
r = await writer.run_sweeper(user_id=uid, agent_id=aid)
r = await writer.run_sweeper(
user_id=uid, agent_id=aid, language=language,
)
results.append({"user_id": uid, "agent_id": aid, **r})
elapsed_ms = round((__import__("time").perf_counter() - t0) * 1000, 2)
@@ -2740,13 +2783,17 @@ class HyMemoryClient:
request_id: Optional[str] = None,
meta: Optional[Dict[str, Any]] = None,
embed_override: Optional[Dict[str, Any]] = None,
language: Optional[str] = None,
) -> Dict[str, Any]:
"""写入记忆(异步)
embed_override: 本次写入的 embedding 端点覆盖(不改 model/dims,只换接入点)。
dict,可含 base_url / api_key / extra_headers / timeout;None = 用 client 配置。
可用于按请求切换任意 OpenAI-compatible embedding endpoint。
language: 本次请求的 prompt 语言码,详见 add()。
"""
# 先校验语言码:非法值直接 ValueError,不占用 request_id 的幂等槽位
language = self._resolve_language(language)
request_id = request_id or str(uuid.uuid4())
replay = await self._claim_write_outcome(
request_id=request_id,
@@ -2770,7 +2817,7 @@ class HyMemoryClient:
source=source,
metadata=metadata, memory_at=memory_at,
scene=scene, workspace_id=workspace_id, branch=branch,
request_id=request_id, meta=meta,
request_id=request_id, meta=meta, language=language,
)
else:
result = await self._async_add_impl(
@@ -2778,7 +2825,7 @@ class HyMemoryClient:
source=source,
metadata=metadata, memory_at=memory_at,
scene=scene, workspace_id=workspace_id, branch=branch,
request_id=request_id, meta=meta,
request_id=request_id, meta=meta, language=language,
)
except BaseException:
await self._mark_write_outcome_unknown(request_id)
@@ -2797,8 +2844,10 @@ class HyMemoryClient:
user_id: str,
agent_id: str = "default_agent",
session_id: str,
language: Optional[str] = None,
) -> Dict[str, Any]:
"""Async counterpart of :meth:`finalize_session`."""
language = self._resolve_language(language)
series = bool(getattr(self._config.recall, "series_episode_enabled", False))
dynamic = bool(
getattr(self._config.recall, "dynamic_episode_enabled", False) or series
@@ -2828,6 +2877,7 @@ class HyMemoryClient:
user_id=user_id,
agent_id=agent_id,
session_id=session_id,
language=language,
)
if finalize is finalize_dynamic_episode_capsules:
finalize_kwargs["summary_concurrency"] = getattr(
@@ -2867,6 +2917,7 @@ class HyMemoryClient:
branch: Optional[str] = None,
request_id: Optional[str] = None,
meta: Optional[Dict[str, Any]] = None,
language: Optional[str] = None,
) -> Dict[str, Any]:
"""写入记忆(异步)
@@ -2875,6 +2926,7 @@ class HyMemoryClient:
None = 自动判定。详见 add() docstring。
workspace_id / branch: Coding memory 路径专用,详见 add() docstring。
meta: 调用方自定义业务 meta,透传存储,详见 add() docstring。
language: 已解析的 canonical 语言码(None = zh/en 自动检测)。
"""
from .utils.source_filter import normalize_source
source = normalize_source(source)
@@ -2973,6 +3025,7 @@ class HyMemoryClient:
request_id=request_id,
source=source,
memory_at=memory_at,
language=language,
)
# Public receipt always identifies the productivity namespace.
resp["variant"] = scene_variant
@@ -3008,13 +3061,13 @@ class HyMemoryClient:
f"[add] writing {preview} user={uid} "
f"agent_id={agent_id} session_id={session_id} mode={self._mode}"
)
logger.info(f"[add] messages detail: {[m.to_dict() for m in chat_messages]}")
logger.debug(f"[add] messages detail: {[m.to_dict() for m in chat_messages]}")
else:
logger.info(
f"[add] writing text({len(content)} chars) user={uid} "
f"agent_id={agent_id} session_id={session_id} mode={self._mode} "
f"content={content}"
f"agent_id={agent_id} session_id={session_id} mode={self._mode}"
)
logger.debug(f"[add] text content={content!r}")
# mode → agent_mode 映射
# lite: 不调 LLM agent
@@ -3038,6 +3091,7 @@ class HyMemoryClient:
# 落库 memory_ops / WRITE_SUMMARY / pipeline_logs 的单一真相源。
request_id=request_id,
client_meta=meta,
language=language,
)
# mode → writer:
@@ -3089,9 +3143,18 @@ class HyMemoryClient:
if self._history_store is not None:
try:
isolation_key = self._build_isolation_key(uid, agent_id)
# 从 content 或 messages 中提取用于记录的文本
# 从 content 或 messages 中提取用于记录的文本。messages 与 L1_RAW
# 同一规则先裁剪再落审计库(工具输出 / 长回复不原样长期保存)。
from .utils.raw_trim import trim_raw_messages
history_content = content or "; ".join(
f"{m.role}: {m.content}" for m in chat_messages
f"{m.role}: {m.content}"
for m in trim_raw_messages(
chat_messages,
tool_max_bytes=getattr(
getattr(self._config, "coding", None),
"tool_result_max_bytes", None,
),
)
)
await self._history_store.record_add(
memory_id=result.memory_id,
@@ -3117,6 +3180,7 @@ class HyMemoryClient:
metadata: Optional[Dict[str, Any]] = None,
memory_at: Optional[datetime] = None,
request_id: Optional[str] = None,
language: Optional[str] = None,
) -> Dict[str, Any]:
"""写入「调用方已抽取好的」记忆:跳过 extract,直接进 reconcile。
@@ -3131,6 +3195,7 @@ class HyMemoryClient:
Args:
contents: 单条事实字符串,或多条事实字符串列表。
language: reconcile prompt 的语言码,语义同 add()。
其余参数语义同 add()。
Returns:
@@ -3142,6 +3207,7 @@ class HyMemoryClient:
contents, user_id=user_id, agent_id=agent_id,
session_id=session_id, source=source, metadata=metadata,
memory_at=memory_at, request_id=request_id,
language=language,
),
include_runtime_diag=True,
)
@@ -3158,8 +3224,10 @@ class HyMemoryClient:
metadata: Optional[Dict[str, Any]] = None,
memory_at: Optional[datetime] = None,
request_id: Optional[str] = None,
language: Optional[str] = None,
) -> Dict[str, Any]:
"""Idempotent async wrapper for extracted-memory writes."""
language = self._resolve_language(language)
request_id = request_id or str(uuid.uuid4())
replay = await self._claim_write_outcome(
request_id=request_id,
@@ -3180,6 +3248,7 @@ class HyMemoryClient:
metadata=metadata,
memory_at=memory_at,
request_id=request_id,
language=language,
)
except BaseException:
await self._mark_write_outcome_unknown(request_id)
@@ -3202,6 +3271,7 @@ class HyMemoryClient:
metadata: Optional[Dict[str, Any]] = None,
memory_at: Optional[datetime] = None,
request_id: Optional[str] = None,
language: Optional[str] = None,
) -> Dict[str, Any]:
"""写入已抽取好的记忆(异步);跳过 extract,直接 reconcile。详见 add_extracted()。"""
request_id = request_id or str(uuid.uuid4())
@@ -3242,6 +3312,7 @@ class HyMemoryClient:
extra=extra,
memory_at=memory_at,
request_id=request_id,
language=language,
)
# lite/pro → MemoryWriter;ultra → System2Writer(内部复用 MemoryWriter)
@@ -4034,14 +4105,14 @@ class HyMemoryClient:
decided_variant = HyMemoryClient._scene_variant(scene)
logger.info(
f"[search] scene explicit={decided_scene} (no LLM judge); "
f"target={effective_target_query!r}"
f"target({len(effective_target_query or '')} chars)"
)
elif not agentic_enabled and not queries_supplied:
# 默认 light search 仅指 query="...":固定 normal,绝不调用 LLM。
decided_scene = "normal"
logger.info(
f"[search] search_mode=default single-query scene=normal (no LLM); "
f"target={target_query!r}"
f"target({len(target_query or '')} chars)"
)
elif agentic_enabled:
# agentic:单/多 query 都强制一次 LLM 完成三分类 + rewrite + 时间解析。
@@ -4076,14 +4147,18 @@ class HyMemoryClient:
rewrite_created_after = ar.get("created_after")
logger.info(
f"[search] agentic scene={decided_scene} via rewrite-LLM "
f"(classify={want_classify}); orig={target_query!r} "
f"effective={effective_target_query!r} rewrite_used={rewrite_used} "
f"(classify={want_classify}); rewrite_used={rewrite_used} "
f"created_after={rewrite_created_after}"
)
logger.debug(
f"[search] agentic rewrite: orig={target_query!r} "
f"effective={effective_target_query!r}"
)
else:
decided_scene = "normal"
logger.info(
f"[search] agentic rewrite-LLM fallback=normal; target={target_query!r}"
f"[search] agentic rewrite-LLM fallback=normal; "
f"target({len(target_query or '')} chars)"
)
elif rewrite_enabled and not _too_long:
# default + queries= 的原逻辑:配置开启时一次 LLM 做 rewrite/time,
@@ -4162,7 +4237,7 @@ class HyMemoryClient:
decided_scene = "normal"
logger.info(
f"[search] default multi-query features disabled; no LLM; "
f"target={target_query!r}"
f"target({len(target_query or '')} chars)"
)
# ────────────────────────────────────────────────────────────
@@ -4215,7 +4290,7 @@ class HyMemoryClient:
else:
logger.info(
f"[search] routing to PRODUCTIVITY path variant={decided_variant}: "
f"target={effective_target_query!r} "
f"target({len(effective_target_query or '')} chars) "
f"workspace_id={workspace_id!r} branch={branch!r}"
)
return await self._search_coding(
@@ -4243,9 +4318,11 @@ class HyMemoryClient:
# Chat 召回链路(原有逻辑)
# ────────────────────────────────────────────────────────────
logger.info(
f"[search] query='{effective_target_query}' user_ids={effective_user_ids} "
f"[search] query({len(effective_target_query or '')} chars) "
f"user_ids={effective_user_ids} "
f"agent_ids={effective_agent_ids} session_ids={effective_session_ids} limit={limit}"
)
logger.debug(f"[search] query={effective_target_query!r}")
# rewrite 抽出的时间下界与用户显式 created_after 取更晚者(更严格)
effective_created_after = created_after
+43 -695
View File
@@ -21,7 +21,12 @@ from typing import List, Optional, Dict, Any, Tuple
from ..agent.llm_provider import LLMProvider
from ..pipelines.base import ChatMessage
from ..utils.lang_detect import is_chinese
from ..prompts import (
get_prompt,
normalize_language,
resolve_language,
with_output_language,
)
from .preproc import (
extract_files,
keep_last_k_user_turns,
@@ -65,332 +70,7 @@ _VERBATIM_STATEMENT_CHARS = 600
# Prompt
# ================================================================
EXTRACT_PROMPT = """\
You extract durable WORKING memories from a task-execution session segment.
A "working" session is any hands-on task where the user operates a computer /
system through tools — this covers BOTH software/code work (writing, debugging,
refactoring, building, deploying, API/library usage) AND system & operations
work (running shell/PowerShell commands, disk/storage analysis or cleanup, file
& folder operations, installing/configuring software, environment/device setup,
hardware/disk health inspection, backup, scheduling/automation, data processing).
You produce TWO layers from the SAME segment:
EXPERIENCES (procedural, "how to do it") — reusable method, extracted only
when it clears the VALUE BAR below. A (task, search_keys, solution,
boundary_envs) bundle.
FACT (episodic, "what happened this time") — EXACTLY ONE record of THIS
session: what the user wanted, how it was gone about, how it came out. No
value bar, but there is never more than one.
The two layers answer different questions and are stored separately. A method
that generalizes goes in EXPERIENCES; the story of this particular session —
only true for this system / this project / right now — goes in the FACT.
═══════════════════════════════════════════════════════════════════
VALUE BAR (EXPERIENCES ONLY) — DO NOT cross this lightly.
═══════════════════════════════════════════════════════════════════
A memory MUST satisfy ALL of:
(a) Solves a problem likely to recur for the same user
(b) Solution is non-trivial: combines multiple pieces of info / steps, OR
carries a decision rationale, OR captures a hard-won workaround, OR is a
reusable procedure/recipe for an operational task (e.g. how to analyze &
clean up disk usage, how to batch-organize files, how to inspect hardware
health, how to set up a scheduled job)
(c) A future agent / user would prefer looking it up over re-discovering it
IMPORTANT — analysis / diagnostic / investigation tasks COUNT:
A task does NOT have to make a change or "fix" something to be worth remembering.
Read-only / analytical / troubleshooting work is valuable when its PROCEDURE or
CONCLUSION is reusable. Explicitly extract:
- Analysis / diagnostic recipes: the reusable procedure + what it revealed
(e.g. "how to analyze which folders eat C-drive space" — the scan commands,
the gotchas hit, and the finding). "It only produced a report, no change"
is NOT a reason to skip — the reusable METHOD is the memory.
- Investigation outcomes, INCLUDING negative / not-found results
(e.g. "searched folder X across C: and D: with rg / fs_search / SQL index —
not found anywhere"). A recorded dead-end saves the next agent from repeating
the same fruitless search, so it clears the bar.
Do NOT reject a memory merely because "the assistant could regenerate the
commands" or "nothing was actually modified". If the procedure or the conclusion
would save future effort, keep it.
DO NOT extract:
- Truly trivial single-step ops with no reusable procedure ("rename one file",
"git status", "run ls once")
- One-off issues already resolved with no transferable lesson
- Information that is already in the repo/system (don't duplicate package.json)
- Per-PR / per-session throwaway tasks unless they encode a reusable pattern
- The act of ASKING THE USER for missing information. Requesting a parameter,
listing what is unknown, or eliciting required fields is conversational
behaviour, not a method: every agent already knows how to ask a question, so
there is nothing to look up later. Never emit an experience whose task or
search_keys are shaped like "how to ask for X", "gather the missing Y",
"eliciting required fields", "what to clarify before starting". Note this
stays out of EXPERIENCES only — that the session ended waiting on the user
is real state and belongs in the FACT.
If the segment made no substantive progress because it stalled waiting on the
user — the assistant only asked for parameters, or was blocked before doing any
real work — emit "experiences": [] and write the FACT alone. Nothing was
learned yet, so there is no method to store. This matters more than it looks:
an experience distilled from a stall reads as a legitimate skill on retrieval,
and a later segment of the same task will match it and ask the user again
instead of proceeding, so the stall reproduces itself for the rest of the
session. Judge substance by what the tools actually did, not by whether the
turn produced fluent text.
Heuristic: if the user (or a future agent doing the same kind of task) would
want this written down instead of figuring it out again, keep it. A multi-step
operational procedure (even a system/ops or read-only analysis/search one, not
just code) usually qualifies. Only drop it if it is a single command any user
would find via `--help` in 30 seconds AND carries no decision/gotcha worth
remembering.
═══════════════════════════════════════════════════════════════════
EXPERIENCE FIELDS
═══════════════════════════════════════════════════════════════════
task — Query-shaped name; phrase as how a user would later ASK.
"publish a release to the xx platform" not
"publish_release_internal"
Self-contained (no "this", "the file", "we").
search_keys — 0 to 5 alternative phrasings capturing important aspects of
solution that `task` alone wouldn't match. Phrase as questions
or topics future users would ASK ABOUT. Distinct from task.
Omit if task already covers everything.
solution — Complete, self-contained content the user actually needs:
commands, credential locations (NOT credential VALUES),
file paths, ordered steps, gotchas, decision rationale.
Aggregate multi-turn end-to-end work into ONE solution if it
forms a coherent task; split into multiple memories only when
the user did unrelated tasks.
boundary_envs — Concrete pinning that solution depends on, multi-line "key: value":
runtime / SDK / library + version
API endpoint or method signature
external service / middleware version
env var NAME (not value)
config file path
host / cluster / region identifier
If solution is environment-agnostic, leave as "" (empty).
Purpose: when the underlying SDK/middleware upgrades, future
consumers can inspect this to decide if solution still applies.
confidence — 0.0 to 1.0
1.0 user explicitly stated this as a rule / instruction
0.7 inferred with good evidence from tool actions
0.5 weakly inferred; one observation; uncertain
═══════════════════════════════════════════════════════════════════
FACT FIELD — EXACTLY ONE per segment, no value bar
═══════════════════════════════════════════════════════════════════
You write ONE fact for the whole segment — a recoverable state record for this
session, not a polished retrospective and never a list of separate findings. A
future agent must be able to resume from it without mistaking a suggestion,
inference, or partial search for something verified.
subject — WHO or WHAT this session is about: the single entity the work was
performed on or for. One short noun phrase, no sentence, no verb.
Pick the most specific entity that the session is *about*, not the
tool it used:
a person's name when the session serves one named person
("Dylan", "Matthew")
the file / module / symbol when the work targets code
("auth/middleware.ts", "parse_plan_text")
the service / host / device / drive for ops work
("D:\\WXWork", "redis-cache-prod", "ci.yml")
This is a HARD KEY, not prose: two sessions about different people
or different files MUST get different subjects, and two sessions
about the same one MUST get the same string, spelled identically.
Prefer the exact identifier as it appears in the trace over a
description of it. If the segment genuinely has no single subject,
output "".
Why it matters: subject is the gate that decides whether a later
session is allowed to overwrite this one as "the same task,
continued". Get it wrong and one person's state silently replaces
another's.
statement — A compact record of THIS session. Two parts, and the second is the
one that gets lost.
Preserve task-scoped givens and concrete constraints instead of
replacing them with a generic evaluation of the outcome. State
each rule once: repeated instruction text competes with the data
the memory must retain.
PART 1 — what happened, about 300 latin characters (~150 CJK).
Start with the specific goal and object ("reclaim disk space taken
by D:\\WXWork", "locate the new_file_search folder"), not a generic
"fix a bug" / "find a file". Then roughly what was inspected, tried,
or changed. Then how it ended, with the semantics of exactly one of:
completed, partially completed, failed, blocked, or
still awaiting verification. Include negative results and dead ends
without overstating them.
PART 2 — the TASK-SCOPED GIVENS: everything the WHOLE task runs on,
even when this segment did not establish it and even when it feels
too elementary to state. Bindings (what each name/symbol/label
stands for), fixed values the task supplies rather than derives,
conventions adopted (indexing base, units, timezone, defaults), and
this segment's own result WITH its value ("x = 7", not "the value
was computed").
Write PART 2 as data, one item per line, values as given:
retry budget = 5; timeout 30s; index starts at 0
step 3 input: n=64, tolerance 1e-6
slot B: type=private, cost 688.7-721.3 per unit
A line should be almost entirely values. Keep the trace's notation —
symbols, units, numbers, spellings — and drop its sentences: strip
the phrases that announce a given rather than being one ("I would
prefer", "it should be somewhere between … and …"). They repeat once
per item, so a 15-item list pays for them 15 times.
Carry them on EVERY fact; never write "as above" or "same as the
previous session" — retrieval returns a handful of facts by
similarity and the referent may not be among them.
Aim under 1200 characters total. If PART 2 does not fit, fix the
notation, never the coverage: statement is the retrieval vector AND
the only field a later segment can read (details is neither embedded
nor injected), so anything cut from it is gone. Compress the wording
around a given; never the given.
The test: if the next segment cannot see the original material, can
it continue from this record alone? Only a yes counts. A missing
binding does not fail loudly — the name still looks meaningful, so
it gets read with its general-usage meaning and the answer that
follows is fluent and about a different question.
LANGUAGE — same language as the session trace: statement is the
retrieval vector and queries arrive in the user's working language,
so another language pays a cross-lingual penalty on every lookup.
EVIDENCE FIDELITY — record only actions actually executed and
evidence explicitly present in the trace. Preserve material
commands, flags, arguments, errors, and numbers verbatim.
NEVER complete a command with customary flags, and never turn
advice or plans into actions:
"ran Get-ChildItem -Force" → "ran Get-ChildItem"
(the trace did not contain `-Force`; do not invent it)
Scope every negative finding to what was actually searched, and
enumerate EVERY explicit coverage limit that could invalidate it,
especially permission failures and depth limits: keep the roots,
filters, and depth that were in effect ("not found under C:\\ and
D:\\ at depth 3; some directories were not covered due to Access
denied"), never widen it to "does not exist". Do not omit one limit
to make the record prettier.
Never promote a partial search to global absence.
A blocked attempt keeps its blocker.
Preserve the distinction among attempted, observed, verified,
inferred, and unresolved — express it naturally ("tried … but was
blocked by Access denied", "suspected …, not yet verified"), not as
labels. Without tool output or explicit confirmation, NEVER claim a
fix succeeded, a cause was established, or a path does not exist.
Do NOT record greetings or pure chatter — that is the only case
getting "fact": null. A segment that merely stated requirements, or
stalled before doing work, still established something.
details — The session's concrete OUTPUT, copied over verbatim. `statement` is
a summary and a summary necessarily drops the individual values; a
later lookup almost always wants one specific value, not the gist.
So whatever addressable result this session produced, reproduce it
here in its original form and structure:
a plan / schedule / itinerary → keep every line, every day, every
slot, and every price exactly as written
a config or code block → keep the block
a command's material output → keep the lines that carry the
findings (sizes, counts, versions, error text)
a structured result (table, list of records) → keep the rows
Preserve the original layout — line breaks, `Day 1:` / key: value
prefixes, units, currency symbols, ids. Do not re-summarize, do not
reorder, do not round numbers, do not translate.
Copy ONLY what is present in the trace. Never fill a gap: if a slot
was left unfilled, leave it unfilled or say so — an invented value
is worse than a missing one, because it will be retrieved and used.
Cap around 4000 characters; if the output is longer, keep the parts
carrying concrete values and drop decorative prose, noting what was
dropped.
"" when the session produced no such artifact (pure investigation
with no findings yet, or a stall waiting on the user).
This field is NOT part of the retrieval vector — `statement` and
`subject` do the finding, `details` supplies the value once found.
So do not write retrieval bait here, and do not duplicate the
summary; write the raw thing.
files — 0..N absolute or repo-relative FILE OR DIRECTORY paths actually
operated on or investigated during this session. Keep investigated
roots such as `C:\\` and `D:\\`. Only include paths that were
actually accessed/searched, not paths merely suggested or planned;
[] if none apply.
═══════════════════════════════════════════════════════════════════
HOST CONTEXT (given by SDK)
═══════════════════════════════════════════════════════════════════
- workspace_id present: {workspace_present}
- branch present: {branch_present}
When no workspace_id is present, the solution cannot rely on unstated
project-local context — spell out any repo / path / service it needs inside
`solution` or `boundary_envs` so the memory stays usable on its own.
═══════════════════════════════════════════════════════════════════
CONVERSATION
═══════════════════════════════════════════════════════════════════
{conversation_block}
═══════════════════════════════════════════════════════════════════
OUTPUT (strict JSON only, no markdown)
═══════════════════════════════════════════════════════════════════
{{
"experiences": [
{{
"task": "...",
"search_keys": ["...", "..."],
"solution": "...",
"boundary_envs": "...",
"confidence": 0.9
}}
],
"fact": {{
"subject": "...",
"statement": "...",
"files": ["..."],
"details": "..."
}}
}}
DEFAULT TO EXTRACTING. When the segment did real work, you MUST output at least
one experience — even if the task looks simple or was a one-shot success, still
capture the reusable essence of what was done (the method / command / how-to).
When the session is thin, the single most valuable experience is enough.
The ONE exception is the stall case above: if the segment did no real work
because it was blocked asking the user for missing information, output
"experiences": [] and write only the fact. Do not manufacture an experience to
satisfy this instruction — a fabricated one ("how to ask for the departure
city") is worse than none, because it is later retrieved and followed. Empty is
allowed ONLY for that reason, never as a shortcut when work was actually done.
"fact" is a SINGLE object, never a list — one record per session.
ALWAYS write the fact. The bar for the fact is not "was this valuable" but "did
anything at all happen", and a segment reaching you has already cleared that.
This includes segments that look like setup rather than work: a single
user message stating requirements, parameters, or a starting state, with no
tools run, is exactly the material a later session cannot reconstruct on its own
— 把需求、参数、起始状态连同情形一起写进 `statement`(紧凑记法,不要照抄对话原句),
不要推给 `details`(`details` 不进检索向量,读取侧也不再注入它)。Use
"fact": null ONLY for greetings and pure chatter with no content whatsoever.
When in doubt, write the fact: an unneeded record costs one row, while a dropped
one costs every later session that needed it and has no way to know it existed.
"""
from ..prompts.en import CODING_EXTRACT_PROMPT as EXTRACT_PROMPT # noqa: E402,F401 (re-export)
# ================================================================
@@ -421,350 +101,7 @@ one costs every later session that needed it and has no way to know it existed.
#
# 修改 prompt 时应使用代表性 coding traces 做回归,确认坐标、决策与验证状态仍保留。
EXTRACT_PROMPT_CODING = """\
╔═══════════════════════════════════════════════════════════════════════════╗
║ HARD CONSTRAINT — read this before anything else, it overrides any ║
║ phrasing later in this prompt that sounds like it wants a narrative. ║
║ ║
║ You are NOT writing a report of what a person did. You are writing what a ║
║ FUTURE AGENT needs in order to act. That agent already has its own task; ║
║ it does not care what this user wanted. It needs to know: what was decided ║
║ here, at which code coordinates, what was tried and rejected, and whether ║
║ it was verified. ║
║ ║
║ Concretely, in every `statement` and every `solution`: ║
║ · The grammatical subject is the CODE or THE DECISION — never "the user" ║
║ and never "the agent". Do not write "The user attempted to…", ║
║ "The user wanted…", "The agent then checked…". ║
║ · Never close with a bookkeeping phrase like "the session was completed", ║
║ "the error was noted", "no solution was found yet". Close with the ║
║ decision and its verification status. If nothing was decided, close ║
║ with what was RULED OUT and the next unexplored lead — that is the ║
║ reusable part. ║
║ · Keep the concrete symptom (error string, failing command, wrong ║
║ output) — that is what a future search will match on. ║
║ ║
║ bad: "The user attempted to write a QTable to RST with header_rows and ║
║ hit a TypeError. The session was completed with the error noted." ║
║ good: "ascii.rst's RST writer rejects header_rows: `tbl.write(..., \ ║
║ format='ascii.rst', header_rows=['name','unit'])` raises TypeError ║
║ because RST.__init__ does not forward it, unlike ascii.fixed_width ║
║ which accepts it. No fix written — the writer needs the same ║
║ header_rows plumbing as fixed_width." ║
╚═══════════════════════════════════════════════════════════════════════════╝
You extract durable WORKING memories from a task-execution session segment.
A "working" session is any hands-on task where the user operates a computer /
system through tools — this covers BOTH software/code work (writing, debugging,
refactoring, building, deploying, API/library usage) AND system & operations
work (running shell/PowerShell commands, disk/storage analysis or cleanup, file
& folder operations, installing/configuring software, environment/device setup,
hardware/disk health inspection, backup, scheduling/automation, data processing).
You produce TWO layers from the SAME segment:
EXPERIENCES (procedural, "how to do it") — reusable method, extracted only
when it clears the VALUE BAR below. A (task, search_keys, solution,
boundary_envs) bundle.
FACT (episodic, "what was established this time") — EXACTLY ONE record of THIS
session. It is NOT a narrative of what the user did. It records, in this order:
the task and its concrete symptom (this is what makes it findable later), the
code coordinates, WHAT WAS SETTLED ON and what was rejected, and how far it was
verified. No value bar, but there is never more than one.
The two layers answer different questions and are stored separately. A method
that generalizes goes in EXPERIENCES; the story of this particular session —
only true for this system / this project / right now — goes in the FACT.
═══════════════════════════════════════════════════════════════════
VALUE BAR (EXPERIENCES ONLY) — DO NOT cross this lightly.
═══════════════════════════════════════════════════════════════════
A memory MUST satisfy ALL of:
(a) Solves a problem likely to recur for the same user
(b) Solution is non-trivial: combines multiple pieces of info / steps, OR
carries a decision rationale, OR captures a hard-won workaround, OR is a
reusable procedure/recipe for an operational task (e.g. how to analyze &
clean up disk usage, how to batch-organize files, how to inspect hardware
health, how to set up a scheduled job)
(c) A future agent / user would prefer looking it up over re-discovering it
IMPORTANT — analysis / diagnostic / investigation tasks COUNT:
A task does NOT have to make a change or "fix" something to be worth remembering.
Read-only / analytical / troubleshooting work is valuable when its PROCEDURE or
CONCLUSION is reusable. Explicitly extract:
- Analysis / diagnostic recipes: the reusable procedure + what it revealed
(e.g. "how to analyze which folders eat C-drive space" — the scan commands,
the gotchas hit, and the finding). "It only produced a report, no change"
is NOT a reason to skip — the reusable METHOD is the memory.
- Investigation outcomes, INCLUDING negative / not-found results
(e.g. "searched folder X across C: and D: with rg / fs_search / SQL index —
not found anywhere"). A recorded dead-end saves the next agent from repeating
the same fruitless search, so it clears the bar.
Do NOT reject a memory merely because "the assistant could regenerate the
commands" or "nothing was actually modified". If the procedure or the conclusion
would save future effort, keep it.
DO NOT extract:
- Truly trivial single-step ops with no reusable procedure ("rename one file",
"git status", "run ls once")
- One-off issues already resolved with no transferable lesson
- Information that is already in the repo/system (don't duplicate package.json)
- Per-PR / per-session throwaway tasks unless they encode a reusable pattern
Heuristic: if the user (or a future agent doing the same kind of task) would
want this written down instead of figuring it out again, keep it. A multi-step
operational procedure (even a system/ops or read-only analysis/search one, not
just code) usually qualifies. Only drop it if it is a single command any user
would find via `--help` in 30 seconds AND carries no decision/gotcha worth
remembering.
═══════════════════════════════════════════════════════════════════
CODE WORK — what a future agent cannot re-derive, so it MUST be captured
═══════════════════════════════════════════════════════════════════
When the segment is software work on a repository, the reusable method is only
half of it. The other half is WHERE the work lives and WHICH VERSION won. Those
cannot be recovered by reading the code later — the code shows the current state,
not which branch it sits on, nor what was already rejected.
This section does NOT ask for a longer record. It asks you to SPEND THE SAME
BUDGET on higher-value tokens: drop restated goals, narration of what the
assistant "then decided to", and pasted code you are only echoing; spend those
characters on the four items below instead. A record that omits the branch but
quotes 30 lines of diff is a bad trade.
1. BRANCH / WORKING LOCATION — verbatim, in the fact `statement`. If the segment
mentions or checks out `feat/x` / `fix/y`, that token MUST appear. An agent
that resumes on the wrong branch produces changes that are silently thrown
away, so this one token outranks every other detail.
2. WHICH VERSION WON — when the user changed their mind, name the winner and mark
the loser in one clause: "settled on A; the interim B was dropped". Not both in
full — one clause. Keeping only the latest is wrong: an agent that meets the
rejected code must recognize it, or it will "fix" things back to the discarded
variant. Phrases like `scratch that / go back to / actually, let's / no, not that`
mark this: what follows is of record, what precedes is superseded.
3. EXACT COORDINATES, NOT PASTED CODE — repo-relative path
(`qutebrowser/utils/debug.py`, never bare `debug.py`) plus the function /
class touched. If a signature changed, give the new signature. If a constant,
regex or literal was decided, quote THAT verbatim — but quote only it, not the
surrounding block. One accurate line beats a paragraph of approximation.
4. VERIFIED vs MERELY WRITTEN — four words is enough ("tests run and passing" /
"confirmed by git diff only, tests not run"). "the edit is in" and "the edit is in
and tested" lead a future agent to do very different things next.
Anti-pattern — summarizing the goal and dropping the coordinates:
bad: "refactored the colour parsing logic to be cleaner."
good: "on feat/q4, in qutebrowser/config/configtypes.py QtColor._parse_value:
settled on `mult = 360.0 if kind == 'h' else 255.0`; the interim version that
used 255.0 for everything was dropped. Tests not run."
(the good one is SHORTER than most bad ones — it just spends every character
on something that cannot be re-derived)
═══════════════════════════════════════════════════════════════════
EXPERIENCE FIELDS
═══════════════════════════════════════════════════════════════════
task — Query-shaped name; phrase as how a user would later ASK.
"publish a release to the xx platform", not "publish_release_internal"
Self-contained (no "this", "the file", "we").
search_keys — 0 to 5 alternative phrasings capturing important aspects of
solution that `task` alone wouldn't match. Phrase as questions
or topics future users would ASK ABOUT. Distinct from task.
Omit if task already covers everything.
solution — Complete, self-contained content the user actually needs:
commands, credential locations (NOT credential VALUES),
file paths, ordered steps, gotchas, decision rationale.
Aggregate multi-turn end-to-end work into ONE solution if it
forms a coherent task; split into multiple memories only when
the user did unrelated tasks.
boundary_envs — Concrete pinning that solution depends on, multi-line "key: value":
runtime / SDK / library + version
API endpoint or method signature
external service / middleware version
env var NAME (not value)
config file path
host / cluster / region identifier
If solution is environment-agnostic, leave as "" (empty).
Purpose: when the underlying SDK/middleware upgrades, future
consumers can inspect this to decide if solution still applies.
confidence — 0.0 to 1.0
1.0 user explicitly stated this as a rule / instruction
0.7 inferred with good evidence from tool actions
0.5 weakly inferred; one observation; uncertain
═══════════════════════════════════════════════════════════════════
FACT FIELD — EXACTLY ONE per segment, no value bar
═══════════════════════════════════════════════════════════════════
You write ONE fact for the whole segment — a recoverable state record for this
session, not a polished retrospective and never a list of separate findings. A
future agent must be able to resume from it without mistaking a suggestion,
inference, or partial search for something verified.
statement — A compact summary of THIS session. It has two jobs at once, and it
must do BOTH: it is the retrieval vector (a future agent finds it by
searching with a fresh issue report, phrased the way a user phrases a
problem), and it is the payload that agent then acts on. So:
OPEN with the task and the symptom, in the words a user would use.
This is what makes it findable — keep the surface form close to how
the problem would be reported. Name the specific goal and object
("reclaim disk space under D:\\WXWork", "adjust the hue multiplier
in QtColor._parse_value on feat/q4", "locate the new_file_search
folder"), never a generic "fix a bug" / "look for a file". Include
the concrete symptom: the error string, the failing command, the
wrong output.
THEN give the coordinates and the decision.
repo-relative path, symbol, and branch for code work. State what was
SETTLED ON, and in the same clause mark what was dropped
("settled on X; the interim Y was dropped"). This is the part that
cannot be re-derived by reading the code — the code only shows the
winner, never that Y was tried and rejected. An agent that meets the
rejected variant later must be able to recognize it, or it will
"fix" the code back to the discarded version.
CLOSE with the evidence status.
Exactly one of: completed, partially completed, failed, blocked, or
still awaiting verification — plus how it was checked ("tests run and
passing" / "confirmed by git diff only, tests not run"). Include
negative results and dead ends without overstating them.
⚠️ Never stop at the narrative. "The user tried X and hit error Y; no
solution found yet" is a wasted record: it retrieves fine but gives a
future agent nothing to act on. If the session genuinely reached no
decision, say what was ruled out and what the next unexplored lead is —
that is the reusable part.
The subject of the sentences should be the code and the decision, not
"the user". 2 to 4 sentences, flowing prose, not labelled sections.
Hard ceiling ~400 latin characters (~200 CJK characters).
EVIDENCE FIDELITY — mandatory:
- Record only actions actually executed and evidence explicitly
present in the trace. Preserve material commands, flags,
arguments, errors, and numbers verbatim. NEVER complete a
command with customary flags or turn advice/plans into actions.
- Preserve the distinction among attempted, observed, verified,
inferred, and unresolved. Do not add those labels to the prose;
express their meaning naturally, e.g. "tried … but was blocked by
Access denied" or "suspect … , not yet verified".
- Without tool output or explicit confirmation, NEVER claim a fix
succeeded, a cause was established, a file/folder does not
exist, or tests passed.
- Scope every negative finding to what was actually searched and
enumerate EVERY explicit coverage limit that could invalidate
it, especially permission failures and maximum search depth:
"not found within the searched parts of C:\\ and D:\\; some
directories were skipped on Access denied, and the search was also
depth-limited". Do not omit one limit to make the
summary prettier. Never promote a partial search to global
absence.
- State causality as certain only when the trace directly verifies
it. Otherwise retain the hedge ("suspected" / "inferred" / "not yet
verified"); do not rewrite an
incorrect test assumption as the system's root cause.
CRITICAL — this text IS the retrieval vector, and it is the only
thing a future query gets matched against. A generic summary is
worthless, because EVERY session looks like a generic summary: it
would match every query and answer none of them. So you MUST name
the concrete anchors that make THIS session findable — real file
paths, function / symbol names, branch names, commands, error
strings, versions, config keys and values, host / service names.
If someone later searches for any one of those specifics, this
summary has to be the thing that comes back.
good: "fixing a failing `npm run build` in CI. Read
.github/workflows/ci.yml and package.json; traced it to webpack
5.74 hitting ERR_OSSL_EVP_UNSUPPORTED under node 18. Bumping the
workflow's node to 20 made the build pass; node 18 still
reproduces locally."
good: "on branch feat/q0, in qutebrowser/utils/debug.py signal_name:
PyQt<5.11 only exposes the name via repr(sig), and repr comes in
both a bracketed and a parenthesised form. Settled on two patterns
tried in order with fullmatch, breaking on the first hit; the
single-regex version was dropped. Confirmed by git diff, tests not
run."
bad: "user wanted a build bug fixed, the agent looked at some config
files, changed some code, and fixed it."
(no path, no command, no error string, no version — nothing
to retrieve on; every session reads exactly like this)
bad: "fixed the signal-name parsing compatibility issue on a feature
branch."
(which branch? which file? which function? which of the two
competing versions won? nothing here survives being
retrieved a week later)
Fidelity examples (bad → good):
"ran Get-ChildItem -Force" → "ran Get-ChildItem"
(the trace did not contain `-Force`; do not invent it)
"confirmed new_file_search does not exist" →
"new_file_search was not found within the searched parts of C:\\ and
D:\\; some directories were skipped on Access denied or by the depth
limit, so the investigation is only partly done"
"identical payloads cause a spurious lock error" →
"the verification script assumed the last writer is always a fixed
rank; that assumption does not hold, so whether there is a lock bug
is still unverified"
Do NOT record greetings or pure chatter. If the segment established
nothing at all, output "fact": null.
files — 0..N absolute or repo-relative FILE OR DIRECTORY paths actually
operated on or investigated during this session. Keep investigated
roots such as `C:\\` and `D:\\`. Only include paths that were
actually accessed/searched, not paths merely suggested or planned;
[] if none apply.
═══════════════════════════════════════════════════════════════════
HOST CONTEXT (given by SDK)
═══════════════════════════════════════════════════════════════════
- workspace_id present: {workspace_present}
- branch present: {branch_present}
When no workspace_id is present, the solution cannot rely on unstated
project-local context — spell out any repo / path / service it needs inside
`solution` or `boundary_envs` so the memory stays usable on its own.
═══════════════════════════════════════════════════════════════════
CONVERSATION
═══════════════════════════════════════════════════════════════════
{conversation_block}
═══════════════════════════════════════════════════════════════════
OUTPUT (strict JSON only, no markdown)
═══════════════════════════════════════════════════════════════════
{{
"experiences": [
{{
"task": "...",
"search_keys": ["...", "..."],
"solution": "...",
"boundary_envs": "...",
"confidence": 0.9
}}
],
"fact": {{
"statement": "...",
"files": ["..."]
}}
}}
MANDATORY: You MUST output at least ONE experience. Do NOT return an empty list.
Even if the task looks simple or is a one-shot success, still capture the
reusable essence of what was done (the method / command / how-to). Only the
single most valuable experience is required when the session is thin — but never
output an empty "experiences" array.
"fact" is a SINGLE object, never a list — one record per session. Use
"fact": null only when the segment is pure chatter that established nothing.
"""
from ..prompts.en import CODING_EXTRACT_PROMPT_CODING as EXTRACT_PROMPT_CODING # noqa: E402,F401 (re-export)
# ================================================================
@@ -839,15 +176,22 @@ class CodingMemoryExtractor:
if str(message.content or "").strip()
)
def _select_prompt_template(self, messages: List[ChatMessage]) -> Tuple[str, str]:
"""Return (template, language) for this write's new messages."""
if not is_chinese(self._language_text(messages)):
return self._prompt_tmpl, "en"
def _select_prompt_template(
self, messages: List[ChatMessage], language: Optional[str] = None,
) -> Tuple[str, str]:
"""Return (template, language code) for this write's new messages.
from .prompts_zh import EXTRACT_PROMPT_CODING_ZH, EXTRACT_PROMPT_ZH
prompt = EXTRACT_PROMPT_CODING_ZH if self.scene == "coding" else EXTRACT_PROMPT_ZH
return prompt, "zh"
Explicit ``language`` wins; otherwise zh/en auto-detection over the new
user text (unchanged behaviour).
"""
code = resolve_language(language, self._language_text(messages))
if code == "en":
return self._prompt_tmpl, code
name = (
"CODING_EXTRACT_PROMPT_CODING" if self.scene == "coding"
else "CODING_EXTRACT_PROMPT"
)
return get_prompt(name, code), code
async def extract(
self,
@@ -861,12 +205,14 @@ class CodingMemoryExtractor:
branch: Optional[str] = None,
session_id: Optional[str] = None,
request_id: str = "",
language: Optional[str] = None,
) -> Tuple[List[CodingMemoryDraft], List[WorkingFactDraft]]:
"""
Args:
messages: 本次 write 的完整新消息;只做单条限长,不裁 turn
history_messages: 之前 write 的历史消息;仅取最近 5 个 user-turn 作消解上下文
user_id / agent_id / workspace_id / branch / session_id: SDK 注入
language: 显式 prompt 语言码;None 时按新 user 文本做 zh/en 自动检测
Returns:
(经验草稿, 事实草稿)。两层都可能为空。只有显式启用
@@ -892,27 +238,26 @@ class CodingMemoryExtractor:
history_block = self._format_conversation(history_window).strip()
new_block = self._format_conversation(new_messages).strip()
prompt_tmpl, prompt_language = self._select_prompt_template(new_messages)
if prompt_language == "zh":
conversation_block = (
"最近 5 个用户轮次(仅作上下文,不要重复抽取):\n"
+ (history_block or "(无)")
+ "\n\n本次新增消息(只从这次完整写入中抽取):\n"
+ (new_block or "(无)")
)
else:
conversation_block = (
"LAST 5 USER TURNS (CONTEXT ONLY; do not extract them again):\n"
+ (history_block or "(none)")
+ "\n\nNEW MESSAGES (extract only from this complete write):\n"
+ (new_block or "(none)")
)
explicit_language = normalize_language(language)
prompt_tmpl, prompt_language = self._select_prompt_template(
new_messages, explicit_language,
)
empty_block = get_prompt("CODING_EMPTY_BLOCK", prompt_language)
conversation_block = get_prompt(
"CODING_CONVERSATION_BLOCK", prompt_language,
).format(
history=history_block or empty_block,
new=new_block or empty_block,
)
prompt = prompt_tmpl.format(
workspace_present=str(bool(workspace_id)).lower(),
branch_present=str(bool(branch)).lower(),
conversation_block=conversation_block,
)
prompt = with_output_language(
prompt, prompt_language, explicit=explicit_language is not None,
)
_t0 = time.perf_counter()
try:
@@ -1166,7 +511,10 @@ class CodingMemoryExtractor:
obj = None
if not isinstance(obj, dict):
logger.warning(f"[coding-extract] failed to parse output: {text!r}")
logger.warning(
f"[coding-extract] failed to parse output ({len(text or '')} chars)"
)
logger.debug(f"[coding-extract] unparsable output: {text!r}")
return [], []
# "memories" 是双层化之前的 key,老 prompt 缓存 / 旧模型可能还这么回
+52 -166
View File
@@ -23,7 +23,12 @@ from typing import Any, Dict, List, Optional, Tuple
from ..agent.llm_provider import LLMProvider
from ..core.embed_service import EmbedService
from ..utils.lang_detect import is_chinese
from ..prompts import (
get_prompt,
normalize_language,
resolve_language,
with_output_language,
)
from .store import CodingMemoryStore
from .types import (
@@ -206,143 +211,10 @@ def _subject_conflict(new_subject: str, old_subject: str) -> bool:
# General subject-isolation rules belong in the prompt; lookalike examples
# encourage pattern matching instead of teaching the rule.
FACT_RECONCILE_PROMPT = """\
You maintain session-level WORKING FACTS. Each statement summarizes one whole
trace/session: the user's task, the rough approach, and the result. It is not an
atomic claim.
Every new session fact below WILL be stored; that is not your decision. Your only
job is to detect task continuation: is the new session continuing to advance the
SAME user task as an existing session fact? If so, SUPERSEDE the older fact so
the links form a cross-session progress chain for that task.
Mark an existing fact as superseded ONLY when there is strong evidence that:
- The subjects match: both records are about the SAME entity — the same
person, the same file/module, the same host/service. A shared subject is
necessary but not sufficient; AND
- The user goal is materially the SAME, not merely in the same topic/project;
AND
- The new trace continues, retries, extends, verifies, or finishes the work
recorded by the existing trace.
Do NOT supersede when:
- The two records name different subjects. Two people in one group, two files
in one module, two hosts in one cluster are PARALLEL, not successive: one
person's state is not a newer version of another's. Where a subject is shown
as (unspecified), fall back to judging by the statement alone.
- The sessions merely mention the same file, symbol, component, repository,
error, or technology but pursue different user goals.
- One task only supplies context or a technique for another task.
- The relationship is uncertain. Be conservative: INSERT is the safe default;
merging unrelated sessions destroys their separate task history.
Treat each statement as evidence-bounded text. Do not reinterpret an inference
inside it as an established cause or use a shared conjecture/error as proof that
the user goals are the same. Your decision is only about task continuation; do
not "correct", complete, or embellish either session record.
Examples:
existing: "The user needed to fix CI's npm run build; traced it to
ERR_OSSL_EVP_UNSUPPORTED under Node 18, workflow not yet changed."
new: "The user continued fixing the same CI build; raised Node in
.github/workflows/ci.yml to 20 and npm run build passed."
→ SUPERSEDE
existing: "The user needed to fix JWT expiry handling in
auth/middleware.ts; refresh logic was added."
new: "The user needed performance logging added to
auth/middleware.ts; a latency metric was introduced."
→ keep both
(same file/project, different user goal)
═══════════════════════════════════════════════════════════════════
NEW SESSION FACTS
═══════════════════════════════════════════════════════════════════
{drafts_block}
═══════════════════════════════════════════════════════════════════
EXISTING SESSION FACTS (per new fact, retrieved by statement similarity)
═══════════════════════════════════════════════════════════════════
{candidates_block}
═══════════════════════════════════════════════════════════════════
OUTPUT (strict JSON only, no markdown)
═══════════════════════════════════════════════════════════════════
{{
"ops": [
{{"draft_idx": 0, "supersedes": ["fact_xxx"], "reason": "continues the same CI fix task"}},
{{"draft_idx": 1, "supersedes": []}}
]
}}
Only use fact ids that appear in the candidate list for that same draft_idx.
If nothing is superseded anywhere, output {{"ops": []}}.
"""
from ..prompts.en import CODING_FACT_RECONCILE_PROMPT as FACT_RECONCILE_PROMPT # noqa: E402,F401 (re-export)
CHAIN_SUMMARY_PROMPT = """\
A user task in a working-memory store has progressed across sessions. Below are
the session summaries in that task's progress chain, oldest first, ending with
the current session.
Summarize how the SAME task progressed across sessions to its current state.
Cover every important stage in chronological order. For each stage, preserve
where available: the actual action or change, its direct result, and any
unresolved issue or limiting condition. End with an explicit current status.
Rules:
- This is compression only. Report ONLY what the entries say; never add or
complete a command, cause, motive, action, test result, success state, or
global negative conclusion.
- Preserve evidence status exactly: attempted is not completed, observed is
not verified, inferred is not established, and unresolved stays unresolved.
Keep uncertainty words and search/permission/coverage limits.
- State the current ending status faithfully (completed, partially completed,
failed, blocked, or still awaiting verification). Never upgrade it based on
a plausible causal story or on a later entry that did not verify it.
- When entries mention a suspected cause and a later result, connect them
causally only if an entry directly says that cause was verified.
- Preserve concrete retrieval anchors: file paths, symbols, commands, error
strings, versions, config keys/values, and important numeric values that
matter to the progress.
- For chains of 2-3 generations, do not over-compress the progression into one
sentence. Prefer 2-4 compact paragraphs when useful, or readable prose with
a clear task goal, stage-by-stage progression, and current status.
- Do not mechanically copy the full history. Omit repetition and low-value
detail while retaining important actions, direct results, and leftovers.
- Do not invent stage headings or dates. Mention a date only when the entries
make it meaningful; bare timestamps are noise.
- If the entries say older revisions were omitted, explicitly state that
earlier history was omitted; do not imply the supplied entries are complete.
- Write in the same language the entries are written in.
- No preamble or enclosing quotes.
Example
entries:
(2026-07-20) The user needed to fix CI's npm run build; traced it to
ERR_OSSL_EVP_UNSUPPORTED under Node 18, workflow not yet
changed.
(2026-07-24) The user continued that CI fix; raised Node in
.github/workflows/ci.yml to 20, but the dependency cache
still reproduced the old error.
(now) The user continued verifying the same fix; after clearing the
npm cache, npm run build passed.
output:
The task goal is fixing CI's npm run build. It was first traced to
ERR_OSSL_EVP_UNSUPPORTED under Node 18, with the workflow not yet changed;
then .github/workflows/ci.yml was raised to Node 20, but the dependency
cache kept the old error present. Finally, after clearing the npm cache,
npm run build passed, so this build fix is currently verified as complete.
═══════════════════════════════════════════════════════════════════
ENTRIES
═══════════════════════════════════════════════════════════════════
{chain_block}
═══════════════════════════════════════════════════════════════════
OUTPUT (the summary itself, nothing else)
═══════════════════════════════════════════════════════════════════
"""
from ..prompts.en import CODING_CHAIN_SUMMARY_PROMPT as CHAIN_SUMMARY_PROMPT # noqa: E402,F401 (re-export)
def _fact_drafts_language_text(drafts: List[WorkingFactDraft]) -> str:
@@ -357,37 +229,34 @@ def _fact_drafts_language_text(drafts: List[WorkingFactDraft]) -> str:
def build_fact_reconcile_prompt(
drafts: List[WorkingFactDraft],
candidates: List[List[WorkingFact]],
language: Optional[str] = None,
) -> Tuple[str, str]:
"""Return the language-matched fact reconcile prompt and language."""
if is_chinese(_fact_drafts_language_text(drafts)):
from .prompts_zh import FACT_RECONCILE_PROMPT_ZH
"""Return the fact reconcile prompt and its language code.
template = FACT_RECONCILE_PROMPT_ZH
language = "zh"
else:
template = FACT_RECONCILE_PROMPT
language = "en"
return template.format(
Explicit ``language`` wins; otherwise zh/en auto-detection over the drafts.
"""
explicit = normalize_language(language)
code = resolve_language(explicit, _fact_drafts_language_text(drafts))
prompt = get_prompt("CODING_FACT_RECONCILE_PROMPT", code).format(
drafts_block=WorkingFactReconciler._format_drafts(drafts),
candidates_block=WorkingFactReconciler._format_candidates(drafts, candidates),
), language
)
return with_output_language(prompt, code, explicit=explicit is not None), code
def build_chain_summary_prompt(
chain_block: str,
*,
current_statement: str,
language: Optional[str] = None,
) -> Tuple[str, str]:
"""Return the language-matched cross-session summary prompt and language."""
if is_chinese(current_statement or chain_block):
from .prompts_zh import CHAIN_SUMMARY_PROMPT_ZH
template = CHAIN_SUMMARY_PROMPT_ZH
language = "zh"
else:
template = CHAIN_SUMMARY_PROMPT
language = "en"
return template.format(chain_block=chain_block), language
"""Return the cross-session summary prompt and its language code."""
explicit = normalize_language(language)
code = resolve_language(explicit, current_statement or chain_block)
prompt = get_prompt("CODING_CHAIN_SUMMARY_PROMPT", code).format(
chain_block=chain_block,
)
return with_output_language(prompt, code, explicit=explicit is not None), code
# ================================================================
@@ -455,9 +324,11 @@ class WorkingFactReconciler:
branch: Optional[str] = None,
source: str = "default",
memory_at: Optional[datetime] = None,
language: Optional[str] = None,
) -> Tuple[List[WorkingFact], List[FactReconcileOp]]:
"""
执行 reconcile + 持久化。memory_at 是这段对话的时间,记到新事实上。
language 为显式 prompt 语言码(None → zh/en 自动检测),同时用于演变总结。
Returns:
(真正落库的新事实, 执行过的 ops)。LLM 挂掉时退化为全 INSERT 不覆盖,
@@ -477,7 +348,7 @@ class WorkingFactReconciler:
)
try:
supersedes = await self._llm_decide(drafts, candidates)
supersedes = await self._llm_decide(drafts, candidates, language=language)
except Exception as e:
logger.warning(
f"[fact-reconcile] LLM decide failed, insert without supersede: {e}"
@@ -495,6 +366,7 @@ class WorkingFactReconciler:
branch=branch,
source=source,
memory_at=memory_at,
language=language,
)
logger.info(
f"[fact-reconcile] drafts={len(drafts)} inserted={len(facts)} "
@@ -570,9 +442,12 @@ class WorkingFactReconciler:
kept.append(fact)
if blocked:
logger.info(
f"[fact-reconcile] subject gate blocked {len(blocked)} candidate(s) "
f"for subject={draft.subject!r}: "
+ ", ".join(f"{f.fact_id}({f.subject!r})" for f in blocked)
f"[fact-reconcile] subject gate blocked {len(blocked)} candidate(s): "
+ ", ".join(f.fact_id for f in blocked)
)
logger.debug(
f"[fact-reconcile] subject gate: draft subject={draft.subject!r} "
f"blocked subjects={[f.subject for f in blocked]!r}"
)
result.append(kept)
return result
@@ -585,13 +460,14 @@ class WorkingFactReconciler:
self,
drafts: List[WorkingFactDraft],
candidates: List[List[WorkingFact]],
language: Optional[str] = None,
) -> Dict[int, List[str]]:
"""返回 {draft_idx: [被它覆盖的旧 fact_id, ...]}。"""
# 一条候选都没有就没得判,省一次 LLM 调用
if not any(candidates):
return {}
prompt, _ = build_fact_reconcile_prompt(drafts, candidates)
prompt, _ = build_fact_reconcile_prompt(drafts, candidates, language=language)
resp = await self.llm_provider.complete(
prompt=prompt,
max_tokens=self.max_tokens,
@@ -635,7 +511,8 @@ class WorkingFactReconciler:
return ordered[-MAX_FACT_CHAIN_ENTRIES:], truncated
async def _summarize_chain(
self, history: List[WorkingFact], current: str, *, truncated: bool = False
self, history: List[WorkingFact], current: str, *, truncated: bool = False,
language: Optional[str] = None,
) -> str:
"""让 LLM 把同一任务的跨 session 进展写成简短摘要。
@@ -656,6 +533,7 @@ class WorkingFactReconciler:
prompt, _ = build_chain_summary_prompt(
"\n".join(rows),
current_statement=current,
language=language,
)
try:
@@ -745,7 +623,10 @@ class WorkingFactReconciler:
obj = None
if not isinstance(obj, dict):
logger.warning(f"[fact-reconcile] failed to parse ops output: {text!r}")
logger.warning(
f"[fact-reconcile] failed to parse ops output ({len(text or '')} chars)"
)
logger.debug(f"[fact-reconcile] unparsable ops output: {text!r}")
return {}
ops_raw = obj.get("ops")
if not isinstance(ops_raw, list):
@@ -784,6 +665,7 @@ class WorkingFactReconciler:
branch: Optional[str] = None,
source: str = "default",
memory_at: Optional[datetime] = None,
language: Optional[str] = None,
) -> Tuple[List[WorkingFact], List[FactReconcileOp]]:
inserted: List[WorkingFact] = []
ops: List[FactReconcileOp] = []
@@ -835,8 +717,11 @@ class WorkingFactReconciler:
# 别的主体的状态不是。闸门关掉时这道复查一并跳过,否则闸门就没关干净。
if self.subject_gate and _subject_conflict(draft.subject, old.subject):
logger.warning(
f"[fact-reconcile] refused cross-subject SUPERSEDE: "
f"{old.fact_id}({old.subject!r}) by draft subject {draft.subject!r}"
f"[fact-reconcile] refused cross-subject SUPERSEDE: {old.fact_id}"
)
logger.debug(
f"[fact-reconcile] cross-subject: old subject={old.subject!r} "
f"draft subject={draft.subject!r}"
)
continue
try:
@@ -864,7 +749,8 @@ class WorkingFactReconciler:
retired, user_id=user_id
)
new_fact.fact_chain_summary = await self._summarize_chain(
history, new_fact.statement, truncated=truncated
history, new_fact.statement, truncated=truncated,
language=language,
)
# 摘要之外再把历代自己的 details 原样过继一份。摘要保叙事,过继保数值,
# 两者不重复:details 本来就不进摘要(_collect_history 只读 statement)。
+21 -10
View File
@@ -195,8 +195,10 @@ async def classify_messages_scene(
result = _parse_scene_json(resp.content)
if result is None:
logger.warning(
f"[coding-judge] failed to parse classify_messages output: {resp.content!r}; defaulting to chat"
f"[coding-judge] failed to parse classify_messages output "
f"({len(resp.content or '')} chars); defaulting to chat"
)
logger.debug(f"[coding-judge] unparsable classify_messages output: {resp.content!r}")
await _store_coding_log(
cache, request_id=request_id, user_id=user_id, agent_id=agent_id,
session_id=session_id, step="CODING_JUDGE", prompt=prompt,
@@ -207,9 +209,8 @@ async def classify_messages_scene(
return "chat"
scene = result["scene"]
parsed = {**result, "is_coding": scene != "chat"}
logger.info(
f"[coding-judge] scene={scene} reason={result.get('reason', '')!r}"
)
logger.info(f"[coding-judge] scene={scene}")
logger.debug(f"[coding-judge] scene={scene} reason={result.get('reason', '')!r}")
await _store_coding_log(
cache, request_id=request_id, user_id=user_id, agent_id=agent_id,
session_id=session_id, step="CODING_JUDGE", prompt=prompt,
@@ -295,8 +296,10 @@ async def classify_queries_scene(
result = _parse_scene_json(resp.content)
if result is None:
logger.warning(
f"[coding-judge] failed to parse classify_queries output: {resp.content!r}; defaulting to chat"
f"[coding-judge] failed to parse classify_queries output "
f"({len(resp.content or '')} chars); defaulting to chat"
)
logger.debug(f"[coding-judge] unparsable classify_queries output: {resp.content!r}")
return "chat"
scene = result["scene"]
logger.info(f"[coding-judge] queries scene={scene}")
@@ -422,15 +425,18 @@ async def classify_and_rewrite_queries(
scene = _parse_scene_label(head)
if scene is None:
logger.warning(
f"[coding-judge] classify_and_rewrite unrecognized head={head!r}; falling back to chat"
f"[coding-judge] classify_and_rewrite unrecognized head "
f"({len(head)} chars); falling back to chat"
)
logger.debug(f"[coding-judge] classify_and_rewrite unrecognized head={head!r}")
return {"scene": "chat", "is_coding": False, "rewrite_query": rewrite, "ok": False}
is_coding = scene != "chat"
logger.info(
f"[coding-judge] classify_and_rewrite scene={scene} "
f"rewrite={rewrite!r}"
f"rewrite({len(rewrite)} chars)"
)
logger.debug(f"[coding-judge] classify_and_rewrite rewrite={rewrite!r}")
return {"scene": scene, "is_coding": is_coding, "rewrite_query": rewrite, "ok": True}
@@ -569,8 +575,9 @@ async def analyze_queries(
parsed_scene = _parse_scene_label(head)
if parsed_scene is None:
logger.warning(
f"[coding-judge] analyze_queries unrecognized head={head!r}; ok=False"
f"[coding-judge] analyze_queries unrecognized head ({len(head)} chars); ok=False"
)
logger.debug(f"[coding-judge] analyze_queries unrecognized head={head!r}")
return {**fallback, "since_raw": ""}
scene = parsed_scene
idx = 1
@@ -589,9 +596,10 @@ async def analyze_queries(
created_after = _since_to_created_after(since_raw, current_time)
logger.info(
f"[coding-judge] analyze scene={scene} rewrite={rewrite!r} "
f"[coding-judge] analyze scene={scene} rewrite({len(rewrite)} chars) "
f"since={since_raw!r} created_after={created_after}"
)
logger.debug(f"[coding-judge] analyze rewrite={rewrite!r}")
return {
"scene": scene, "is_coding": scene != "chat", "rewrite_query": rewrite,
"created_after": created_after, "since_raw": since_raw, "ok": True,
@@ -614,7 +622,10 @@ def _since_to_created_after(since_raw: str, current_time: str) -> Optional[float
return dt.timestamp()
except (ValueError, TypeError):
continue
logger.warning(f"[coding-judge] analyze_queries unparsable SINCE={since_raw!r}")
logger.warning(
f"[coding-judge] analyze_queries unparsable SINCE ({len(since_raw)} chars)"
)
logger.debug(f"[coding-judge] analyze_queries unparsable SINCE={since_raw!r}")
return None
+18
View File
@@ -11,6 +11,7 @@ Coding Memory 预处理工具(规则,零 LLM)
使用方式与 scene 路由见 docs/scene_coding_usage.md。
"""
import re
from typing import Any, Dict, List, Optional, Tuple
from ..pipelines.base import ChatMessage
@@ -63,6 +64,13 @@ DEFAULT_MAX_TOOL_RESULT_BYTES = 2048
DEFAULT_HEAD_BYTES = 1024
DEFAULT_TAIL_BYTES = 512
# 本模块自己产出的省略标记。截断结果会再次经过截断(落库前一次、读历史时又一次),
# 认出自己的标记才能保证幂等:已截断的文本再截一次原样返回,而不是把省略计数改写。
_TOOL_OMIT_MARKER_RE = re.compile(r"\n\n\[\.\.\.\d+ bytes omitted by truncate\.\.\.\]\n\n")
# 标记本身的字节数上限(数字位数有限),用于判断「已截断且只超出一个标记的长度」。
_TOOL_OMIT_MARKER_MAX_BYTES = 128
_ASSISTANT_OMIT_MARKER_RE = re.compile(r" \[\.\.\.\d+ chars omitted\.\.\.\] ")
def truncate_tool_result_text(
text: str,
@@ -81,6 +89,12 @@ def truncate_tool_result_text(
raw = text.encode("utf-8")
if len(raw) <= max_bytes:
return text
# 已截断过(max_bytes 很小时标记本身会让结果略超上限):原样返回,保证幂等。
if (
len(raw) <= max_bytes + _TOOL_OMIT_MARKER_MAX_BYTES
and _TOOL_OMIT_MARKER_RE.search(text)
):
return text
# 自定义 max_bytes 可能小于默认 head+tail(1024+512)。按原比例缩放内容窗口,
# 给省略标记预留空间,不能因为窗口和过大而把超限原文直接返回。
marker_budget = min(96, max(24, max_bytes // 4))
@@ -161,6 +175,10 @@ def truncate_assistant_text(
text = "" if text is None else str(text)
if len(text) <= head_chars + tail_chars:
return text
# 已截断过:中段恰好是本函数的省略标记 → 原样返回,保证幂等(否则会把
# 省略计数改写成标记自身的长度)。
if _ASSISTANT_OMIT_MARKER_RE.fullmatch(text[head_chars:len(text) - tail_chars]):
return text
omitted = len(text) - head_chars - tail_chars
return (
text[:head_chars]
+9 -741
View File
@@ -1,745 +1,13 @@
# -*- coding: utf-8 -*-
"""HY Memory working/coding 链路使用的中文提示词模板。"""
"""HY Memory working/coding 链路使用的中文提示词模板(兼容入口)。
EXTRACT_PROMPT_ZH = """\
你需要从一段任务执行会话中抽取可长期保留的 WORKING 记忆。
“working”会话是指用户通过工具操作计算机或系统的任何实操任务——既包括软件/代码
工作(编写、调试、重构、构建、部署、API/library 使用),也包括系统与运维工作
(运行 shell/PowerShell 命令、磁盘/存储分析或清理、文件与文件夹操作、安装/配置
软件、环境/设备设置、硬件/磁盘健康检查、备份、调度/自动化、数据处理)。
你需要从同一段会话中产出两个层次:
EXPERIENCES(过程性,“如何做”)——可复用的方法;只有达到下述 VALUE BAR 时才
抽取。每条是一个 (task, search_keys, solution, boundary_envs) 组合。
FACT(情节性,“这次发生了什么”)——关于本次会话的恰好一条记录:用户想做什么、
大致如何推进、最终结果如何。它不受价值门槛限制,但绝不能超过一条。
这两个层次回答不同问题,并分别存储。可泛化的方法放入 EXPERIENCES;只对这个系统、
这个项目、此时此刻成立的本次会话经历放入 FACT。
═══════════════════════════════════════════════════════════════════
VALUE BAR(仅适用于 EXPERIENCES)——不要轻易越过此门槛。
═══════════════════════════════════════════════════════════════════
一条记忆必须同时满足以下全部条件:
(a) 解决了同一用户以后很可能再次遇到的问题
(b) 解决方案并非琐碎操作:它组合了多项信息/多个步骤,或者包含决策理由,或者记录了
来之不易的 workaround,或者是某项运维任务的可复用流程/配方(例如如何分析并
清理磁盘占用、如何批量整理文件、如何检查硬件健康、如何设置 scheduled job)
(c) 未来的 agent / 用户会更愿意查阅它,而不是重新摸索
重要——分析、诊断、调查任务也算:
一项任务不必实际修改或“修复”任何东西才值得记忆。只读、分析、故障排查工作,只要其
流程或结论可复用,就有价值。明确抽取:
- 分析/诊断配方:可复用的流程以及它揭示的结果(例如“如何分析哪些文件夹占用了
C-drive 空间”——扫描命令、遇到的坑和最终发现)。“只生成了报告,没有修改”
不是跳过理由——可复用的方法本身就是记忆。
- 调查结果,包括否定结果或未找到结果(例如“使用 rg / fs_search / SQL index
在 C: 和 D: 中搜索 folder X——任何位置都未找到”)。记录一条死路可以避免下一个
agent 重复同样无果的搜索,因此达到了价值门槛。
不要仅仅因为“assistant 可以重新生成这些命令”或“实际上没有修改任何东西”就拒绝
一条记忆。只要流程或结论能节省未来的工作,就保留它。
不要抽取:
- 没有可复用流程、真正琐碎的单步操作(“rename one file”、“git status”、
“run ls once”)
- 已经解决且没有可迁移经验的一次性问题
- repo/system 中已经存在的信息(不要重复 package.json)
- 每个 PR / 每次会话的一次性任务,除非其中编码了可复用模式
- 向用户询问缺失信息这一行为。请求参数、列出未知项或引导用户提供必填字段属于
对话行为,不是方法:每个 agent 都知道如何提问,因此以后没有任何值得查阅的内容。
绝不要产出 task 或 search_keys 形如“how to ask for X”、“gather the missing Y”、
“eliciting required fields”、“what to clarify before starting”的 experience。
注意,这条限制只适用于 EXPERIENCES——会话最终停在等待用户的状态是真实状态,
应写入 FACT。
如果该段会话因为等待用户而没有取得任何实质进展——assistant 只是询问参数,或者在
开展任何实际工作前就被阻塞——输出 "experiences": [],只写 FACT。此时尚未学到任何
东西,因此没有方法可存。这一点比看起来更重要:从停滞中提炼出的 experience 在检索时
会像一项真正的技能;同一任务的后续片段会匹配它,再次向用户提问而不是继续推进,导致
这种停滞在余下会话中不断自我复制。应根据工具实际做了什么判断是否有实质进展,而不是
根据该轮是否生成了流畅文本。
启发式标准:如果用户(或未来执行同类任务的 agent)希望把它记下来,而不是以后再重新
摸索,就保留。多步骤运维流程通常符合条件——包括 system/ops 或只读分析/搜索流程,
不局限于代码。只有当它只是任何用户都能在 30 秒内通过 `--help` 找到的单条命令,并且
不包含值得记忆的决策或坑点时,才丢弃。
═══════════════════════════════════════════════════════════════════
EXPERIENCE 字段
═══════════════════════════════════════════════════════════════════
task — 采用查询式名称;写成用户以后会如何提问。
使用“publish a release to the xx platform”,不要使用
“publish_release_internal”。
必须自包含(不要使用“this”、“the file”、“we”)。
search_keys — 0 到 5 种替代表述,用来覆盖仅靠 `task` 无法匹配、但对 solution 很
重要的方面。写成未来用户会询问的问题或主题,且必须与 task 有区别。
如果 task 已覆盖全部内容,则省略。
solution — 用户实际需要的完整、自包含内容:命令、凭据位置(不是凭据值)、
文件路径、有序步骤、坑点、决策理由。如果多轮端到端工作构成一个连贯
任务,应聚合成一个 solution;只有用户执行了互不相关的任务时才拆成
多条记忆。
boundary_envs — solution 所依赖的具体限定条件,使用多行“key: value”:
runtime / SDK / library + version
API endpoint 或 method signature
external service / middleware version
env var NAME(不是值)
config file path
host / cluster / region identifier
如果 solution 与环境无关,填写 ""(空字符串)。
目的:底层 SDK/middleware 升级后,未来使用者可以据此判断 solution
是否仍然适用。
confidence — 0.0 到 1.0
1.0 用户明确将其陈述为规则/指令
0.7 根据工具操作及充分证据推断
0.5 弱推断;只有一次观察;不确定
═══════════════════════════════════════════════════════════════════
FACT 字段——每段会话恰好一条,无价值门槛
═══════════════════════════════════════════════════════════════════
你要为整段会话写一条 FACT——它是本次会话可恢复的状态记录,不是精心润色的回顾,也
绝不是分散发现的列表。未来 agent 必须能够从中继续推进,且不会把建议、推断或局部搜索
误认为已经验证的事实。
subject — 本次会话围绕谁或什么展开:工作所针对或服务的单一实体。使用简短名词
短语,不要写句子,不要使用动词。选择会话真正围绕的最具体实体,而不是
所使用的工具:
会话服务于某个具名人员时,使用人名
(“Dylan”、“Matthew”)
工作针对代码时,使用 file / module / symbol
(“auth/middleware.ts”、“parse_plan_text”)
运维工作使用 service / host / device / drive
(“D:\\WXWork”、“redis-cache-prod”、“ci.yml”)
这是 HARD KEY,不是散文:关于不同人员或不同文件的两次会话必须得到不同
subject;关于同一实体的两次会话必须得到拼写完全一致的同一字符串。
优先使用 trace 中出现的精确标识符,而不是对它的描述。如果该段确实没有
单一 subject,输出 ""。
这很重要:subject 是决定后续会话是否可以按“同一任务的继续”覆盖本条
记录的闸门。填错后,一个人的状态可能静默替换另一个人的状态。
statement — 本次会话的紧凑记录。它分为两部分,而第二部分最容易丢失。
保留任务范围内的给定量和具体约束,不要用泛泛的结果评价取代它们。
每条规则只写一次:重复的指令文字会挤占记忆真正需要保留的数据。
PART 1——发生了什么,约 300 个拉丁字符(约 150 个 CJK 字符)。
从具体目标和对象开始(“reclaim disk space taken by D:\\WXWork”、
“locate the new_file_search folder”),不要写泛泛的“fix a bug”/
“find a file”。然后概述检查、尝试或修改了什么。最后说明结束状态,
且语义必须精确对应以下之一:completed、partially completed、failed、
blocked、still awaiting verification。纳入否定结果和死路,但不要夸大。
PART 2——TASK-SCOPED GIVENS:整个任务赖以运行的全部给定信息,即使并非
本段会话建立,或看起来基础到不值得写。包括绑定关系(每个 name/symbol/
label 代表什么)、任务提供而不是推导出的固定值、采用的约定(indexing
base、units、timezone、defaults),以及本段自己的结果和值(写“x = 7”,
不要只写“the value was computed”)。
将 PART 2 写成数据,每行一项,值保持给定形式:
retry budget = 5; timeout 30s; index starts at 0
step 3 input: n=64, tolerance 1e-6
slot B: type=private, cost 688.7-721.3 per unit
每一行几乎都应由值构成。保留 trace 的 notation——symbols、units、
numbers、spellings——删掉其句子外壳:去掉那些只是宣告“这是给定量”的
短语,而非给定量本身(“I would prefer”、“it should be somewhere
between … and …”)。这些短语会在每项重复;一个 15 项列表会为它们
支付 15 次空间成本。
每一条 FACT 都要携带这些信息;绝不要写“as above”或“same as the
previous session”——检索只会按相似度返回少量 facts,被指代项可能不在
其中。
总长度尽量控制在 1200 字符以内。如果 PART 2 放不下,应优化 notation,
绝不能削减覆盖范围:statement 是 retrieval vector,也是后续 segment
唯一能读到的字段(details 既不做 embedding,也不会被注入),因此从中
删除的内容就永久丢失。压缩给定量周围的措辞,绝不要压缩给定量本身。
检验标准:如果下一段看不到原始材料,能否仅凭此记录继续任务?只有肯定
答案才合格。缺失 binding 不会明显报错——该名称仍看起来有意义,于是会
按其通用含义被理解,后续答案可以很流畅,却回答了另一个问题。
LANGUAGE——使用与 session trace 相同的语言:statement 是 retrieval
vector,而查询会使用用户的工作语言;改用另一种语言会让每次检索都付出
跨语言匹配损失。
EVIDENCE FIDELITY——只记录 trace 中实际执行的操作和明确存在的证据。
对关键命令、flags、arguments、errors、numbers 保持原样。
绝不要按惯例补全命令 flags,也不要把建议或计划写成已经执行的动作:
“ran Get-ChildItem -Force” → “ran Get-ChildItem”
(trace 中没有 `-Force`,不要杜撰)
每个否定发现都必须限定在实际搜索范围内,并逐一列出可能使其失效的所有
明确覆盖限制,尤其是 permission failures 和 depth limits:保留当时的
roots、filters、depth(“not found under C:\\ and D:\\ at depth 3;
some directories were not covered due to Access denied”),绝不要扩大为
“does not exist”。不要为了让记录更漂亮而漏掉任何限制。
绝不要把局部搜索提升为全局不存在。
被阻塞的尝试必须保留其 blocker。
保留 attempted、observed、verified、inferred、unresolved 之间的区别;
用自然语言表达(“tried … but was blocked by Access denied”、
“suspected …, not yet verified”),不要把它们当标签。没有工具输出或
明确确认时,绝不要声称修复已成功、原因已确定或路径不存在。
不要记录问候或纯闲聊——只有这种情况才输出 "fact": null。仅仅陈述了
要求,或尚未开展工作就停滞的会话,仍然建立了信息。
details — 本次会话的具体 OUTPUT,逐字复制。`statement` 是摘要,而摘要必然会
丢掉各个具体值;后续检索几乎总是在寻找某个具体值,而非大意。因此,只要
会话产出了可寻址的结果,就以原始形式和结构复现在这里:
plan / schedule / itinerary → 每一行、每一天、每个 slot、每个 price
都严格按原文保留
config 或 code block → 保留整个 block
command 的关键 output → 保留承载发现的行(sizes、counts、versions、
error text)
structured result(table、records list)→ 保留各行
保留原始布局——line breaks、`Day 1:` / key: value prefixes、units、
currency symbols、ids。不要再次概括,不要重排,不要舍入数字,不要翻译。
只复制 trace 中实际存在的内容。绝不要填补空缺:如果某个 slot 没有填,
就保持未填或明确说明——杜撰的值比缺失的值更糟,因为它会被检索并使用。
长度控制在约 4000 字符;如果输出更长,保留承载具体值的部分,删去装饰性
文字,并注明删掉了什么。
如果本次会话未产生此类 artifact(尚无发现的纯调查,或等待用户导致的
停滞),填写 ""。
此字段不属于 retrieval vector——`statement` 和 `subject` 负责找到
记录,`details` 在找到后提供具体值。因此不要在此堆砌检索诱饵,也不要
重复摘要;只写原始内容。
files — 0..N 个本次会话中实际操作或调查过的绝对或 repo-relative FILE OR
DIRECTORY paths。保留调查过的 roots,例如 `C:\\` 和 `D:\\`。只包含
实际访问/搜索过的路径,不包含仅建议或计划访问的路径;不适用时填写 []。
═══════════════════════════════════════════════════════════════════
HOST CONTEXT(由 SDK 提供)
═══════════════════════════════════════════════════════════════════
- workspace_id present: {workspace_present}
- branch present: {branch_present}
没有 workspace_id 时,solution 不能依赖未声明的项目本地上下文——必须在 `solution`
或 `boundary_envs` 中明确写出其所需的 repo / path / service,使记忆能够独立使用。
═══════════════════════════════════════════════════════════════════
CONVERSATION
═══════════════════════════════════════════════════════════════════
{conversation_block}
═══════════════════════════════════════════════════════════════════
OUTPUT(仅输出严格 JSON,不要 markdown)
═══════════════════════════════════════════════════════════════════
{{
"experiences": [
{{
"task": "...",
"search_keys": ["...", "..."],
"solution": "...",
"boundary_envs": "...",
"confidence": 0.9
}}
],
"fact": {{
"subject": "...",
"statement": "...",
"files": ["..."],
"details": "..."
}}
}}
默认应进行抽取。当该段会话确实完成了实际工作时,必须输出至少一条 experience——即使
任务看似简单或一次成功,也要记录所做工作的可复用本质(method / command / how-to)。
会话内容较少时,只保留最有价值的一条 experience 即可。
唯一例外是上述停滞情况:如果该段因为缺少信息、被阻塞在向用户提问而没有开展实际工作,
输出 "experiences": [],只写 fact。不要为了满足本指令而杜撰 experience——杜撰的
“how to ask for the departure city”比没有记忆更糟,因为以后会被检索并照做。只有这个
原因允许空数组;实际开展过工作时,绝不能把空数组当捷径。
"fact" 是单个对象,绝不是列表——每次会话一条记录。
始终写 fact。fact 的门槛不是“这是否有价值”,而是“是否发生了任何事情”;能够到达
你的 segment 已经越过这条门槛。这包括看起来只是 setup 而非工作的片段:仅有一条用户
消息陈述 requirements、parameters 或 starting state,即使没有运行工具,也恰恰是后续
会话无法自行重建的材料——把需求、参数、起始状态连同情形一起写进 `statement`(紧凑
记法,不要照抄对话原句),不要推给 `details`(`details` 不进检索向量,读取侧也不再
注入它)。只有完全没有内容的问候和纯闲聊才使用 "fact": null。有疑问时应写 fact:
多余记录只多占一行,而漏掉记录会伤害每一次需要它、却无从知道它曾存在的后续会话。
文本已迁到 ``hy_memory/prompts/zh_hans.py``(``CODING_*``),这里保留旧名字。
"""
EXTRACT_PROMPT_CODING_ZH = """\
╔═══════════════════════════════════════════════════════════════════════════╗
║ HARD CONSTRAINT——先阅读本节;它覆盖本提示词后文任何看似要求叙事的措辞。 ║
║ ║
║ 你不是在撰写某个人做过什么的报告。你要写的是 FUTURE AGENT 为采取行动所需要 ║
║ 的信息。该 agent 已有自己的任务;它不关心这个用户想要什么。它需要知道:此处 ║
║ 做出了什么决定、位于哪些代码坐标、尝试并否决了什么,以及是否经过验证。 ║
║ ║
║ 具体来说,每个 `statement` 和每个 `solution` 都必须遵守: ║
║ · 语法主语必须是 CODE 或 THE DECISION——绝不能是“the user”,也绝不能是 ║
║ “the agent”。不要写“The user attempted to…”、“The user wanted…”、 ║
║ “The agent then checked…”。 ║
║ · 绝不要用“the session was completed”、“the error was noted”、 ║
║ “no solution was found yet”之类流水账短语收尾。结尾应写决定及其验证 ║
║ 状态。如果没有做出决定,就以已排除的内容和下一条尚未探索的线索收尾——这才 ║
║ 是可复用部分。 ║
║ · 保留具体症状(error string、failing command、wrong output)——未来搜索 ║
║ 正是通过这些内容命中。 ║
║ ║
║ bad: “The user attempted to write a QTable to RST with header_rows and ║
║ hit a TypeError. The session was completed with the error noted.” ║
║ good: “ascii.rst's RST writer rejects header_rows: ║
║ `tbl.write(..., format='ascii.rst', header_rows=['name','unit'])` ║
║ raises TypeError because RST.__init__ does not forward it, unlike ║
║ ascii.fixed_width which accepts it. No fix written — the writer ║
║ needs the same header_rows plumbing as fixed_width.” ║
╚═══════════════════════════════════════════════════════════════════════════╝
你需要从一段任务执行会话中抽取可长期保留的 WORKING 记忆。
“working”会话是指用户通过工具操作计算机或系统的任何实操任务——既包括软件/代码
工作(编写、调试、重构、构建、部署、API/library 使用),也包括系统与运维工作
(运行 shell/PowerShell 命令、磁盘/存储分析或清理、文件与文件夹操作、安装/配置
软件、环境/设备设置、硬件/磁盘健康检查、备份、调度/自动化、数据处理)。
你需要从同一段会话中产出两个层次:
EXPERIENCES(过程性,“如何做”)——可复用的方法;只有达到下述 VALUE BAR 时才
抽取。每条是一个 (task, search_keys, solution, boundary_envs) 组合。
FACT(情节性,“这次确立了什么”)——关于本次会话的恰好一条记录。它不是用户做了
什么的叙事,而要按以下顺序记录:任务及其具体症状(使它以后能被检索到)、代码坐标、
最终采用了什么以及拒绝了什么、验证推进到什么程度。它不受价值门槛限制,但绝不能
超过一条。
这两个层次回答不同问题,并分别存储。可泛化的方法放入 EXPERIENCES;只对这个系统、
这个项目、此时此刻成立的本次会话经历放入 FACT。
═══════════════════════════════════════════════════════════════════
VALUE BAR(仅适用于 EXPERIENCES)——不要轻易越过此门槛。
═══════════════════════════════════════════════════════════════════
一条记忆必须同时满足以下全部条件:
(a) 解决了同一用户以后很可能再次遇到的问题
(b) 解决方案并非琐碎操作:它组合了多项信息/多个步骤,或者包含决策理由,或者记录了
来之不易的 workaround,或者是某项运维任务的可复用流程/配方(例如如何分析并
清理磁盘占用、如何批量整理文件、如何检查硬件健康、如何设置 scheduled job)
(c) 未来的 agent / 用户会更愿意查阅它,而不是重新摸索
重要——分析、诊断、调查任务也算:
一项任务不必实际修改或“修复”任何东西才值得记忆。只读、分析、故障排查工作,只要其
流程或结论可复用,就有价值。明确抽取:
- 分析/诊断配方:可复用的流程以及它揭示的结果(例如“如何分析哪些文件夹占用了
C-drive 空间”——扫描命令、遇到的坑和最终发现)。“只生成了报告,没有修改”
不是跳过理由——可复用的方法本身就是记忆。
- 调查结果,包括否定结果或未找到结果(例如“使用 rg / fs_search / SQL index
在 C: 和 D: 中搜索 folder X——任何位置都未找到”)。记录一条死路可以避免下一个
agent 重复同样无果的搜索,因此达到了价值门槛。
不要仅仅因为“assistant 可以重新生成这些命令”或“实际上没有修改任何东西”就拒绝
一条记忆。只要流程或结论能节省未来的工作,就保留它。
不要抽取:
- 没有可复用流程、真正琐碎的单步操作(“rename one file”、“git status”、
“run ls once”)
- 已经解决且没有可迁移经验的一次性问题
- repo/system 中已经存在的信息(不要重复 package.json)
- 每个 PR / 每次会话的一次性任务,除非其中编码了可复用模式
启发式标准:如果用户(或未来执行同类任务的 agent)希望把它记下来,而不是以后再重新
摸索,就保留。多步骤运维流程通常符合条件——包括 system/ops 或只读分析/搜索流程,
不局限于代码。只有当它只是任何用户都能在 30 秒内通过 `--help` 找到的单条命令,并且
不包含值得记忆的决策或坑点时,才丢弃。
═══════════════════════════════════════════════════════════════════
CODE WORK——未来 agent 无法重新推导的信息,因此必须捕获
═══════════════════════════════════════════════════════════════════
当该段是针对 repository 的软件工作时,可复用方法只占一半。另一半是工作位于哪里,
以及哪个版本胜出。这些无法通过以后阅读代码恢复——代码只能展示当前状态,无法说明它
位于哪个 branch,也无法说明哪些方案已被否决。
本节不是要求写得更长,而是要求把相同预算花在价值更高的 token 上:删掉重复目标、
assistant“随后决定”做什么的叙事,以及只是复述的粘贴代码;把这些字符用于下面四项。
一条遗漏 branch、却引用 30 行 diff 的记录是糟糕的取舍。
1. BRANCH / WORKING LOCATION——逐字写入 fact `statement`。如果该段提到或 checkout
了 `feat/x` / `fix/y`,这个 token 必须出现。在错误 branch 上恢复工作的 agent
会产生随后被静默丢弃的改动,因此这一个 token 的优先级高于其他任何细节。
2. WHICH VERSION WON——当用户改变主意时,用一个分句指出胜出者并标记落败者:
“settled on A; the interim B was dropped”。不要完整保留两个版本,只需一个分句。
只保留最新版本是错误的:以后遇到被否决代码的 agent 必须识别它,否则会把代码
“修复”回已经丢弃的变体。`scratch that / go back to / actually, let's / no, not that`
之类措辞就是标志:后面的内容才是正式结论,前面的内容已被 superseded。
3. EXACT COORDINATES, NOT PASTED CODE——使用 repo-relative path
(`qutebrowser/utils/debug.py`,绝不要只写 `debug.py`)以及触及的 function /
class。如果 signature 发生变化,给出新的 signature。如果决定的是 constant、regex
或 literal,逐字引用它——但只引用这一项,不要引用周围 block。准确的一行胜过一段
近似描述。
4. VERIFIED vs MERELY WRITTEN——四个词就足够(“tests run and passing”/
“confirmed by git diff only, tests not run”)。“the edit is in”和“the edit is in
and tested”会让未来 agent 采取完全不同的下一步。
反例——只总结目标,却丢掉坐标:
bad: “refactored the colour parsing logic to be cleaner.”
good: “on feat/q4, in qutebrowser/config/configtypes.py QtColor._parse_value:
settled on `mult = 360.0 if kind == 'h' else 255.0`; the interim version that
used 255.0 for everything was dropped. Tests not run.”
(good 往往比 bad 还短——它只是把每个字符都用在无法重新推导的信息上)
═══════════════════════════════════════════════════════════════════
EXPERIENCE 字段
═══════════════════════════════════════════════════════════════════
task — 采用查询式名称;写成用户以后会如何提问。
使用“publish a release to the xx platform”,不要使用
“publish_release_internal”。
必须自包含(不要使用“this”、“the file”、“we”)。
search_keys — 0 到 5 种替代表述,用来覆盖仅靠 `task` 无法匹配、但对 solution 很
重要的方面。写成未来用户会询问的问题或主题,且必须与 task 有区别。
如果 task 已覆盖全部内容,则省略。
solution — 用户实际需要的完整、自包含内容:命令、凭据位置(不是凭据值)、
文件路径、有序步骤、坑点、决策理由。如果多轮端到端工作构成一个连贯
任务,应聚合成一个 solution;只有用户执行了互不相关的任务时才拆成
多条记忆。
boundary_envs — solution 所依赖的具体限定条件,使用多行“key: value”:
runtime / SDK / library + version
API endpoint 或 method signature
external service / middleware version
env var NAME(不是值)
config file path
host / cluster / region identifier
如果 solution 与环境无关,填写 ""(空字符串)。
目的:底层 SDK/middleware 升级后,未来使用者可以据此判断 solution
是否仍然适用。
confidence — 0.0 到 1.0
1.0 用户明确将其陈述为规则/指令
0.7 根据工具操作及充分证据推断
0.5 弱推断;只有一次观察;不确定
═══════════════════════════════════════════════════════════════════
FACT 字段——每段会话恰好一条,无价值门槛
═══════════════════════════════════════════════════════════════════
你要为整段会话写一条 FACT——它是本次会话可恢复的状态记录,不是精心润色的回顾,也
绝不是分散发现的列表。未来 agent 必须能够从中继续推进,且不会把建议、推断或局部搜索
误认为已经验证的事实。
statement — 本次会话的紧凑摘要。它同时承担两个职责,且两者都必须完成:它既是
retrieval vector(未来 agent 用一份新 issue report 搜索它;措辞方式
与用户报告问题的方式一致),也是该 agent 找到后要据以行动的 payload。
因此:
开头写任务和症状,使用用户会使用的措辞。
这是保证可检索性的关键——表述应尽量贴近问题被报告时的表面形式。
点明具体目标和对象(“reclaim disk space under D:\\WXWork”、
“adjust the hue multiplier in QtColor._parse_value on feat/q4”、
“locate the new_file_search folder”),绝不要写泛泛的“fix a bug”/
“look for a file”。包含具体症状:error string、failing command、
wrong output。
然后给出坐标和决定。
对代码工作写 repo-relative path、symbol 和 branch。说明最终 SETTLED ON
什么,并在同一分句标记丢弃了什么(“settled on X; the interim Y was
dropped”)。这部分无法通过阅读代码重新推导——代码只展示胜出者,从不
展示 Y 曾被尝试后否决。以后遇到被否决变体的 agent 必须能识别它,
否则会把代码“修复”回已丢弃版本。
结尾写证据状态。
必须精确对应以下之一:completed、partially completed、failed、blocked、
still awaiting verification;还要写明如何检查(“tests run and passing”/
“confirmed by git diff only, tests not run”)。纳入否定结果和死路,
但不要夸大。
⚠️ 绝不要停留在叙事层。“The user tried X and hit error Y; no solution found
yet”是一条浪费的记录:它容易被检索到,却不能指导未来 agent 行动。如果
本次确实没有得出决定,就说明排除了什么、下一条尚未探索的线索是什么——
这才是可复用部分。
句子的主语应是 code 和 decision,而不是“the user”。使用 2 到 4 个连贯
句子,不要写带标签的小节。硬上限约 400 个拉丁字符(约 200 个 CJK 字符)。
EVIDENCE FIDELITY——强制要求:
- 只记录 trace 中实际执行的操作和明确存在的证据。对关键 commands、flags、
arguments、errors、numbers 保持原样。绝不要按惯例补全 command flags,
也不要把 advice/plans 写成已执行的动作。
- 保留 attempted、observed、verified、inferred、unresolved 之间的区别。
不要把这些词作为标签写进正文;自然表达其含义,例如“tried … but was
blocked by Access denied”或“suspect …, not yet verified”。
- 没有工具输出或明确确认时,绝不要声称修复成功、原因已经确定、file/folder
不存在或 tests passed。
- 每个否定发现都必须限定在实际搜索范围内,并逐一列出可能使其失效的所有
明确覆盖限制,尤其是 permission failures 和 maximum search depth:
“not found within the searched parts of C:\\ and D:\\; some directories
were skipped on Access denied, and the search was also depth-limited”。
不要为了摘要更漂亮而漏掉任何限制。绝不要把局部搜索提升为全局不存在。
- 只有 trace 直接验证了因果关系时,才能确定地陈述因果。否则保留限定词
(“suspected”/“inferred”/“not yet verified”);不要把错误的测试假设
改写成系统的 root cause。
关键——这段文字就是 retrieval vector,也是未来 query 唯一会匹配的内容。
泛泛摘要毫无价值,因为每次会话看起来都像泛泛摘要:它会匹配所有查询,
却无法回答任何一个。因此必须写出使本次会话可检索的具体 anchors——真实
file paths、function / symbol names、branch names、commands、error strings、
versions、config keys and values、host / service names。以后有人搜索其中任一
具体项时,本摘要必须能被返回。
good: “fixing a failing `npm run build` in CI. Read
.github/workflows/ci.yml and package.json; traced it to webpack
5.74 hitting ERR_OSSL_EVP_UNSUPPORTED under node 18. Bumping the
workflow's node to 20 made the build pass; node 18 still
reproduces locally.”
good: “on branch feat/q0, in qutebrowser/utils/debug.py signal_name:
PyQt<5.11 only exposes the name via repr(sig), and repr comes in
both a bracketed and a parenthesised form. Settled on two patterns
tried in order with fullmatch, breaking on the first hit; the
single-regex version was dropped. Confirmed by git diff, tests not
run.”
bad: “user wanted a build bug fixed, the agent looked at some config
files, changed some code, and fixed it.”
(没有 path、command、error string、version——没有任何可检索锚点;
每次会话都可以写成这样)
bad: “fixed the signal-name parsing compatibility issue on a feature
branch.”
(哪个 branch?哪个 file?哪个 function?两个竞争版本中谁胜出?
一周后检索到它时,没有任何信息仍然有用)
忠实度示例(bad → good):
“ran Get-ChildItem -Force” → “ran Get-ChildItem”
(trace 中没有 `-Force`,不要杜撰)
“confirmed new_file_search does not exist” →
“new_file_search was not found within the searched parts of C:\\ and
D:\\; some directories were skipped on Access denied or by the depth
limit, so the investigation is only partly done”
“identical payloads cause a spurious lock error” →
“the verification script assumed the last writer is always a fixed
rank; that assumption does not hold, so whether there is a lock bug
is still unverified”
不要记录问候或纯闲聊。如果该段完全没有建立任何信息,输出 "fact": null。
files — 0..N 个本次会话中实际操作或调查过的绝对或 repo-relative FILE OR
DIRECTORY paths。保留调查过的 roots,例如 `C:\\` 和 `D:\\`。只包含
实际访问/搜索过的路径,不包含仅建议或计划访问的路径;不适用时填写 []。
═══════════════════════════════════════════════════════════════════
HOST CONTEXT(由 SDK 提供)
═══════════════════════════════════════════════════════════════════
- workspace_id present: {workspace_present}
- branch present: {branch_present}
没有 workspace_id 时,solution 不能依赖未声明的项目本地上下文——必须在 `solution`
或 `boundary_envs` 中明确写出其所需的 repo / path / service,使记忆能够独立使用。
═══════════════════════════════════════════════════════════════════
CONVERSATION
═══════════════════════════════════════════════════════════════════
{conversation_block}
═══════════════════════════════════════════════════════════════════
OUTPUT(仅输出严格 JSON,不要 markdown)
═══════════════════════════════════════════════════════════════════
{{
"experiences": [
{{
"task": "...",
"search_keys": ["...", "..."],
"solution": "...",
"boundary_envs": "...",
"confidence": 0.9
}}
],
"fact": {{
"statement": "...",
"files": ["..."]
}}
}}
强制要求:必须输出至少一条 experience,不要返回空列表。即使任务看似简单或一次成功,
也要记录所做工作的可复用本质(method / command / how-to)。会话内容较少时,只需要
最有价值的一条 experience——但绝不能输出空的 "experiences" array。
"fact" 是单个对象,绝不是列表——每次会话一条记录。只有该段是没有建立任何信息的
纯闲聊时,才使用 "fact": null。
"""
RECONCILE_PROMPT_ZH = """\
你需要决定如何将新的 coding memory drafts 与用户已有的 memories 合并。
对每条新 draft,选择以下一种 action:
ADD — 没有已有 memory 描述相同的任务/方面
UPDATE — 某条已有 memory 描述相同任务;用新 draft 替换它
(solution 可能已经变化,env 可能已经升级,等等)
SKIP — 与已有内容重复;没有新信息
此外,当新内容使已有候选失效或作废时,可以针对已有 candidates 输出 DELETE ops:
DELETE — 新 conversation 明确 supersedes/cancels/invalidates 某条已有 memory,
且无法通过 UPDATE 修复。示例:
* 用户说“don't use X anymore”,而 X 只存在于另一条独立 memory 中
(不是本次正在 ADD/UPDATE 的那条)
* candidate 的 boundary_envs(service / endpoint / SDK)已经消失,
且新 conversation 确认其已 deprecated
规则:
- “Same task”是指已有 memory 和新 draft 都可以作为同一个未来用户查询的有效答案。
- 如果 solution 不同,优先 UPDATE(不保留历史版本)。
- 如果 task 和 solution 都高度一致,选择 SKIP。
- 只有在存在明确证据表明 candidate 已 invalid/obsolete 时才 DELETE。绝不要仅仅因为
candidate 看起来“stale”就 DELETE。
- 产生 DELETE 时,输出的 ops 数量可以超过 drafts 数量。
═══════════════════════════════════════════════════════════════════
NEW DRAFTS
═══════════════════════════════════════════════════════════════════
{drafts_block}
═══════════════════════════════════════════════════════════════════
EXISTING CANDIDATES(按每条 draft 的 task similarity 检索)
═══════════════════════════════════════════════════════════════════
{candidates_block}
═══════════════════════════════════════════════════════════════════
OUTPUT(仅输出严格 JSON,不要 markdown)
═══════════════════════════════════════════════════════════════════
{{
"ops": [
{{"draft_idx": 0, "action": "ADD"}},
{{"draft_idx": 1, "action": "UPDATE", "target_experience_id": "exp_xxx", "reason": "..."}},
{{"draft_idx": 1, "action": "DELETE", "target_experience_id": "exp_yyy", "reason": "user explicitly deprecated Y"}},
{{"draft_idx": 2, "action": "SKIP", "reason": "duplicate"}}
]
}}
如果没有 drafts,输出 {{"ops": []}}。
"""
FACT_RECONCILE_PROMPT_ZH = """\
你负责维护 session-level WORKING FACTS。每条 statement 都概括一整段 trace/session:
用户的任务、大致方法和结果。它不是一条 atomic claim。
下面每条新的 session fact 都一定会被存储;这不是你要决定的事情。你的唯一任务是识别
task continuation:新 session 是否在继续推进某条已有 session fact 所记录的同一个用户
任务?如果是,就将较旧 fact 标记为 SUPERSEDE,使这些链接为该任务构成跨 session 的
进展链。
只有在存在充分证据同时满足以下条件时,才将已有 fact 标记为 superseded:
- subjects 匹配:两条记录围绕同一个实体——同一个 person、同一个 file/module、
同一个 host/service。subject 相同是必要条件,但不是充分条件;并且
- 用户目标在实质上相同,而不只是处于同一 topic/project;并且
- 新 trace 在继续、重试、扩展、验证或完成已有 trace 所记录的工作。
以下情况不要 supersede:
- 两条记录的 subjects 不同。同一 group 中的两个人、同一 module 中的两个 files、
同一 cluster 中的两个 hosts 是并行关系,不是先后关系:一个人的状态不是另一个人
状态的新版本。如果 subject 显示为 (unspecified),退回到只根据 statement 判断。
- 两次 session 只是提到了相同 file、symbol、component、repository、error 或
technology,但追求不同的用户目标。
- 一个任务只是为另一个任务提供 context 或 technique。
- 关系不确定。应保守判断:INSERT 是安全默认值;合并无关 session 会毁掉它们各自
独立的任务历史。
把每条 statement 当作有证据边界的文本。不要把其中的 inference 重新解释为已经确定的
cause,也不要把共同的 conjecture/error 当作用户目标相同的证据。你的判断仅涉及 task
continuation;不要“纠正”、补全或润色任何一条 session 记录。
示例:
existing: “需要修复 CI 的 npm run build;已追踪到 Node 18 下的
ERR_OSSL_EVP_UNSUPPORTED,workflow 尚未修改。”
new: “继续修复同一个 CI build;已将 .github/workflows/ci.yml 中的 Node
提升到 20,且 npm run build 通过。”
→ SUPERSEDE
existing: “需要修复 auth/middleware.ts 中的 JWT expiry handling;已添加
refresh logic。”
new: “需要在 auth/middleware.ts 中添加 performance logging;已引入
latency metric。”
→ 两者都保留
(同一 file/project,不同用户目标)
═══════════════════════════════════════════════════════════════════
NEW SESSION FACTS
═══════════════════════════════════════════════════════════════════
{drafts_block}
═══════════════════════════════════════════════════════════════════
EXISTING SESSION FACTS(按每条新 fact 的 statement similarity 检索)
═══════════════════════════════════════════════════════════════════
{candidates_block}
═══════════════════════════════════════════════════════════════════
OUTPUT(仅输出严格 JSON,不要 markdown)
═══════════════════════════════════════════════════════════════════
{{
"ops": [
{{"draft_idx": 0, "supersedes": ["fact_xxx"], "reason": "continues the same CI fix task"}},
{{"draft_idx": 1, "supersedes": []}}
]
}}
只能使用同一个 draft_idx 的 candidate list 中实际出现的 fact ids。
如果任何位置都没有 fact 被 superseded,输出 {{"ops": []}}。
"""
CHAIN_SUMMARY_PROMPT_ZH = """\
working-memory store 中的某个用户任务已跨多个 sessions 推进。下面是该任务进展链中的
session summaries,按从最早到最新排列,最后一条是当前 session。
总结同一个任务如何跨 sessions 推进到当前状态。按时间顺序覆盖每个重要阶段。对每个
阶段,在有信息时保留:实际 action 或 change、其 direct result,以及任何 unresolved
issue 或 limiting condition。最后明确写出当前状态。
规则:
- 这只是压缩。只能报告 entries 实际陈述的内容;绝不要新增或补全 command、cause、
motive、action、test result、success state 或 global negative conclusion。
- 精确保留证据状态:attempted 不等于 completed,observed 不等于 verified,inferred
不等于 established,unresolved 必须保持 unresolved。保留 uncertainty words 以及
search/permission/coverage limits。
- 忠实陈述当前结束状态(completed、partially completed、failed、blocked 或
still awaiting verification)。绝不要因为某个看似合理的因果故事,或后续 entry
并未验证的内容而提升状态。
- 如果 entries 提到 suspected cause 和后续 result,只有某条 entry 直接说明该 cause
已被 verified 时,才可在两者之间建立因果联系。
- 保留具体 retrieval anchors:file paths、symbols、commands、error strings、versions、
config keys/values,以及对进展重要的 numeric values。
- 对 2-3 代的 chains,不要把进展过度压缩成一句话。适合时优先使用 2-4 个紧凑段落,
或使用可读性良好的 prose,清楚呈现 task goal、逐阶段 progression 和 current status。
- 不要机械复制完整 history。在保留重要 actions、direct results 和 leftovers 的同时,
省略重复及低价值细节。
- 不要杜撰 stage headings 或 dates。只有 entries 使日期有实际意义时才提及日期;
单独的 timestamps 是噪声。
- 如果 entries 说明更早 revisions 已被省略,必须明确指出 earlier history was omitted;
不要暗示所提供的 entries 是完整历史。
- 使用与 entries 相同的语言。
- 不要写前言,也不要用引号包裹输出。
示例
entries:
(2026-07-20) 需要修复 CI 的 npm run build;已追踪到 Node 18 下的
ERR_OSSL_EVP_UNSUPPORTED,workflow 尚未修改。
(2026-07-24) 继续修复同一个 CI 问题;已将 .github/workflows/ci.yml 中的 Node
提升到 20,但 dependency cache 仍复现旧错误。
(now) 继续验证同一修复;清除 npm cache 后,npm run build 通过。
output:
任务目标是修复 CI 的 npm run build。最初将问题追踪到 Node 18 下的
ERR_OSSL_EVP_UNSUPPORTED,当时尚未修改 workflow;随后将
.github/workflows/ci.yml 提升到 Node 20,但 dependency cache 仍使旧错误继续
出现。最后,清除 npm cache 后 npm run build 通过,因此该 build 修复目前已验证完成。
═══════════════════════════════════════════════════════════════════
ENTRIES
═══════════════════════════════════════════════════════════════════
{chain_block}
═══════════════════════════════════════════════════════════════════
OUTPUT(只输出 summary 本身,不要其他内容)
═══════════════════════════════════════════════════════════════════
"""
from ..prompts.zh_hans import ( # noqa: F401
CODING_CHAIN_SUMMARY_PROMPT as CHAIN_SUMMARY_PROMPT_ZH,
CODING_EXTRACT_PROMPT as EXTRACT_PROMPT_ZH,
CODING_EXTRACT_PROMPT_CODING as EXTRACT_PROMPT_CODING_ZH,
CODING_FACT_RECONCILE_PROMPT as FACT_RECONCILE_PROMPT_ZH,
CODING_RECONCILE_PROMPT as RECONCILE_PROMPT_ZH,
)
+44 -75
View File
@@ -21,7 +21,12 @@ from typing import Any, Dict, List, Optional, Tuple
from ..agent.llm_provider import LLMProvider
from ..core.embed_service import EmbedService
from ..utils.lang_detect import is_chinese
from ..prompts import (
get_prompt,
normalize_language,
resolve_language,
with_output_language,
)
from .judge import _store_coding_log
from .store import CodingMemoryStore
from .types import CodingMemory, CodingMemoryDraft, ReconcileOp
@@ -50,57 +55,7 @@ def _resolve_configured_temperature(llm_provider) -> Optional[float]:
# Prompt
# ================================================================
RECONCILE_PROMPT = """\
You decide how to merge new coding memory drafts with the user's existing memories.
For each new draft, decide one of:
ADD — no existing memory describes the same task/aspect
UPDATE — an existing memory describes the same task; the new draft replaces it
(solution may have changed, env may have upgraded, etc.)
SKIP — duplicate of existing; nothing new
Additionally, you may emit DELETE ops on existing candidates when the new
content makes them obsolete or invalid:
DELETE — the new conversation EXPLICITLY supersedes/cancels/invalidates an
existing memory that cannot be repaired via UPDATE. Examples:
* user says "don't use X anymore" and X is only present as a
separate memory (not the one being added/updated)
* candidate's boundary_envs (service / endpoint / SDK) is gone
and the new conversation confirms the deprecation
Rules:
- "Same task" means the existing memory and the new draft would both be valid
answers to the same future user query.
- If solution differs, prefer UPDATE (we don't keep history).
- If both task and solution match closely, SKIP.
- DELETE only when there is EXPLICIT evidence the candidate is invalid/obsolete.
Never DELETE just because a candidate looks "stale".
- Output ops can outnumber drafts when DELETEs are produced.
═══════════════════════════════════════════════════════════════════
NEW DRAFTS
═══════════════════════════════════════════════════════════════════
{drafts_block}
═══════════════════════════════════════════════════════════════════
EXISTING CANDIDATES (per draft, retrieved by task similarity)
═══════════════════════════════════════════════════════════════════
{candidates_block}
═══════════════════════════════════════════════════════════════════
OUTPUT (strict JSON only, no markdown)
═══════════════════════════════════════════════════════════════════
{{
"ops": [
{{"draft_idx": 0, "action": "ADD"}},
{{"draft_idx": 1, "action": "UPDATE", "target_experience_id": "exp_xxx", "reason": "..."}},
{{"draft_idx": 1, "action": "DELETE", "target_experience_id": "exp_yyy", "reason": "user explicitly deprecated Y"}},
{{"draft_idx": 2, "action": "SKIP", "reason": "duplicate"}}
]
}}
If there are no drafts, output {{"ops": []}}.
"""
from ..prompts.en import CODING_RECONCILE_PROMPT as RECONCILE_PROMPT # noqa: E402,F401 (re-export)
def _drafts_language_text(drafts: List[CodingMemoryDraft]) -> str:
@@ -119,24 +74,23 @@ def _drafts_language_text(drafts: List[CodingMemoryDraft]) -> str:
def build_experience_reconcile_prompt(
drafts: List[CodingMemoryDraft],
candidates_per_draft: List[List[CodingMemory]],
language: Optional[str] = None,
) -> Tuple[str, str]:
"""Return the language-matched experience reconcile prompt and language."""
"""Return the experience reconcile prompt and its language code.
Explicit ``language`` wins; otherwise zh/en auto-detection over the drafts.
"""
drafts_block = CodingMemoryReconciler._format_drafts(drafts)
candidates_block = CodingMemoryReconciler._format_candidates(
drafts, candidates_per_draft,
)
if is_chinese(_drafts_language_text(drafts)):
from .prompts_zh import RECONCILE_PROMPT_ZH
template = RECONCILE_PROMPT_ZH
language = "zh"
else:
template = RECONCILE_PROMPT
language = "en"
return template.format(
explicit = normalize_language(language)
code = resolve_language(explicit, _drafts_language_text(drafts))
prompt = get_prompt("CODING_RECONCILE_PROMPT", code).format(
drafts_block=drafts_block,
candidates_block=candidates_block,
), language
)
return with_output_language(prompt, code, explicit=explicit is not None), code
# ================================================================
@@ -186,6 +140,7 @@ class CodingMemoryReconciler:
user_id: str = "",
agent_id: str = "default_agent",
session_id: str = "",
language: Optional[str] = None,
) -> List[ReconcileOp]:
"""
执行 reconcile + 持久化。返回执行的 ReconcileOp[](含成功执行的 ADD/UPDATE/DELETE/SKIP)。
@@ -214,7 +169,7 @@ class CodingMemoryReconciler:
_decide_resp = None
try:
ops, _decide_prompt, _decide_raw, _decide_resp = await self._llm_decide(
drafts, candidates_per_draft
drafts, candidates_per_draft, language=language,
)
except Exception as e:
logger.warning(f"[coding-reconcile] LLM decide failed, fallback to ADD-all: {e}")
@@ -311,9 +266,12 @@ class CodingMemoryReconciler:
self,
drafts: List[CodingMemoryDraft],
candidates_per_draft: List[List[CodingMemory]],
language: Optional[str] = None,
) -> Tuple[List[ReconcileOp], str, str, Any]:
"""返回 (ops, prompt, raw_response, llm_resp),供上层打 CODING_RECONCILE log。"""
prompt, _ = build_experience_reconcile_prompt(drafts, candidates_per_draft)
prompt, _ = build_experience_reconcile_prompt(
drafts, candidates_per_draft, language=language,
)
resp = await self.llm_provider.complete(
prompt=prompt,
@@ -388,7 +346,10 @@ class CodingMemoryReconciler:
obj = None
if not obj or not isinstance(obj, dict):
logger.warning(f"[coding-reconcile] failed to parse ops output: {text!r}")
logger.warning(
f"[coding-reconcile] failed to parse ops output ({len(text or '')} chars)"
)
logger.debug(f"[coding-reconcile] unparsable ops output: {text!r}")
return []
ops_raw = obj.get("ops") or []
if not isinstance(ops_raw, list):
@@ -441,10 +402,10 @@ class CodingMemoryReconciler:
for op in ops:
if op.action == "ADD":
if op.draft_idx is None or not (0 <= op.draft_idx < len(drafts)):
logger.warning(f"[coding-reconcile] drop invalid ADD: {op.to_dict()}")
logger.warning(f"[coding-reconcile] drop invalid ADD: {_op_ref(op)}")
continue
if op.draft_idx in consumed:
logger.warning(f"[coding-reconcile] drop ADD on consumed draft: {op.to_dict()}")
logger.warning(f"[coding-reconcile] drop ADD on consumed draft: {_op_ref(op)}")
continue
d = drafts[op.draft_idx]
memory = self._draft_to_new_memory(d)
@@ -458,18 +419,18 @@ class CodingMemoryReconciler:
elif op.action == "UPDATE":
if op.draft_idx is None or not (0 <= op.draft_idx < len(drafts)):
logger.warning(f"[coding-reconcile] drop invalid UPDATE: {op.to_dict()}")
logger.warning(f"[coding-reconcile] drop invalid UPDATE: {_op_ref(op)}")
continue
if not op.target_experience_id:
logger.warning(f"[coding-reconcile] drop UPDATE missing target: {op.to_dict()}")
logger.warning(f"[coding-reconcile] drop UPDATE missing target: {_op_ref(op)}")
continue
if not self._target_was_offered(op, candidates_per_draft):
logger.warning(
f"[coding-reconcile] drop UPDATE outside authority scope: {op.to_dict()}"
f"[coding-reconcile] drop UPDATE outside authority scope: {_op_ref(op)}"
)
continue
if op.draft_idx in consumed:
logger.warning(f"[coding-reconcile] drop UPDATE on consumed draft: {op.to_dict()}")
logger.warning(f"[coding-reconcile] drop UPDATE on consumed draft: {_op_ref(op)}")
continue
d = drafts[op.draft_idx]
memory = self._draft_to_new_memory(d, experience_id=op.target_experience_id)
@@ -488,17 +449,17 @@ class CodingMemoryReconciler:
elif op.action == "DELETE":
if not op.target_experience_id:
logger.warning(f"[coding-reconcile] drop DELETE missing target: {op.to_dict()}")
logger.warning(f"[coding-reconcile] drop DELETE missing target: {_op_ref(op)}")
continue
if not self._target_was_offered(op, candidates_per_draft):
logger.warning(
f"[coding-reconcile] drop DELETE outside authority scope: {op.to_dict()}"
f"[coding-reconcile] drop DELETE outside authority scope: {_op_ref(op)}"
)
continue
# DELETE 需要知道 user_id;通过 draft_idx(如有)或第一个 draft 推断
user_id = self._infer_user_id(op, drafts)
if not user_id:
logger.warning(f"[coding-reconcile] drop DELETE no user_id context: {op.to_dict()}")
logger.warning(f"[coding-reconcile] drop DELETE no user_id context: {_op_ref(op)}")
continue
try:
ok = await self.store.delete(op.target_experience_id, user_id=user_id)
@@ -585,6 +546,14 @@ class CodingMemoryReconciler:
# Helpers
# ================================================================
def _op_ref(op: ReconcileOp) -> str:
"""op 的日志引用:只带动作与编号,不带 LLM 写的 reason。"""
return (
f"action={op.action} draft_idx={op.draft_idx} "
f"target={op.target_experience_id}"
)
def _truncate(s: str, n: int) -> str:
if not s:
return ""
+7 -2
View File
@@ -605,7 +605,11 @@ class CodingMemoryStore:
await self._upsert_vec_keys(memory)
logger.info(
f"[coding-store] inserted experience_id={memory.experience_id} "
f"task={memory.task!r} keys={1 + len(memory.search_keys)}"
f"keys={1 + len(memory.search_keys)}"
)
logger.debug(
f"[coding-store] inserted experience_id={memory.experience_id} "
f"task={memory.task!r}"
)
async def update(self, memory: CodingMemory) -> None:
@@ -1002,7 +1006,8 @@ class CodingMemoryStore:
fact.created_at = datetime.utcnow()
await _run_in_pool(self._upsert_fact_sql_sync, fact)
await self._upsert_fact_vec(fact)
logger.info(
logger.info(f"[coding-store] inserted fact_id={fact.fact_id}")
logger.debug(
f"[coding-store] inserted fact_id={fact.fact_id} "
f"statement={fact.statement!r}"
)
+7 -1
View File
@@ -81,10 +81,13 @@ class CodingWriter:
request_id: Optional[str] = None,
source: str = "default",
memory_at: Optional[datetime] = None,
language: Optional[str] = None,
) -> Dict[str, Any]:
"""
working/coding 段写入端到端。memory_at 是对话时间,记到这次写出的经验和
事实上;不传则用收到请求的时间(与 chat 链路一致)。返回响应 dict(兼容 client.add 返回结构):
事实上;不传则用收到请求的时间(与 chat 链路一致)。language 是显式 prompt
语言码,透传给抽取、经验 reconcile、事实 reconcile 与演变总结;None 时各步
沿用 zh/en 自动检测。返回响应 dict(兼容 client.add 返回结构):
{
"success": True,
"scene": "productivity",
@@ -151,6 +154,7 @@ class CodingWriter:
branch=branch,
session_id=session_id,
request_id=request_id,
language=language,
)
finally:
# 历史服务于下一次 write;即使本次没跨过价值门槛,也不能丢掉上下文。
@@ -187,6 +191,7 @@ class CodingWriter:
user_id=user_id,
agent_id=agent_id,
session_id=session_id or "",
language=language,
)
if drafts else []
)
@@ -204,6 +209,7 @@ class CodingWriter:
branch=branch,
source=source,
memory_at=memory_at,
language=language,
)
experience_ids = [op.target_experience_id for op in ops
+21
View File
@@ -260,6 +260,7 @@ class EmbedderConfig:
max_retries: int = None # 最大重试次数
retry_delay: float = None # 重试间隔(秒)
timeout: int = None # 请求超时(秒)
batch_size: int = None # 单次 HTTP 请求最多带多少条 input(embed_batch 按此切块)
# 内部平台扩展(extra_headers / extra_body 会透传给 OpenAI client)
extra_headers: Optional[Dict[str, str]] = None
@@ -294,6 +295,10 @@ class EmbedderConfig:
# 请求切断并制造重试风暴。SDK 内层重试已关闭,15s 是每个显式 attempt
# 的真实上限;部署可通过 MEMORY_EMBEDDER_TIMEOUT 覆盖。
self.timeout = _get_env_int("MEMORY_EMBEDDER_TIMEOUT", 15)
if self.batch_size is None:
# 大批量(如重新 embed 整个库)不能一次 HTTP 发几千条:多数服务对
# input 数组有上限,超了直接 400。按 batch_size 切块逐块发送。
self.batch_size = _get_env_int("MEMORY_EMBEDDER_BATCH_SIZE", 64)
if self.extra_headers is None:
val = os.getenv("MEMORY_EMBEDDER_EXTRA_HEADERS")
if val:
@@ -1259,6 +1264,12 @@ class MemoryConfig:
enable_agent: bool = False
debug: bool = False
metrics_enabled: bool = True
# LLM prompt 语言(client 默认值;env: MEMORY_LANGUAGE)。canonical 码见
# hy_memory.prompts.SUPPORTED_LANGUAGES(en / zh-Hans / zh-Hant / ja / es / de /
# pt / fr / ko / ru / id),也接受别名 zh、zh-CN → zh-Hans,zh-TW、zh-HK →
# zh-Hant,pt-BR → pt。add() 等的 per-call language 优先于它。None(默认)=
# 沿用 zh/en 自动检测。显式指定时记忆正文用该语言书写。未知码抛 ValueError。
language: Optional[str] = None
# Runtime-only HTTP transport factory. MemHub uses this for request-cost
# budgeting without restricting the embedded SDK's local store access.
http_transport_factory: Optional[Callable[[], Any]] = field(
@@ -1273,6 +1284,11 @@ class MemoryConfig:
self.enable_agent = _get_env_bool("MEMORY_ENABLE_AGENT", False)
self.debug = _get_env_bool("MEMORY_DEBUG", False)
self.metrics_enabled = _get_env_bool("MEMORY_METRICS", True)
if self.language is None:
self.language = os.getenv("MEMORY_LANGUAGE", "").strip() or None
from .prompts import normalize_language
self.language = normalize_language(self.language)
@classmethod
def from_env(cls) -> "MemoryConfig":
@@ -1323,6 +1339,10 @@ class MemoryConfig:
config.metrics_enabled = config_dict.get(
"metrics_enabled", config.metrics_enabled
)
if config_dict.get("language") is not None:
from .prompts import normalize_language
config.language = normalize_language(config_dict["language"])
return config
@@ -1381,6 +1401,7 @@ class MemoryConfig:
"enable_graph": self.enable_graph,
"enable_agent": self.enable_agent,
"metrics_enabled": self.metrics_enabled,
"language": self.language,
}
def print_config(self) -> None:
+120 -23
View File
@@ -38,10 +38,12 @@ import importlib.util
import json
from json import JSONDecodeError
import logging
import random
import threading
import time
from ..config import MemoryConfig
from ..embedding import EmbeddingDimensionMismatchError
from .workload import embedding_slot
from ..utils.log_setup import get_request_id
@@ -80,7 +82,7 @@ def _count_tokens(text: str) -> int:
try:
import tiktoken
return len(tiktoken.get_encoding("cl100k_base").encode(text))
return len(tiktoken.get_encoding("cl100k_base").encode(text, disallowed_special=()))
except Exception:
return len(text) // _CHARS_PER_TOKEN_FALLBACK + 1
@@ -98,7 +100,8 @@ def _truncate_for_embedding(text: str, limit: int = _EMBED_MAX_TOKENS) -> str:
import tiktoken
enc = tiktoken.get_encoding("cl100k_base")
ids = enc.encode(text)
# Input is ordinary user text, not a tokenizer control-token stream.
ids = enc.encode(text, disallowed_special=())
if len(ids) <= limit:
return text
dropped = len(ids) - limit
@@ -186,6 +189,41 @@ def _redact_headers(headers: Optional[Dict[str, Any]]) -> Optional[Dict[str, Any
return out
# Retry-After 的上限:服务端给出超长等待(或恶意值)时不让单次调用挂几十分钟。
_RETRY_AFTER_MAX_S = 60.0
def _retry_after_seconds(exc: Exception) -> Optional[float]:
"""从 429 异常的响应头里取 Retry-After(秒数或 HTTP 日期),取不到返回 None。"""
response = getattr(exc, "response", None)
headers = getattr(response, "headers", None)
if not headers:
return None
try:
raw = headers.get("retry-after") or headers.get("Retry-After")
except Exception:
return None
if raw is None:
return None
raw = str(raw).strip()
try:
seconds = float(raw)
except ValueError:
try:
from email.utils import parsedate_to_datetime
from datetime import datetime, timezone
when = parsedate_to_datetime(raw)
if when.tzinfo is None:
when = when.replace(tzinfo=timezone.utc)
seconds = (when - datetime.now(timezone.utc)).total_seconds()
except Exception:
return None
if seconds != seconds: # NaN
return None
return min(max(0.0, seconds), _RETRY_AFTER_MAX_S)
def _embeddings_api_url(base_url: str) -> str:
"""OpenAI SDK posts to ``{base_url}/embeddings`` (base_url usually ends with /v1)."""
base = (base_url or "").rstrip("/")
@@ -204,7 +242,8 @@ class EmbedServiceConfig:
api_key: str = ""
base_url: str = ""
embedding_dims: int = 1536
batch_size: int = 100
# embed_batch 单次 HTTP 最多带多少条 input;超出按块顺序发送
batch_size: int = 64
enable_cache: bool = True
cache_size: int = 10000
max_retries: int = 5
@@ -286,6 +325,7 @@ class EmbedService:
max_retries=embedder.max_retries if embedder.max_retries is not None else 5,
retry_delay=embedder.retry_delay if embedder.retry_delay is not None else 3.0,
timeout=embedder.timeout if embedder.timeout is not None else 60,
batch_size=max(1, int(getattr(embedder, "batch_size", None) or 64)),
queue_batch_size=int(os.getenv("MEMORY_EMBED_BATCH_SIZE", "8")),
queue_batch_window_ms=float(os.getenv("MEMORY_EMBED_BATCH_WINDOW_MS", "100")),
extra_headers=embedder.extra_headers,
@@ -639,6 +679,8 @@ class EmbedService:
self,
texts: List[str],
use_cache: bool = True,
*,
max_retries: Optional[int] = None,
) -> List[List[float]]:
"""
批量向量化
@@ -646,17 +688,23 @@ class EmbedService:
Args:
texts: 文本列表
use_cache: 是否使用缓存
max_retries: 本次调用的重试次数;None 用配置值。长批量任务(重新
embed 整个库)可以给更宽的重试预算,不影响在线路径。
Returns:
向量列表
"""
vectors, _metas = await self.embed_batch_with_meta(texts, use_cache=use_cache)
vectors, _metas = await self.embed_batch_with_meta(
texts, use_cache=use_cache, max_retries=max_retries,
)
return vectors
async def embed_batch_with_meta(
self,
texts: List[str],
use_cache: bool = True,
*,
max_retries: Optional[int] = None,
) -> Tuple[List[List[float]], List[Dict[str, Any]]]:
"""批量向量化,同时返回每条的 provider metadata(非 vector)。"""
if not texts:
@@ -681,7 +729,12 @@ class EmbedService:
indices_to_embed.append(i)
if texts_to_embed:
vectors, batch_metas = await self._embed_texts_with_meta(texts_to_embed)
# max_retries 只在显式给出时透传:保持 `_embed_texts_with_meta(texts)`
# 这个既有调用形状不变(测试与下游替身按它打桩)。
retry_kwargs = {} if max_retries is None else {"max_retries": max_retries}
vectors, batch_metas = await self._embed_texts_with_meta(
texts_to_embed, **retry_kwargs,
)
for j, vector in enumerate(vectors):
idx = indices_to_embed[j]
@@ -699,9 +752,9 @@ class EmbedService:
return results, metas
async def _embed_texts_with_meta(
self, texts: List[str]
self, texts: List[str], *, max_retries: Optional[int] = None,
) -> Tuple[List[List[float]], List[Dict[str, Any]]]:
"""实际的向量化调用:**一次 HTTP 带走整批**;返回 (vectors, metas)。
"""实际的向量化调用:**一次 HTTP 带走一块**;返回 (vectors, metas)。
早期实现是 `for text in texts: await _embed_openai([text])` 逐条串行,
原意是"避免 batch size 限制",但代价极大:线上一批 8 条 × ~5.6s 串行
@@ -718,21 +771,34 @@ class EmbedService:
截断在这里做(发送前逐条过 `_truncate_for_embedding`):所有 embedding
路径(embed / embed_batch / embed_queued / *_with_meta)最终都落到这个
方法 —— 放在各入口会漏。理由与阈值见模块头部 `_EMBED_MAX_TOKENS`。
切块:一次带走整批只适用于小批。几千条的 input 数组会被多数服务直接
400(input 上限),所以按 `batch_size`(默认 64)切块,块之间顺序发送。
不超过一块时行为与切块前逐字相同(`batch_mode="single_request"`)。
"""
if not texts:
return [], []
safe_texts = [_truncate_for_embedding(t) for t in texts]
vectors, meta = await self._embed_openai(safe_texts)
base_meta = dict(meta or {})
base_meta["batch_mode"] = "single_request"
base_meta["batch_input_count"] = len(safe_texts)
chunk = max(1, int(self._embed_config.batch_size or 64))
vectors: List[List[float]] = []
all_metas: List[Dict[str, Any]] = []
for idx in range(len(safe_texts)):
m = dict(base_meta)
m["batch_index"] = idx
all_metas.append(m)
chunk_count = (len(safe_texts) + chunk - 1) // chunk
for start in range(0, len(safe_texts), chunk):
part = safe_texts[start:start + chunk]
retry_kwargs = {} if max_retries is None else {"max_retries": max_retries}
part_vectors, meta = await self._embed_openai(part, **retry_kwargs)
base_meta = dict(meta or {})
base_meta["batch_mode"] = "single_request" if chunk_count == 1 else "chunked"
base_meta["batch_input_count"] = len(part)
if chunk_count > 1:
base_meta["batch_chunk_index"] = start // chunk
base_meta["batch_chunk_count"] = chunk_count
for idx in range(len(part)):
m = dict(base_meta)
m["batch_index"] = start + idx
all_metas.append(m)
vectors.extend(part_vectors)
self._last_embed_meta = all_metas[-1] if all_metas else {}
return vectors, all_metas
@@ -753,16 +819,22 @@ class EmbedService:
"""
打一条 INFO 级 embedding 请求日志(含 request_id / provider / model / api / 入参 / 出参)。
出参只含 vector 维度,不含向量数值。request_id 同时写入 message,
出参只含 vector 维度,不含向量数值。入参的 input 是待向量化的原文
(对话 / 记忆 / 查询),INFO 只记每条的字符数(input_chars),原文仅在
DEBUG 单独输出。request_id 同时写入 message,
并依赖 log formatter 的 %(request_id)s(setup_logging 后生效)。
"""
texts = request_payload.get("input")
info_request = {k: v for k, v in request_payload.items() if k != "input"}
if isinstance(texts, list):
info_request["input_chars"] = [len(str(t)) for t in texts]
payload = {
"request_id": get_request_id(),
"status": status,
"provider": self._embed_config.provider,
"model": self._embed_config.model,
"api": api_url,
"request": request_payload,
"request": info_request,
"response": response_payload,
}
try:
@@ -770,9 +842,13 @@ class EmbedService:
except Exception:
body = str(payload)
logger.info(f"[embed] {body}")
if texts is not None and logger.isEnabledFor(logging.DEBUG):
logger.debug(
f"[embed] input: {json.dumps(texts, ensure_ascii=False, default=str)}"
)
async def _embed_openai(
self, texts: List[str]
self, texts: List[str], *, max_retries: Optional[int] = None,
) -> Tuple[List[List[float]], Dict[str, Any]]:
"""
使用 OpenAI 兼容接口向量化
@@ -862,7 +938,9 @@ class EmbedService:
"endpoint_overridden": bool(_ov),
}
_max_retries = max(0, int(self._embed_config.max_retries or 0))
_max_retries = max(0, int(
(self._embed_config.max_retries or 0) if max_retries is None else max_retries
))
_retry_delay = max(0.0, float(self._embed_config.retry_delay or 0.0))
last_exc: Optional[Exception] = None
@@ -908,6 +986,17 @@ class EmbedService:
f"got={len(vectors)}"
)
dims = [len(v) for v in vectors]
expected_dims = self._embed_config.embedding_dims
if expected_dims:
bad = [d for d in dims if d != int(expected_dims)]
if bad:
# 宽度不对的向量写进集合会把集合的维度钉死在错误值上(Chroma
# 首次 add 定维),或在已有集合上整批失败。确定性错误,不重试。
raise EmbeddingDimensionMismatchError(
"embedding dimension mismatch: "
f"expected={int(expected_dims)} got={bad[0]} "
f"model={self._embed_config.model}"
)
meta = {
"http_status": raw.status_code,
"response_headers": resp_headers,
@@ -943,6 +1032,8 @@ class EmbedService:
},
)
return vectors, meta
except EmbeddingDimensionMismatchError:
raise
except Exception as e:
last_exc = e
_err_name = type(e).__name__
@@ -976,12 +1067,18 @@ class EmbedService:
if _retryable and attempt < _max_retries:
# 逐出可能卡死的 client,下个 attempt 重建干净连接(恢复原防御语义)
self._evict_client(_eff_base_url, _eff_api_key, _eff_timeout)
_delay = _retry_delay
_retry_after = _retry_after_seconds(e) if _status == 429 else None
if _retry_after is not None:
# 限流时服务端明确告诉了要等多久:按它等,再加一点抖动,
# 避免多个 worker 在同一时刻一起醒来再次撞限流。
_delay = max(_delay, _retry_after) + random.uniform(0, 0.25 * _retry_after)
logger.warning(
f"[embed] attempt {attempt + 1}/{_max_retries + 1} failed "
f"({_err_name}): {_err_msg[:200]}; retry in {_retry_delay}s"
f"({_err_name}): {_err_msg[:200]}; retry in {_delay:.2f}s"
)
if _retry_delay > 0:
await asyncio.sleep(_retry_delay)
if _delay > 0:
await asyncio.sleep(_delay)
continue
self._log_embed_call(
status="error",
+18
View File
@@ -22,6 +22,7 @@ import logging
import json
import threading
import time
import weakref
from ..models.memory import MemoryNode, MemoryLayer, MemoryStatus
from ..config import MemoryConfig
@@ -77,6 +78,21 @@ async def _run_in_kuzu_pool(func, *args, **kwargs):
return await run_in_executor_guarded(_kuzu_executor, _measured)
# 本进程里已打开、尚未 close 的 store。drop_graph 据此拒绝删掉一个正被
# 活着的 client 使用的图库。WeakSet:store 被回收后自动消失。
_LIVE_STORES: "weakref.WeakSet[KuzuGraphStore]" = weakref.WeakSet()
def open_graph_paths() -> List[str]:
"""Real paths of the graph databases live stores in this process hold open."""
import os
return [
os.path.realpath(str(store._db_path))
for store in list(_LIVE_STORES)
if store._db is not None
]
def kuzu_executor_snapshot() -> Dict[str, Any]:
with _kuzu_stats_lock:
return {
@@ -350,6 +366,7 @@ class KuzuGraphStore(GraphStoreBase):
)
self._db = db
self._conn = conn
_LIVE_STORES.add(self)
except asyncio.TimeoutError:
logger.warning(
f"GraphStore init timed out after 5s (Kuzu file lock contention). "
@@ -2441,6 +2458,7 @@ class KuzuGraphStore(GraphStoreBase):
async def close(self) -> None:
"""关闭连接"""
conn, db = self._conn, self._db
_LIVE_STORES.discard(self)
self._conn = None
self._db = None
self._available = False
+14 -6
View File
@@ -89,6 +89,17 @@ def passes_created_after(
logger = logging.getLogger(__name__)
def physical_collection_name(base_name: Optional[str], dims: Optional[int]) -> str:
"""The collection name a store opens for ``base_name`` at ``dims``.
The dims suffix keeps two embedding widths from sharing one collection.
`hy_memory.reembed.vector_space_names` uses this too, so callers never
re-derive the rule.
"""
base = base_name or "agent_memories"
return f"{base}_{dims}" if dims else base
class VectorStoreBase(ABC):
"""
向量存储抽象基类
@@ -98,13 +109,10 @@ class VectorStoreBase(ABC):
def __init__(self, config: MemoryConfig):
self.config = config
base_name = config.vector_store.collection_name or "agent_memories"
dims = config.vector_store.embedding_dims
# 自动拼接维度后缀,避免不同 embedding 模型写入同一 collection 导致维度冲突
if dims:
self._collection_name = f"{base_name}_{dims}"
else:
self._collection_name = base_name
self._collection_name = physical_collection_name(
config.vector_store.collection_name, config.vector_store.embedding_dims,
)
_supports_retrieval_clauses: bool = False
_supports_narrow_episode_queries: bool = False
+29 -1
View File
@@ -9,10 +9,12 @@ Agent Memory V2 - VectorStore (Chroma)
- 同步 API → _run_in_vdb_pool() 包装为异步
"""
from typing import Optional, List, Dict, Any
from typing import Optional, List, Dict, Any, Set, Tuple
import concurrent.futures
import functools
import logging
import os
import weakref
from ..models.memory import (
MemoryNode, MemoryLayer, MemoryStatus,
@@ -33,6 +35,30 @@ _vdb_executor = concurrent.futures.ThreadPoolExecutor(
)
# 本进程里已 initialize、尚未 close 的 store。drop_vector_space 据此拒绝删掉
# 一个正被活着的 client 使用的集合。WeakSet:store 被回收后自动消失。
_LIVE_STORES: "weakref.WeakSet[ChromaVectorStore]" = weakref.WeakSet()
def _location_key(vs_config) -> Tuple[str, str]:
if getattr(vs_config, "host", None):
return ("http", f"{vs_config.host}:{vs_config.port or 8000}")
return ("local", os.path.realpath(str(vs_config.persist_directory or "")))
def open_collection_names(location: Tuple[str, str]) -> Set[str]:
"""Collections that live stores in this process hold open at ``location``."""
names: Set[str] = set()
for store in list(_LIVE_STORES):
if store._client is None or _location_key(store.config.vector_store) != location:
continue
names.add(store._collection_name)
if store._entity_collection is not None:
from ..pipelines._retrieval.config import entity_collection_name
names.add(entity_collection_name(store._collection_name))
return names
async def _run_in_vdb_pool(func, *args, **kwargs):
"""Run a synchronous Chroma call in the dedicated VDB executor."""
call = functools.partial(func, *args, **kwargs) if args or kwargs else func
@@ -88,6 +114,7 @@ class ChromaVectorStore(VectorStoreBase):
return client, collection
self._client, self._collection = await _run_in_vdb_pool(_init)
_LIVE_STORES.add(self)
logger.debug(f"VectorStore initialized (Chroma), collection={self._collection_name}")
# ================================================================
@@ -561,6 +588,7 @@ class ChromaVectorStore(VectorStoreBase):
async def close(self) -> None:
"""关闭客户端"""
# Chroma PersistentClient 无需显式关闭
_LIVE_STORES.discard(self)
self._client = None
self._collection = None
self._entity_collection = None
+21 -57
View File
@@ -64,59 +64,15 @@ def _topic_lock(graph_store, isolation_key: str, topic_id: str) -> asyncio.Lock:
return lock
TOPIC_SUMMARY_PROMPT = """You are summarizing everything known about ONE topic.
## Topic
{topic_name}
## Memories under this topic (oldest → newest)
{memories}
---
Write a single dense summary of what is known about the user with respect to this
topic.
Rules:
* 2–4 sentences, third person, starting with "The user ..." (or the named person
if the memories are about someone else).
* Cover the topic as a whole — state the current situation, and mention how it
developed only when the change matters.
* Preserve concrete details: names, places, numbers, dates.
* Do NOT invent anything that is not in the memories.
* Do NOT enumerate the memories one by one; synthesize them.
* Output language must match the language of the memories.
Return ONLY the summary text, no preamble, no JSON, no quotes."""
TOPIC_SUMMARY_PROMPT_ZH = """你在为**一个主题**总结已知的全部信息。
## 主题
{topic_name}
## 该主题下的记忆(时间正序)
{memories}
---
写一段关于用户在这个主题上的高密度摘要。
要求:
* 2–4 句,第三人称,以"用户…"开头(若记忆讲的是别人,则以那个人的名字开头)。
* 覆盖整个主题 —— 说明当前状况;只在变化本身重要时才提它是怎么演变过来的。
* 保留具体细节:人名、地点、数字、日期。
* 绝不编造记忆里没有的内容。
* 不要逐条罗列记忆,要综合成一段。
* 输出语言与记忆的语言一致。
只返回摘要正文,不要任何前言、不要 JSON、不要引号。"""
# 多语言版本见 hy_memory/prompts/<language>.py;旧名字保留为别名。
from ...prompts import ( # noqa: E402
get_prompt,
normalize_language,
resolve_language,
with_output_language,
)
from ...prompts.en import TOPIC_SUMMARY_PROMPT # noqa: E402,F401 (re-export)
from ...prompts.zh_hans import TOPIC_SUMMARY_PROMPT as TOPIC_SUMMARY_PROMPT_ZH # noqa: E402,F401 (re-export)
async def find_latest_topic_summary(
@@ -193,9 +149,13 @@ async def maybe_summarize_topics(
max_concurrency: int = 3,
temperature: Optional[float] = None,
trace=None,
language: Optional[str] = None,
) -> Dict[str, Any]:
"""对本次写入触达的 Topic,按需有界并发生成 / 刷新 summary。
language 为显式 prompt 语言码(随触发本次写入的请求);None 时按记忆内容
做 zh/en 自动检测。
触发条件:Topic 下 active 记忆数 ≥ `min_memories`,且比上次总结时多了 ≥ `step` 条。
这样既不会每写一条就重算(贵),也不会一直不更新(陈旧)。
@@ -254,6 +214,7 @@ async def maybe_summarize_topics(
topic_name=topic_names.get(topic_id) or topic_id,
memories=[m.get("content") or "" for m in mems],
temperature=temperature,
language=language,
)
if trace is not None:
await trace(
@@ -317,15 +278,18 @@ async def maybe_summarize_topics(
async def _generate(
*, llm_provider, topic_name: str, memories: List[str],
temperature: Optional[float] = None,
language: Optional[str] = None,
) -> tuple:
"""返回 (摘要文本, 实际 prompt, token 用量) —— 后两个供埋点,线上要能复盘。"""
from ...utils.lang_detect import is_chinese
body = "\n".join(f"- {c}" for c in memories if c)
if not body:
return "", "", {}
tpl = TOPIC_SUMMARY_PROMPT_ZH if is_chinese(body) else TOPIC_SUMMARY_PROMPT
prompt = tpl.format(topic_name=topic_name, memories=body)
explicit = normalize_language(language)
code = resolve_language(explicit, body)
prompt = get_prompt("TOPIC_SUMMARY_PROMPT", code).format(
topic_name=topic_name, memories=body,
)
prompt = with_output_language(prompt, code, explicit=explicit is not None)
resp = await llm_provider.complete(
prompt=prompt,
max_tokens=400,
@@ -115,11 +115,16 @@ class ReconcileHybridRetriever:
fused_max = max(fused_all) if fused_all else 0.0
base = (
f"[reconcile-hybrid][{req}] pool={pool_size} q_terms={q_terms!r} "
f"[reconcile-hybrid][{req}] pool={pool_size} q_terms={len(q_terms)} "
f"raw_bm25 max={raw_max:.4f} nonzero={raw_nonzero}/{len(raw_bm25)} "
f"bm25_norm max={norm_max:.4f} nonzero={norm_nonzero}/{len(bm25_norms)} "
f"sem_max={sem_max:.4f} fused_max={fused_max:.4f} "
f"kept={kept}/{len(fused_all)} min_score={self._cfg.min_score} "
f"query({len(query_text or '')} chars)"
)
# query 是待对账的新记忆原文,词项与预览只进 DEBUG
logger.debug(
f"[reconcile-hybrid][{req}] q_terms={q_terms!r} "
f"query={q_preview!r} lemmatized={lem_preview!r}"
)
+54 -15
View File
@@ -95,21 +95,48 @@ class ToolCall:
# [John]: 我也喜欢茶
# 名字单独占一行、整段只出现一次;[user] 标签**不换成名字**,视角信息留在标签里。
# extract 用这一行给 L0 basic_info 取 name(用户不必在对话里自报姓名)。
# 各语言的模板在 hy_memory/prompts/<language>.py 的 USER_NAME_LINE;这两个旧名字保留。
_USER_NAME_LINE_ZH = "用户名叫{name}"
_USER_NAME_LINE_EN = "The user's name is {name}"
_USER_NAME_LINE_RE = re.compile(
r"^(?:用户名叫|The user's name is)\s*(.+?)\s*$"
)
_user_name_line_res: Optional[List["re.Pattern"]] = None
def render_user_name_line(name: str, is_zh: bool) -> str:
"""渲染文首的用户名字行;name 为空返回空串。"""
def _user_name_line_patterns() -> List["re.Pattern"]:
"""每种语言一个匹配名字行的正则(L1_RAW 里可能存着任意语言的名字行)。"""
global _user_name_line_res
if _user_name_line_res is None:
from ..prompts import SUPPORTED_LANGUAGES, get_prompt
patterns = []
for code in SUPPORTED_LANGUAGES:
try:
tpl = get_prompt("USER_NAME_LINE", code)
except (ImportError, AttributeError):
continue
prefix, _, suffix = tpl.partition("{name}")
patterns.append(re.compile(
r"^" + re.escape(prefix.strip()) + r"\s*(.+?)\s*"
+ re.escape(suffix.strip()) + r"$"
))
_user_name_line_res = patterns
return _user_name_line_res
def render_user_name_line(
name: str, is_zh: bool = False, *, language: Optional[str] = None,
) -> str:
"""渲染文首的用户名字行;name 为空返回空串。
language(语言码)优先;未给时按 is_zh 在 zh-Hans / en 间选(旧行为)。
"""
name = (name or "").strip()
if not name:
return ""
tpl = _USER_NAME_LINE_ZH if is_zh else _USER_NAME_LINE_EN
return tpl.format(name=name)
from ..prompts import get_prompt
code = language or ("zh-Hans" if is_zh else "en")
return get_prompt("USER_NAME_LINE", code).format(name=name)
def parse_user_name_line(line: str) -> Optional[str]:
@@ -118,8 +145,12 @@ def parse_user_name_line(line: str) -> Optional[str]:
L1_RAW 存的就是 get_flat_content() 的产物,回读历史时要靠这个把名字行识别出来,
否则它会被当成一条无前缀的裸消息(见 MemoryWriter._get_recent_history)。
"""
m = _USER_NAME_LINE_RE.match((line or "").strip())
return m.group(1) if m else None
text = (line or "").strip()
for pattern in _user_name_line_patterns():
m = pattern.match(text)
if m:
return m.group(1)
return None
@dataclass
@@ -257,6 +288,11 @@ class WriteRequest:
branch: Optional[str] = None
# 分支名。同 workspace_id。
# 显式 prompt 语言码(canonical,见 hy_memory.prompts.SUPPORTED_LANGUAGES)。
# 由 client 入口按「per-call > MemoryConfig.language」解析后注入;None 时
# 抽取 / reconcile / 画像等各步沿用 zh/en 自动检测。
language: Optional[str] = None
# ---- 便捷方法 ----
def has_messages(self) -> bool:
@@ -286,8 +322,8 @@ class WriteRequest:
[John]: 我也喜欢茶
无 name 时与历史行为完全一致(`[user]: ...` / `[assistant]: ...`)。
名字行只在 role=user 带 name 时出现,且整段只出现一次;语言按消息内容判定
(与 extractor 选 zh/en prompt 同一套判据)。
名字行只在 role=user 带 name 时出现,且整段只出现一次;语言取显式
language,未指定时按消息内容判定(与 extractor 选 zh/en prompt 同一套判据)。
"""
if not self.messages:
return self.content or ""
@@ -296,11 +332,14 @@ class WriteRequest:
name = self.user_name()
if name:
from ..utils.lang_detect import is_chinese
if self.language:
name_line = render_user_name_line(name, language=self.language)
else:
from ..utils.lang_detect import is_chinese
name_line = render_user_name_line(
name, is_chinese(messages=self.messages)
)
name_line = render_user_name_line(
name, is_chinese(messages=self.messages)
)
if name_line:
lines.insert(0, name_line)
+2 -1
View File
@@ -692,9 +692,10 @@ class CrossDomainSweeper:
)
logger.info(
f"[sweeper] Created L6 core {core_id[:12]}: '{core_content[:60]}' "
f"[sweeper] Created L6 core {core_id[:12]} "
f"from {len(cluster_ids)} basics, evidence_inherited={len(inherited_evidence_ids)}"
)
logger.debug(f"[sweeper] L6 core {core_id[:12]} content={core_content[:60]!r}")
return 1, 0, [core_id]
except Exception as e:
+2 -1
View File
@@ -3587,8 +3587,9 @@ class BaseReadPipeline(ReadPipeline):
f"hidden_l0={len(hidden_l0_hits)} fwd_l6={len(forward_l6)} rev_l6={len(reverse_l6)} "
f"fused_l6={len(fused_l6)} graph_intent={len(graph_intention_hits)} "
f"returned={len(final_results)} "
f"query='{request.query[:80]}'"
f"query({len(request.query or '')} chars)"
)
logger.debug(f"[read/{self.VERSION}] query={request.query[:80]!r}")
response.success = True
+82 -369
View File
@@ -979,171 +979,16 @@ def s2_agent_skip_reason(materials: Dict[str, Any]) -> Optional[str]:
return None
# ====================================================================
# System Prompt (dual language)
# System Prompt (multilingual: hy_memory/prompts/<language>.py)
# ====================================================================
SYSTEM2_AGENT_PROMPT_ZH = """你是一个认知加工 Agent,负责从用户的记忆数据中演化高层认知结构。
## 记忆分层架构
用户的记忆存储在两个系统中:
### VDB(向量数据库)— 你只能读取,不能修改
| 层级 | 说明 |
|------|------|
| L0 BASIC_INFO | 用户基础属性(姓名、年龄、职业等)|
| L1 RAW | 原始对话记录 |
| L2 FACT | 原子事实(从对话中提取的客观事实)|
| L3 SUMMARY | 会话摘要 |
| L4 IDENTITY | 身份画像(性格特征、偏好、习惯)|
| L5 KNOWLEDGE | 领域知识 |
### Graph(知识图谱)— 你是唯一的写入者
| 层级 | 说明 |
|------|------|
| L6 SCHEMA | 行为模式 — 域内、原子化、创建后不可变 |
## L6 Schema 是什么?
Schema 是从多次独立经历归纳出的长期行为模式,包含以下三个要素:
- **Circumstance(场景)**:该模式发生的领域/话题/场景。Schema 必须限定在其场景内,不要跨域泛化。
- **Pattern(模式)**:用户在该场景下的惯常行为、思维方式或行动倾向。
- **Insight(洞察)**:底层心理驱动力或心智模型。
### 内容格式
行为模式用一句话组合三要素:
"当[场景]时,用户[模式]——反映了[洞察]。"
对与情绪、治疗、压力、哀伤或创伤有关的长期模式,不要把具体人生背景压缩成
空泛人格词。优先保留这三部分:
"Trigger/background(持久触发背景) → emotional response(反复情绪反应) → observed coping resource(用户已实际使用的应对资源)。"
只能写入 user-owned 证据中出现的触发背景、反应和应对方式;助手建议不是用户已观察到的应对资源。
### 规则
- **原子化**:一个 Schema 只包含一个模式。两个不同模式 → 两个 Schema。
- **不可变**:Schema 创建后内容永远不变。不要修改已有 Schema 的内容。
- **累积证据**:新事实只是在支持已有表述 → 调 `add_evidence`。如果新证据会让旧画像的当前状态产生实质遗漏、错误或误导,使用 `revise_schema` 创建新版本,让旧版本保留审计但退出 active 检索。
- **域内约束**:不要跨域。跨域抽象由系统的其他流程处理。
- **事实与模式分层**:单条明确身份、健康状态、设备使用、隶属关系、项目或偏好属于 L2 状态,不创建 L6。L6 必须由至少两个独立 raw/session 中的 user-owned 证据支持。
- **主体安全**:同一原始写入可能包含他人资料或示例。只有材料明确归因于当前用户时才创建用户 Schema;否则对该材料不执行任何操作。
- **候选覆盖**:source-context 审查时,每条候选事实必须二选一:用于 `create_schema`/`add_evidence` 的 evidence,或用 `skip_fact` 明确跳过。原始上下文只用于确认主体或补回该事实遗漏的细节,不得据此创建无关 Schema。
- **证据门槛**:创建 L6 必须有 user-owned evidence,且这些证据合计代表至少两个独立 raw/session。
一个被 S1 合并的 L2 事实可以代表多次独立写入,应检查它的 `source_raw_memory_ids`,不要只数节点。
助手建议不能作为用户模式的独立证据。
## 工作流程
输入材料分两部分(仅当已形成聚类时才会调用你):
- **聚类结果(Cluster Results)**:主输入,按语义相似度归组的事实
- **未聚类事实(补充)**:未进入任一聚类的事实,优先级低于聚类结果,酌情审视即可
对于每组事实(无论聚类还是未聚类):
1. **搜索已有 Schema**:如果已有 Schema 已覆盖该模式,使用 `add_evidence`。只有加入新证据后旧表述会产生实质遗漏、错误或误导时才使用 `revise_schema`。
2. **创建新 Schema**:仅当没有已有 Schema 覆盖时。一句话,三要素。
3. **建立关系**:两个 Schema 主题相关 → 用 `add_edge`(RELATED_TO)。
## 原则
- 所有 Graph 操作必须通过 tools 执行
- Schema 内容创建后不可变——绝不重新创建已存在的 Schema
- 优先 `add_evidence` 而非创建重复节点
- 宁可不创建,也不要创建低质量节点
- 一个 Schema = 一个域内原子模式
- 打标签时优先使用已有标签列表中的标签"""
SYSTEM2_AGENT_PROMPT_EN = """You are a cognitive processing Agent. Your job is to evolve higher-order cognitive structures from a user's raw memory data.
## Memory Architecture
### VDB (Vector Database) — read-only
| Layer | Description |
|-------|-------------|
| L0 BASIC_INFO | User attributes (name, age, occupation) |
| L1 RAW | Original conversation transcripts |
| L2 FACT | Atomic facts extracted from conversations |
| L3 SUMMARY | Session summaries |
| L4 IDENTITY | Identity traits, preferences, habits |
| L5 KNOWLEDGE | Domain knowledge |
### Graph (Knowledge Graph) — you are the sole writer
| Layer | Description |
|-------|-------------|
| L6 SCHEMA | Behavioral patterns — domain-bound, atomic, immutable once created |
## What is an L6 Schema?
A Schema is one cross-event behavioral pattern in the long-term user model and
contains the following three components:
- **Circumstance**: The domain/topic/situation where this pattern is observed. A Schema MUST stay within its circumstance — do NOT generalize across domains. Cross-domain abstraction is handled separately by the system.
- **Pattern**: The user's habitual behavior, thinking style, or action tendency in this circumstance.
- **Insight**: The underlying psychological driver or mental model.
### Content format
Use a single sentence combining all three:
"When [circumstance], the user [pattern] — reflecting [insight]."
For durable emotional, therapy, stress, grief, or trauma-related patterns, do
not erase the specific autobiographical background into a generic personality
label. Prefer an anchored structure that preserves:
"Trigger/background -> recurring emotional response -> observed coping resource."
Only include triggers, responses, and coping resources present in user-owned
evidence. Assistant advice is not an observed user coping resource.
### Rules
- **Atomic**: One pattern per Schema. Two distinct patterns → two Schemas.
- **Immutable**: Once created, content NEVER changes.
- **Evidence only**: New facts that merely support existing wording → call
`add_evidence`. If new evidence materially changes, corrects, or narrows the
current durable state, use `revise_schema` so the old version is retained for
audit but removed from active retrieval.
- **Domain-bound**: Stay within the observed domain. Cross-domain abstraction is handled by a separate system process.
- **Separate facts from patterns**: A single explicit identity, health state,
equipment use, affiliation, project, or preference belongs in L2 state memory,
not L6. L6 requires user-owned evidence representing at least two independent
raw writes or sessions. One reconciled L2 fact may represent multiple writes;
inspect its source_raw_memory_ids rather than counting fact nodes.
- **Subject safety**: Facts sharing a source chunk may describe other people or
examples. Create a user Schema only when the material explicitly attributes
the pattern to the user. Otherwise output no operation for that material.
- **Candidate coverage**: In a source-context review, account for every listed
candidate fact: either use it as evidence in `create_schema`/`add_evidence`,
or emit `skip_fact`. Raw context may confirm its subject or restore omitted
detail, but must not seed unrelated Schemas.
- **Evidence threshold**: Creating L6 requires one or more user-owned evidence
facts representing at least two independent raw writes or sessions.
Assistant advice does not establish a user pattern by itself.
### Good vs Bad examples
✅ "When cooking, the user strictly follows recipe steps and precisely measures ingredients — reflecting a need for external structure to manage uncertainty."
✅ "When gaming, the user always checks walkthroughs to achieve 100% completion — driven by low tolerance for incompleteness."
✅ "After an unexpected mentor loss, mortality reminders can bring grief back suddenly; the user pauses to reflect and takes quiet walks to regain steadiness."
❌ "The user approaches music as a medium for creative synthesis and cultural bridge-building, consistently blending diverse musical traditions..." (multiple patterns merged, too broad)
❌ "The user is passionate about many things and values quality." (no circumstance, too vague)
❌ "The user is meaning-driven and resilient." (erases the durable trigger and observed coping evidence)
## Your Workflow
Input materials (you are only invoked when at least one cluster exists):
- **Cluster Results**: Primary input — facts grouped by semantic similarity
- **Unclustered Facts (supplemental)**: Facts not in any cluster; lower priority than cluster results
For each group of facts (clustered or unclustered):
1. **Search existing Schemas**: If an existing Schema already captures this pattern, call `add_evidence`. Use `revise_schema` only when the old wording would become materially incomplete or misleading after adding the new evidence.
2. **Create new Schema**: Only if no existing Schema covers this pattern. One sentence, three components.
3. **Build relationships**: If two Schemas are thematically related, use `add_edge` (RELATED_TO).
## Principles
- All Graph operations MUST go through tools
- Schema content is IMMUTABLE — never recreate what already exists
- Prefer `add_evidence` over creating duplicates
- Prefer not creating over creating low-quality nodes
- One Schema = one atomic pattern in one domain
- When tagging, prefer reusing tags from the existing tags list"""
from ..prompts import ( # noqa: E402
get_prompt,
normalize_language,
with_output_language,
)
from ..prompts.zh_hans import SYSTEM2_AGENT_PROMPT as SYSTEM2_AGENT_PROMPT_ZH # noqa: E402,F401 (re-export)
from ..prompts.en import SYSTEM2_AGENT_PROMPT as SYSTEM2_AGENT_PROMPT_EN # noqa: E402,F401 (re-export)
# backward compat alias
SYSTEM2_AGENT_PROMPT = SYSTEM2_AGENT_PROMPT_ZH
@@ -2095,10 +1940,14 @@ async def run_system2_agent(
tool_executor: System2ToolExecutor,
config: MemoryConfig,
max_iterations: int = 10, # kept for backward compat, unused in single-call mode
language: Optional[str] = None,
) -> Dict[str, Any]:
"""
Phase 2: System 2 Agent — Single LLM call + JSON output + post-process execution.
``language``: explicit prompt language code (overrides config.language);
None and no config default → zh/en detection over the materials.
Instead of multi-turn tool-calling loop, this version:
1. Puts all materials (facts + existing graph) into the prompt
2. Asks LLM to output a JSON array of operations
@@ -2120,9 +1969,17 @@ async def run_system2_agent(
# Build LLMProvider from config
llm_provider = LLMProvider(config)
# Detect language from materials content
lang = _detect_language(materials)
system_prompt = _build_single_call_system_prompt(lang)
# Prompt language: explicit language (evolve(language=...) or
# MemoryConfig.language) wins; otherwise detect zh/en from the materials.
explicit_language = normalize_language(language) or normalize_language(
getattr(config, "language", None)
)
lang = explicit_language or _s2_language_code(_detect_language(materials))
system_prompt = with_output_language(
_build_single_call_system_prompt(lang),
lang,
explicit=explicit_language is not None,
)
user_msg = _build_materials_message(materials, lang=lang)
messages = [
@@ -2393,101 +2250,30 @@ async def run_system2_agent(
}
def _build_single_call_system_prompt(lang: str) -> str:
"""Build system prompt for single-call mode (includes output format instructions)."""
base = SYSTEM2_AGENT_PROMPT_ZH if lang == "zh" else SYSTEM2_AGENT_PROMPT_EN
def _s2_language_code(lang: str) -> str:
"""旧的 "zh" / "en" 标记映射到语言码;其余值按语言码处理。"""
if lang == "zh":
output_instructions = """
return "zh-Hans"
return lang or "en"
## 输出格式
不要调用 tools。直接输出一个 JSON 数组,包含你决定执行的操作。用 ```json 代码块包裹。
def _build_single_call_system_prompt(lang: str) -> str:
"""Build system prompt for single-call mode (includes output format instructions).
可用操作:
1. **create_schema** — 创建新的 L6 行为 Schema
```{"op": "create_schema", "schema_kind": "behavior_pattern", "content": "供用户阅读的简洁画像", "retrieval_text": "用于检索的行为与适用场景", "evidence_list": ["fact_node_id_1", "fact_node_id_2"], "tags": ["tag1"]}```
2. **add_evidence** — 给已有 Schema 追加证据(不修改内容)
```{"op": "add_evidence", "node_id": "existing_schema_node_id", "evidence_list": ["new_fact_id_1"]}```
3. **revise_schema** — 新证据使已有 Schema 的当前状态发生实质变化时,创建新版本并让旧版本退出默认检索。不要用于可共存的细节补充。
```{"op": "revise_schema", "node_id": "old_schema_id", "schema_kind": "explicit_state", "content": "更新后的当前画像", "retrieval_text": "更新后的检索文本", "evidence_list": ["new_fact_id_1"], "tags": ["tag1"], "reason": "明确说明旧画像为何会误导"}```
4. **add_edge** — 建立两个 Schema 之间的关系
```{"op": "add_edge", "source_id": "schema_id_1", "target_id": "schema_id_2", "reason": "关系原因"}```
5. **skip_fact** — 明确标记某条候选事实已审查但不创建 Schema。仅用于他人事实、临时事件、不能证明用户特征的助手建议,或已被现有 Schema 完整覆盖的事实。
```{"op": "skip_fact", "node_id": "candidate_fact_id", "reason": "不是稳定的用户模式"}```
## 输出约定
1. 输出必须是 JSON 数组,用 ```json 代码块包裹。
2. 如果数据不足以得出可靠结论,输出 `[]`。在 source-context review 中,
`[]` 表示已审查该组全部候选,且都不足以支持稳定 Schema。
3. 代码块外不要有任何其他文字。
4. evidence_list 中的 id 必须来自材料中提供的 fact node_id。
5. add_evidence / revise_schema 的 node_id 必须来自已有 Graph 节点的 id。
6. 对 source-context 审查,必须让每条候选事实出现在 evidence_list 或 skip_fact 中。"""
else:
output_instructions = """
## Output Format
Do NOT call tools. Output a JSON array of operations directly, wrapped in a ```json code block.
Available operations:
1. **create_schema** — Create a new L6 behavioral Schema
```{"op": "create_schema", "schema_kind": "behavior_pattern", "content": "concise user-facing pattern", "retrieval_text": "behavior plus relevant situations", "evidence_list": ["fact_node_id_1", "fact_node_id_2"], "tags": ["tag1"]}```
2. **add_evidence** — Add evidence to an existing Schema (content stays immutable)
```{"op": "add_evidence", "node_id": "existing_schema_node_id", "evidence_list": ["new_fact_id_1"]}```
3. **revise_schema** — Create a new version when new evidence materially changes the current state and retire the old version from active retrieval. Do not use it for compatible detail.
```{"op": "revise_schema", "node_id": "old_schema_id", "schema_kind": "explicit_state", "content": "updated current profile", "retrieval_text": "updated retrieval text", "evidence_list": ["new_fact_id_1"], "tags": ["tag1"], "reason": "why the old wording would mislead"}```
4. **add_edge** — Create a relationship between two Schemas
```{"op": "add_edge", "source_id": "schema_id_1", "target_id": "schema_id_2", "reason": "relationship reason"}```
5. **skip_fact** — Explicitly mark a candidate fact as reviewed without creating
a Schema. Use only for facts about another person, transient events, assistant
advice that does not establish a user trait, or facts already fully covered.
```{"op": "skip_fact", "node_id": "candidate_fact_id", "reason": "not a durable user pattern"}```
## Output Contract
1. Output MUST be a JSON array wrapped in a ```json code block.
2. If data is insufficient for reliable conclusions, output `[]`. In a
source-context review, `[]` means every candidate in that review was examined
and none supports a durable Schema.
3. NO text outside the code block.
4. evidence_list IDs must come from the fact node_ids provided in the materials.
5. add_evidence / revise_schema node_id must come from existing Graph node IDs.
6. For a source-context review, every candidate fact must appear in an evidence_list or skip_fact."""
``lang`` is a language code (``en`` / ``zh-Hans`` / ``ja`` ...; legacy ``zh``
means zh-Hans). Every fragment comes from hy_memory/prompts/<language>.py.
"""
code = _s2_language_code(lang)
base = get_prompt("SYSTEM2_AGENT_PROMPT", code)
output_instructions = get_prompt("SYSTEM2_OUTPUT_INSTRUCTIONS", code)
if _S2_SCHEMA_EVIDENCE_GATE_ENABLED:
if lang == "zh":
output_instructions += """
output_instructions += get_prompt("SYSTEM2_SCHEMA_EVIDENCE_GATE", code)
7. L6 只表示跨事件行为模式。create_schema 必须使用 user-owned 证据,且证据合计代表至少两个独立 raw/session。当 `source_raw_memory_ids` 包含多个独立写入时,一个合并后的 L2 事实也可以满足门槛。
8. 单条明确身份、健康状态、设备使用、隶属关系、项目或偏好不要创建 L6;这些由 canonical L2 状态通道维护。"""
else:
output_instructions += """
7. L6 represents cross-event behavioral patterns only. create_schema must use user-owned evidence representing at least two independent raw writes or sessions. A single reconciled fact can qualify when source_raw_memory_ids contains multiple independent writes.
8. Do not create L6 for a single explicit identity, health state, equipment use, affiliation, project, or preference. Canonical L2 state projection owns those facts."""
# Remove the old "All Graph operations MUST go through tools" principle
base = base.replace("- 所有 Graph 操作必须通过 tools 执行\n", "")
base = base.replace("- All Graph operations MUST go through tools\n", "")
# Remove old tool-calling instructions from workflow
base = base.replace("调 `add_evidence` 添加证据", "使用 add_evidence 操作")
base = base.replace("call `add_evidence`", "use add_evidence operation")
base = base.replace("调 `add_evidence`", "使用 add_evidence")
base = base.replace("用 `add_edge`(RELATED_TO)", "使用 add_edge 操作")
base = base.replace("use `add_edge` (RELATED_TO)", "use add_edge operation")
# Remove the old "All Graph operations MUST go through tools" principle and
# the tool-calling wording from the workflow (single-call mode has no tools).
for old, new in get_prompt("SYSTEM2_SINGLE_CALL_REPLACEMENTS", code):
base = base.replace(old, new)
return base + output_instructions
@@ -2516,7 +2302,8 @@ def _parse_operations_json(text: str) -> Optional[List[Dict[str, Any]]]:
return result
return None
except json.JSONDecodeError:
logger.warning(f"[S2-agent] JSON parse failed: {json_str[:200]}")
logger.warning(f"[S2-agent] JSON parse failed ({len(json_str)} chars)")
logger.debug(f"[S2-agent] unparsable output: {json_str[:200]!r}")
return None
@@ -2553,10 +2340,14 @@ def _detect_language(materials: Dict[str, Any]) -> str:
def _build_materials_message(materials: Dict[str, Any], lang: str = "zh") -> str:
"""将预处理好的 materials 格式化为 Agent 能理解的 user message"""
is_zh = (lang == "zh")
"""将预处理好的 materials 格式化为 Agent 能理解的 user message
parts = [("## 预处理材料\n" if is_zh else "## Preprocessed Materials\n")]
``lang`` 为语言码(旧值 "zh" 视作 zh-Hans);文案见 prompts/<language>.py 的
SYSTEM2_MATERIALS。
"""
text = get_prompt("SYSTEM2_MATERIALS", _s2_language_code(lang))
parts = [text["title"]]
stats = materials.get("stats", {})
fresh = stats.get("fresh_facts", 0)
@@ -2565,55 +2356,35 @@ def _build_materials_message(materials: Dict[str, Any], lang: str = "zh") -> str
graph_total = stats.get("graph_total_schemas", 0)
recalled_schemas = stats.get("graph_schemas_recalled", 0)
if is_zh:
parts.append(f"### 统计\n- VDB 事实总数: {stats.get('total_facts_all', 0)}")
parts.append(f"- 新鲜事实(待处理): {fresh}(已被引用排除={evidenced})")
parts.append(f"- 聚类数: {stats.get('clusters_found', 0)}")
parts.append(
f"- 聚类内事实: {stats.get('clustered_facts_count', 0)},"
f"未聚类补充: {stats.get('unclustered_facts_count', 0)}"
)
parts.append(f"- Graph Schema 总数: {stats.get('graph_total_schemas', 0)}(本次召回={recalled_schemas})")
if graph_total == 0:
parts.append("- Graph 状态: **空**(首次认知加工)\n")
else:
parts.append("")
parts.append(text["stats"].format(
total=stats.get('total_facts_all', 0),
fresh=fresh,
evidenced=evidenced,
clusters=stats.get('clusters_found', 0),
clustered=stats.get('clustered_facts_count', 0),
unclustered=stats.get('unclustered_facts_count', 0),
graph_total=stats.get('graph_total_schemas', 0),
recalled=recalled_schemas,
))
if graph_total == 0:
parts.append(text["graph_status_empty"])
else:
parts.append(f"### Statistics\n- Total VDB facts: {stats.get('total_facts_all', 0)}")
parts.append(f"- Fresh facts (to process): {fresh} (evidenced_excluded={evidenced})")
parts.append(f"- Clusters found: {stats.get('clusters_found', 0)}")
parts.append(
f"- Facts in clusters: {stats.get('clustered_facts_count', 0)}, "
f"unclustered supplement: {stats.get('unclustered_facts_count', 0)}"
)
parts.append(f"- Graph Schemas total: {stats.get('graph_total_schemas', 0)} (recalled={recalled_schemas})")
if graph_total == 0:
parts.append("- Graph status: **EMPTY** (first cognitive processing run)\n")
else:
parts.append("")
parts.append("")
# Clusters
clusters = materials.get("clusters", [])
clustered_ids = set()
if clusters:
parts.append(("### 聚类结果\n" if is_zh else "### Cluster Results\n"))
parts.append(text["clusters_header"])
for i, c in enumerate(clusters):
n_facts = len(c['facts'])
group_type = c.get("group_type", "semantic_cluster")
if group_type == "holistic_therapy_review":
label = (
f"{n_facts} 条跨会话用户事实;只归纳有证据的长期情绪/治疗模式"
if is_zh else
f"{n_facts} cross-session user facts; only evidence-backed durable emotional/therapy patterns"
)
label = text["label_holistic"].format(n=n_facts)
elif group_type == "source_context_review":
label = (
f"{n_facts} 条候选事实;原始写入仅用于主体与细节核验"
if is_zh else
f"{n_facts} candidate facts; raw write is only for subject/detail validation"
)
label = text["label_source_context"].format(n=n_facts)
else:
label = f"{n_facts} 条,主题" if is_zh else f"{n_facts} facts, topic"
label = text["label_topic"].format(n=n_facts)
parts.append(f"**Cluster {i}** ({label}: {c['centroid_text']})")
for f in c["facts"]:
source_ids = list(f.get("source_raw_memory_ids") or [])
@@ -2627,8 +2398,7 @@ def _build_materials_message(materials: Dict[str, Any], lang: str = "zh") -> str
)
clustered_ids.add(f['node_id'])
if group_type == "source_context_review" and c.get("raw_contexts"):
raw_label = "候选事实对应的原始片段" if is_zh else "Relevant raw excerpt per candidate"
parts.append(f"\n**{raw_label}:**")
parts.append(f"\n**{text['raw_label']}:**")
for item in c["raw_contexts"]:
parts.append(
f"- candidate_id={item.get('node_id')}, "
@@ -2637,30 +2407,12 @@ def _build_materials_message(materials: Dict[str, Any], lang: str = "zh") -> str
)
parts.append("")
if materials.get("holistic_therapy_review"):
parts.append(
(
"这是一次有边界的夜间全局整理。最多创建 "
f"{_S2_HOLISTIC_THERAPY_MAX_SCHEMAS} 条 anchored therapy Schema。"
"优先联合不同 raw/session 中的持久 trigger/background、"
"反复 emotional response 和用户已实际使用的 coping resource/decision constraint。"
"证据不足时输出 []。\n"
if is_zh else
"This is one bounded nightly holistic consolidation pass. Create at most "
f"{_S2_HOLISTIC_THERAPY_MAX_SCHEMAS} anchored therapy Schemas. "
"Prefer joining durable trigger/background, recurring emotional response, "
"and observed coping resource or decision constraint across different raw writes/sessions. "
"Before create_schema, union raw_sources for its evidence IDs and require at least two "
"distinct raw source IDs; two facts from one raw write are one event. "
"Output [] when evidence is insufficient.\n"
)
)
parts.append(text["holistic_note"].format(
max_schemas=_S2_HOLISTIC_THERAPY_MAX_SCHEMAS,
))
companion_hints = materials.get("independent_source_companions") or {}
if companion_hints:
parts.append(
"**独立来源 companion 提示(只是候选,不是结论):**"
if is_zh else
"**Independent-source companion hints (candidates, not conclusions):**"
)
parts.append(text["companion_header"])
for node_id, companions in companion_hints.items():
parts.append(
f"- {node_id} -> "
@@ -2676,14 +2428,8 @@ def _build_materials_message(materials: Dict[str, Any], lang: str = "zh") -> str
all_facts = materials.get("unprocessed_facts", [])
unclustered = [f for f in all_facts if f["node_id"] not in clustered_ids]
if clusters and unclustered:
header = "### 未聚类事实(补充,次要)\n" if is_zh else "### Unclustered Facts (supplemental)\n"
hint = (
"以下事实未进入上述任一聚类,可在完成聚类主题加工后酌情审视;优先级低于聚类结果。\n"
if is_zh
else "Facts below were not grouped into any cluster above. Review after cluster themes; lower priority than cluster results.\n"
)
parts.append(header)
parts.append(hint)
parts.append(text["unclustered_header"])
parts.append(text["unclustered_hint"])
for f in unclustered:
parts.append(f" - [{f['layer']}] {f['content']} (id={f['node_id']})")
parts.append("")
@@ -2691,10 +2437,7 @@ def _build_materials_message(materials: Dict[str, Any], lang: str = "zh") -> str
# Existing graph nodes
forward = materials.get("graph_forward", [])
if forward:
if is_zh:
parts.append(f"### 相关 Graph 节点(通过 cluster centroid 向量召回,共 {len(forward)} 条)\n")
else:
parts.append(f"### Related Graph Nodes (retrieved via cluster centroid search, {len(forward)} hits)\n")
parts.append(text["forward_header"].format(count=len(forward)))
for g in forward:
sim = g.get("similarity", "")
sim_str = f", sim={sim}" if sim else ""
@@ -2704,30 +2447,17 @@ def _build_materials_message(materials: Dict[str, Any], lang: str = "zh") -> str
)
parts.append("")
elif graph_total == 0:
if is_zh:
parts.append("### 已有 Graph 节点\n当前 Graph 为空,这是首次认知加工。无需搜索 Graph,直接根据聚类结果创建 Schema。\n")
else:
parts.append("### Existing Graph Nodes\nThe Graph is currently EMPTY — this is the first cognitive processing run. Do NOT search the Graph (it will return nothing). Proceed directly to create Schemas from the cluster results.\n")
parts.append(text["graph_empty"])
else:
# Graph 有节点但 centroid search 没召回(可能 threshold 太高或语义距离远)
if is_zh:
parts.append(
f"### 已有 Graph 节点\n"
f"Graph 中共有 {stats.get('graph_total_schemas', 0)} 个 Schema,"
f"但 centroid 向量召回未命中相关节点。请根据聚类结果判断是否需要创建新 Schema。\n"
)
else:
parts.append(
f"### Existing Graph Nodes\n"
f"The Graph has {stats.get('graph_total_schemas', 0)} Schemas, "
f"but centroid-based vector recall found no matches. Decide based on cluster results whether new Schemas are needed.\n"
)
parts.append(text["graph_no_recall"].format(
graph_total=stats.get('graph_total_schemas', 0),
))
# Reverse hits
reverse = materials.get("graph_reverse", [])
if reverse:
parts.append(("### Graph 反向引用(已有 Schema 引用的 VDB 事实)\n" if is_zh
else "### Graph Reverse References (VDB facts referenced by existing Schemas)\n"))
parts.append(text["reverse_header"])
for r in reverse:
parts.append(
f" - [{r['layer']}] {r['content']} "
@@ -2738,26 +2468,9 @@ def _build_materials_message(materials: Dict[str, Any], lang: str = "zh") -> str
# Existing tags(指导 Agent 复用已有 tag,减少发散)
existing_tags = materials.get("existing_tags", [])
if existing_tags:
if is_zh:
parts.append("### 已有标签(请优先从中选择,避免创建同义标签)\n")
parts.append(f" {', '.join(existing_tags)}\n")
else:
parts.append("### Existing Tags (prefer reusing these over creating synonyms)\n")
parts.append(f" {', '.join(existing_tags)}\n")
parts.append(text["tags_header"])
parts.append(f" {', '.join(existing_tags)}\n")
if is_zh:
parts.append(
"请分析以上材料,输出操作 JSON 数组。"
"决定是否需要创建新的 Schema,或为已有 Schema 追加 evidence。"
"如果数据不足以得出可靠结论,输出空数组 `[]`。"
"打标签时请优先使用已有标签列表中的标签。"
)
else:
parts.append(
"Analyze the materials above and output a JSON array of operations. "
"Decide whether to create new Schemas, or add evidence to existing ones. "
"If the data is insufficient for reliable conclusions, output an empty array `[]`. "
"When tagging, prefer reusing tags from the existing tags list above."
)
parts.append(text["closing"])
return "\n".join(parts)
+25 -1
View File
@@ -243,6 +243,7 @@ class System2Writer(WritePipeline):
materials=materials,
tool_executor=tool_executor,
config=self.config,
language=payload.get("language"),
)
# 计算 timing
@@ -508,6 +509,7 @@ class System2Writer(WritePipeline):
materials=batch,
tool_executor=tool_executor,
config=self.config,
language=payload.get("language"),
)
batch_record = {
"batch_index": batch_index,
@@ -632,6 +634,7 @@ class System2Writer(WritePipeline):
materials=holistic_materials,
tool_executor=tool_executor,
config=self.config,
language=payload.get("language"),
)
if holistic_result.get("success"):
await self._mark_holistic_therapy_reviewed(
@@ -1044,11 +1047,26 @@ class System2Writer(WritePipeline):
agent_id: str,
llm_call,
request_id: str = "",
language: Optional[str] = None,
) -> Dict[str, Any]:
"""在 System2 Agent 之后运行跨域突破性归纳"""
"""在 System2 Agent 之后运行跨域突破性归纳
sweeper 的 prompt 只有英文版(要求与输入 schema 同语言)。显式 language
时在每个 prompt 末尾追加该语言的输出规则,让产出用该语言书写。
"""
if not _SWEEPER_ENABLED:
return {"skipped": True, "reason": "SWEEPER_ENABLED=false"}
if language:
from ..prompts import with_output_language
_base_call = llm_call
async def llm_call(prompt: str) -> str:
return await _base_call(
with_output_language(prompt, language, explicit=True)
)
try:
from .cross_domain_sweeper import CrossDomainSweeper
@@ -1119,6 +1137,7 @@ class System2Writer(WritePipeline):
self,
user_id: str,
agent_id: str = "default_agent",
language: Optional[str] = None,
) -> Dict[str, Any]:
"""
手动触发 cross-domain sweeper(同步等待完成)。
@@ -1145,6 +1164,7 @@ class System2Writer(WritePipeline):
agent_id=agent_id,
llm_call=self._get_llm_call(),
request_id=request_id,
language=language,
)
result["request_id"] = request_id
return result
@@ -1154,6 +1174,7 @@ class System2Writer(WritePipeline):
user_id: str,
agent_id: str = "default_agent",
max_batches: Optional[int] = None,
language: Optional[str] = None,
) -> Dict[str, Any]:
"""
手动触发 System 2 认知加工(由调用方/App 层控制触发时机)。
@@ -1186,6 +1207,8 @@ class System2Writer(WritePipeline):
"content": "",
"created_at": datetime.now().isoformat(),
"max_batches": max_batches,
# 显式 prompt 语言码(None → System 2 按 materials 做 zh/en 检测)
"language": language,
}
results = await self._run_system2_workers(payload)
@@ -1204,6 +1227,7 @@ class System2Writer(WritePipeline):
agent_id=agent_id,
llm_call=self._get_llm_call(),
request_id=request_id,
language=language,
)
agent_success = bool((results.get("system2_agent") or {}).get("success", True))
+70 -20
View File
@@ -280,6 +280,22 @@ class MemoryWriter(WritePipeline):
custom["source_raw_memory_ids"] = source_ids
return custom
def _flat_trimmed_raw(self, request: WriteRequest) -> str:
"""把 request.messages 按 raw 统一规则裁剪后拼成 L1_RAW 文本。
只裁落库副本;request.messages 不动(本次 extractor 输入仍用完整 messages)。
"""
import copy as _copy
from ..utils.raw_trim import trim_raw_messages
_coding_cfg = getattr(getattr(self, "config", None), "coding", None)
_trimmed_req = _copy.copy(request)
_trimmed_req.messages = trim_raw_messages(
request.messages,
tool_max_bytes=getattr(_coding_cfg, "tool_result_max_bytes", None),
)
return _trimmed_req.get_flat_content()
@staticmethod
def _attach_client_meta(custom: Dict[str, Any], request: Optional[WriteRequest]) -> None:
"""给新建节点挂上调用方 meta(单份 list)。
@@ -981,6 +997,7 @@ class MemoryWriter(WritePipeline):
# 这是一次 LLM 调用:prompt / 产出必须能在线上追到,
# 否则 badcase 只能看到"摘要内容不对"却拿不到输入
trace=self._make_trace_sink(request, "EPISODE_SUMMARY"),
language=request.language,
)
if nid:
logger.info(
@@ -1194,6 +1211,7 @@ class MemoryWriter(WritePipeline):
self._make_trace_sink(request, "TOPIC_SUMMARY")
if request is not None else None
),
language=getattr(request, "language", None),
)
except Exception as e:
logger.warning(f"[topic-summary] failed (non-fatal): {e}")
@@ -1270,6 +1288,7 @@ class MemoryWriter(WritePipeline):
graph_store=getattr(self, "_graph_store", None),
new_memory_embeddings=precomputed_embeddings,
source=request.source,
language=request.language,
)
if not recon_result.success:
@@ -2554,9 +2573,14 @@ class MemoryWriter(WritePipeline):
try:
# 参数校验
content = request.content
# raw 在写入时就按统一规则裁好(见 utils/raw_trim.py):工具输出、长
# assistant 回复不再原样长期存储;历史拼接直接用存下来的文本,不再二次裁剪。
# request.messages 保持原样——本次 extractor 输入仍是完整 messages。
_raw_trimmed = False
if not content and request.has_messages():
content = request.get_flat_content()
content = self._flat_trimmed_raw(request)
request.content = content
_raw_trimmed = True
if not content:
response.error_code = 400
@@ -2627,6 +2651,9 @@ class MemoryWriter(WritePipeline):
vector_store = await self._get_vector_store()
layer_enum = MemoryLayer.from_string(suggested_layer)
_raw_custom: Dict[str, Any] = {}
if _raw_trimmed:
from ..utils.raw_trim import RAW_TRIMMED_KEY
_raw_custom[RAW_TRIMMED_KEY] = True
self._attach_client_meta(_raw_custom, request)
mem_node = MemoryNode(
user_id=request.user_id,
@@ -2690,6 +2717,7 @@ class MemoryWriter(WritePipeline):
vector_store, request.user_id, request.agent_id,
exclude_memory_id=memory_id,
max_turns=_hist_turns * 2, # k 轮对话 = k*2 条 message
language=request.language,
)
_recent_history_ms = (_t_recent_history and (datetime.now() - _t_recent_history).total_seconds() * 1000) or 0.0
_recent_history_count = len([line for line in (_history_context or "").splitlines() if line.strip()])
@@ -2770,6 +2798,7 @@ class MemoryWriter(WritePipeline):
user_id=request.user_id,
agent_id=request.agent_id,
session_id=request.session_id or "default_session",
language=request.language,
)
if finalize is finalize_dynamic_episode_capsules:
finalize_kwargs["summary_concurrency"] = getattr(
@@ -2929,13 +2958,18 @@ class MemoryWriter(WritePipeline):
agent_id: Optional[str] = None,
exclude_memory_id: str = "",
max_turns: int = 20,
max_chars_assistant: int = 500,
max_chars_assistant: Optional[int] = None,
language: Optional[str] = None,
) -> str:
"""
获取该用户最近的 L1_RAW 对话记录,按轮次拆分后拼成 history context。
language:显式 prompt 语言码,用于渲染名字行;None 时按内容判 zh/en。
每条 L1_RAW 的 content 是多轮对话([user]: ...\n[assistant]: ...)。
拆成单条 message 后,user 消息完整保留,assistant 消息截取前 max_chars_assistant 字符。
raw 在写入时已按 utils/raw_trim.py 的规则裁好(custom[RAW_TRIMMED_KEY]),
这里拆成单条 message 后直接拼接,不再二次裁剪。该规则之前写入的旧 raw
没有这个标记,仍按同一(幂等)规则裁剪后再拼,避免旧的长回复 / 工具输出
撑大 prompt。max_chars_assistant 只作用于这些旧 raw,默认同写入规则。
最终取最近 max_turns 条 message(注意单位是 message 条数,非对话轮数;
调用方按 k 轮 × 2 传入)。
@@ -2977,23 +3011,33 @@ class MemoryWriter(WritePipeline):
# 避免把正文里形如 "[注]: xxx" 的普通文本误判成消息边界。
import re as _re
from .base import parse_user_name_line
_KNOWN_ROLES = {"user", "assistant", "system", "tool"}
# 带方括号的任意 label(含对方真人名字)
_labeled_prefix = _re.compile(r"^\[([^\]\n]{1,32})\]:\s?")
# 无方括号的已知 role(历史数据容错,沿用旧正则的 `\[?...\]?` 宽容度)
_bare_role_prefix = _re.compile(r"^(user|assistant|system|tool):\s?")
# (label, content) 元组列表;label 为 role 或对方真人名字
parsed: List[Tuple[str, str]] = []
from ..utils.raw_trim import (
DEFAULT_RAW_ASSISTANT_MAX_CHARS,
RAW_TRIMMED_KEY,
trim_raw_body,
)
if max_chars_assistant is None:
max_chars_assistant = DEFAULT_RAW_ASSISTANT_MAX_CHARS
_coding_cfg = getattr(getattr(self, "config", None), "coding", None)
_tool_max_bytes = getattr(_coding_cfg, "tool_result_max_bytes", None)
# (label, content, 是否旧 raw 需兼容裁剪) 元组列表;label 为 role 或对方真人名字
parsed: List[Tuple[str, str, bool]] = []
user_name: str = ""
for node in nodes:
content = node.content or ""
_legacy = not bool((node.custom or {}).get(RAW_TRIMMED_KEY))
for _line_idx, line in enumerate(content.split("\n")):
m = _labeled_prefix.match(line) or _bare_role_prefix.match(line)
if m:
label = m.group(1)
body = line[m.end():]
parsed.append((label, body))
parsed.append((label, body, _legacy))
continue
# 名字行只可能是每个 node content 的第一行(get_flat_content 插在文首)。
# 逐 node 判断而非用 `not parsed`:parsed 跨 node 累积,第二个 node
@@ -3006,36 +3050,41 @@ class MemoryWriter(WritePipeline):
continue
if parsed:
# 续行:拼回上一条 message(保留换行)
prev_label, prev_body = parsed[-1]
parsed[-1] = (prev_label, prev_body + "\n" + line)
prev_label, prev_body, prev_legacy = parsed[-1]
parsed[-1] = (prev_label, prev_body + "\n" + line, prev_legacy)
else:
# 开头就是无前缀行(异常/老数据):当成 user 消息兜底
parsed.append(("user", line))
parsed.append(("user", line, _legacy))
# 取最近 max_turns 条 message(按 message 截断,绝不从消息中间切开)
recent = parsed[-max_turns:] if len(parsed) > max_turns else parsed
if not recent:
return ""
# 格式化:非 user 侧整条截断(含对方真人),user 完整保留
# 格式化:新 raw 直接拼接;旧 raw 按写入同一规则兼容裁剪
all_messages: List[str] = []
if user_name:
from .base import render_user_name_line
from ..utils.lang_detect import is_chinese
_name_line = render_user_name_line(
user_name,
is_chinese("\n".join(b for _, b in recent)),
_name_line = (
render_user_name_line(user_name, language=language)
if language else
render_user_name_line(
user_name,
is_chinese("\n".join(b for _, b, _ in recent)),
)
)
if _name_line:
all_messages.append(_name_line)
for label, body in recent:
for label, body, legacy in recent:
if legacy:
body = trim_raw_body(
label, body,
assistant_max_chars=max_chars_assistant,
tool_max_bytes=_tool_max_bytes,
)
body = body.strip()
# 截断口径:assistant + 对方真人(非已知 role 的 label)。
# user 完整保留;system / tool 沿用历史行为(不截断)。
_truncate = label == "assistant" or label not in _KNOWN_ROLES
if _truncate and len(body) > max_chars_assistant:
body = body[:max_chars_assistant] + "..."
all_messages.append(f"[{label}]: {body}")
return "\n".join(all_messages)
@@ -3267,6 +3316,7 @@ class MemoryWriter(WritePipeline):
existing_tags=existing_tags,
history_context=history_context,
basic_profile_fields=_basic_profile_fields,
language=request.language,
)
extract_timing["extract_total_ms"] = round(
(datetime.now() - _t_extract_total).total_seconds() * 1000, 2
+242
View File
@@ -0,0 +1,242 @@
"""Prompt registry: every multilingual LLM prompt, keyed by language code.
One module per language (``en.py``, ``zh_hans.py``, ``ja.py`` ...) defines the
same set of names (``PROMPT_NAMES``). Call sites look prompts up with
``get_prompt(name, language)`` instead of importing a language module, so a
new language only needs a new module and an entry in ``_MODULES``.
Language selection (``resolve_language``):
* explicit ``language`` (per call, else ``MemoryConfig.language`` /
``MEMORY_LANGUAGE``) wins and is validated by ``normalize_language``;
* otherwise the existing zh/en auto-detection runs unchanged
(``utils.lang_detect.is_chinese`` → ``zh-Hans`` or ``en``). Auto-detection
never picks any of the other languages.
When the language is explicit, ``with_output_language`` appends
``OUTPUT_LANGUAGE_DIRECTIVE`` so memories are written in that language even
if the conversation uses another one. Auto-detected requests keep the
"same language as the input" rule and their prompts are byte-identical to
earlier releases.
"""
from __future__ import annotations
import importlib
from types import ModuleType
from typing import Any, Dict, Optional, Tuple
# Canonical codes, in documentation order.
SUPPORTED_LANGUAGES: Tuple[str, ...] = (
"en",
"zh-Hans",
"zh-Hant",
"ja",
"es",
"de",
"pt",
"fr",
"ko",
"ru",
"id",
)
# Accepted aliases (matched case-insensitively, "_" treated as "-").
LANGUAGE_ALIASES: Dict[str, str] = {
"zh": "zh-Hans",
"zh-cn": "zh-Hans",
"zh-sg": "zh-Hans",
"zh-hans-cn": "zh-Hans",
"zh-tw": "zh-Hant",
"zh-hk": "zh-Hant",
"zh-mo": "zh-Hant",
"zh-hant-tw": "zh-Hant",
"zh-hant-hk": "zh-Hant",
"pt-br": "pt",
"pt-pt": "pt",
"en-us": "en",
"en-gb": "en",
}
_MODULES: Dict[str, str] = {
"en": "en",
"zh-Hans": "zh_hans",
"zh-Hant": "zh_hant",
"ja": "ja",
"es": "es",
"de": "de",
"pt": "pt",
"fr": "fr",
"ko": "ko",
"ru": "ru",
"id": "id",
}
# Every language module must define all of these.
PROMPT_NAMES: Tuple[str, ...] = (
# chat extraction
"EXTRACT_SYSTEM_PROMPT",
"STRUCTURE_METADATA_PROMPT",
"EXTRACT_FEW_SHOT",
"EXTRACT_PROMPT",
"EXTRACT_PROMPT_NO_INTENTION",
"NO_INTENTION_SECTION_PATTERN",
"NO_INTENTION_MEMORY_SECTION",
"NO_INTENTION_REPLACEMENTS",
"INTENTION_DISABLED_NOTE",
"USER_NAME_LINE",
# chat reconcile
"SEARCH_QUERY_PROMPT",
"RECONCILE_PROMPT",
"RECONCILE_STRUCTURE_METADATA_PROMPT",
"RECONCILE_FEW_SHOT",
# working / coding memory
"CODING_EXTRACT_PROMPT",
"CODING_EXTRACT_PROMPT_CODING",
"CODING_CONVERSATION_BLOCK",
"CODING_EMPTY_BLOCK",
"CODING_RECONCILE_PROMPT",
"CODING_FACT_RECONCILE_PROMPT",
"CODING_CHAIN_SUMMARY_PROMPT",
# summaries
"EPISODE_SUMMARY_PROMPT",
"TOPIC_SUMMARY_PROMPT",
# System 2 evolve
"SYSTEM2_AGENT_PROMPT",
"SYSTEM2_OUTPUT_INSTRUCTIONS",
"SYSTEM2_SCHEMA_EVIDENCE_GATE",
"SYSTEM2_SINGLE_CALL_REPLACEMENTS",
"SYSTEM2_MATERIALS",
# explicit-language output rule
"OUTPUT_LANGUAGE_DIRECTIVE",
"LANGUAGE_NAME",
)
# Prompts rendered with str.format(); the value lists the expected fields.
# The rest are appended verbatim (their braces are literal JSON).
FORMAT_FIELDS: Dict[str, Tuple[str, ...]] = {
"EXTRACT_PROMPT": ("content", "current_date", "last_messages", "memory_at"),
"EXTRACT_PROMPT_NO_INTENTION": ("content", "current_date", "last_messages", "memory_at"),
"USER_NAME_LINE": ("name",),
"SEARCH_QUERY_PROMPT": ("new_memories",),
"RECONCILE_PROMPT": ("current_date", "existing_memories", "few_shot_section", "new_memories"),
"CODING_EXTRACT_PROMPT": ("branch_present", "conversation_block", "workspace_present"),
"CODING_EXTRACT_PROMPT_CODING": ("branch_present", "conversation_block", "workspace_present"),
"CODING_CONVERSATION_BLOCK": ("history", "new"),
"CODING_RECONCILE_PROMPT": ("candidates_block", "drafts_block"),
"CODING_FACT_RECONCILE_PROMPT": ("candidates_block", "drafts_block"),
"CODING_CHAIN_SUMMARY_PROMPT": ("chain_block",),
"EPISODE_SUMMARY_PROMPT": ("memories",),
"TOPIC_SUMMARY_PROMPT": ("memories", "topic_name"),
}
# Fields of each SYSTEM2_MATERIALS entry (all other entries have none).
SYSTEM2_MATERIALS_FIELDS: Dict[str, Tuple[str, ...]] = {
"title": (),
"stats": ("clustered", "clusters", "evidenced", "fresh", "graph_total", "recalled", "total", "unclustered"),
"graph_status_empty": (),
"clusters_header": (),
"label_holistic": ("n",),
"label_source_context": ("n",),
"label_topic": ("n",),
"raw_label": (),
"holistic_note": ("max_schemas",),
"companion_header": (),
"unclustered_header": (),
"unclustered_hint": (),
"forward_header": ("count",),
"graph_empty": (),
"graph_no_recall": ("graph_total",),
"reverse_header": (),
"tags_header": (),
"closing": (),
}
_loaded: Dict[str, ModuleType] = {}
def _supported_text() -> str:
aliases = ", ".join(f"{k}->{v}" for k, v in LANGUAGE_ALIASES.items())
return f"supported: {', '.join(SUPPORTED_LANGUAGES)} (aliases: {aliases})"
def normalize_language(language: Optional[str]) -> Optional[str]:
"""Return the canonical code for ``language``; None/"" stays None.
Raises ValueError for an unknown code, listing the supported codes.
"""
if language is None:
return None
if not isinstance(language, str):
raise ValueError(
f"language must be a string, got {type(language).__name__}; "
+ _supported_text()
)
raw = language.strip()
if not raw:
return None
key = raw.replace("_", "-").lower()
for code in SUPPORTED_LANGUAGES:
if code.lower() == key:
return code
if key in LANGUAGE_ALIASES:
return LANGUAGE_ALIASES[key]
raise ValueError(f"unsupported language {language!r}; " + _supported_text())
def detect_language_code(content: str = "", messages=None) -> str:
"""The legacy automatic choice: ``zh-Hans`` if Chinese, else ``en``."""
from ..utils.lang_detect import is_chinese
return "zh-Hans" if is_chinese(content, messages=messages) else "en"
def resolve_language(
language: Optional[str] = None,
content: str = "",
messages=None,
) -> str:
"""Explicit language if given (validated), else zh/en auto-detection."""
code = normalize_language(language)
if code is not None:
return code
return detect_language_code(content, messages=messages)
def _module(language: str) -> ModuleType:
code = normalize_language(language) or "en"
mod = _loaded.get(code)
if mod is None:
mod = importlib.import_module(f"{__name__}.{_MODULES[code]}")
_loaded[code] = mod
return mod
def get_prompt(name: str, language: Optional[str] = "en") -> Any:
"""Look up prompt ``name`` for ``language`` (canonical code or alias)."""
if name not in PROMPT_NAMES:
raise KeyError(f"unknown prompt {name!r}")
return getattr(_module(language or "en"), name)
def with_output_language(
prompt: str, language: Optional[str], explicit: bool,
) -> str:
"""Append the output-language rule when the language was set explicitly."""
if not explicit or not language:
return prompt
return prompt + get_prompt("OUTPUT_LANGUAGE_DIRECTIVE", language)
__all__ = [
"SUPPORTED_LANGUAGES",
"LANGUAGE_ALIASES",
"PROMPT_NAMES",
"FORMAT_FIELDS",
"SYSTEM2_MATERIALS_FIELDS",
"normalize_language",
"detect_language_code",
"resolve_language",
"get_prompt",
"with_output_language",
]
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
+1777
View File
File diff suppressed because it is too large. Load diff
+24
View File
@@ -21,6 +21,11 @@ API Endpoints:
GET /healthz — Health check (static, process alive)
GET /health — Liveness probe (pings internal event loop; 503 if loop blocked)
GET /info — Server info
/add and /add_extracted accept an optional "language" (en, zh-Hans, zh-Hant, ja,
es, de, pt, fr, ko, ru, id, or an alias such as zh-CN / zh-TW / pt-BR). An
unknown code returns 400. /search has no language option: the search-time
judge / rewrite prompts are English-only and keep the query's own language.
"""
import asyncio
@@ -100,6 +105,20 @@ def _get_client():
return _client
def _language_arg(handler: BaseHTTPRequestHandler, body: Dict, kwargs: Dict) -> bool:
"""Validate optional ``language`` and add it to kwargs; False after a 400 reply."""
if body.get("language") in (None, ""):
return True
from .prompts import normalize_language
try:
kwargs["language"] = normalize_language(body["language"])
except ValueError as e:
_json_response(handler, 400, {"error": str(e)})
return False
return True
def _json_response(handler: BaseHTTPRequestHandler, status: int, data: Any):
"""Send a JSON response."""
body = json.dumps(data, ensure_ascii=False, default=str).encode("utf-8")
@@ -436,6 +455,9 @@ class MemoryHTTPHandler(BaseHTTPRequestHandler):
kwargs["memory_at"] = datetime.fromisoformat(body["memory_at"])
except Exception:
pass
# language: explicit prompt / memory language (see HyMemoryClient.add)
if not _language_arg(self, body, kwargs):
return
result = client.add(data, **kwargs)
_json_response(self, 200, result)
@@ -462,6 +484,8 @@ class MemoryHTTPHandler(BaseHTTPRequestHandler):
kwargs["memory_at"] = datetime.fromisoformat(body["memory_at"])
except Exception:
pass
if not _language_arg(self, body, kwargs):
return
result = client.add_extracted(contents, **kwargs)
_json_response(self, 200, result)
+4 -1
View File
@@ -4,6 +4,7 @@ from __future__ import annotations
import asyncio
import concurrent.futures
import contextvars
import functools
from concurrent.futures import Executor
from typing import Any, Callable, Optional
@@ -39,7 +40,9 @@ async def run_in_executor_guarded(
async def to_thread_guarded(func: Callable[..., Any], *args: Any, **kwargs: Any) -> Any:
"""Guarded equivalent of ``asyncio.to_thread`` using the default pool."""
return await run_in_executor_guarded(None, func, *args, **kwargs)
context = contextvars.copy_context()
call = functools.partial(func, *args, **kwargs)
return await run_in_executor_guarded(None, context.run, call)
def concurrent_result_guarded(
+2 -1
View File
@@ -1,6 +1,7 @@
# -*- coding: utf-8 -*-
"""
Pipeline Log Writer — 环节级 JSONL 日志(始终开启,与 SQLite Trace 分离)。
Pipeline Log Writer — 环节级 JSONL 日志(默认开启,MEMORY_PIPELINE_LOG_ENABLED=false 关闭;
与 SQLite Trace 分离)。
目录: {log_dir}/{subdir}/{date}.log
每行: request_id, namespace, user_id, agent_id, step, prompt, content, parsed,
+71 -30
View File
@@ -1,16 +1,20 @@
# -*- coding: utf-8 -*-
"""
Pipeline 可观测性:Log(文件 JSONL,始终开启)与 Trace(SQLite pipeline_logs,可关闭)。
Pipeline 可观测性:Log(文件 JSONL,默认开启,可关闭)与 Trace(SQLite pipeline_logs,可关闭)。
- Log:运维 tail/grep,每个环节即时一行,不依赖后续步骤成功与否。
- Trace:Inspector / 调试,落库 pipeline_logs(需 MEMORY_CACHE_BACKEND=sqlite 等持久化后端)。
环境变量:
MEMORY_PIPELINE_LOG_ENABLED — Log 写 JSONL 文件,默认 true;false 时完全不写
MEMORY_PIPELINE_TRACE_ENABLED — Trace 落库,默认 true;OpenClaw C 端可设 false
MEMORY_MEMORY_OPERATIONS_ENABLED — memory_operations 审计表落库,默认 false
MEMORY_LOG_DIR — Log 目录(优先)
MEMORY_TRACE_LOG_DIR — Log 目录(兼容旧名)
Log 与 Trace 都关闭时不包装 store_pipeline_log,而是换成立即返回 True 的
空操作:热路径不做任何计时、脱敏或后台任务,也不写库。
子目录({log_dir}/{subdir}/{date}.log)由 PipelineLogWriter 从每条记录的
user_id 前缀(第一个 "__" 左侧)切分;无前缀回落 "default"。该前缀是
通用 namespace,可用于项目、部署或数据分桶。
@@ -94,6 +98,16 @@ def is_pipeline_trace_enabled() -> bool:
)
def is_pipeline_log_enabled() -> bool:
"""是否把 pipeline step 写成 JSONL 文件({log_dir}/<subdir>/<date>.log)。默认开启。"""
return os.getenv("MEMORY_PIPELINE_LOG_ENABLED", "true").strip().lower() in (
"true",
"1",
"yes",
"on",
)
def is_memory_operations_enabled() -> bool:
"""是否将知识库变动写入 memory_operations 审计表。默认关闭(false)。"""
return os.getenv("MEMORY_MEMORY_OPERATIONS_ENABLED", "false").lower() in (
@@ -125,7 +139,7 @@ def redact_content(value: str) -> str:
def resolve_pipeline_log_dir() -> str:
"""
JSONL pipeline log 根目录(始终使用,有默认值)。
JSONL pipeline log 根目录(MEMORY_PIPELINE_LOG_ENABLED 开启时使用,有默认值)。
优先级: MEMORY_LOG_DIR > MEMORY_TRACE_LOG_DIR > {MEMORY_DATA_DIR}/logs/pipeline
"""
@@ -228,6 +242,7 @@ def wrap_store_pipeline_log(
) -> Callable[..., Any]:
"""包装 cache.store_pipeline_log:计时同步、JSONL/DB 异步落盘,不堵主请求。
- log_writer 为 None 时不写 JSONL(MEMORY_PIPELINE_LOG_ENABLED=false)
- step-to-step elapsed_ms / event_at_ms 在调用瞬间打点(与业务墙钟对齐)
- event_at_ms:调用方传入非 0 值则直接复用(App 已在自己被调用的那一刻
同步打点,比这里更接近真实事件时刻);传 0 / 不传才用本地 time.time()
@@ -410,7 +425,8 @@ def wrap_store_pipeline_log(
try:
loop = asyncio.get_running_loop()
# JSONL 用线程池,与 loop 无亲和,留在当前 loop。
_track_task(loop.create_task(_persist_jsonl()))
if log_writer is not None:
_track_task(loop.create_task(_persist_jsonl()))
# DB trace 用 cache pool(可能 aiomysql,绑定 owner_loop)。若当前不在
# owner_loop(如 search 走独立 loop),派发回 owner_loop 执行,避免跨 loop。
if trace_enabled:
@@ -446,25 +462,26 @@ def wrap_store_pipeline_log(
)
except RuntimeError:
# 无 running loop(极少见):退化为同步写,保证不丢日志
try:
log_writer.write_step(
request_id=request_id,
user_id=user_id,
agent_id=agent_id,
session_id=session_id,
step=step,
prompt=prompt,
response=response,
parsed=parsed,
memory_ids=mem_ids,
elapsed_ms=step_elapsed_ms,
prompt_tokens=prompt_tokens,
completion_tokens=completion_tokens,
total_tokens=total_tokens,
event_at_ms=event_at_ms,
)
except Exception:
pass
if log_writer is not None:
try:
log_writer.write_step(
request_id=request_id,
user_id=user_id,
agent_id=agent_id,
session_id=session_id,
step=step,
prompt=prompt,
response=response,
parsed=parsed,
memory_ids=mem_ids,
elapsed_ms=step_elapsed_ms,
prompt_tokens=prompt_tokens,
completion_tokens=completion_tokens,
total_tokens=total_tokens,
event_at_ms=event_at_ms,
)
except Exception:
pass
if trace_enabled:
try:
result = original_store(
@@ -496,6 +513,11 @@ def wrap_store_pipeline_log(
return _store_pipeline_observability
async def _noop_store_pipeline_log(*args: Any, **kwargs: Any) -> bool:
"""Log 与 Trace 都关闭时的 store_pipeline_log:不计时、不写文件、不写库。"""
return True
def install_pipeline_observability(
cache: Any,
*,
@@ -503,6 +525,9 @@ def install_pipeline_observability(
) -> None:
"""给 cache 安装 JSONL + Trace 包装(fire-and-forget)。
MEMORY_PIPELINE_LOG_ENABLED=false 时不写 JSONL;若 Trace 也关闭,
store_pipeline_log 被换成空操作(不包装、不写库)。
在 cache 所属 loop(write loop / _LoopThread)上调用,本函数会捕获该 loop
作为 owner_loop 传给包装器:search 走独立 loop 后,从 search loop 触发的 DB
落库会被派发回这个 owner_loop,避免跨 loop 用 aiomysql 污染连接池。
@@ -514,15 +539,31 @@ def install_pipeline_observability(
except RuntimeError:
owner_loop = None
log_dir = resolve_pipeline_log_dir()
log_writer = PipelineLogWriter(log_dir)
log_enabled = is_pipeline_log_enabled()
trace_enabled = is_pipeline_trace_enabled()
logger.info(
f"[{log_label}] Pipeline log (JSONL) enabled: {log_dir}/<subdir>/<date>.log "
f"(subdir from user_id prefix before '__', else default) ; "
f"pipeline trace (DB)={'on' if trace_enabled else 'off'} ; "
f"persist=async(fire-and-forget)"
)
if not log_enabled and not trace_enabled:
logger.info(
f"[{log_label}] Pipeline log (JSONL) disabled ; pipeline trace (DB)=off ; "
f"store_pipeline_log=no-op"
)
cache.store_pipeline_log = _noop_store_pipeline_log
return
log_writer = None
if log_enabled:
log_dir = resolve_pipeline_log_dir()
log_writer = PipelineLogWriter(log_dir)
logger.info(
f"[{log_label}] Pipeline log (JSONL) enabled: {log_dir}/<subdir>/<date>.log "
f"(subdir from user_id prefix before '__', else default) ; "
f"pipeline trace (DB)={'on' if trace_enabled else 'off'} ; "
f"persist=async(fire-and-forget)"
)
else:
logger.info(
f"[{log_label}] Pipeline log (JSONL) disabled ; "
f"pipeline trace (DB)=on ; persist=async(fire-and-forget)"
)
cache.store_pipeline_log = wrap_store_pipeline_log(
cache.store_pipeline_log,
log_writer=log_writer,
+91
View File
@@ -0,0 +1,91 @@
# -*- coding: utf-8 -*-
"""
原始对话(raw)落库前的统一裁剪规则(规则,零 LLM)。
raw 会被长期保存(MemHub 已取消 raw 的保留期限;lite 模式下 raw 是唯一存储),
不裁剪会把工具输出、大段粘贴日志原样存下来。因此 raw 在**写入时**就裁好,
读取方(extractor 的「最近 k 条」历史注入)直接拼接,不再二次裁剪。
规则按说话人标签(ChatMessage.speaker_label() / L1_RAW 里的 ``[label]``):
- ``user``:完整保留;
- ``system``:完整保留(与历史注入原口径一致);
- ``tool``:头尾保留、中段替换为 ``[...N bytes omitted by truncate...]``,
上限为 ``coding.tool_result_max_bytes``(env ``MEMORY_CODING_TOOL_RESULT_MAX_BYTES``,
默认 2048),与 coding 链路同一个函数;
- 其余(``assistant``、对方真人名字等非已知 role 的 label):strip 后超过
``assistant_max_chars``(默认 500,即历史注入原来的 ``max_chars_assistant``)
时截成前 N 个字符 + ``...``。
所有规则确定且幂等:对已裁剪的文本再裁一次结果不变。
"""
from typing import List, Optional
from ..pipelines.base import ChatMessage
# 历史注入原 `_get_recent_history(max_chars_assistant=500)` 的默认值,现为唯一来源。
DEFAULT_RAW_ASSISTANT_MAX_CHARS = 500
# L1_RAW 节点 custom 上的标记:内容已按本模块规则在写入时裁剪。
# 历史拼接据此区分「新 raw 直接拼接」与「本规则之前写入的旧 raw 需兼容裁剪」。
RAW_TRIMMED_KEY = "raw_trimmed"
# 不裁剪的标签(tool 单独处理)
_KEEP_LABELS = frozenset({"user", "system"})
def trim_raw_body(
label: str,
body: str,
*,
assistant_max_chars: int = DEFAULT_RAW_ASSISTANT_MAX_CHARS,
tool_max_bytes: Optional[int] = None,
) -> str:
"""按说话人标签裁剪单条消息正文。幂等。"""
if not isinstance(body, str):
body = "" if body is None else str(body)
if label in _KEEP_LABELS:
return body
if label == "tool":
from ..coding.preproc import (
DEFAULT_MAX_TOOL_RESULT_BYTES,
truncate_tool_result_text,
)
return truncate_tool_result_text(
body, max_bytes=int(tool_max_bytes or DEFAULT_MAX_TOOL_RESULT_BYTES),
)
stripped = body.strip()
if len(stripped) > assistant_max_chars:
return stripped[:assistant_max_chars] + "..."
return body
def trim_raw_messages(
messages: List[ChatMessage],
*,
assistant_max_chars: int = DEFAULT_RAW_ASSISTANT_MAX_CHARS,
tool_max_bytes: Optional[int] = None,
) -> List[ChatMessage]:
"""返回裁剪后的新列表;不修改入参(messages 仍要原样给 extractor)。"""
out: List[ChatMessage] = []
for m in messages:
new_content = trim_raw_body(
m.speaker_label(),
m.content,
assistant_max_chars=assistant_max_chars,
tool_max_bytes=tool_max_bytes,
)
if new_content == m.content:
out.append(m)
continue
out.append(ChatMessage(
role=m.role,
content=new_content,
name=m.name,
tool_calls=list(m.tool_calls),
tool_call_id=m.tool_call_id,
tool_name=m.tool_name,
))
return out
+2 -1
View File
@@ -36,7 +36,8 @@ def count_tokens(text: str) -> int:
enc = _get_encoder()
if enc:
try:
return len(enc.encode(text))
# Quoted control-token spellings are ordinary input text.
return len(enc.encode(text, disallowed_special=()))
except Exception as e:
logger.debug(f"[token_count] encode failed, char fallback: {e}")
return max(1, len(text) // 4)
+2 -2
View File
@@ -13,8 +13,8 @@
接口先补 SDK。Runtime writers 停止后的 App 启动迁移可直接操作底层存储,
但不得把迁移专用方法加入公共 Client。
3. MemHub **不存 memory 实体**;**可以存 `memory_id`** 做组织/回执。原文进入 **SDK raw**。
4. SDK raw 只保留滚动三天;一条 raw 对应一个完整 turn,只含真人 user 与
assistant,tool 不持久化。提取后的高层 memory 不受该窗口影响。
4. SDK raw 全部保留,任何模式(lite / pro / ultra)都不删除;一条 raw 对应一个
完整 turn,只含真人 user 与 assistant,tool 不持久化。
5. `source_raw_ids` 是 SDK 内部溯源关系,不是产品 Evidence 合同;记忆详情不提供
“原始依据”或按 raw ID 展开原文的入口。
+6 -4
View File
@@ -29,10 +29,12 @@ prompt/response、密钥、访问令牌或完整本地路径。结构化日志
文件。详细合同见
[`docs/contracts/diagnostic-logging.md`](docs/contracts/diagnostic-logging.md)。
用户显式把诊断级别切换为 `debug` 时,Core 会启用 HY pipeline trace,并将
`MEMORY_LOG_LLM_CONTENT` 设为 `true`。该模式会把完整模型 prompt/response 写入本地调试
日志,其中可能包含获授权的对话正文和模型输出;它只应用于有意识的故障排查,不应长期
开启,也不应在未经检查和脱敏时分享相关日志。切回 `info` 会停止后续正文记录,但不会
HY pipeline 日志与 trace 在任何级别下都关闭。HY SDK 的文本日志只写入
`hy-memory/governed-v2/logs/hy_memory.log`,其中的 WARNING 及以上同时写入 `core.log`;
SDK 不在 INFO 及以上级别记录对话或记忆正文。用户显式把诊断级别切换为 `debug` 时,Core 将
`MEMORY_LOG_LLM_CONTENT` 设为 `true`。该模式会把完整模型 prompt/response 写入
`hy_memory.log` 的 DEBUG 行,其中可能包含获授权的对话正文和模型输出;它只应用于有意识的
故障排查,不应长期开启,也不应在未经检查和脱敏时分享相关日志。切回 `info` 会停止后续正文记录,但不会
自动删除此前已写入的调试文件。
具体行为以当前源码、网络请求实现和发布说明为准。发现实现与本说明不一致时,请按
+4 -4
View File
@@ -18,13 +18,13 @@ If you run into a problem, please open an Issue with the version, platform,repro
## 发行与平台合同
源码当前固定使用公开 PyPI 正式版 `hy-memory==2.0.7`。MemHub 源码版本为 `1.0.5`
(build 14),GitHub 仓库为
源码当前固定使用公开 PyPI 正式版 `hy-memory==2.0.10`。MemHub 源码版本为 `1.0.5`
(build 36),GitHub 仓库为
[`Tencent-Hunyuan/Hy-Memory`](https://github.com/Tencent-Hunyuan/Hy-Memory),npm 包名为
`memhub-cli`。当前代码架构见
[`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md)。
`v1.0.5` 的发行目标是 Linux x64/arm64 Runtime 与 macOS arm64/x64 App 和 Runtime。
`v1.0.5` 的发行目标是 Linux x64/arm64 Runtime 与 macOS arm64 App 和 Runtime。
Linux 制品由 GitHub Actions 在原生架构 runner 上构建并完成生命周期验收;macOS
制品由发行流水线完成 Developer ID 签名、公证和验收后手动上传。Windows 不支持。
只有同时出现在 GitHub Release 与 `release-manifest.json` 中、且通过公开校验的制品才属于
@@ -118,7 +118,7 @@ MemHub 的 SQLite 数据默认位于 `~/.one-memory`,不在 npm 包或 Runtime
## Hy Memory
SDK 依赖固定为公开 PyPI 正式版 `hy-memory==2.0.7`。SDK 配置、权限边界及 search
SDK 依赖固定为公开 PyPI 正式版 `hy-memory==2.0.10`。SDK 配置、权限边界及 search
返回格式见 [`MEMORY.md`](MEMORY.md)。HY search 返回的 `content` 已包含 SDK 统一生成的
逻辑时间和演化上下文;调用方不应再次转换时间戳或重复拼接 `When`。
+1 -1
View File
@@ -2352,7 +2352,7 @@
{
"SPDXID": "SPDXRef-Package-hy-memory",
"name": "hy-memory",
"versionInfo": "2.0.7",
"versionInfo": "2.0.10",
"downloadLocation": "NOASSERTION",
"filesAnalyzed": false,
"licenseConcluded": "Apache-2.0",
+1 -1
View File
@@ -9,7 +9,7 @@ Per-package license texts are not reproduced here yet; the license
column is filled in as part of release review, and the SBOM records
`NOASSERTION` until then rather than guessing.
## hy-memory 2.0.7
## hy-memory 2.0.10
License: Apache-2.0
@@ -315,6 +315,8 @@ describe("catalog exit codes", () => {
"models",
"memories",
"profile",
"settings",
"evolve",
"graph-width",
"host",
"recall",
+45 -1
View File
@@ -162,10 +162,54 @@ it("never emits a secret from a malformed manually written event", async () => {
command: "logs",
state: "empty",
events: 0,
invalid_lines: 1,
invalid_lines: 0,
unrecognized_lines: 1,
});
});
// After an upgrade the files still hold an older build's records. They are
// whole JSON lines the reader leaves out, not damage, so the text output must
// not warn about them; only a line that is not JSON is called damaged.
it("warns only about lines that are not JSON, not about older records", async () => {
const dataDir = await fixture();
const older = {
schema_version: 0,
service: "desktop",
component: "desktop.lifecycle",
event: "desktop.started",
};
await writeFile(
path.join(dataDir, "logs", "diagnostic-desktop.jsonl"),
`${JSON.stringify(older)}\n${JSON.stringify({ ...older, schema_version: 1 })}\n`,
{ mode: 0o600 },
);
let stdout = "";
let stderr = "";
vi.spyOn(process.stdout, "write").mockImplementation((value) => {
stdout += String(value);
return true;
});
vi.spyOn(process.stderr, "write").mockImplementation((value) => {
stderr += String(value);
return true;
});
expect(await run(["logs"], { ONE_MEMORY_DATA_DIR: dataDir })).toBe(0);
expect(stdout).toContain("core.start_failed");
expect(stdout).not.toContain("损坏");
expect(stderr).toBe("");
await writeFile(
path.join(dataDir, "logs", "diagnostic-cli.jsonl"),
'{"schema_version": 1, "event": \n',
{ mode: 0o600 },
);
stdout = "";
expect(await run(["logs"], { ONE_MEMORY_DATA_DIR: dataDir })).toBe(0);
expect(stdout).toContain("跳过 1 行损坏的诊断记录。");
expect(stderr).toBe("memhub: 跳过 1 行损坏的诊断记录。\n");
});
it("rejects invalid filters without touching the log directory", async () => {
const dataDir = path.join(
await mkdtemp(path.join(os.tmpdir(), "om-cli-no-log-")),
+566 -38
View File
@@ -122,6 +122,14 @@ import {
doctorExitCode,
renderDoctorText,
} from "./doctor-report.js";
import {
activateWithEmbeddingSwitch,
cleanupSpace,
listSpaces,
type SwitchIo,
switchCancel,
switchStatus,
} from "./embedding-switch-flow.js";
import {
EXIT_FAILED,
EXIT_OK,
@@ -188,10 +196,31 @@ import {
removeManagedCliLink,
rollbackManagedCliLink,
} from "./managed-cli-link.js";
import {
type FeaturesDocument,
featuresExitCode,
parseFeaturesCommand,
renderFeaturesText,
runFeaturesCommand,
} from "./memory-features-flow.js";
import {
type EvolveDocument,
evolveExitCode,
type ModeDocument,
modeExitCode,
parseEvolveCommand,
parseSettingsCommand,
renderEvolveText,
renderModeText,
runEvolveCommand,
runModeCommand,
} from "./memory-mode-flow.js";
import {
addMemory,
DEFAULT_USER_ID,
isMemoryId,
isRequestId,
isUserId,
MEMORY_KINDS,
MEMORY_SCENES,
MEMORY_VARIANTS,
@@ -209,6 +238,7 @@ import {
renderMemoryAddText,
renderMemoryDetailText,
renderMemoryPageText,
userIdParameters,
validMemoryText,
} from "./memory-report.js";
import {
@@ -226,7 +256,6 @@ import {
renderModelReportText,
} from "./model-report.js";
import {
activateModelProfile,
editModelProfile,
type ModelWriteDocument,
modelWriteExitCode,
@@ -322,6 +351,8 @@ export interface RunControls {
watchSignal?: AbortSignal;
watchWait?: (milliseconds: number, signal: AbortSignal) => Promise<void>;
watchNow?: () => string;
/** `memhub evolve --wait` polls on this; tests inject an instant one. */
sleep?: (milliseconds: number) => Promise<void>;
readStdin?: () => Promise<Buffer | null>;
stdinIsTTY?: boolean;
/** Injectable onboarding prompter so the wizard can be driven in tests
@@ -538,6 +569,23 @@ async function runCommand(
environment = { ...environment, ONE_MEMORY_DATA_DIR: lifted.dataDir };
}
}
// `--user-id <id>` is global to the memory commands: it names whose
// memories are read or written. It is lifted before dispatch so each
// command's own parser keeps seeing exactly the arguments it always did.
// `sources connect` gives the flag its own meaning and parses it itself.
let userId: string | undefined;
if (USER_ID_COMMANDS.has(arguments_[0] ?? "")) {
const lifted = liftUserIdOption(arguments_);
if (lifted === null) {
return reject(
findCommand(arguments_[0] ?? ""),
asJson,
"--user-id 无效:只能是 1–64 位字母、数字、点、下划线或连字符。",
);
}
arguments_ = lifted.arguments_;
userId = lifted.userId;
}
environment = await environmentWithManagedDataDir(environment);
const command = arguments_[0];
if (command === undefined) return reject(undefined, asJson, "缺少命令。");
@@ -776,9 +824,11 @@ async function runCommand(
const attachment = await attachToRuntimeOrNull(environment);
if (attachment === null) return runtimeUnavailable(descriptor, asJson);
if (parsed.kind === "status") {
const receipt = await readMemoryWrite(attachment, parsed.id).catch(
() => undefined,
);
const receipt = await readMemoryWrite(
attachment,
parsed.id,
userId,
).catch(() => undefined);
if (receipt === undefined) {
return emitFailure(asJson, "write status", "memory_write_read_failed");
}
@@ -812,6 +862,7 @@ async function runCommand(
? {}
: { memory_at: parsed.memory_at }),
...(parsed.priority === undefined ? {} : { priority: parsed.priority }),
...(userId === undefined ? {} : { user_id: userId }),
});
if (outcome.state !== "accepted") {
emit(
@@ -957,9 +1008,12 @@ async function runCommand(
state: "empty",
events: 0,
invalid_lines: report.invalid_lines,
unrecognized_lines: report.unrecognized_lines,
})}\n`
: renderLogText(report.events, report.invalid_lines),
);
// Only lines that are not JSON are damage. Records an older build wrote
// are left out without a warning: after an upgrade they are expected.
if (report.invalid_lines) {
process.stderr.write(
`${PRODUCT}: 跳过 ${report.invalid_lines} 行损坏的诊断记录。\n`,
@@ -1582,6 +1636,7 @@ async function runCommand(
limit: parsed.limit,
character_budget: parsed.characterBudget,
timeout_ms: parsed.timeoutMs,
...userIdParameters(userId),
})
: ({
schema_version: 2,
@@ -1655,6 +1710,18 @@ async function runCommand(
);
if (apiKey === undefined) return usage();
}
if (
parsed.kind === "spaces" &&
parsed.cleanup !== undefined &&
!parsed.confirmed
) {
return emitConfirmPreview(asJson, {
command: "models spaces",
target: parsed.cleanup,
effect: `删除已停用的向量空间 ${parsed.cleanup}(不可恢复;正在使用的空间不能删除)`,
retry: `${PRODUCT} models spaces --cleanup ${parsed.cleanup} --yes`,
});
}
if (parsed.kind === "remove" && !parsed.confirmed) {
return emitConfirmPreview(asJson, {
command: "models remove",
@@ -1677,6 +1744,50 @@ async function runCommand(
return connected.exitCode;
}
const { reader, operations } = connected;
// Ctrl-C while following a switch stops watching, never the switch.
const interrupt = new AbortController();
const switchIo: SwitchIo = {
reader,
operations,
emit: (text) => {
emit(text);
},
asJson,
interactive:
!asJson && (controls.stdinIsTTY ?? process.stdin.isTTY) === true,
confirm: async (question) => {
const prompter =
controls.wizardPrompter ?? (await createTerminalPrompter());
try {
return /^(y|yes)$/i.test((await prompter.ask(question)).trim());
} finally {
if ("close" in prompter && typeof prompter.close === "function") {
prompter.close();
}
}
},
...(controls.watchWait ? { wait: controls.watchWait } : {}),
signal: controls.watchSignal ?? interrupt.signal,
};
const withInterrupt = async (run: () => Promise<number>) => {
const onInterrupt = () => interrupt.abort();
process.once("SIGINT", onInterrupt);
try {
return await run();
} finally {
process.removeListener("SIGINT", onInterrupt);
}
};
if (parsed.kind === "switch") {
return parsed.action === "status"
? await switchStatus(switchIo)
: await switchCancel(switchIo);
}
if (parsed.kind === "spaces") {
return parsed.cleanup === undefined
? await listSpaces(switchIo)
: await cleanupSpace(parsed.cleanup, switchIo);
}
if (parsed.kind === "add") {
const created = await createModelProfile(
config,
@@ -1691,33 +1802,66 @@ async function runCommand(
emitFlowDocument(created, asJson);
return flowExitCode(created);
}
const activated = await activateModelProfile(
String(created.receipt.profile_id),
// Core activates only a probed profile: probe first, then the same
// path as `models activate`, so an embedding change shows its plan.
const profileId = String(created.receipt.profile_id);
const probe = await probeUntilSettled(
reader,
operations,
profileId,
controls.watchWait,
);
const document = {
...created,
activation:
activated.state === "committed"
? { state: "committed" }
: { state: activated.state, code: activated.code },
};
const probed =
probe.state === "completed" &&
Boolean(
(
(probe.receipt.result as Record<string, unknown> | undefined)
?.capabilities as Record<string, unknown> | undefined
)?.embedding_available,
);
emit(
asJson
? `${JSON.stringify(document)}\n`
? `${JSON.stringify({
...created,
probe: {
state: probe.state,
...("operation_id" in probe && probe.operation_id
? { operation_id: probe.operation_id }
: {}),
...("code" in probe ? { code: probe.code } : {}),
},
})}\n`
: `${renderFlowText(created)}${
activated.state === "committed"
? "已设为当前使用的模型配置。\n"
: `设为当前使用未完成:${activated.state}(${activated.code})\n`
probed
? "探测通过。\n"
: `探测未通过(${"code" in probe ? probe.code : probe.state}),没有设为当前使用;用 ${PRODUCT} models test ${profileId} 查看原因。\n`
}`,
);
return activated.state === "committed" ? EXIT_OK : EXIT_FAILED;
if (!probed) return EXIT_FAILED;
return await withInterrupt(() =>
activateWithEmbeddingSwitch(
profileId,
{ confirmed: parsed.confirmed, follow: parsed.follow },
switchIo,
),
);
}
if (parsed.kind === "edit" || parsed.kind === "activate") {
const document =
parsed.kind === "edit"
? await editModelProfile(parsed.profileId, config, apiKey, operations)
: await activateModelProfile(parsed.profileId, operations);
if (parsed.kind === "activate") {
return await withInterrupt(() =>
activateWithEmbeddingSwitch(
parsed.profileId,
{ confirmed: parsed.confirmed, follow: parsed.follow },
switchIo,
),
);
}
if (parsed.kind === "edit") {
const document = await editModelProfile(
parsed.profileId,
config,
apiKey,
operations,
);
emitModelWriteDocument(document, asJson);
return modelWriteExitCode(document);
}
@@ -1780,6 +1924,16 @@ async function runCommand(
}
if (named === "sources" && arguments_[1] !== "list") {
const parsed = parseSourcesCommand(arguments_.slice(1));
if (
!parsed &&
arguments_.some((item) => item === "--user-id" || item === "--root")
) {
return reject(
descriptor,
asJson,
"--user-id 和 --root 只能用于接入单个 hermes 来源(Hermes profile),用户 ID 只能是 1–64 位字母、数字、点、下划线或连字符。",
);
}
if (!parsed) return usage();
if (parsed.kind === "receipt") {
const connected = await connectRuntime(environment);
@@ -1981,12 +2135,17 @@ async function runCommand(
return sourceProgressExitCode(document);
}
if (named === "profile") {
const profileFlags = arguments_.slice(1);
if (
arguments_.filter((item) => item === "--json").length > 1 ||
arguments_.slice(1).some((value) => value !== "--json")
profileFlags.filter((item) => item === "--json").length > 1 ||
profileFlags.filter((item) => item === "--versions").length > 1 ||
profileFlags.some((value) => value !== "--json" && value !== "--versions")
) {
return usage();
}
if (profileFlags.includes("--versions")) {
return await profileVersionsCommand(environment, { asJson, userId });
}
const connected = await connectRuntime(environment, {
readerTimeoutMs: 15_000,
});
@@ -2004,7 +2163,11 @@ async function runCommand(
);
return connected.exitCode;
}
const outcome = await connected.reader<unknown>("/api/v2/user-profile");
const profileUser = userIdParameters(userId);
const outcome =
profileUser.user_id === undefined
? await connected.reader<unknown>("/api/v2/user-profile")
: await connected.reader<unknown>("/api/v2/user-profile", profileUser);
const value =
outcome.state === "ok"
? (outcome.data as {
@@ -2109,7 +2272,7 @@ async function runCommand(
}
const { reader, operations } = connected;
if (parsed.kind === "list") {
const document = await readMemoryPage(reader, parsed.query);
const document = await readMemoryPage(reader, parsed.query, userId);
emit(
asJson
? `${JSON.stringify(document)}\n`
@@ -2122,6 +2285,7 @@ async function runCommand(
reader,
parsed.memoryId,
parsed.namespace,
userId,
);
emit(
asJson
@@ -2133,8 +2297,8 @@ async function runCommand(
if (parsed.kind === "add" || parsed.kind === "add-status") {
const document =
parsed.kind === "add"
? await addMemory(operations, text ?? "", parsed.requestId)
: await readMemoryAddStatus(reader, parsed.requestId);
? await addMemory(operations, text ?? "", parsed.requestId, userId)
: await readMemoryAddStatus(reader, parsed.requestId, userId);
emit(
asJson
? `${JSON.stringify(document)}\n`
@@ -2149,9 +2313,13 @@ async function runCommand(
content: text,
...namespaceParameters(parsed.namespace),
...(parsed.task === undefined ? {} : { task: parsed.task }),
...userIdParameters(userId),
})
: await operations.delete<unknown>(
withQuery(memoryPath, namespaceParameters(parsed.namespace)),
withQuery(memoryPath, {
...namespaceParameters(parsed.namespace),
...userIdParameters(userId),
}),
);
const document = {
schema_version: 2,
@@ -2177,6 +2345,89 @@ async function runCommand(
);
return outcome.state === "committed" ? EXIT_OK : EXIT_FAILED;
}
if (
named === "settings" &&
arguments_.slice(1).find((value) => value !== "--json") === "features"
) {
const parsed = parseFeaturesCommand(
arguments_
.slice(1)
.filter((value, index, all) => index !== all.indexOf("features")),
);
if (!parsed) return usage();
const connected = await connectRuntime(environment);
if (!connected.ok) {
const document: FeaturesDocument = {
schema_version: 1,
product: PRODUCT,
command: "settings features",
state: "unavailable",
code: connected.code,
};
emit(
asJson ? `${JSON.stringify(document)}\n` : renderFeaturesText(document),
);
return connected.exitCode;
}
const document = await runFeaturesCommand(
parsed,
connected.reader,
connected.operations,
);
emit(
asJson ? `${JSON.stringify(document)}\n` : renderFeaturesText(document),
);
return featuresExitCode(document);
}
if (named === "settings") {
const parsed = parseSettingsCommand(arguments_.slice(1));
if (!parsed) return usage();
const connected = await connectRuntime(environment);
if (!connected.ok) {
const document: ModeDocument = {
schema_version: 1,
product: PRODUCT,
command: "settings mode",
state: "unavailable",
code: connected.code,
};
emit(asJson ? `${JSON.stringify(document)}\n` : renderModeText(document));
return connected.exitCode;
}
const document = await runModeCommand(
parsed,
connected.reader,
connected.operations,
);
emit(asJson ? `${JSON.stringify(document)}\n` : renderModeText(document));
return modeExitCode(document);
}
if (named === "evolve") {
const parsed = parseEvolveCommand(arguments_.slice(1));
if (!parsed) return usage();
const connected = await connectRuntime(environment);
if (!connected.ok) {
const document: EvolveDocument = {
schema_version: 1,
product: PRODUCT,
command: "evolve",
state: "unavailable",
code: connected.code,
};
emit(
asJson ? `${JSON.stringify(document)}\n` : renderEvolveText(document),
);
return connected.exitCode;
}
const document = await runEvolveCommand(parsed, {
userId: userIdParameters(userId).user_id,
reader: connected.reader,
operations: connected.operations,
sleep: controls.sleep,
});
emit(asJson ? `${JSON.stringify(document)}\n` : renderEvolveText(document));
return evolveExitCode(document);
}
if (named === "graph-width") {
const parsed = parseGraphWidthCommand(arguments_.slice(1));
if (!parsed) return usage();
@@ -3675,13 +3926,21 @@ async function connectSources(
asJson,
);
}
if (parsed.importHistory) {
// Another user's Hermes profile is granted and imported in one connect
// request: only that request can carry a user and a root.
if (
parsed.importHistory ||
parsed.userId !== undefined ||
parsed.root !== undefined
) {
const document = await connectSourcesWithImport(reader, operations, {
sourceIds: parsed.sourceIds,
confirmOverrides: parsed.confirmOverrides,
...(parsed.requestId === undefined
? {}
: { requestId: parsed.requestId }),
...(parsed.userId === undefined ? {} : { userId: parsed.userId }),
...(parsed.root === undefined ? {} : { root: parsed.root }),
});
emitFlowDocument(document, asJson);
return flowExitCode(document);
@@ -3834,6 +4093,108 @@ async function emitModelReceipt(
/** Attach to a verified Runtime, telling a stopped one (exit 3) from one
* that is up but unusable (exit 1). */
/**
* `memhub profile --versions`: the long-term profile's L0 version chains,
* latest first, so a value that replaced a better one can be found by id.
*/
async function profileVersionsCommand(
environment: NodeJS.ProcessEnv,
options: { asJson: boolean; userId?: string },
): Promise<number> {
const fail = (state: string, code: string, exitCode: number): number => {
emit(
options.asJson
? `${JSON.stringify({ schema_version: 2, product: PRODUCT, state, code })}\n`
: `画像版本读取失败:${code}\n`,
);
return exitCode;
};
const connected = await connectRuntime(environment, {
readerTimeoutMs: 15_000,
});
if (!connected.ok) {
return fail("unavailable", connected.code, connected.exitCode);
}
const outcome = await connected.reader<unknown>(
"/api/v2/user-profile/versions",
userIdParameters(options.userId),
);
const chains = outcome.state === "ok" ? profileChains(outcome.data) : null;
if (outcome.state !== "ok" || chains === null) {
return fail(
outcome.state === "ok" ? "invalid" : outcome.state,
outcome.state === "ok" ? "response_invalid" : outcome.code,
EXIT_FAILED,
);
}
if (options.asJson) {
emit(
`${JSON.stringify({
schema_version: 2,
product: PRODUCT,
state: "ok",
...(outcome.data as object),
})}\n`,
);
return EXIT_OK;
}
if (chains.length === 0) {
emit("尚未形成用户画像版本。\n");
return EXIT_OK;
}
const lines: string[] = [];
for (const chain of chains) {
lines.push(`画像 ${chain.memory_id}(${chain.status})`);
for (const version of chain.versions) {
const when =
typeof version.gmt_created === "number"
? new Date(version.gmt_created * 1000).toISOString()
: "时间未知";
lines.push(
` ${version.current ? "*" : " "} ${version.memory_id} ${when} ${version.content}`,
);
}
}
lines.push(
"* 表示当前生效的版本;删除错误版本:memhub memories delete <memory-id> --yes",
);
emit(`${lines.join("\n")}\n`);
return EXIT_OK;
}
type ProfileChainView = {
memory_id: string;
status: string;
versions: {
memory_id: string;
content: string;
gmt_created?: number | null;
current: boolean;
}[];
};
function profileChains(value: unknown): ProfileChainView[] | null {
if (!value || typeof value !== "object") return null;
const chains = (value as { chains?: unknown }).chains;
if (!Array.isArray(chains)) return null;
const valid = chains.every(
(chain) =>
chain &&
typeof chain === "object" &&
typeof (chain as ProfileChainView).memory_id === "string" &&
typeof (chain as ProfileChainView).status === "string" &&
Array.isArray((chain as ProfileChainView).versions) &&
(chain as ProfileChainView).versions.every(
(version) =>
version &&
typeof version.memory_id === "string" &&
typeof version.content === "string" &&
typeof version.current === "boolean",
),
);
return valid ? (chains as ProfileChainView[]) : null;
}
async function connectRuntime(
environment: NodeJS.ProcessEnv,
options?: { readerTimeoutMs?: number },
@@ -4101,6 +4462,8 @@ const SDK_MIGRATION_FAILURES: Record<string, string> = {
"核对发现新版本还读不到你的全部记忆,没有切换数据源。",
sdk_migration_model_profile_required:
"迁移需要先有可用的模型配置,用来核对新版本能否读回记忆。请先用 memhub models setup 完成配置,再运行 memhub restart 重试。",
sdk_migration_vector_profile_missing:
"记忆的向量来自一个已删除的模型配置,当前配置读不回它们,所以没有迁移。请重新添加并启用那个模型配置,再运行 memhub restart 重试。",
};
const MIGRATION_ISSUES_URL =
@@ -5022,6 +5385,8 @@ type ModelsCommand =
configFile: string;
apiKeyFile?: string;
activate: boolean;
confirmed: boolean;
follow?: boolean;
}
| {
kind: "edit";
@@ -5029,7 +5394,14 @@ type ModelsCommand =
configFile: string;
apiKeyFile?: string;
}
| { kind: "activate"; profileId: string }
| {
kind: "activate";
profileId: string;
confirmed: boolean;
follow?: boolean;
}
| { kind: "switch"; action: "status" | "cancel" }
| { kind: "spaces"; cleanup?: string; confirmed: boolean }
| { kind: "test"; profileId: string }
| { kind: "receipt"; operationId: string }
| { kind: "remove"; profileId: string; confirmed: boolean };
@@ -5070,16 +5442,46 @@ function parseModelsCommand(rest: readonly string[]): ModelsCommand | null {
const flags = readFlags(
tokens.slice(1),
["--config-file", "--api-key-file"],
["--activate"],
["--activate", "--yes", "--wait", "--no-wait"],
);
const configFile = flags?.values.get("--config-file");
if (!flags || configFile === undefined) return null;
const follow = followFlag(flags.switches);
if (follow === null) return null;
const activate = flags.switches.has("--activate");
// The switch flags only mean something when the add also activates.
if (
!activate &&
["--yes", "--wait", "--no-wait"].some((flag) => flags.switches.has(flag))
) {
return null;
}
const apiKeyFile = flags.values.get("--api-key-file");
return {
kind: "add",
configFile,
...(apiKeyFile === undefined ? {} : { apiKeyFile }),
activate: flags.switches.has("--activate"),
activate,
confirmed: flags.switches.has("--yes"),
...(follow === undefined ? {} : { follow }),
};
}
if (action === "switch") {
if (tokens.length !== 2 || (target !== "status" && target !== "cancel")) {
return null;
}
return { kind: "switch", action: target };
}
if (action === "spaces") {
const flags = readFlags(tokens.slice(1), ["--cleanup"], ["--yes"]);
if (!flags) return null;
const cleanup = flags.values.get("--cleanup");
if (cleanup === undefined && flags.switches.has("--yes")) return null;
if (cleanup !== undefined && !OPERATION_ID.test(cleanup)) return null;
return {
kind: "spaces",
...(cleanup === undefined ? {} : { cleanup }),
confirmed: flags.switches.has("--yes"),
};
}
if (target === undefined || !OPERATION_ID.test(target)) return null;
@@ -5100,8 +5502,21 @@ function parseModelsCommand(rest: readonly string[]): ModelsCommand | null {
};
}
if (action === "activate") {
const flags = readFlags(tokens.slice(2), [], ["--confirm", "--yes"]);
return flags ? { kind: "activate", profileId: target } : null;
const flags = readFlags(
tokens.slice(2),
[],
["--confirm", "--yes", "--wait", "--no-wait"],
);
if (!flags) return null;
const follow = followFlag(flags.switches);
if (follow === null) return null;
return {
kind: "activate",
profileId: target,
// `--confirm` stays the hidden alias of `--yes`.
confirmed: flags.switches.has("--yes") || flags.switches.has("--confirm"),
...(follow === undefined ? {} : { follow }),
};
}
if (action === "test") {
return tokens.length === 2 ? { kind: "test", profileId: target } : null;
@@ -5124,6 +5539,35 @@ function parseModelsCommand(rest: readonly string[]): ModelsCommand | null {
return null;
}
/** `--wait` / `--no-wait`: undefined when neither, null when both. */
function followFlag(switches: ReadonlySet<string>): boolean | undefined | null {
const wait = switches.has("--wait");
const noWait = switches.has("--no-wait");
if (wait && noWait) return null;
return wait ? true : noWait ? false : undefined;
}
/** Start a probe and read its receipt until it settles (about two minutes). */
async function probeUntilSettled(
reader: CoreReader,
operations: CoreOperations,
profileId: string,
wait?: (milliseconds: number, signal: AbortSignal) => Promise<void>,
): Promise<ModelProbeFlowDocument> {
let document = await startModelProbe(reader, operations, profileId);
const pause =
wait ??
((milliseconds: number) =>
new Promise<void>((resolve) => setTimeout(resolve, milliseconds)));
const signal = new AbortController().signal;
for (let attempt = 0; attempt < 120; attempt += 1) {
if (document.state !== "pending" || !document.operation_id) break;
await pause(1_000, signal);
document = await readModelProbeReceipt(reader, document.operation_id);
}
return document;
}
type SourcesCommand =
| {
kind: "ids";
@@ -5131,14 +5575,23 @@ type SourcesCommand =
importHistory: boolean;
confirmOverrides: boolean;
requestId?: string;
/** `--user-id`: whose grant (Hermes only). Absent for `local`. */
userId?: string;
/** `--root`: an absolute root other than the candidate (Hermes only). */
root?: string;
}
| { kind: "all"; importHistory: boolean; confirmOverrides: boolean }
| { kind: "receipt"; requestId: string };
const SOURCE_ID_SHAPE = /^[a-z0-9][a-z0-9-]{0,127}$/;
/** Only Hermes keeps one history per profile, so only Hermes may be
* connected for another user or from another root. */
const MULTI_USER_SOURCES = new Set(["hermes"]);
/** `sources connect <id> [<id> ...] | --all-detected [--import]
* [--confirm-overrides] [--request-id <uuid>]` and `sources receipt <id>`.
* [--confirm-overrides] [--request-id <uuid>]
* [--user-id <id>] [--root <path>]` and `sources receipt <id>`.
*
* Sources may be named positionally or with repeated `--source` (the old
* `init --source` spelling is rewritten to this command). */
@@ -5154,6 +5607,8 @@ function parseSourcesCommand(rest: readonly string[]): SourcesCommand | null {
const sourceIds: string[] = [];
const switches = new Set<string>();
let requestId: string | undefined;
let userId: string | undefined;
let root: string | undefined;
for (let index = 1; index < tokens.length; index += 1) {
const token = tokens[index];
if (["--import", "--confirm-overrides", "--all-detected"].includes(token)) {
@@ -5161,6 +5616,21 @@ function parseSourcesCommand(rest: readonly string[]): SourcesCommand | null {
switches.add(token);
continue;
}
if (token === "--user-id" || token === "--root") {
const value = tokens[index + 1];
if (value === undefined || value.startsWith("--")) return null;
if (token === "--user-id") {
if (userId !== undefined || !isUserId(value)) return null;
userId = value;
} else {
if (root !== undefined) return null;
const expanded = expandRootPath(value);
if (expanded === null) return null;
root = expanded;
}
index += 1;
continue;
}
if (token === "--source" || token === "--request-id") {
const value = tokens[index + 1];
if (value === undefined || value.startsWith("--")) return null;
@@ -5184,10 +5654,23 @@ function parseSourcesCommand(rest: readonly string[]): SourcesCommand | null {
const confirmOverrides = switches.has("--confirm-overrides");
if (
requestId !== undefined &&
(!OPERATION_ID.test(requestId) || !importHistory)
(!OPERATION_ID.test(requestId) ||
!(importHistory || userId !== undefined || root !== undefined))
) {
return null;
}
// `--user-id local` is the default and changes nothing.
if (userId === DEFAULT_USER_ID) userId = undefined;
if (userId !== undefined || root !== undefined) {
// One Hermes profile per connect: the user and root name one grant.
if (
switches.has("--all-detected") ||
sourceIds.length !== 1 ||
!MULTI_USER_SOURCES.has(sourceIds[0])
) {
return null;
}
}
if (switches.has("--all-detected")) {
// One names exact sources, the other means "whatever is detected".
if (sourceIds.length > 0 || requestId !== undefined) return null;
@@ -5200,9 +5683,24 @@ function parseSourcesCommand(rest: readonly string[]): SourcesCommand | null {
importHistory,
confirmOverrides,
...(requestId === undefined ? {} : { requestId }),
...(userId === undefined ? {} : { userId }),
...(root === undefined ? {} : { root }),
};
}
/** `--root` as an absolute path: a leading `~` is the home directory, a
* relative path is taken from the working directory. Core expands nothing. */
function expandRootPath(value: string): string | null {
if (value.length === 0 || /[\0\r\n]/.test(value)) return null;
const expanded =
value === "~"
? os.homedir()
: value.startsWith("~/")
? path.join(os.homedir(), value.slice(2))
: value;
return path.resolve(expanded);
}
function parseGraphWidthCommand(
rest: readonly string[],
): { decision: GraphWidthDecision; confirmed: boolean } | null {
@@ -6596,6 +7094,36 @@ const OWN_DATA_DIR_COMMANDS = new Set([
/** Remove one `--data-dir <absolute path>` from argv; null when it is
* repeated, missing its value, or not an absolute path. */
/** Memory commands that honour the global `--user-id`. */
const USER_ID_COMMANDS = new Set([
"memories",
"recall",
"profile",
"write",
"evolve",
]);
/**
* Take `--user-id <id>` out of the arguments. The default user (`local`) is
* returned as undefined, so a default request names no user at all and stays
* byte-identical to one sent before multi-user support.
*/
function liftUserIdOption(
arguments_: readonly string[],
): { arguments_: string[]; userId?: string } | null {
const index = arguments_.indexOf("--user-id");
if (index === -1) return { arguments_: [...arguments_] };
if (arguments_.indexOf("--user-id", index + 1) !== -1) return null;
const value = arguments_[index + 1];
if (value === undefined || value.startsWith("--") || !isUserId(value)) {
return null;
}
return {
arguments_: [...arguments_.slice(0, index), ...arguments_.slice(index + 2)],
...(value === DEFAULT_USER_ID ? {} : { userId: value }),
};
}
function liftDataDirOption(
arguments_: readonly string[],
): { arguments_: string[]; dataDir?: string } | null {
+99 -11
View File
@@ -68,6 +68,12 @@ const JSON_OPTION: CommandOption = {
flag: "--json",
summary: "以 JSON 输出到 stdout,供脚本与 Agent 消费。",
};
/** Global to the memory commands: whose memories are read or written. */
const USER_ID_OPTION: CommandOption = {
flag: "--user-id <id>",
summary:
"读写哪个用户的记忆,默认 local(App 用的就是 local)。1–64 位字母、数字、点、下划线或连字符;其他用户的记忆来自用 sources connect hermes --user-id 接入的 Hermes profile。",
};
const WATCH_OPTION: CommandOption = {
flag: "--watch",
summary: "串行持续刷新;Ctrl-C 只停止观察,不停止 Runtime。",
@@ -252,15 +258,25 @@ export const COMMANDS: readonly CommandDescriptor[] = [
{
name: "sources",
summary:
"列出来源,直接接入来源,或按 request id 查接入回执。connect 不加 --import 只授权(处理保持冻结:不导入历史、不扫描新会话);加 --import 同时导入历史并开启持续同步,返回 request id。目录来自环境变量覆盖的来源必须加 --confirm-overrides。",
"列出来源,直接接入来源,或按 request id 查接入回执。connect 不加 --import 只授权(处理保持冻结:不导入历史、不扫描新会话);加 --import 同时导入历史并开启持续同步,返回 request id。目录来自环境变量覆盖的来源必须加 --confirm-overrides。Hermes 的每个 profile 是一个独立用户:用 --user-id 和 --root 接入,授权并导入历史(会自动排除 ~/.hermes/profiles/**,各用户互不读取)。",
usage:
"memhub sources list [--json]\n memhub sources connect <id> [<id> ...] [--import] [--confirm-overrides] [--request-id <uuid>] [--json]\n memhub sources connect --all-detected [--import] [--confirm-overrides] [--json]\n memhub sources receipt <request-id> [--json]",
"memhub sources list [--json]\n memhub sources connect <id> [<id> ...] [--import] [--confirm-overrides] [--request-id <uuid>] [--json]\n memhub sources connect hermes --user-id <id> --root <path> [--import] [--request-id <uuid>] [--json]\n memhub sources connect --all-detected [--import] [--confirm-overrides] [--json]\n memhub sources receipt <request-id> [--json]",
options: [
{
flag: "--import",
summary:
"接入后立即导入历史会话并开启持续同步;不加时只授权,处理保持冻结。",
},
{
flag: "--user-id <id>",
summary:
"只用于 hermes:这条接入属于哪个用户(默认 local)。和 --root 一起接入一个 Hermes profile,例如 --user-id wife --root ~/.hermes/profiles/wife;其他来源只能是 local。",
},
{
flag: "--root <path>",
summary:
"只用于 hermes:读取的根目录(~ 会展开为主目录)。默认是 Hermes 自己的目录 ~/.hermes。",
},
{
flag: "--all-detected",
summary:
@@ -345,9 +361,9 @@ export const COMMANDS: readonly CommandDescriptor[] = [
{
name: "models",
summary:
"管理生成模型配置。add/test 直接提交并返回操作 ID(异步,用 receipt 查结果);edit/activate 直接执行;remove 不可逆,需 --yes。<profile-id>/<operation-id> 都是 UUID;脚本模式的凭据只从受控文件读取。",
"管理生成模型配置。add/test 直接提交并返回操作 ID(异步,用 receipt 查结果);edit 直接执行;activate 换了向量模型(模型或维度)时先显示切换计划,需 --yes 确认后重新生成全部向量,switch 查看或取消、spaces 查看或清理旧向量数据;remove 不可逆,需 --yes。<profile-id>/<operation-id> 都是 UUID;脚本模式的凭据只从受控文件读取。",
usage:
"memhub models list [--json]\n memhub models show <profile-id> [--json]\n memhub models add --config-file <path> [--api-key-file <path>] [--activate] [--json]\n memhub models test <profile-id> [--json]\n memhub models receipt <operation-id> [--json]\n memhub models edit <profile-id> --config-file <path> [--api-key-file <path>] [--json]\n memhub models activate <profile-id> [--json]\n memhub models remove <profile-id> [--yes] [--json]",
"memhub models list [--json]\n memhub models show <profile-id> [--json]\n memhub models add --config-file <path> [--api-key-file <path>] [--activate [--yes] [--wait|--no-wait]] [--json]\n memhub models test <profile-id> [--json]\n memhub models receipt <operation-id> [--json]\n memhub models edit <profile-id> --config-file <path> [--api-key-file <path>] [--json]\n memhub models activate <profile-id> [--yes] [--wait|--no-wait] [--json]\n memhub models switch <status|cancel> [--json]\n memhub models spaces [--cleanup <space-id> --yes] [--json]\n memhub models remove <profile-id> [--yes] [--json]",
alternative_usage: "memhub models setup",
options: [
{
@@ -361,7 +377,17 @@ export const COMMANDS: readonly CommandDescriptor[] = [
},
{
flag: "--activate",
summary: "add 成功后把新配置设为当前使用。",
summary:
"add 成功后先探测,再按 activate 的规则设为当前使用(向量模型变化时同样要确认切换)。",
},
{
flag: "--wait | --no-wait",
summary:
"activate 开始向量模型切换后是否跟随进度;终端里默认跟随,Ctrl-C 只停止显示。",
},
{
flag: "--cleanup <space-id>",
summary: "spaces:立即删除一个已停用的向量空间(不可恢复,需 --yes)。",
},
DESTRUCTIVE_CONFIRM_OPTION,
JSON_OPTION,
@@ -373,7 +399,7 @@ export const COMMANDS: readonly CommandDescriptor[] = [
summary:
"列出、搜索、查看、添加、修改或物理删除记忆(对话、工作、编码三个场景)。add 和 App 的「添加记忆」是同一个入口;delete 不可逆,需 --yes。",
usage:
"memhub memories list [--scene <all|chat|working|coding>] [--source <id>] [--query <text>] [--status <state>] [--offset <n>] [--limit <n>] [--json]\n memhub memories show <memory-id> [--kind <kind>] [--variant <variant>] [--json]\n memhub memories add <text> [--request-id <uuid>] [--json]\n memhub memories add --content-file <path> [--request-id <uuid>] [--json]\n memhub memories add-status <request-id> [--json]\n memhub memories update <memory-id> --content-file <path> [--kind <kind>] [--variant <variant>] [--task <text>] [--json]\n memhub memories delete <memory-id> [--kind <kind>] [--variant <variant>] [--yes] [--json]",
"memhub memories list [--scene <all|chat|working|coding>] [--source <id>] [--query <text>] [--status <state>] [--offset <n>] [--limit <n>] [--user-id <id>] [--json]\n memhub memories show <memory-id> [--kind <kind>] [--variant <variant>] [--user-id <id>] [--json]\n memhub memories add <text> [--request-id <uuid>] [--user-id <id>] [--json]\n memhub memories add --content-file <path> [--request-id <uuid>] [--user-id <id>] [--json]\n memhub memories add-status <request-id> [--user-id <id>] [--json]\n memhub memories update <memory-id> --content-file <path> [--kind <kind>] [--variant <variant>] [--task <text>] [--user-id <id>] [--json]\n memhub memories delete <memory-id> [--kind <kind>] [--variant <variant>] [--yes] [--user-id <id>] [--json]",
options: [
{
flag: "--scene <all|chat|working|coding>",
@@ -411,15 +437,75 @@ export const COMMANDS: readonly CommandDescriptor[] = [
"add 的幂等键;用同一个 id 重试只会读回第一次的结果,不会重复添加。不给时自动生成并输出。",
},
DESTRUCTIVE_CONFIRM_OPTION,
USER_ID_OPTION,
JSON_OPTION,
],
exit_codes: RUNTIME_EXITS,
},
{
name: "profile",
summary: "读取 SDK 组装后的用户长期画像(L0 + 已启用的长期画像补充)。",
usage: "memhub profile [--json]",
options: [JSON_OPTION],
summary:
"读取 SDK 组装后的用户长期画像(L0 + 已启用的长期画像补充);--versions 列出 L0 画像的版本链(最新在前,* 为当前版本)。",
usage: "memhub profile [--versions] [--user-id <id>] [--json]",
options: [
{
flag: "--versions",
summary:
"列出 L0 基本信息的版本链(get_user_basic_info),用于找出覆盖了正确值的错误版本。",
},
USER_ID_OPTION,
JSON_OPTION,
],
exit_codes: RUNTIME_EXITS,
},
{
name: "settings",
summary:
"查看或切换记忆模式(lite / pro / ultra)。lite 不调用模型,只在原始对话里检索;pro 由模型提取记忆(System 1);ultra 在 pro 之上每晚自动进化(System 2,使用知识图谱)。任何模式都保留全部原始对话。切换不迁移数据:已处理的不会重做,排队中的和新的按新模式处理;需 --yes,不加时只显示计划。环境变量 ONE_MEMORY_GRAPH_STORE 会强制 ultra。settings features 查看或开关 SDK 的可选检索与提取功能(全局设置,默认全部保持 SDK 默认值):graph_rerank 图谱重排(仅 ultra)、same_series_timeline 同系列时间线(需 graph_rerank)、episode_recall 片段召回(pro / ultra,需先开 session_episodes 写入片段)、session_episodes 片段写入、episode_summary 片段摘要(每个片段一次模型调用)、intention 意图提取(默认开)、few_shot 提取示例;set 需 --yes,不加时只显示计划;reset 恢复默认值。",
usage:
"memhub settings mode [lite|pro|ultra] [--yes] [--json]\n" +
" memhub settings features [--json]\n" +
" memhub settings features set <name>=<true|false> [...] [--yes] [--json]\n" +
" memhub settings features reset [<name> ...] [--json]",
options: [
{
flag: "mode [lite|pro|ultra]",
summary: "不带模式只显示当前模式;带模式时先显示切换计划。",
},
{
flag: "features set <name>=<true|false>",
summary:
"开关一个或多个 SDK 功能;功能在当前模式下不生效时会注明原因(例如 graph_rerank 需要 ultra)。",
},
{
flag: "features reset [<name> ...]",
summary: "把指定功能(不写则全部)恢复为 SDK 默认值。",
},
CONFIRM_OPTION,
JSON_OPTION,
],
exit_codes: RUNTIME_EXITS,
},
{
name: "evolve",
summary:
"立即为一个用户运行记忆进化(System 2,仅 ultra 模式)。在后台运行并返回运行 ID;--wait 等待结果。同一用户同时只运行一次;每晚的自动进化由 Runtime 按设置安排。",
usage:
"memhub evolve [--user-id <id>] [--agent-id <id>] [--max-batches <n>] [--wait] [--json]",
options: [
USER_ID_OPTION,
{
flag: "--agent-id <id>",
summary:
"进化哪个 agent 的记忆,默认 all-apps(MemHub 写入记忆用的 agent)。",
},
{
flag: "--max-batches <n>",
summary: "每轮最多处理的批次数(1–1000),默认由 SDK 决定。",
},
{ flag: "--wait", summary: "等待这次进化结束再返回。" },
JSON_OPTION,
],
exit_codes: RUNTIME_EXITS,
},
{
@@ -475,7 +561,7 @@ export const COMMANDS: readonly CommandDescriptor[] = [
summary:
"从原始对话与已沉淀记忆统一检索上下文;本地只读,不调用模型、不触发补索引。文本输出标明每条是「原始对话摘录」还是「记忆」(JSON 里是 kind: raw|memory)。",
usage:
"memhub recall <query> [--kind <raw|memory>] [--source <id>] [--limit <n>] [--character-budget <n>] [--timeout-ms <n>] [--json]\n memhub recall --query-file <path> [...]\n memhub recall --stdin [...]",
"memhub recall <query> [--kind <raw|memory>] [--source <id>] [--limit <n>] [--character-budget <n>] [--timeout-ms <n>] [--user-id <id>] [--json]\n memhub recall --query-file <path> [...]\n memhub recall --stdin [...]",
options: [
{
flag: "--query-file <path>",
@@ -502,6 +588,7 @@ export const COMMANDS: readonly CommandDescriptor[] = [
flag: "--timeout-ms <n>",
summary: "Core 查询时限 50..30000 毫秒,默认 2000。",
},
USER_ID_OPTION,
JSON_OPTION,
],
exit_codes: RUNTIME_EXITS,
@@ -650,7 +737,7 @@ export const COMMANDS: readonly CommandDescriptor[] = [
summary:
"提交一段 OpenAI 格式的对话(messages)到写入队列,异步加工成记忆;或按 request id 查询状态。request id 是幂等键:不给时自动生成并输出,重试请带上同一个。新增一条结论式记忆请用 memories add。",
usage:
"memhub write submit --messages-file <path> --session <id> [--source <id>] [--request-id <id>] [--memory-at <RFC3339>] [--live] [--json]\n memhub write status <request-id> [--json]",
"memhub write submit --messages-file <path> --session <id> [--source <id>] [--request-id <id>] [--memory-at <RFC3339>] [--live] [--user-id <id>] [--json]\n memhub write status <request-id> [--user-id <id>] [--json]",
options: [
{
flag: "--messages-file <path>",
@@ -666,6 +753,7 @@ export const COMMANDS: readonly CommandDescriptor[] = [
},
{ flag: "--memory-at <RFC3339>", summary: "对话发生时间,必须带时区。" },
{ flag: "--live", summary: "优先处理。" },
USER_ID_OPTION,
JSON_OPTION,
],
exit_codes: RUNTIME_EXITS,
@@ -76,6 +76,8 @@ const COMMANDS = [
"models",
"memories",
"profile",
"settings",
"evolve",
"graph-width",
"host",
"recall",
@@ -633,6 +635,8 @@ describe("documented commands", () => {
"receipt",
"edit",
"activate",
"switch",
"spaces",
"remove",
],
// Source processing is one Freeze/Thaw switch under `sync`; the obsolete
+8 -1
View File
@@ -10,6 +10,8 @@ export interface ContextSearchInput {
limit: number;
character_budget: number;
timeout_ms: number;
/** Whose memories to search. Omitted for the default user (`local`). */
user_id?: string;
}
/** The CLI's JSON document mirrors Core's SDK-native context schema v2. */
@@ -95,7 +97,12 @@ export function contextExitCode(document: ContextSearchDocument): number {
}
export function renderContextText(document: ContextSearchDocument): string {
if (document.state === "unavailable") return "暂时无法查记忆,请稍后重试。\n";
if (document.state === "unavailable") {
// No lexical fallback: without the embedding service nothing is searched.
return document.code === "embedding_unavailable"
? "向量服务不可用,无法检索;请检查向量模型配置。\n"
: "暂时无法查记忆,请稍后重试。\n";
}
if (document.state === "denied") return "当前应用没有查记忆权限。\n";
if (document.state === "invalid") return "查记忆返回了无法识别的结果。\n";
if (document.state === "rejected") {
+40 -2
View File
@@ -8,15 +8,26 @@ import axios from "axios";
*/
const READ_ONLY_PATHS = [
"/api/v1/call-events",
// Read-only: the registry plus a scan of the vector store's collection names.
"/api/v1/embedding-spaces",
"/api/v1/embedding-switches/current",
"/api/v1/import-jobs",
"/api/v1/live-sync",
"/api/v1/memory-processing",
"/api/v1/model-profiles",
"/api/v1/source-grants",
"/api/v1/sources",
"/api/v2/home-summary",
"/api/v2/memory-library",
"/api/v2/user-profile",
"/api/v2/user-profile/versions",
"/api/v1/settings/memory-mode",
"/api/v1/settings/memory-features",
"/api/v2/evolve",
] as const;
/** One evolve run's receipt (`memhub evolve --wait`). */
const EVOLVE_RUN =
/^\/api\/v2\/evolve\/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/;
const SDK_MEMORY_DETAIL = /^\/api\/v2\/memories\/[A-Za-z0-9][\w.-]{0,127}$/;
/** Status of one `POST /api/v2/memories` addition, by its request id. */
const MEMORY_ADDITION =
@@ -155,10 +166,23 @@ export function createCoreReader(attachment: {
code: detailCode(response.data, response.status),
};
}
return { state: "unavailable" as const, code: `http_${response.status}` };
return {
state: "unavailable" as const,
code: unavailableCode(response.data, response.status),
};
};
}
/** A 503 that names `embedding_unavailable` keeps it: the search could not
* embed its query, which needs another fix than "try again later". */
function unavailableCode(value: unknown, status: number): string {
return status === 503 && detailCode(value, status) === EMBEDDING_UNAVAILABLE
? EMBEDDING_UNAVAILABLE
: `http_${status}`;
}
export const EMBEDDING_UNAVAILABLE = "embedding_unavailable";
/** Core's stable `detail` code when it sent one, else the bare status. */
function detailCode(value: unknown, status: number): string {
const detail =
@@ -210,7 +234,10 @@ export function createCoreContextQuery(attachment: {
if (response.status === 404) {
return { state: "not_found" as const, code: "http_404" };
}
return { state: "unavailable" as const, code: `http_${response.status}` };
return {
state: "unavailable" as const,
code: unavailableCode(response.data, response.status),
};
};
}
@@ -219,6 +246,7 @@ export function isReadOnlyPath(path: string): boolean {
if (IMPORT_JOB.test(path)) return true;
if (OPERATION_RECEIPT.test(path)) return true;
if (MEMORY_ADDITION.test(path)) return true;
if (EVOLVE_RUN.test(path)) return true;
if (!SDK_MEMORY_DETAIL.test(path)) return false;
const segment = path.slice("/api/v2/memories/".length);
return !(RESERVED_MEMORY_SEGMENTS as readonly string[]).includes(segment);
@@ -310,6 +338,8 @@ export async function submitMemoryWrite(
messages: readonly OpenAIMessage[];
memory_at?: string;
priority?: "live" | "normal";
/** Whose memory this is. Omitted for the default user (`local`). */
user_id?: string;
},
): Promise<MemoryWriteOutcome> {
let response: { status: number; data: unknown };
@@ -327,6 +357,9 @@ export async function submitMemoryWrite(
...(request.priority === undefined
? {}
: { priority: request.priority }),
...(request.user_id === undefined || request.user_id === "local"
? {}
: { user_id: request.user_id }),
},
{
headers: { Authorization: `Bearer ${attachment.apiToken}` },
@@ -469,11 +502,16 @@ export async function deleteModelProfile(
export async function readMemoryWrite(
attachment: RuntimeAttachmentLike,
requestId: string,
/** A receipt belongs to the user that submitted it; omit for `local`. */
userId?: string,
): Promise<MemoryWriteReceipt | null> {
try {
const response = await axios.get<MemoryWriteReceipt>(
`${baseUrlOf(attachment)}/api/v2/memory-writes/${encodeURIComponent(requestId)}`,
{
...(userId === undefined || userId === "local"
? {}
: { params: { user_id: userId } }),
headers: { Authorization: `Bearer ${attachment.apiToken}` },
timeout: 10_000,
proxy: false,
+45 -6
View File
@@ -6,9 +6,10 @@ const WRITE_PATHS = [
] as const;
/** `POST /api/v2/memories` adds one memory (the app's 添加记忆). */
const MEMORY_ADD_PATH = "/api/v2/memories";
/** One SDK memory; delete names its namespace in the query string. */
/** One SDK memory; delete names its namespace, and a non-default user, in
* the query string. */
const SDK_MEMORY_PATH =
/^\/api\/v2\/memories\/[A-Za-z0-9][\w.-]{0,127}(?:\?kind=(?:chat|experience|fact)(?:&variant=(?:working|coding))?)?$/;
/^\/api\/v2\/memories\/[A-Za-z0-9][\w.-]{0,127}(?:\?(?:kind=(?:chat|experience|fact)(?:&variant=(?:working|coding))?(?:&user_id=[A-Za-z0-9._-]{1,64})?|user_id=[A-Za-z0-9._-]{1,64}))?$/;
const MODEL_PROBE_PATH =
/^\/api\/v1\/model-profiles\/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}\/probe$/;
const MODEL_PROFILE_PATH =
@@ -25,12 +26,32 @@ const SOURCE_IMPORT_JOB_PATH = "/api/v1/import-jobs";
const IMPORT_JOB_ACTION_PATH =
/^\/api\/v1\/import-jobs\/[A-Za-z0-9][\w.-]{0,127}\/(?:pause|resume|cancel)$/;
const GRAPH_WIDTH_PATH = "/api/v1/graph-width";
/** `memhub evolve` starts a run; `memhub settings mode <m> --yes` sets the mode. */
const EVOLVE_PATH = "/api/v2/evolve";
const MEMORY_MODE_PATH = "/api/v1/settings/memory-mode";
/** `memhub settings features set|reset` sets the SDK features. */
const MEMORY_FEATURES_PATH = "/api/v1/settings/memory-features";
const ONBOARDING_COMPLETE_PATH = "/api/v1/onboarding/complete";
const EMBEDDING_SWITCH_PATHS = [
"/api/v1/embedding-switches",
"/api/v1/embedding-switches/plan",
] as const;
const EMBEDDING_SWITCH_CANCEL =
/^\/api\/v1\/embedding-switches\/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}\/cancel$/;
const EMBEDDING_SPACE_PATH =
/^\/api\/v1\/embedding-spaces\/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/;
const MAX_BYTES = 4 * 1024 * 1024;
export type OperationOutcome<T> =
| { state: "committed"; data: T }
| { state: "rejected"; code: string; http_status: number }
| {
state: "rejected";
code: string;
http_status: number;
/** Core's refusal body, when it carries more than the code (a switch
* plan rides along with `embedding_switch_required`). */
body?: unknown;
}
| { state: "unknown"; code: string };
export interface CoreOperations {
@@ -96,7 +117,10 @@ export function createCoreOperations(attachment: {
!SOURCE_AUTHORIZE_PATH.test(path) &&
!SOURCE_REVOKE_PATH.test(path) &&
path !== GRAPH_WIDTH_PATH &&
path !== EVOLVE_PATH &&
path !== ONBOARDING_COMPLETE_PATH &&
!(EMBEDDING_SWITCH_PATHS as readonly string[]).includes(path) &&
!EMBEDDING_SWITCH_CANCEL.test(path) &&
!MODEL_PROFILE_ACTIVATE.test(path)
) {
throw new Error("core_operation_path_not_allowed");
@@ -114,11 +138,18 @@ export function createCoreOperations(attachment: {
if (response.status >= 200 && response.status < 300) {
return { state: "committed", data: response.data as T };
}
if (response.status >= 400 && response.status < 500) {
if (
(response.status >= 400 && response.status < 500) ||
// Core answers a refusal it can name with 503 (`sdk_reembed_unavailable`)
// or 507 (`insufficient_disk_space`) on the switch routes.
([503, 507].includes(response.status) &&
(EMBEDDING_SWITCH_PATHS as readonly string[]).includes(path))
) {
return {
state: "rejected",
code: safeErrorCode(response.data, response.status),
http_status: response.status,
body: response.data,
};
}
return {
@@ -131,7 +162,11 @@ export function createCoreOperations(attachment: {
body: Record<string, unknown>,
signal?: AbortSignal,
) {
if (!SDK_MEMORY_PATH.test(path) || path.includes("?")) {
if (
path !== MEMORY_MODE_PATH &&
path !== MEMORY_FEATURES_PATH &&
(!SDK_MEMORY_PATH.test(path) || path.includes("?"))
) {
throw new Error("core_operation_path_not_allowed");
}
let response: { status: number; data: unknown };
@@ -193,7 +228,11 @@ export function createCoreOperations(attachment: {
};
},
async delete<T>(path: string, signal?: AbortSignal) {
if (!SDK_MEMORY_PATH.test(path) && !MODEL_PROFILE_PATH.test(path)) {
if (
!SDK_MEMORY_PATH.test(path) &&
!MODEL_PROFILE_PATH.test(path) &&
!EMBEDDING_SPACE_PATH.test(path)
) {
throw new Error("core_operation_path_not_allowed");
}
let response: { status: number; data: unknown };
@@ -0,0 +1,488 @@
import { beforeEach, describe, expect, it, vi } from "vitest";
import { run } from "./cli.js";
import { renderContextText } from "./context-search.js";
import { describeEmbeddingSwitchSummary } from "./embedding-switch-flow.js";
import { composeStatusReport, renderStatusText } from "./status-report.js";
const transport = vi.hoisted(() => ({
get: vi.fn(),
post: vi.fn(),
put: vi.fn(),
patch: vi.fn(),
delete: vi.fn(),
}));
vi.mock("axios", () => ({
default: { create: vi.fn(() => transport), get: vi.fn() },
}));
const runtime = vi.hoisted(() => ({
inspectRuntimeStatus: vi.fn(),
discoverRuntimeAttachment: vi.fn(),
}));
vi.mock("@one-memory/runtime-control", async (importOriginal) => ({
...(await importOriginal<typeof import("@one-memory/runtime-control")>()),
...runtime,
}));
const PROFILE = "44444444-4444-4444-8444-444444444444";
const JOB = "55555555-5555-4555-8555-555555555555";
const SPACE = "66666666-6666-4666-8666-666666666666";
function plan(overrides: Record<string, unknown> = {}) {
return {
needed: true,
reason: "embedding_identity_changed",
plan_token: "token-1",
source: {
space_id: "77777777-7777-4777-8777-777777777777",
model: "bge-small-zh-v1.5",
dims: 512,
collection_base: "memhub_aaaaaaaaaaaaaaaa",
vectors: 12480,
},
target: {
space_id: null,
model: "text-embedding-v3",
dims: 1024,
collection_base: "",
},
target_profile_id: PROFILE,
target_revision: 5,
target_provider_kind: "openai_compatible",
target_local: false,
sources: [],
total_vectors: 12480,
users: 2,
graph_vectors: null,
estimate_s: 120,
requests: 195,
disk_bytes_needed: 70 * 1024 * 1024,
disk_bytes_free: 10 * 1024 * 1024 * 1024,
retention_days: 14,
blockers: [],
...overrides,
};
}
function job(
state: string,
percent: number,
extra: Record<string, unknown> = {},
) {
return {
job_id: JOB,
kind: "switch",
state,
source: plan().source,
target: { ...plan().target, space_id: SPACE },
stray_space_ids: [],
target_profile_id: PROFILE,
target_revision: 5,
requested_by: "cli",
progress: {
phase: state === "reembedding" ? "vectors" : "prepare",
done: Math.round((12480 * percent) / 100),
total: 12480,
percent,
eta_s: 60,
},
cancel_requested: false,
error_code: null,
attempt: 1,
created_at: "T1",
updated_at: "T1",
finished_at: state === "done" ? "T2" : null,
...extra,
};
}
beforeEach(() => {
vi.clearAllMocks();
for (const method of Object.values(transport)) method.mockReset();
runtime.inspectRuntimeStatus.mockResolvedValue({
state: "running",
code: "runtime_verified",
instanceId: "instance-1",
host: "127.0.0.1",
port: 8765,
version: "1.2.3",
});
runtime.discoverRuntimeAttachment.mockResolvedValue({
instanceId: "instance-1",
apiToken: "synthetic-runtime-token",
host: "127.0.0.1",
port: 8765,
servicePid: 4242,
health: {
status: "ok",
service: "one-memory-core",
version: "1.2.3",
instance_id: "instance-1",
},
});
});
describe("models activate with an embedding change", () => {
it("prints the plan and exits 2 without --yes on a non-terminal", async () => {
transport.post.mockImplementation(async (url: string) =>
url === "/api/v1/embedding-switches/plan"
? { status: 200, data: plan() }
: { status: 404, data: {} },
);
const result = await capturedRun(["models", "activate", PROFILE]);
expect(result.code).toBe(2);
expect(result.stdout).toContain("将更换向量模型");
expect(result.stdout).toContain("bge-small-zh-v1.5(512 维)");
expect(result.stdout).toContain("text-embedding-v3(1024 维)");
expect(result.stdout).toContain("12,480");
expect(result.stdout).toContain("重新运行并加上 --yes 以确认");
expect(transport.post.mock.calls.map(([url]) => url)).toEqual([
"/api/v1/embedding-switches/plan",
]);
});
it("starts the switch with --yes and emits JSON progress lines", async () => {
transport.post.mockImplementation(async (url: string) => {
if (url === "/api/v1/embedding-switches/plan") {
return { status: 200, data: plan() };
}
if (url === "/api/v1/embedding-switches") {
return { status: 202, data: job("preparing", 0) };
}
return { status: 404, data: {} };
});
const answers = [
{
phase: "running",
writes_paused: true,
job: job("reembedding", 50),
last_finished: null,
},
{
phase: "idle",
writes_paused: false,
job: null,
last_finished: job("done", 100),
},
];
transport.get.mockImplementation(async (url: string) =>
url === "/api/v1/embedding-switches/current"
? { status: 200, data: { schema_version: 1, ...answers.shift() } }
: { status: 404, data: {} },
);
const result = await capturedRun(
["models", "activate", PROFILE, "--yes", "--wait", "--json"],
{ watchWait: async () => undefined },
);
expect(result.code).toBe(0);
const lines = result.stdout
.trim()
.split("\n")
.map((line) => JSON.parse(line));
expect(lines.map((line) => line.state)).toEqual([
"preparing",
"reembedding",
"done",
"done",
]);
expect(lines[1]).toMatchObject({ percent: 50, job_id: JOB });
expect(lines.at(-1)).toMatchObject({ result: "switched" });
expect(transport.post).toHaveBeenCalledWith(
"/api/v1/embedding-switches",
{
profile_id: PROFILE,
expected_revision: 5,
plan_token: "token-1",
requested_by: "cli",
},
undefined,
);
});
it("refuses a plan with blockers and starts nothing", async () => {
transport.post.mockResolvedValue({
status: 200,
data: plan({ blockers: ["sdk_reembed_unavailable"] }),
});
const result = await capturedRun([
"models",
"activate",
PROFILE,
"--yes",
"--json",
]);
expect(result.code).toBe(1);
expect(JSON.parse(result.stdout)).toMatchObject({
state: "blocked",
code: "sdk_reembed_unavailable",
});
expect(transport.post).toHaveBeenCalledTimes(1);
});
it("says nothing is re-embedded when only generation changed", async () => {
transport.post.mockImplementation(async (url: string) =>
url === "/api/v1/embedding-switches/plan"
? {
status: 200,
data: plan({ needed: false, reason: "same_embedding" }),
}
: { status: 200, data: profileResponse() },
);
const result = await capturedRun(["models", "activate", PROFILE]);
expect(result.code).toBe(0);
expect(result.stdout).toContain("向量模型未变化,无需重新生成向量。");
});
it("reads the plan carried by a 409 embedding_switch_required", async () => {
transport.post.mockImplementation(async (url: string) =>
url.endsWith("/activate")
? {
status: 409,
data: { detail: "embedding_switch_required", plan: plan() },
}
: { status: 404, data: { detail: "Not Found" } },
);
const result = await capturedRun(["models", "activate", PROFILE, "--json"]);
expect(result.code).toBe(2);
expect(JSON.parse(result.stdout)).toMatchObject({
state: "confirmation_required",
code: "embedding_switch_required",
});
});
});
describe("models switch and spaces", () => {
it("reports no switch, then cancels the running one", async () => {
transport.get.mockResolvedValueOnce({
status: 200,
data: {
schema_version: 1,
phase: "idle",
writes_paused: false,
job: null,
last_finished: null,
},
});
const idle = await capturedRun(["models", "switch", "status"]);
expect(idle.code).toBe(0);
expect(idle.stdout).toContain("没有进行中的向量模型切换");
transport.get.mockResolvedValueOnce({
status: 200,
data: {
schema_version: 1,
phase: "running",
writes_paused: true,
job: job("reembedding", 20),
last_finished: null,
},
});
transport.post.mockResolvedValue({
status: 200,
data: job("reembedding", 20, { cancel_requested: true }),
});
const cancelled = await capturedRun([
"models",
"switch",
"cancel",
"--json",
]);
expect(cancelled.code).toBe(0);
expect(transport.post).toHaveBeenCalledWith(
`/api/v1/embedding-switches/${JOB}/cancel`,
{},
undefined,
);
expect(JSON.parse(cancelled.stdout)).toMatchObject({
cancel_requested: true,
});
});
it("lists spaces and needs --yes before cleaning one up", async () => {
transport.get.mockResolvedValue({
status: 200,
data: {
schema_version: 1,
live_space_id: "x",
spaces: [
{
space_id: SPACE,
state: "retired",
model: "bge-small-zh-v1.5",
dims: 512,
vectors: 10,
keep_until: "2026-11-15T00:00:00+00:00",
},
{
space_id: "88888888-8888-4888-8888-888888888888",
state: "dropped",
model: "m",
dims: 8,
},
],
scanned: [],
scan_error: null,
},
});
const listed = await capturedRun(["models", "spaces"]);
expect(listed.code).toBe(0);
expect(listed.stdout).toContain("已停用");
expect(listed.stdout).toContain("保留到 2026-11-15");
expect(listed.stdout).not.toContain("88888888");
const preview = await capturedRun(["models", "spaces", "--cleanup", SPACE]);
expect(preview.code).toBe(0);
expect(preview.stdout).toContain("--yes");
expect(transport.delete).not.toHaveBeenCalled();
transport.delete.mockResolvedValue({
status: 200,
data: { schema_version: 1 },
});
const dropped = await capturedRun([
"models",
"spaces",
"--cleanup",
SPACE,
"--yes",
]);
expect(dropped.code).toBe(0);
expect(transport.delete).toHaveBeenCalledWith(
`/api/v1/embedding-spaces/${SPACE}`,
undefined,
);
});
});
describe("status and search wording", () => {
it("shows the switch line in memhub status", async () => {
const reader = async (path: string) =>
path === "/api/v1/memory-processing"
? {
state: "ok" as const,
data: {
pending: 3,
fresh_pending: 1,
backlog_pending: 2,
outside_window_pending: 0,
running: 0,
waiting_for_model: 0,
completed: 9,
failed: 0,
embedding_switch: {
job_id: JOB,
kind: "switch",
state: "reembedding",
percent: 50,
eta_s: 60,
},
},
}
: { state: "unavailable" as const, code: "http_503" };
const report = await composeStatusReport({
dataDir: "/synthetic/data",
runtime: {
state: "running",
code: "runtime_verified",
instanceId: "instance-1",
host: "127.0.0.1",
port: 8765,
} as never,
background: null,
reader: reader as never,
});
const text = renderStatusText(report, "MemHub");
expect(text).toContain("向量模型:切换中 50%,记忆整理已暂停");
expect(
describeEmbeddingSwitchSummary({
kind: "merge",
state: "reembedding",
percent: 30,
}),
).toBe("正在合并旧向量数据 30%,记忆整理已暂停");
expect(
describeEmbeddingSwitchSummary({
kind: "merge",
state: "scanning",
percent: 0,
}),
).toBe("正在检查向量数据…");
});
it("names an unavailable embedding service instead of 'try later'", () => {
expect(
renderContextText({
schema_version: 2,
product: "memhub",
state: "unavailable",
code: "embedding_unavailable",
items: [],
}),
).toContain("向量服务不可用");
});
});
function profileResponse() {
return {
profile_id: PROFILE,
name: "Synthetic",
provider_kind: "openai_compatible",
base_url: "https://example.invalid/v1",
model: "synthetic-model-2",
embedding_model: "text-embedding-v3",
embedding_provider_kind: "openai_compatible",
embedding_base_url: "https://example.invalid/v1",
embedding_dimensions: 1024,
has_api_key: true,
credential_state: "available",
timeout_seconds: 30,
max_concurrency: 1,
context_window: null,
disable_thinking: true,
capabilities: {
endpoint_reachable: true,
model_available: true,
structured_output: true,
embedding_available: true,
},
revision: 5,
is_active: true,
created_at: "2026-09-19T00:00:00Z",
updated_at: "2026-09-19T00:00:00Z",
};
}
async function capturedRun(
arguments_: string[],
controls: Parameters<typeof run>[2] = {},
) {
let stdout = "";
let stderr = "";
const stdoutSpy = vi
.spyOn(process.stdout, "write")
.mockImplementation((chunk) => {
stdout += String(chunk);
return true;
});
const stderrSpy = vi
.spyOn(process.stderr, "write")
.mockImplementation((chunk) => {
stderr += String(chunk);
return true;
});
try {
return {
code: await run(
arguments_,
{ ONE_MEMORY_DATA_DIR: "/synthetic/data" },
{ stdinIsTTY: false, ...controls },
),
stdout,
stderr,
};
} finally {
stdoutSpy.mockRestore();
stderrSpy.mockRestore();
}
}
@@ -0,0 +1,847 @@
/**
* Switching the embedding model from the CLI: `models activate` when the
* embedding changes, `models switch status|cancel`, `models spaces`.
*
* Core owns every decision (`/api/v1/embedding-switches*`, contract
* `docs/contracts/embedding-switch-v1.md`). A change of embedding model or
* width is never applied by activation alone: Core plans a switch, the user
* confirms it, Core re-embeds every memory into a new space and only then
* makes it live. The CLI shows the plan, asks, starts the job and follows it.
*/
import type { CoreReader } from "./core-client.js";
import type {
CoreDeleteOperations,
CoreOperations,
OperationOutcome,
} from "./core-operations.js";
import { EXIT_FAILED, EXIT_OK, EXIT_USAGE } from "./exit-codes.js";
import {
activateModelProfile,
type ModelWriteDocument,
modelProfileResult,
renderModelWriteText,
} from "./model-write-flows.js";
import { PRODUCT } from "./version.js";
const UUID = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/;
const CODE = /^[a-z0-9_.:-]{1,128}$/;
const TERMINAL = new Set(["done", "rolled_back"]);
const POLL_MS = 1_000;
export interface SpaceSummary {
space_id: string | null;
model: string | null;
dims: number | null;
collection_base: string;
vectors?: number | null;
}
export interface EmbeddingSwitchPlan {
needed: boolean;
reason: string;
plan_token: string;
source: SpaceSummary;
target: SpaceSummary;
target_profile_id: string;
target_revision: number;
target_provider_kind: string;
target_local: boolean;
total_vectors: number;
users: number | null;
graph_vectors: number | null;
estimate_s: number | null;
requests: number | null;
disk_bytes_needed: number;
disk_bytes_free: number | null;
retention_days: number;
blockers: string[];
}
export interface SwitchProgress {
phase: string;
done: number;
total: number;
percent: number;
eta_s: number | null;
}
export interface EmbeddingSwitchJob {
job_id: string;
kind: "switch" | "merge";
state: string;
source: SpaceSummary;
target: SpaceSummary;
progress: SwitchProgress;
cancel_requested: boolean;
error_code: string | null;
finished_at: string | null;
}
export interface SwitchCurrent {
phase: "idle" | "scanning" | "running";
writes_paused: boolean;
job: EmbeddingSwitchJob | null;
last_finished: EmbeddingSwitchJob | null;
}
export interface SwitchIo {
reader: CoreReader;
operations: CoreOperations & CoreDeleteOperations;
emit: (text: string) => void;
asJson: boolean;
/** A terminal is attached: may prompt, and follows progress by default. */
interactive: boolean;
confirm?: (question: string) => Promise<boolean>;
wait?: (milliseconds: number, signal: AbortSignal) => Promise<void>;
signal?: AbortSignal;
}
export interface ActivateOptions {
confirmed: boolean;
/** `--wait` / `--no-wait`; undefined follows on a terminal only. */
follow?: boolean;
}
// ---------------------------------------------------------------------------
// Parsing: bounded projections of Core's answers, never passthrough.
// ---------------------------------------------------------------------------
function record(value: unknown): Record<string, unknown> | null {
return value && typeof value === "object" && !Array.isArray(value)
? (value as Record<string, unknown>)
: null;
}
function int(value: unknown): number | null {
return typeof value === "number" && Number.isSafeInteger(value) && value >= 0
? value
: null;
}
function str(value: unknown, max = 256): string | null {
return typeof value === "string" && value.length <= max ? value : null;
}
function space(value: unknown): SpaceSummary | null {
const item = record(value);
if (!item) return null;
return {
space_id: str(item.space_id, 64),
model: str(item.model),
dims: int(item.dims),
collection_base: str(item.collection_base) ?? "",
vectors: int(item.vectors),
};
}
export function parsePlan(value: unknown): EmbeddingSwitchPlan | null {
const plan = record(value);
if (!plan || typeof plan.needed !== "boolean") return null;
const source = space(plan.source);
const target = space(plan.target);
const token = str(plan.plan_token, 128);
const profile = str(plan.target_profile_id, 128);
const revision = int(plan.target_revision);
if (!source || !target || !token || !profile || revision === null) {
return null;
}
const blockers = Array.isArray(plan.blockers)
? plan.blockers.filter(
(item): item is string => typeof item === "string" && CODE.test(item),
)
: [];
return {
needed: plan.needed,
reason: str(plan.reason, 64) ?? "",
plan_token: token,
source,
target,
target_profile_id: profile,
target_revision: revision,
target_provider_kind: str(plan.target_provider_kind, 64) ?? "",
target_local: plan.target_local === true,
total_vectors: int(plan.total_vectors) ?? 0,
users: int(plan.users),
graph_vectors: int(plan.graph_vectors),
estimate_s: int(plan.estimate_s),
requests: int(plan.requests),
disk_bytes_needed: int(plan.disk_bytes_needed) ?? 0,
disk_bytes_free: int(plan.disk_bytes_free),
retention_days: int(plan.retention_days) ?? 14,
blockers,
};
}
export function parseJob(value: unknown): EmbeddingSwitchJob | null {
const job = record(value);
if (!job) return null;
const jobId = str(job.job_id, 64);
const state = str(job.state, 32);
const source = space(job.source);
const target = space(job.target);
const progress = record(job.progress) ?? {};
if (!jobId || !state || !source || !target) return null;
if (job.kind !== "switch" && job.kind !== "merge") return null;
const percent =
typeof progress.percent === "number" && Number.isFinite(progress.percent)
? Math.min(100, Math.max(0, progress.percent))
: 0;
return {
job_id: jobId,
kind: job.kind,
state,
source,
target,
progress: {
phase: str(progress.phase, 32) ?? "prepare",
done: int(progress.done) ?? 0,
total: int(progress.total) ?? 0,
percent,
eta_s: int(progress.eta_s),
},
cancel_requested: job.cancel_requested === true,
error_code:
typeof job.error_code === "string" && CODE.test(job.error_code)
? job.error_code
: null,
finished_at: str(job.finished_at, 64),
};
}
export function parseCurrent(value: unknown): SwitchCurrent | null {
const current = record(value);
if (!current) return null;
const phase = current.phase;
if (phase !== "idle" && phase !== "scanning" && phase !== "running") {
return null;
}
return {
phase,
writes_paused: current.writes_paused === true,
job: current.job == null ? null : parseJob(current.job),
last_finished:
current.last_finished == null ? null : parseJob(current.last_finished),
};
}
// ---------------------------------------------------------------------------
// Wording
// ---------------------------------------------------------------------------
const BLOCKER_TEXT: Record<string, string> = {
embedding_switch_running: "上一次向量模型切换还在进行,请等待完成或先取消。",
embedding_probe_required: "新配置还没有通过探测;先运行 memhub models test。",
sdk_reembed_unavailable:
"当前运行时的 SDK 不支持重新生成向量,请先升级 MemHub。",
embedding_switch_requires_migration:
"还有旧版本的向量数据没有迁移完,请先完成升级迁移。",
embedding_dimensions_unknown:
"新配置的向量维度未知;先探测或在配置里写明 embedding_dimensions。",
insufficient_disk_space: "磁盘空间不足,无法容纳新旧两份向量数据。",
};
const FAILURE_TEXT: Record<string, string> = {
writes_did_not_drain: "等待正在进行的记忆整理结束超时",
embedding_service_failed: "向量服务连续请求失败",
embedding_dimensions_mismatch: "向量服务返回的维度与探测到的不一致",
embedding_verification_failed: "新向量没有通过校验",
target_profile_changed: "切换期间目标配置被修改",
embedding_switch_cancelled: "已取消",
model_secret_unavailable: "读不到向量服务的 API Key",
secure_storage_unavailable: "系统钥匙串不可用",
embedding_switch_clients_busy: "记忆图谱一直被占用,无法重新生成",
};
const STATE_TEXT: Record<string, string> = {
preparing: "准备中(等待记忆整理暂停)",
reembedding: "重新生成向量",
verifying: "校验新向量",
committing: "正式切换",
failed: "正在回滚",
done: "已完成",
rolled_back: "已回滚",
};
export function blockerText(code: string): string {
return BLOCKER_TEXT[code] ?? code;
}
function spaceLabel(item: SpaceSummary, local?: boolean): string {
const dims = item.dims === null ? "维度未知" : `${item.dims} 维`;
return `${item.model ?? "未知模型"}(${dims}${local ? ",本地" : ""})`;
}
function duration(seconds: number | null): string {
if (seconds === null) return "未知";
if (seconds < 60) return `${Math.max(1, seconds)} 秒`;
return `${Math.round(seconds / 60)} 分钟`;
}
function megabytes(bytes: number | null): string {
return bytes === null ? "未知" : `${Math.ceil(bytes / (1024 * 1024))} MB`;
}
export function renderPlanText(plan: EmbeddingSwitchPlan): string {
const lines = [
`将更换向量模型:${spaceLabel(plan.source)} → ${spaceLabel(plan.target, plan.target_local)}`,
`需要用新模型重新生成 ${plan.total_vectors.toLocaleString("en-US")} 条向量` +
`${plan.users === null ? "" : `(${plan.users} 个用户)`}` +
`,预计约 ${duration(plan.estimate_s)}` +
`${plan.requests === null ? ",在本机计算,不产生费用" : `,约 ${plan.requests} 次请求`}` +
`,额外磁盘约 ${megabytes(plan.disk_bytes_needed)}。`,
];
if (plan.graph_vectors) {
lines.push(`记忆图谱另有 ${plan.graph_vectors} 条向量一并重新生成。`);
}
if (
plan.source.dims !== null &&
plan.source.dims === plan.target.dims &&
plan.source.model !== plan.target.model
) {
lines.push("新旧模型维度相同,但向量不能通用,仍需重新生成。");
}
lines.push(
"切换期间记忆整理暂停,检索继续使用旧模型;完成并校验通过后才会生效。",
`旧的向量数据保留 ${plan.retention_days} 天,可用 memhub models spaces 清理。`,
);
for (const blocker of plan.blockers) {
lines.push(`无法开始:${blockerText(blocker)}`);
}
return `${lines.join("\n")}\n`;
}
export function renderProgressLine(job: EmbeddingSwitchJob): string {
const what = job.kind === "merge" ? "正在合并旧向量数据" : "正在切换向量模型";
const state = STATE_TEXT[job.state] ?? job.state;
if (job.state === "reembedding" && job.progress.total > 0) {
const eta =
job.progress.eta_s === null
? ""
: ` 预计还需 ${duration(job.progress.eta_s)}`;
return `${what}:${state} ${job.progress.done.toLocaleString("en-US")}/${job.progress.total.toLocaleString("en-US")} ${Math.floor(job.progress.percent)}%${eta}`;
}
return `${what}:${state}`;
}
export function renderOutcomeText(job: EmbeddingSwitchJob): string {
if (job.state === "done") {
return job.kind === "merge"
? "旧向量数据已合并,记忆整理已恢复。\n"
: `已切换到 ${job.target.model ?? "新模型"}。\n`;
}
const code = job.error_code ?? "embedding_switch_failed";
if (code === "embedding_switch_cancelled") {
return `已取消切换,继续使用 ${job.source.model ?? "旧模型"}。\n`;
}
return `切换未完成,已恢复使用 ${job.source.model ?? "旧模型"},数据未受影响。原因:${FAILURE_TEXT[code] ?? code}(${code})\n`;
}
/** One line for `memhub status`, from `memory-processing.embedding_switch`. */
export function describeEmbeddingSwitchSummary(value: unknown): string | null {
const summary = record(value);
if (!summary) return null;
const percent =
typeof summary.percent === "number" && Number.isFinite(summary.percent)
? Math.floor(summary.percent)
: 0;
if (summary.state === "scanning") return "正在检查向量数据…";
if (summary.kind === "merge") {
return `正在合并旧向量数据 ${percent}%,记忆整理已暂停`;
}
if (summary.state === "failed") return "切换未完成,正在恢复旧向量模型";
return `切换中 ${percent}%,记忆整理已暂停`;
}
// ---------------------------------------------------------------------------
// Flows
// ---------------------------------------------------------------------------
function document(
command: string,
fields: Record<string, unknown>,
): Record<string, unknown> {
return { schema_version: 1, product: PRODUCT, command, ...fields };
}
function emitDocument(io: SwitchIo, value: Record<string, unknown>): void {
io.emit(`${JSON.stringify(value)}\n`);
}
async function requestPlan(
profileId: string,
io: SwitchIo,
): Promise<
| { state: "plan"; plan: EmbeddingSwitchPlan }
| { state: "unsupported" }
| { state: "failed"; code: string }
> {
const outcome = await io.operations.post<unknown>(
"/api/v1/embedding-switches/plan",
{ profile_id: profileId },
);
if (outcome.state === "committed") {
const plan = parsePlan(outcome.data);
// Not a plan: activation itself still refuses an embedding change.
return plan ? { state: "plan", plan } : { state: "unsupported" };
}
if (outcome.state === "rejected") {
// A Core before the registry has no plan route: activation as before.
if (outcome.http_status === 404 && outcome.code === "http_404") {
return { state: "unsupported" };
}
const plan = parsePlan(record(outcome.body)?.plan);
if (plan) return { state: "plan", plan };
return { state: "failed", code: outcome.code };
}
return { state: "failed", code: outcome.code };
}
function emitActivation(io: SwitchIo, result: ModelWriteDocument): number {
io.emit(
io.asJson ? `${JSON.stringify(result)}\n` : renderModelWriteText(result),
);
return result.state === "committed" ? EXIT_OK : EXIT_FAILED;
}
/**
* `memhub models activate <id> [--yes] [--wait|--no-wait] [--json]`.
*
* Asks Core for the plan first. No embedding change: activate as before.
* An embedding change: show the plan, confirm (`--yes`, or a prompt on a
* terminal; exit 2 otherwise), start the switch and follow it.
*/
export async function activateWithEmbeddingSwitch(
profileId: string,
options: ActivateOptions,
io: SwitchIo,
): Promise<number> {
if (!UUID.test(profileId)) {
return emitActivation(
io,
await activateModelProfile(profileId, io.operations),
);
}
const planned = await requestPlan(profileId, io);
if (planned.state === "unsupported") {
const activated = await activateOrPlan(profileId, io);
return activated.state === "plan"
? confirmAndSwitch(activated.plan, options, io)
: emitActivation(io, activated.result);
}
if (planned.state === "failed") {
return reject(io, "models activate", planned.code);
}
const plan = planned.plan;
if (!plan.needed) {
if (!io.asJson && plan.reason === "same_embedding") {
io.emit("向量模型未变化,无需重新生成向量。\n");
}
const activated = await activateOrPlan(profileId, io);
if (activated.state === "plan") {
return confirmAndSwitch(activated.plan, options, io);
}
return emitActivation(io, activated.result);
}
return confirmAndSwitch(plan, options, io);
}
async function activateOrPlan(
profileId: string,
io: SwitchIo,
): Promise<
| { state: "plan"; plan: EmbeddingSwitchPlan }
| { state: "activated"; result: ModelWriteDocument }
> {
const result: OperationOutcome<unknown> = await io.operations.post<unknown>(
`/api/v1/model-profiles/${encodeURIComponent(profileId)}/activate`,
{},
);
if (
result.state === "rejected" &&
result.code === "embedding_switch_required"
) {
const plan = parsePlan(record(result.body)?.plan);
if (plan) return { state: "plan", plan };
}
return { state: "activated", result: modelProfileResult(profileId, result) };
}
function reject(io: SwitchIo, command: string, code: string): number {
if (io.asJson) {
emitDocument(io, document(command, { state: "rejected", code }));
} else {
io.emit(`${command} 未完成:${blockerText(code)}(${code})\n`);
}
return EXIT_FAILED;
}
async function confirmAndSwitch(
plan: EmbeddingSwitchPlan,
options: ActivateOptions,
io: SwitchIo,
): Promise<number> {
if (plan.blockers.length > 0) {
if (io.asJson) {
emitDocument(
io,
document("models activate", {
state: "blocked",
code: plan.blockers[0],
plan,
}),
);
} else {
io.emit(renderPlanText(plan));
}
return EXIT_FAILED;
}
if (!options.confirmed) {
const ask = io.interactive && !io.asJson ? io.confirm : undefined;
if (ask === undefined) {
if (io.asJson) {
emitDocument(
io,
document("models activate", {
state: "confirmation_required",
code: "embedding_switch_required",
plan,
next_step: `${PRODUCT} models activate ${plan.target_profile_id} --yes`,
}),
);
} else {
io.emit(`${renderPlanText(plan)}重新运行并加上 --yes 以确认\n`);
}
return EXIT_USAGE;
}
io.emit(renderPlanText(plan));
if (!(await ask("继续?[y/N] "))) {
io.emit("未开始切换,仍在使用原来的向量模型。\n");
return EXIT_OK;
}
}
const started = await io.operations.post<unknown>(
"/api/v1/embedding-switches",
{
profile_id: plan.target_profile_id,
expected_revision: plan.target_revision,
plan_token: plan.plan_token,
requested_by: "cli",
},
);
if (started.state !== "committed") {
return reject(io, "models activate", started.code);
}
const job = parseJob(started.data);
if (!job) return reject(io, "models activate", "embedding_switch_invalid");
const follow = options.follow ?? io.interactive;
if (!follow) {
if (io.asJson) {
emitDocument(io, document("models activate", progressFields(job)));
} else {
io.emit(
`已开始切换向量模型(${job.job_id})。用 memhub models switch status 查看进度。\n`,
);
}
return EXIT_OK;
}
return followJob(job, io, "models activate");
}
function progressFields(job: EmbeddingSwitchJob): Record<string, unknown> {
return {
state: job.state,
job_id: job.job_id,
kind: job.kind,
phase: job.progress.phase,
percent: job.progress.percent,
done: job.progress.done,
total: job.progress.total,
eta_s: job.progress.eta_s,
...(job.error_code ? { code: job.error_code } : {}),
};
}
async function readCurrent(io: SwitchIo): Promise<SwitchCurrent | null> {
const outcome = await io.reader<unknown>(
"/api/v1/embedding-switches/current",
);
return outcome.state === "ok" ? parseCurrent(outcome.data) : null;
}
/** Poll `current` until the job ends or the user stops watching. */
async function followJob(
started: EmbeddingSwitchJob,
io: SwitchIo,
command: string,
): Promise<number> {
const wait = io.wait ?? defaultWait;
const signal = io.signal ?? new AbortController().signal;
let last = "";
let job = started;
const show = (value: EmbeddingSwitchJob) => {
const line = io.asJson
? JSON.stringify(document(command, progressFields(value)))
: renderProgressLine(value);
if (line !== last) {
io.emit(`${line}\n`);
last = line;
}
};
show(job);
if (!io.asJson) {
io.emit(
"(Ctrl-C 只停止显示;用 `memhub models switch cancel` 取消切换)\n",
);
}
while (!TERMINAL.has(job.state)) {
try {
await wait(POLL_MS, signal);
} catch {
// Aborted below.
}
if (signal.aborted) {
if (!io.asJson) {
io.emit(
"已停止显示进度;切换仍在后台进行。用 `memhub models switch cancel` 取消。\n",
);
}
return EXIT_OK;
}
const current = await readCurrent(io);
if (current === null) continue;
const next =
current.job?.job_id === job.job_id
? current.job
: current.last_finished?.job_id === job.job_id
? current.last_finished
: null;
if (next === null) continue;
job = next;
show(job);
}
if (io.asJson) {
emitDocument(
io,
document(command, {
...progressFields(job),
result: job.state === "done" ? "switched" : "rolled_back",
}),
);
} else {
io.emit(renderOutcomeText(job));
}
return job.state === "done" ? EXIT_OK : EXIT_FAILED;
}
function defaultWait(milliseconds: number, signal: AbortSignal): Promise<void> {
return new Promise((resolve, reject) => {
if (signal.aborted) {
reject(new Error("aborted"));
return;
}
const timer = setTimeout(() => {
signal.removeEventListener("abort", onAbort);
resolve();
}, milliseconds);
const onAbort = () => {
clearTimeout(timer);
reject(new Error("aborted"));
};
signal.addEventListener("abort", onAbort, { once: true });
});
}
/** `memhub models switch status [--json]`. */
export async function switchStatus(io: SwitchIo): Promise<number> {
const outcome = await io.reader<unknown>(
"/api/v1/embedding-switches/current",
);
if (outcome.state !== "ok")
return reject(io, "models switch status", outcome.code);
const current = parseCurrent(outcome.data);
if (!current) {
return reject(io, "models switch status", "embedding_switch_invalid");
}
if (io.asJson) {
emitDocument(
io,
document("models switch status", {
state: current.phase,
writes_paused: current.writes_paused,
job: current.job ? progressFields(current.job) : null,
last_finished: current.last_finished
? progressFields(current.last_finished)
: null,
}),
);
return EXIT_OK;
}
if (current.job) {
io.emit(
`${renderProgressLine(current.job)}\n记忆整理已暂停,检索不受影响。\n`,
);
} else if (current.phase === "scanning") {
io.emit("正在检查向量数据…\n");
} else if (current.last_finished) {
io.emit(
`没有进行中的向量模型切换。上一次:${renderOutcomeText(current.last_finished)}`,
);
} else {
io.emit("没有进行中的向量模型切换。\n");
}
return EXIT_OK;
}
/** `memhub models switch cancel [--json]`: cancel the running job, if any. */
export async function switchCancel(io: SwitchIo): Promise<number> {
const current = await readCurrent(io);
if (current === null) {
return reject(io, "models switch cancel", "embedding_switch_unavailable");
}
if (!current.job) {
return reject(io, "models switch cancel", "embedding_switch_not_found");
}
const outcome = await io.operations.post<unknown>(
`/api/v1/embedding-switches/${current.job.job_id}/cancel`,
{},
);
if (outcome.state !== "committed") {
return reject(io, "models switch cancel", outcome.code);
}
const job = parseJob(outcome.data) ?? current.job;
if (io.asJson) {
emitDocument(
io,
document("models switch cancel", {
...progressFields(job),
cancel_requested: true,
}),
);
} else {
io.emit("已请求取消;当前批次完成后回滚,继续使用原来的向量模型。\n");
}
return EXIT_OK;
}
interface SpaceRow {
space_id: string;
state: string;
model: string | null;
dims: number | null;
vectors: number | null;
keep_until: string | null;
}
function spaceRows(value: unknown): SpaceRow[] | null {
const body = record(value);
if (!body || !Array.isArray(body.spaces)) return null;
const rows: SpaceRow[] = [];
for (const item of body.spaces) {
const row = record(item);
const spaceId = str(row?.space_id, 64);
const state = str(row?.state, 16);
if (!row || !spaceId || !state) return null;
if (state === "dropped") continue;
rows.push({
space_id: spaceId,
state,
model: str(row.model),
dims: int(row.dims) ?? int(row.name_dims),
vectors: int(row.vectors),
keep_until: str(row.keep_until, 64),
});
}
return rows;
}
const SPACE_STATE: Record<string, string> = {
live: "使用中",
building: "正在生成",
stray: "待合并",
retired: "已停用",
};
/** `memhub models spaces [--json]`. */
export async function listSpaces(io: SwitchIo): Promise<number> {
const outcome = await io.reader<unknown>("/api/v1/embedding-spaces");
if (outcome.state !== "ok") return reject(io, "models spaces", outcome.code);
const rows = spaceRows(outcome.data);
if (!rows) return reject(io, "models spaces", "embedding_spaces_invalid");
const body = record(outcome.data) ?? {};
const unregistered = Array.isArray(body.scanned)
? body.scanned
.map(record)
.filter(
(item): item is Record<string, unknown> =>
item !== null && item.state === "unregistered",
)
.map((item) => ({
collection_base: str(item.collection_base) ?? "",
vectors: int(item.vectors) ?? 0,
}))
: [];
if (io.asJson) {
emitDocument(
io,
document("models spaces", {
state: "ok",
spaces: rows,
unregistered,
scan_error:
typeof body.scan_error === "string" && CODE.test(body.scan_error)
? body.scan_error
: null,
}),
);
return EXIT_OK;
}
const lines = ["向量空间:"];
for (const row of rows) {
const dims = row.dims === null ? "维度未知" : `${row.dims} 维`;
const keep =
row.state === "retired" && row.keep_until
? `,保留到 ${row.keep_until.slice(0, 10)}`
: "";
lines.push(
` ${row.space_id} ${SPACE_STATE[row.state] ?? row.state} ${row.model ?? "未知模型"}(${dims}) ${row.vectors ?? "?"} 条${keep}`,
);
}
for (const item of unregistered) {
lines.push(
` (未登记)${item.collection_base} ${item.vectors} 条,下次启动时合并`,
);
}
if (rows.some((row) => row.state === "retired")) {
lines.push(
"清理已停用的空间:memhub models spaces --cleanup <space-id> --yes",
);
}
io.emit(`${lines.join("\n")}\n`);
return EXIT_OK;
}
/** `memhub models spaces --cleanup <space-id> --yes`: drop a retired space now. */
export async function cleanupSpace(
spaceId: string,
io: SwitchIo,
): Promise<number> {
if (!UUID.test(spaceId)) {
return reject(io, "models spaces", "embedding_space_not_found");
}
const outcome = await io.operations.delete<unknown>(
`/api/v1/embedding-spaces/${spaceId}`,
);
if (outcome.state !== "committed") {
return reject(io, "models spaces", outcome.code);
}
if (io.asJson) {
emitDocument(
io,
document("models spaces", { state: "dropped", space_id: spaceId }),
);
} else {
io.emit(`已清理旧向量数据:${spaceId}\n`);
}
return EXIT_OK;
}
+149 -17
View File
@@ -510,15 +510,98 @@ describe("models add", () => {
expect(result.stdout + result.stderr).not.toContain("SYNTHETIC_LOOSE_KEY");
});
it("activates the new profile with --activate", async () => {
it("probes the new profile before activating it with --activate", async () => {
const configFile = await modelConfigFile();
transport.post.mockImplementation(async (url: string) =>
url === "/api/v1/model-profiles"
? { status: 200, data: PROFILE }
: url === "/api/v1/model-profiles/profile-1/activate"
? { status: 200, data: { ...PROFILE, is_active: true } }
: { status: 404, data: {} },
let probeId: string | undefined;
transport.get.mockImplementation(async (url: string) => {
if (url === "/api/v1/model-profiles") {
return { status: 200, data: [PROBE_PROFILE] };
}
const match = /^\/api\/v1\/model-probe-operations\/(.+)$/.exec(url);
if (match) {
return {
status: 200,
data:
probeId === undefined
? probeNotFound(match[1])
: probeReceiptPassing(match[1]),
};
}
return { status: 404, data: {} };
});
transport.post.mockImplementation(async (url: string, body: unknown) => {
if (url === "/api/v1/model-profiles") {
return { status: 200, data: PROBE_PROFILE };
}
if (url === `/api/v1/model-profiles/${PROBE_PROFILE.profile_id}/probe`) {
probeId = (body as { operation_id: string }).operation_id;
return { status: 200, data: probePending(probeId) };
}
if (url === "/api/v1/embedding-switches/plan") {
return { status: 200, data: samePlan() };
}
if (
url === `/api/v1/model-profiles/${PROBE_PROFILE.profile_id}/activate`
) {
return {
status: 200,
data: {
...PROBE_PROFILE,
is_active: true,
embedding_model: "synthetic-embedding",
},
};
}
return { status: 404, data: {} };
});
const result = await capturedRun(
["models", "add", "--config-file", configFile, "--activate", "--json"],
{ watchWait: async () => undefined },
);
expect(result.code).toBe(0);
const lines = result.stdout
.trim()
.split("\n")
.map((line) => JSON.parse(line));
expect(lines[0].probe).toMatchObject({ state: "completed" });
expect(lines.at(-1)).toMatchObject({ state: "committed" });
expect(transport.post.mock.calls.map(([url]) => url)).toEqual([
"/api/v1/model-profiles",
`/api/v1/model-profiles/${PROBE_PROFILE.profile_id}/probe`,
"/api/v1/embedding-switches/plan",
`/api/v1/model-profiles/${PROBE_PROFILE.profile_id}/activate`,
]);
});
it("does not activate a profile whose probe fails", async () => {
const configFile = await modelConfigFile();
let probeId: string | undefined;
transport.get.mockImplementation(async (url: string) => {
if (url === "/api/v1/model-profiles") {
return { status: 200, data: [PROBE_PROFILE] };
}
const match = /^\/api\/v1\/model-probe-operations\/(.+)$/.exec(url);
if (match) {
return {
status: 200,
data:
probeId === undefined
? probeNotFound(match[1])
: probeReceipt(match[1]),
};
}
return { status: 404, data: {} };
});
transport.post.mockImplementation(async (url: string, body: unknown) => {
if (url === "/api/v1/model-profiles") {
return { status: 200, data: PROBE_PROFILE };
}
if (url.endsWith("/probe")) {
probeId = (body as { operation_id: string }).operation_id;
return { status: 200, data: probeReceipt(probeId) };
}
return { status: 404, data: {} };
});
const result = await capturedRun([
"models",
"add",
@@ -527,14 +610,12 @@ describe("models add", () => {
"--activate",
"--json",
]);
expect(result.code).toBe(0);
expect(JSON.parse(result.stdout).activation).toEqual({
state: "committed",
});
expect(transport.post.mock.calls.map(([url]) => url)).toEqual([
"/api/v1/model-profiles",
"/api/v1/model-profiles/profile-1/activate",
]);
expect(result.code).toBe(1);
expect(
transport.post.mock.calls.some(([url]) =>
String(url).endsWith("/activate"),
),
).toBe(false);
});
it("recovers an unknown create through its operation receipt", async () => {
@@ -752,6 +833,54 @@ function probePending(operationId: string) {
};
}
function samePlan() {
const space = {
space_id: null,
model: "synthetic-embedding",
dims: 8,
collection_base: "",
};
return {
needed: false,
reason: "same_embedding",
plan_token: "token-1",
source: space,
target: space,
target_profile_id: PROBE_PROFILE.profile_id,
target_revision: 3,
target_provider_kind: "openai_compatible",
target_local: false,
blockers: [],
};
}
function probeReceiptPassing(operationId: string) {
const receipt = probeReceipt(operationId);
return {
...receipt,
result: {
...receipt.result,
stages: [
{
stage: "embedding",
ok: true,
code: "ok",
latency_ms: 1,
embedding_dimensions: 8,
},
],
capabilities: {
endpoint_reachable: true,
model_available: true,
structured_output: true,
embedding_available: true,
},
embedding_width: 8,
embedding_switch_required: false,
},
};
}
function probeNotFound(operationId: string) {
return {
schema_version: 1,
@@ -819,7 +948,10 @@ function importJob() {
};
}
async function capturedRun(argv: string[]) {
async function capturedRun(
argv: string[],
controls: Parameters<typeof run>[2] = {},
) {
let stdout = "";
let stderr = "";
const originalOut = process.stdout.write;
@@ -833,7 +965,7 @@ async function capturedRun(argv: string[]) {
return true;
}) as typeof process.stderr.write;
try {
const code = await run(argv, {});
const code = await run(argv, {}, controls);
return { code, stdout, stderr };
} finally {
process.stdout.write = originalOut;
+29 -7
View File
@@ -110,9 +110,18 @@ export async function connectSourcesWithImport(
sourceIds: readonly string[];
confirmOverrides: boolean;
requestId?: string;
/** Whose grant this is. Omitted for the default user (`local`). */
userId?: string;
/** An absolute root other than the source's candidate (a Hermes
* profile). Core reads its identity itself, so the source's candidate
* root id and revision are not sent for it. */
root?: string;
},
): Promise<FlowDocument> {
const sourceIds = input.sourceIds;
// A second user's grant is not a first run, and a custom root has no
// candidate the app showed: both are a plain connect.
const profile = input.root !== undefined || input.userId !== undefined;
if (
sourceIds.length < 1 ||
sourceIds.length > 32 ||
@@ -134,9 +143,14 @@ export async function connectSourcesWithImport(
return invalid("source_unavailable");
}
const items = selected.map(([candidate]) => candidate);
const needsOverride = items
.filter((item) => item.requires_override_confirmation)
.map((item) => item.source_id);
// A custom root is the caller's own choice, not the environment override
// the confirmation exists for.
const needsOverride =
input.root !== undefined
? []
: items
.filter((item) => item.requires_override_confirmation)
.map((item) => item.source_id);
if (needsOverride.length > 0 && !input.confirmOverrides) {
return {
schema_version: 1,
@@ -154,18 +168,22 @@ export async function connectSourcesWithImport(
request = buildSourceImportRequest({
sourceIds: [...sourceIds],
sourceRoots: Object.fromEntries(
items.map((item) => [item.source_id, item.candidate_root_id]),
items.flatMap((item) =>
input.root === undefined
? [[item.source_id, item.candidate_root_id]]
: [],
),
),
sourceCandidateRevisions: Object.fromEntries(
items.flatMap((item) =>
item.candidate_revision === undefined
item.candidate_revision === undefined || input.root !== undefined
? []
: [[item.source_id, item.candidate_revision]],
),
),
confirmEnvironmentOverrides: needsOverride.length > 0,
fullRescan: false,
onboarding: true,
onboarding: !profile,
enableSync: true,
requestId,
});
@@ -175,7 +193,11 @@ export async function connectSourcesWithImport(
}
throw error;
}
const result = await operations.post<unknown>(request.path, request.body);
const result = await operations.post<unknown>(request.path, {
...request.body,
...(input.userId === undefined ? {} : { user_id: input.userId }),
...(input.root === undefined ? {} : { root: input.root }),
});
if (result.state === "rejected") {
return rejected(requestId, result.code);
}
+6
View File
@@ -107,7 +107,11 @@ export async function readLogReport(
query: LogQuery,
): Promise<{
events: DiagnosticEventV1[];
/** Lines that are not JSON: the only ones `memhub logs` calls damaged. */
invalid_lines: number;
/** Whole JSON lines the reader leaves out, such as an older build's
* records. Nothing is wrong with them, so the text output says nothing. */
unrecognized_lines: number;
failure?: "log_directory_unreadable";
}> {
// `readDiagnosticEvents` treats an unreadable logs path as "no events". That
@@ -119,6 +123,7 @@ export async function readLogReport(
return {
events: [],
invalid_lines: 0,
unrecognized_lines: 0,
failure: "log_directory_unreadable",
};
}
@@ -129,6 +134,7 @@ export async function readLogReport(
return {
events: [],
invalid_lines: 0,
unrecognized_lines: 0,
failure: "log_directory_unreadable",
};
}
@@ -0,0 +1,318 @@
import { beforeEach, describe, expect, it, vi } from "vitest";
import { run } from "./cli.js";
import { parseFeaturesCommand } from "./memory-features-flow.js";
import { composeStatusReport, renderStatusText } from "./status-report.js";
const transport = vi.hoisted(() => ({
get: vi.fn(),
post: vi.fn(),
put: vi.fn(),
patch: vi.fn(),
delete: vi.fn(),
}));
vi.mock("axios", () => ({
default: {
create: vi.fn(() => transport),
get: vi.fn(),
post: vi.fn(),
isAxiosError: () => false,
},
}));
const runtime = vi.hoisted(() => ({
inspectRuntimeStatus: vi.fn(),
discoverRuntimeAttachment: vi.fn(),
}));
vi.mock("@one-memory/runtime-control", async (importOriginal) => ({
...(await importOriginal<typeof import("@one-memory/runtime-control")>()),
...runtime,
}));
const DEFAULTS: Record<string, boolean> = {
graph_rerank: false,
same_series_timeline: false,
episode_recall: false,
session_episodes: false,
episode_summary: false,
intention: true,
few_shot: false,
};
function featuresState(
configured: Record<string, boolean> = {},
mode = "pro",
extra: Record<string, unknown> = {},
) {
const nonDefault: Record<string, boolean> = {};
const features = Object.entries(DEFAULTS).map(([name, fallback]) => {
const enabled = configured[name] ?? fallback;
if (enabled !== fallback) nonDefault[name] = enabled;
const needsUltra = name === "graph_rerank";
const reason = !enabled
? "disabled"
: needsUltra && mode !== "ultra"
? "requires_ultra"
: null;
return {
name,
enabled,
configured: name in configured ? configured[name] : null,
default: fallback,
active: reason === null,
reason,
side: "read",
requires_mode: needsUltra ? "ultra" : "pro",
requires_feature: null,
cost: "none",
sdk_field: `recall.${name}`,
env: `MEMORY_${name.toUpperCase()}`,
description: name,
};
});
return { mode, features, non_default: nonDefault, ...extra };
}
beforeEach(() => {
vi.clearAllMocks();
for (const method of Object.values(transport)) method.mockReset();
runtime.inspectRuntimeStatus.mockResolvedValue({
state: "running",
code: "runtime_verified",
instanceId: "instance-1",
host: "127.0.0.1",
port: 8765,
version: "1.2.3",
});
runtime.discoverRuntimeAttachment.mockResolvedValue({
instanceId: "instance-1",
apiToken: "synthetic-runtime-token",
host: "127.0.0.1",
port: 8765,
servicePid: 4242,
health: {
status: "ok",
service: "one-memory-core",
version: "1.2.3",
instance_id: "instance-1",
},
});
transport.get.mockImplementation(async (url: string) =>
url === "/api/v1/settings/memory-features"
? { status: 200, data: featuresState() }
: { status: 404, data: {} },
);
});
describe("memhub settings features", () => {
it("lists every feature with its value and whether it acts", async () => {
const result = await capturedRun(["settings", "features", "--json"]);
expect(result.code).toBe(0);
const json = JSON.parse(result.stdout);
expect(json.state).toBe("ok");
expect(json.features.features).toHaveLength(7);
expect(transport.put).not.toHaveBeenCalled();
const text = (await capturedRun(["settings", "features"])).stdout;
expect(text).toContain("graph_rerank:关(默认)");
expect(text).toContain("intention:开(默认)");
expect(text).toContain("先开启 session_episodes");
});
it("prints the plan and changes nothing without --yes", async () => {
const result = await capturedRun([
"settings",
"features",
"set",
"graph_rerank=true",
"episode_recall=true",
]);
expect(result.code).toBe(0);
expect(transport.put).not.toHaveBeenCalled();
expect(result.stdout).toContain(
"memhub settings features set graph_rerank=true episode_recall=true --yes",
);
});
it("sets features with --yes and reports the ones inactive in this mode", async () => {
transport.put.mockResolvedValue({
status: 200,
data: featuresState({ graph_rerank: true, episode_recall: true }, "pro", {
changed: ["graph_rerank", "episode_recall"],
}),
});
const result = await capturedRun([
"settings",
"features",
"set",
"graph_rerank=true",
"episode_recall=true",
"--yes",
]);
expect(result.code).toBe(0);
expect(transport.put).toHaveBeenCalledWith(
"/api/v1/settings/memory-features",
{ graph_rerank: true, episode_recall: true },
undefined,
);
expect(result.stdout).toContain("已更新:graph_rerank、episode_recall");
expect(result.stdout).toContain(
"graph_rerank:开(已设置,当前模式下不生效(需要 ultra))",
);
});
it("does nothing when the values are already set", async () => {
transport.get.mockResolvedValue({
status: 200,
data: featuresState({ few_shot: true }),
});
const result = await capturedRun([
"settings",
"features",
"set",
"few_shot=true",
"--yes",
]);
expect(result.code).toBe(0);
expect(result.stdout).toContain("没有需要改动的设置");
expect(transport.put).not.toHaveBeenCalled();
});
it("resets the named features, or every set one, to the default", async () => {
transport.get.mockResolvedValue({
status: 200,
data: featuresState({ few_shot: true, intention: false }),
});
transport.put.mockResolvedValue({
status: 200,
data: featuresState({}, "pro", { changed: ["intention", "few_shot"] }),
});
expect(
(await capturedRun(["settings", "features", "reset", "few_shot"])).code,
).toBe(0);
expect(transport.put).toHaveBeenLastCalledWith(
"/api/v1/settings/memory-features",
{ few_shot: null },
undefined,
);
const all = await capturedRun(["settings", "features", "reset", "--json"]);
expect(JSON.parse(all.stdout).state).toBe("committed");
expect(transport.put).toHaveBeenLastCalledWith(
"/api/v1/settings/memory-features",
{ intention: null, few_shot: null },
undefined,
);
});
it("refuses unknown names and malformed values before the Runtime", async () => {
for (const bad of [
["settings", "features", "set"],
["settings", "features", "set", "local_rerank=true"],
["settings", "features", "set", "graph_rerank=yes"],
["settings", "features", "set", "graph_rerank"],
["settings", "features", "set", "few_shot=true", "few_shot=false"],
["settings", "features", "reset", "nope"],
["settings", "features", "toggle"],
]) {
const result = await capturedRun(bad);
expect(result.code, bad.join(" ")).toBe(2);
}
expect(transport.put).not.toHaveBeenCalled();
});
it("surfaces a 422 from the Runtime as rejected", async () => {
transport.put.mockResolvedValue({
status: 422,
data: { detail: "invalid_request" },
});
const result = await capturedRun([
"settings",
"features",
"set",
"few_shot=true",
"--yes",
"--json",
]);
expect(result.code).toBe(1);
expect(JSON.parse(result.stdout).state).toBe("rejected");
});
it("parses the forms the help shows", () => {
expect(parseFeaturesCommand([])).toEqual({ kind: "show" });
expect(parseFeaturesCommand(["set", "intention=false", "--yes"])).toEqual({
kind: "set",
values: { intention: false },
confirmed: true,
});
expect(parseFeaturesCommand(["reset"])).toEqual({
kind: "reset",
names: [],
});
});
});
describe("status pipeline features", () => {
it("shows the features that differ from the defaults", async () => {
const processing = {
pending: 0,
fresh_pending: 0,
backlog_pending: 0,
outside_window_pending: 0,
running: 0,
waiting_for_model: 0,
completed: 3,
failed: 0,
blocked_stages: [],
pipeline: {
mode: "pro",
configured_mode: "pro",
forced_by_env: false,
last_evolve_at: null,
graph_build: { state: "idle", users: {} },
evolve: { mode: "pro", schedule: null, running: [], users: {} },
features: { episode_recall: true, intention: false },
},
};
const report = await composeStatusReport({
dataDir: "/tmp/isolated",
runtime: { state: "running", code: "ok", host: "127.0.0.1", port: 1 },
background: null,
reader: (async (path: string) =>
path === "/api/v1/memory-processing"
? { state: "ok", data: processing }
: { state: "unavailable", code: "x" }) as never,
});
expect(report.pipeline?.features).toEqual({
episode_recall: true,
intention: false,
});
const text = renderStatusText(report, "MemHub");
expect(text).toContain("SDK 功能:episode_recall=开、intention=关");
});
});
async function capturedRun(arguments_: string[]) {
let stdout = "";
let stderr = "";
const stdoutSpy = vi
.spyOn(process.stdout, "write")
.mockImplementation((chunk) => {
stdout += String(chunk);
return true;
});
const stderrSpy = vi
.spyOn(process.stderr, "write")
.mockImplementation((chunk) => {
stderr += String(chunk);
return true;
});
try {
return {
code: await run(arguments_, { ONE_MEMORY_DATA_DIR: "/synthetic/data" }),
stdout,
stderr,
};
} finally {
stdoutSpy.mockRestore();
stderrSpy.mockRestore();
}
}
+273
View File
@@ -0,0 +1,273 @@
/**
* `memhub settings features [set <name>=<true|false> ... | reset [<name> ...]]`.
*
* The SDK's optional retrieval and extraction features (graph rerank, episode
* recall, the episode write side, intention, few-shot). One global setting in
* the Runtime; an unset feature keeps the SDK default, so an install nobody
* touched behaves as before. `set` prints its plan first and needs `--yes`,
* like `settings mode`, because some features spend model calls; `reset`
* only returns features to their defaults and applies at once.
*/
import type { CoreReader } from "./core-client.js";
import type { CorePutOperations } from "./core-operations.js";
import { PRODUCT } from "./version.js";
export const MEMORY_FEATURES = [
"graph_rerank",
"same_series_timeline",
"episode_recall",
"session_episodes",
"episode_summary",
"intention",
"few_shot",
] as const;
export type MemoryFeature = (typeof MEMORY_FEATURES)[number];
const FEATURE_LABEL: Record<MemoryFeature, string> = {
graph_rerank: "图谱重排(沿知识图谱的边传播支持度;仅 ultra)",
same_series_timeline:
"同系列时间线(同一系列的记忆按时间放在一起;需图谱重排)",
episode_recall:
"片段召回(补入同一片段的记忆并多通道打分;pro / ultra,需片段写入)",
session_episodes: "片段写入(同一会话提取的记忆归入同一片段)",
episode_summary: "片段摘要(片段结束时总结一次;每个片段一次模型调用)",
intention: "意图提取(随事实一起提取用户的计划与意图)",
few_shot: "提取示例(在提取提示里附上示例;不加调用,输入更长)",
};
const REASON_TEXT: Record<string, string> = {
disabled: "未开启",
requires_pro: "lite 模式下不生效(需要 pro 或 ultra)",
requires_ultra: "当前模式下不生效(需要 ultra)",
requires_graph_rerank: "需要先开启并生效 graph_rerank",
};
export type FeaturesCommand =
| { kind: "show" }
| {
kind: "set";
values: Partial<Record<MemoryFeature, boolean>>;
confirmed: boolean;
}
| { kind: "reset"; names: MemoryFeature[] };
function isFeature(value: string): value is MemoryFeature {
return (MEMORY_FEATURES as readonly string[]).includes(value);
}
/** Arguments after `settings features`; null = usage. */
export function parseFeaturesCommand(
values: readonly string[],
): FeaturesCommand | null {
const rest = values.filter((value) => value !== "--json");
if (rest.length === 0) return { kind: "show" };
const [verb, ...tail] = rest;
const yes = tail.filter((value) => value === "--yes").length;
if (yes > 1) return null;
const positional = tail.filter((value) => value !== "--yes");
if (verb === "set") {
if (positional.length === 0) return null;
const parsed: Partial<Record<MemoryFeature, boolean>> = {};
for (const item of positional) {
const match = /^([a-z_]+)=(true|false)$/.exec(item);
if (!match || !isFeature(match[1]) || match[1] in parsed) return null;
parsed[match[1]] = match[2] === "true";
}
return { kind: "set", values: parsed, confirmed: yes === 1 };
}
if (verb === "reset") {
if (new Set(positional).size !== positional.length) return null;
if (!positional.every(isFeature)) return null;
return { kind: "reset", names: positional as MemoryFeature[] };
}
return null;
}
export interface FeatureState {
name: MemoryFeature;
enabled: boolean;
configured: boolean | null;
default: boolean;
active: boolean;
reason: string | null;
cost: string;
requires_mode: string;
description: string;
}
export interface FeaturesState {
mode: string;
features: FeatureState[];
non_default: Partial<Record<MemoryFeature, boolean>>;
changed?: MemoryFeature[];
}
function featuresState(value: unknown): FeaturesState | null {
if (!value || typeof value !== "object") return null;
const item = value as Record<string, unknown>;
if (
typeof item.mode !== "string" ||
!Array.isArray(item.features) ||
!item.non_default ||
typeof item.non_default !== "object"
) {
return null;
}
for (const feature of item.features) {
if (
!feature ||
typeof feature !== "object" ||
typeof (feature as FeatureState).name !== "string" ||
typeof (feature as FeatureState).enabled !== "boolean" ||
typeof (feature as FeatureState).active !== "boolean"
) {
return null;
}
}
return item as unknown as FeaturesState;
}
export type FeaturesDocument = {
schema_version: 1;
product: string;
command: "settings features";
} & (
| { state: "ok"; features: FeaturesState }
| {
state: "preview";
features: FeaturesState;
target: Partial<Record<MemoryFeature, boolean>>;
applied: false;
next_step: string;
}
| { state: "unchanged"; features: FeaturesState }
| { state: "committed"; features: FeaturesState; changed: MemoryFeature[] }
| { state: "rejected" | "unknown" | "unavailable"; code: string }
);
const BASE = {
schema_version: 1 as const,
product: PRODUCT,
command: "settings features" as const,
};
const PATH = "/api/v1/settings/memory-features";
export async function runFeaturesCommand(
command: FeaturesCommand,
reader: CoreReader,
operations: CorePutOperations,
): Promise<FeaturesDocument> {
const read = await reader<unknown>(PATH);
if (read.state !== "ok") {
return { ...BASE, state: "unavailable", code: read.code };
}
const current = featuresState(read.data);
if (current === null) {
return { ...BASE, state: "unavailable", code: "response_invalid" };
}
if (command.kind === "show") {
return { ...BASE, state: "ok", features: current };
}
let body: Record<string, unknown>;
if (command.kind === "set") {
const configured = new Map(
current.features.map((feature) => [feature.name, feature.configured]),
);
const differs = Object.entries(command.values).some(
([name, value]) => configured.get(name as MemoryFeature) !== value,
);
if (!differs) return { ...BASE, state: "unchanged", features: current };
if (!command.confirmed) {
const pairs = Object.entries(command.values)
.map(([name, value]) => `${name}=${value}`)
.join(" ");
return {
...BASE,
state: "preview",
features: current,
target: command.values,
applied: false,
next_step: `${PRODUCT} settings features set ${pairs} --yes`,
};
}
body = { ...command.values };
} else {
const names =
command.names.length > 0
? command.names
: current.features
.filter((feature) => feature.configured !== null)
.map((feature) => feature.name);
if (
names.every(
(name) =>
current.features.find((feature) => feature.name === name)
?.configured === null,
)
) {
return { ...BASE, state: "unchanged", features: current };
}
body = Object.fromEntries(names.map((name) => [name, null]));
}
const result = await operations.put<unknown>(PATH, body);
if (result.state !== "committed") {
return { ...BASE, state: result.state, code: result.code };
}
const changed = featuresState(result.data);
if (changed === null) {
return { ...BASE, state: "unknown", code: "response_invalid" };
}
return {
...BASE,
state: "committed",
features: changed,
changed: changed.changed ?? [],
};
}
export function featuresExitCode(document: FeaturesDocument): number {
return ["ok", "preview", "unchanged", "committed"].includes(document.state)
? 0
: 1;
}
function featureLine(feature: FeatureState): string {
const label = FEATURE_LABEL[feature.name] ?? feature.name;
const value = feature.enabled ? "开" : "关";
const origin = feature.configured === null ? "默认" : "已设置";
const activity =
feature.enabled && !feature.active && feature.reason
? `,${REASON_TEXT[feature.reason] ?? feature.reason}`
: "";
return `${feature.name}:${value}(${origin}${activity})— ${label}`;
}
export function renderFeaturesText(document: FeaturesDocument): string {
if ("code" in document) {
return `SDK 功能未能${document.state === "unavailable" ? "读取" : "修改"}:${document.code}\n`;
}
const state = document.features;
const lines = [`当前模式:${state.mode}`];
for (const feature of state.features) lines.push(featureLine(feature));
lines.push(
"片段召回(episode_recall)只对带片段的记忆有用:先开启 session_episodes,之后提取的记忆才会带片段。",
);
if (document.state === "unchanged") lines.push("没有需要改动的设置。");
if (document.state === "preview") {
lines.push("将设置:");
for (const [name, value] of Object.entries(document.target)) {
lines.push(`- ${name} = ${value ? "开" : "关"}`);
}
lines.push(`未执行任何改动。确认后运行:${document.next_step}`);
}
if (document.state === "committed") {
lines.push(
document.changed.length > 0
? `已更新:${document.changed.join("、")}。正在进行的写入按原设置完成,之后的写入和检索使用新设置。`
: "已保存,生效的值没有变化。",
);
}
return `${lines.join("\n")}\n`;
}
+309
View File
@@ -0,0 +1,309 @@
import { beforeEach, describe, expect, it, vi } from "vitest";
import { run } from "./cli.js";
import { modeEffects, parseSettingsCommand } from "./memory-mode-flow.js";
import { composeStatusReport, renderStatusText } from "./status-report.js";
const transport = vi.hoisted(() => ({
get: vi.fn(),
post: vi.fn(),
put: vi.fn(),
patch: vi.fn(),
delete: vi.fn(),
}));
vi.mock("axios", () => ({
default: {
create: vi.fn(() => transport),
get: vi.fn(),
post: vi.fn(),
isAxiosError: () => false,
},
}));
const runtime = vi.hoisted(() => ({
inspectRuntimeStatus: vi.fn(),
discoverRuntimeAttachment: vi.fn(),
}));
vi.mock("@one-memory/runtime-control", async (importOriginal) => ({
...(await importOriginal<typeof import("@one-memory/runtime-control")>()),
...runtime,
}));
const RUN_ID = "11111111-2222-4333-8444-555555555555";
function modeState(mode: string, configured = mode) {
return {
mode,
configured_mode: configured,
forced_by_env: false,
modes: ["lite", "pro", "ultra"],
graph_build: { state: "idle", users: {} },
};
}
function receipt(state: string, extra: Record<string, unknown> = {}) {
return {
run_id: RUN_ID,
user_id: "wife",
agent_id: "all-apps",
state,
trigger: "manual",
started_at: "2026-11-01T03:00:00Z",
rounds: state === "running" ? 0 : 2,
tasks_processed: state === "running" ? 0 : 5,
batch_failures: 0,
capped: false,
elapsed_ms: 10,
results: { schemas: ["never printed"] },
...extra,
};
}
beforeEach(() => {
vi.clearAllMocks();
for (const method of Object.values(transport)) method.mockReset();
runtime.inspectRuntimeStatus.mockResolvedValue({
state: "running",
code: "runtime_verified",
instanceId: "instance-1",
host: "127.0.0.1",
port: 8765,
version: "1.2.3",
});
runtime.discoverRuntimeAttachment.mockResolvedValue({
instanceId: "instance-1",
apiToken: "synthetic-runtime-token",
host: "127.0.0.1",
port: 8765,
servicePid: 4242,
health: {
status: "ok",
service: "one-memory-core",
version: "1.2.3",
instance_id: "instance-1",
},
});
transport.get.mockImplementation(async (url: string) =>
url === "/api/v1/settings/memory-mode"
? { status: 200, data: modeState("pro") }
: { status: 404, data: {} },
);
});
describe("memhub settings mode", () => {
it("shows the current mode without changing it", async () => {
const result = await capturedRun(["settings", "mode", "--json"]);
expect(result.code).toBe(0);
expect(JSON.parse(result.stdout).mode.mode).toBe("pro");
expect(transport.put).not.toHaveBeenCalled();
});
it("prints the plan and changes nothing without --yes", async () => {
const result = await capturedRun(["settings", "mode", "lite"]);
expect(result.code).toBe(0);
expect(transport.put).not.toHaveBeenCalled();
expect(result.stdout).toContain("更高模式生成的记忆仍保存着");
expect(result.stdout).toContain("memhub settings mode lite --yes");
const json = JSON.parse(
(await capturedRun(["settings", "mode", "ultra", "--json"])).stdout,
);
expect(json.state).toBe("preview");
expect(json.applied).toBe(false);
expect(json.effects).toEqual(["existing_memories_added_to_graph"]);
});
it("changes the mode with --yes and reports what changed", async () => {
transport.put.mockResolvedValue({
status: 200,
data: {
...modeState("lite"),
previous_mode: "pro",
effects: ["higher_mode_memories_hidden"],
},
});
const result = await capturedRun(["settings", "mode", "lite", "--yes"]);
expect(result.code).toBe(0);
expect(transport.put).toHaveBeenCalledWith(
"/api/v1/settings/memory-mode",
{ mode: "lite" },
undefined,
);
expect(result.stdout).toContain("已切换");
expect(result.stdout).toContain("原始对话:全部保留");
});
it("does nothing for the mode already set, and refuses bad input", async () => {
const same = await capturedRun(["settings", "mode", "pro", "--yes"]);
expect(same.code).toBe(0);
expect(same.stdout).toContain("已经是这个模式");
expect(transport.put).not.toHaveBeenCalled();
for (const bad of [
["settings"],
["settings", "mode", "max"],
["settings", "mode", "--yes"],
["settings", "mode", "lite", "pro"],
["settings", "language", "en"],
]) {
expect((await capturedRun(bad)).code, bad.join(" ")).toBe(2);
}
});
it("plans the same effects Core reports", () => {
expect(modeEffects("lite", "pro")).toEqual([
"earlier_raw_not_extracted",
"lite_search_stops",
]);
expect(modeEffects("ultra", "lite")).toEqual([
"higher_mode_memories_hidden",
"evolve_stopped",
]);
expect(modeEffects("pro", "pro")).toEqual([]);
expect(parseSettingsCommand(["mode", "ultra", "--yes"])).toEqual({
kind: "set",
mode: "ultra",
confirmed: true,
});
});
});
describe("memhub evolve", () => {
it("starts a run for the named user and returns at once", async () => {
transport.post.mockResolvedValue({ status: 202, data: receipt("running") });
const result = await capturedRun([
"evolve",
"--user-id",
"wife",
"--agent-id",
"all-apps",
"--max-batches",
"3",
"--json",
]);
expect(result.code).toBe(0);
expect(transport.post).toHaveBeenCalledWith(
"/api/v2/evolve",
{ user_id: "wife", agent_id: "all-apps", max_batches: 3 },
undefined,
);
const document = JSON.parse(result.stdout);
expect(document.state).toBe("running");
expect(document.run.results).toBeUndefined();
});
it("follows the run with --wait until it ends", async () => {
transport.post.mockResolvedValue({ status: 202, data: receipt("running") });
const reads = [receipt("running"), receipt("succeeded")];
transport.get.mockImplementation(async (url: string) =>
url === `/api/v2/evolve/${RUN_ID}`
? { status: 200, data: reads.shift() }
: { status: 404, data: {} },
);
const result = await capturedRun(["evolve", "--wait"], {
sleep: async () => undefined,
});
expect(result.code).toBe(0);
expect(transport.post).toHaveBeenCalledWith(
"/api/v2/evolve",
{},
undefined,
);
expect(result.stdout).toContain("进化完成");
expect(result.stdout).toContain("2 轮");
});
it("names the mode when Core refuses outside ultra", async () => {
transport.post.mockResolvedValue({
status: 409,
data: { detail: "memory_mode_not_ultra" },
});
const result = await capturedRun(["evolve"]);
expect(result.code).toBe(1);
expect(result.stdout).toContain("memhub settings mode ultra --yes");
expect((await capturedRun(["evolve", "--max-batches", "0"])).code).toBe(2);
expect((await capturedRun(["evolve", "--now"])).code).toBe(2);
});
});
describe("status pipeline", () => {
it("reports pipeline.mode and the last evolve from memory-processing", async () => {
const processing = {
pending: 0,
fresh_pending: 0,
backlog_pending: 0,
outside_window_pending: 0,
running: 0,
waiting_for_model: 0,
completed: 3,
failed: 0,
blocked_stages: [],
pipeline: {
mode: "ultra",
configured_mode: "ultra",
forced_by_env: false,
last_evolve_at: "2026-11-01T03:00:05+08:00",
graph_build: { state: "succeeded", users: {} },
evolve: {
mode: "ultra",
schedule: { enabled: true, time: "03:00" },
running: [],
users: {
local: {
last_run_at: "2026-11-01T03:00:05+08:00",
last_result: { state: "succeeded", tasks_processed: 4 },
last_error: null,
},
},
},
},
};
const report = await composeStatusReport({
dataDir: "/tmp/isolated",
runtime: { state: "running", code: "ok", host: "127.0.0.1", port: 1 },
background: null,
reader: (async (path: string) =>
path === "/api/v1/memory-processing"
? { state: "ok", data: processing }
: { state: "unavailable", code: "x" }) as never,
});
expect(report.pipeline?.mode).toBe("ultra");
expect(report.pipeline?.last_evolve_at).toBe("2026-11-01T03:00:05+08:00");
expect(report.pipeline?.evolve_users.local?.state).toBe("succeeded");
const text = renderStatusText(report, "MemHub");
expect(text).toContain("记忆模式:ultra");
expect(text).toContain("每日 03:00 自动进化");
});
});
async function capturedRun(
arguments_: string[],
controls: Parameters<typeof run>[2] = {},
) {
let stdout = "";
let stderr = "";
const stdoutSpy = vi
.spyOn(process.stdout, "write")
.mockImplementation((chunk) => {
stdout += String(chunk);
return true;
});
const stderrSpy = vi
.spyOn(process.stderr, "write")
.mockImplementation((chunk) => {
stderr += String(chunk);
return true;
});
try {
return {
code: await run(
arguments_,
{ ONE_MEMORY_DATA_DIR: "/synthetic/data" },
controls,
),
stdout,
stderr,
};
} finally {
stdoutSpy.mockRestore();
stderrSpy.mockRestore();
}
}
+372
View File
@@ -0,0 +1,372 @@
/**
* `memhub settings mode` and `memhub evolve`.
*
* The mode (lite / pro / ultra) decides what the SDK does with every
* conversation, and switching it migrates nothing: what one mode made is not
* shown by a lower one, and raw kept by lite is not extracted until it is
* imported again. So a change prints its plan first and needs `--yes`, like
* every other command whose effect a person cannot simply take back.
*
* `memhub evolve` starts System 2 (ultra only) and, with `--wait`, follows
* the run until it ends.
*/
import type { CoreReader } from "./core-client.js";
import type { CoreOperations, CorePutOperations } from "./core-operations.js";
import { PRODUCT } from "./version.js";
export const MEMORY_MODES = ["lite", "pro", "ultra"] as const;
export type MemoryMode = (typeof MEMORY_MODES)[number];
export type ModeEffect =
| "earlier_raw_not_extracted"
| "higher_mode_memories_hidden"
| "existing_memories_added_to_graph"
| "evolve_stopped"
| "lite_search_stops"
| "graph_backfill_already_running";
const MODE_SUMMARY: Record<MemoryMode, string> = {
lite: "lite:不调用模型,保留全部原始对话,只在原始对话里检索",
pro: "pro:模型从对话中提取记忆(System 1)",
ultra: "ultra:pro + 每晚自动进化(System 2,使用知识图谱)",
};
const EFFECT_TEXT: Record<ModeEffect, string> = {
earlier_raw_not_extracted:
"lite 下已处理完的对话不会再被提取;只有尚未处理的和之后的新对话会由模型整理成记忆,检索立即改为记忆检索",
higher_mode_memories_hidden:
"更高模式生成的记忆仍保存着,但在这个模式下不会显示",
existing_memories_added_to_graph:
"已有记忆会在后台加入知识图谱(每个用户一次)",
evolve_stopped: "每晚的自动进化(System 2)停止",
lite_search_stops: "lite 只在原始对话里的检索立即停止,改为检索记忆",
graph_backfill_already_running: "知识图谱的补建已在进行中",
};
/** The same rule Core applies (`memory_mode_effects`), for the plan. */
export function modeEffects(
previous: MemoryMode,
next: MemoryMode,
): ModeEffect[] {
if (previous === next) return [];
const rank: Record<MemoryMode, number> = { lite: 0, pro: 1, ultra: 2 };
const effects: ModeEffect[] = [];
if (previous === "lite") {
effects.push("earlier_raw_not_extracted", "lite_search_stops");
}
if (rank[next] < rank[previous]) effects.push("higher_mode_memories_hidden");
if (previous === "ultra") effects.push("evolve_stopped");
if (next === "ultra") effects.push("existing_memories_added_to_graph");
return effects;
}
export type ModeCommand =
| { kind: "show" }
| { kind: "set"; mode: MemoryMode; confirmed: boolean };
/** `settings mode [lite|pro|ultra] [--yes]` after `settings`; null = usage. */
export function parseSettingsCommand(
values: readonly string[],
): ModeCommand | null {
const rest = values.filter((value) => value !== "--json");
if (rest[0] !== "mode") return null;
const tail = rest.slice(1);
const confirmed = tail.includes("--yes");
const positional = tail.filter((value) => value !== "--yes");
if (tail.filter((value) => value === "--yes").length > 1) return null;
if (positional.length === 0) return confirmed ? null : { kind: "show" };
if (positional.length !== 1) return null;
const mode = positional[0] as MemoryMode;
if (!(MEMORY_MODES as readonly string[]).includes(mode)) return null;
return { kind: "set", mode, confirmed };
}
export interface ModeState {
mode: MemoryMode;
configured_mode: MemoryMode;
forced_by_env: boolean;
graph_build?: { state?: string } | null;
previous_mode?: MemoryMode;
effects?: ModeEffect[];
}
function modeState(value: unknown): ModeState | null {
if (!value || typeof value !== "object") return null;
const item = value as Record<string, unknown>;
if (
!(MEMORY_MODES as readonly unknown[]).includes(item.mode) ||
!(MEMORY_MODES as readonly unknown[]).includes(item.configured_mode) ||
typeof item.forced_by_env !== "boolean"
) {
return null;
}
return item as unknown as ModeState;
}
export type ModeDocument = {
schema_version: 1;
product: string;
command: "settings mode";
} & (
| { state: "ok"; mode: ModeState }
| {
state: "preview";
mode: ModeState;
target: MemoryMode;
effects: ModeEffect[];
applied: false;
next_step: string;
}
| { state: "unchanged"; mode: ModeState }
| { state: "committed"; mode: ModeState; effects: ModeEffect[] }
| { state: "rejected" | "unknown" | "unavailable"; code: string }
);
const BASE = {
schema_version: 1 as const,
product: PRODUCT,
command: "settings mode" as const,
};
export async function runModeCommand(
command: ModeCommand,
reader: CoreReader,
operations: CorePutOperations,
): Promise<ModeDocument> {
const read = await reader<unknown>("/api/v1/settings/memory-mode");
if (read.state !== "ok") {
return { ...BASE, state: "unavailable", code: read.code };
}
const current = modeState(read.data);
if (current === null) {
return { ...BASE, state: "unavailable", code: "response_invalid" };
}
if (command.kind === "show") return { ...BASE, state: "ok", mode: current };
if (current.configured_mode === command.mode) {
return { ...BASE, state: "unchanged", mode: current };
}
if (!command.confirmed) {
return {
...BASE,
state: "preview",
mode: current,
target: command.mode,
effects: modeEffects(current.mode, command.mode),
applied: false,
next_step: `${PRODUCT} settings mode ${command.mode} --yes`,
};
}
const result = await operations.put<unknown>("/api/v1/settings/memory-mode", {
mode: command.mode,
});
if (result.state !== "committed") {
return { ...BASE, state: result.state, code: result.code };
}
const changed = modeState(result.data);
if (changed === null) {
return { ...BASE, state: "unknown", code: "response_invalid" };
}
return {
...BASE,
state: "committed",
mode: changed,
effects: changed.effects ?? [],
};
}
export function modeExitCode(document: ModeDocument): number {
return ["ok", "preview", "unchanged", "committed"].includes(document.state)
? 0
: 1;
}
export function renderModeText(document: ModeDocument): string {
if ("code" in document) {
return `记忆模式未能${document.state === "unavailable" ? "读取" : "修改"}:${document.code}\n`;
}
const mode = document.mode;
const lines = [`当前模式:${MODE_SUMMARY[mode.mode]}`];
if (mode.forced_by_env) {
lines.push(
`设置里是 ${mode.configured_mode};环境变量 ONE_MEMORY_GRAPH_STORE 强制使用 ultra。`,
);
}
lines.push("原始对话:全部保留(任何模式都不删除)");
if (document.state === "unchanged") {
lines.push("已经是这个模式,未做改动。");
}
if (document.state === "preview") {
lines.push(`将切换到:${MODE_SUMMARY[document.target]}`);
for (const effect of document.effects)
lines.push(`- ${EFFECT_TEXT[effect]}`);
lines.push(`未执行任何改动。确认后运行:${document.next_step}`);
}
if (document.state === "committed") {
lines.push("已切换。");
for (const effect of document.effects)
lines.push(`- ${EFFECT_TEXT[effect]}`);
}
return `${lines.join("\n")}\n`;
}
export type EvolveCommand = {
agentId?: string;
maxBatches?: number;
wait: boolean;
};
/** `evolve [--agent-id <id>] [--max-batches <n>] [--wait]`; null = usage. */
export function parseEvolveCommand(
values: readonly string[],
): EvolveCommand | null {
const parsed: EvolveCommand = { wait: false };
const rest = values.filter((value) => value !== "--json");
for (let index = 0; index < rest.length; index += 1) {
const flag = rest[index];
if (flag === "--wait" && !parsed.wait) {
parsed.wait = true;
} else if (flag === "--agent-id" && parsed.agentId === undefined) {
const value = rest[index + 1];
if (!value || !/^[A-Za-z0-9._:-]{1,128}$/.test(value)) return null;
parsed.agentId = value;
index += 1;
} else if (flag === "--max-batches" && parsed.maxBatches === undefined) {
const value = Number(rest[index + 1]);
if (!Number.isInteger(value) || value < 1 || value > 1000) return null;
parsed.maxBatches = value;
index += 1;
} else {
return null;
}
}
return parsed;
}
export type EvolveDocument = {
schema_version: 1;
product: string;
command: "evolve";
} & (
| { state: "running" | "succeeded" | "failed"; run: Record<string, unknown> }
| { state: "rejected" | "unknown" | "unavailable"; code: string }
);
const RUN_ID = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/;
function evolveRun(value: unknown): Record<string, unknown> | null {
if (!value || typeof value !== "object") return null;
const run = value as Record<string, unknown>;
if (
typeof run.run_id !== "string" ||
!RUN_ID.test(run.run_id) ||
!["running", "succeeded", "failed"].includes(String(run.state))
) {
return null;
}
return run;
}
export async function runEvolveCommand(
command: EvolveCommand,
options: {
userId?: string;
reader: CoreReader;
operations: CoreOperations;
sleep?: (ms: number) => Promise<void>;
pollMs?: number;
},
): Promise<EvolveDocument> {
const base = {
schema_version: 1 as const,
product: PRODUCT,
command: "evolve" as const,
};
const body: Record<string, unknown> = {};
if (options.userId !== undefined) body.user_id = options.userId;
if (command.agentId !== undefined) body.agent_id = command.agentId;
if (command.maxBatches !== undefined) body.max_batches = command.maxBatches;
const started = await options.operations.post<unknown>(
"/api/v2/evolve",
body,
);
if (started.state !== "committed") {
return { ...base, state: started.state, code: started.code };
}
let run = evolveRun(started.data);
if (run === null)
return { ...base, state: "unknown", code: "response_invalid" };
const sleep =
options.sleep ??
((ms: number) => new Promise((done) => setTimeout(done, ms)));
while (command.wait && run.state === "running") {
await sleep(options.pollMs ?? 2_000);
const read = await options.reader<unknown>(
`/api/v2/evolve/${run.run_id}`,
options.userId === undefined ? {} : { user_id: options.userId },
);
if (read.state !== "ok") {
return { ...base, state: "unavailable", code: read.code };
}
const next = evolveRun(read.data);
if (next === null)
return { ...base, state: "unknown", code: "response_invalid" };
run = next;
}
return {
...base,
state: run.state as "running" | "succeeded" | "failed",
run: projectRun(run),
};
}
/** Counts and codes; SDK result bodies stay in the Core receipt. */
function projectRun(run: Record<string, unknown>): Record<string, unknown> {
const keep = [
"run_id",
"user_id",
"agent_id",
"state",
"trigger",
"started_at",
"finished_at",
"rounds",
"tasks_processed",
"batch_failures",
"cycle_status",
"should_continue",
"continuation_reason",
"remaining_clusters",
"remaining_actionable_clusters",
"remaining_fresh_facts",
"capped",
"elapsed_ms",
"error_code",
"failure_kind",
];
return Object.fromEntries(
keep.filter((key) => key in run).map((key) => [key, run[key]]),
);
}
export function evolveExitCode(document: EvolveDocument): number {
return document.state === "running" || document.state === "succeeded" ? 0 : 1;
}
export function renderEvolveText(document: EvolveDocument): string {
if ("code" in document) {
return document.code === "memory_mode_not_ultra"
? "进化只在 ultra 模式下可用。切换:memhub settings mode ultra --yes\n"
: `进化未能开始:${document.code}\n`;
}
const run = document.run;
if (document.state === "running") {
return `进化已开始(${run.run_id}),在后台运行;加 --wait 等待结果。\n`;
}
if (document.state === "failed") {
return `进化失败(${run.run_id}):${String(run.error_code ?? "unknown")}\n`;
}
return `进化完成(${run.run_id}):${run.rounds} 轮,处理 ${run.tasks_processed} 项${
run.cycle_status ? `,周期 ${run.cycle_status}` : ""
}${run.capped ? ",已到轮数上限,可再次运行继续" : ""}\n`;
}
+40 -3
View File
@@ -14,6 +14,25 @@ import { type CoreReader, RESERVED_MEMORY_SEGMENTS } from "./core-client.js";
import type { CoreOperations } from "./core-operations.js";
import { PRODUCT } from "./version.js";
/** A MemHub user id, the same shape Core accepts. `local` is the default. */
const USER_ID = /^[A-Za-z0-9._-]{1,64}$/;
export const DEFAULT_USER_ID = "local";
export function isUserId(value: string): boolean {
return USER_ID.test(value);
}
/**
* The `user_id` to send, or nothing. The default user is never named on the
* wire, so a request without `--user-id` stays exactly what it was before
* multi-user support and still works against an older Core.
*/
export function userIdParameters(userId?: string): { user_id?: string } {
return userId === undefined || userId === DEFAULT_USER_ID
? {}
: { user_id: userId };
}
/** Output budget: a long memory is shortened visibly, never dropped silently. */
const CONTENT_BUDGET = 240;
@@ -101,10 +120,12 @@ export interface MemoryDetailDocument {
export async function readMemoryPage(
reader: CoreReader,
query: MemoryQuery,
userId?: string,
): Promise<MemoryPageDocument> {
const parameters: Record<string, string | number> = {
offset: query.offset,
limit: query.limit,
...userIdParameters(userId),
};
if (query.status !== undefined) parameters.statuses = query.status;
if (query.source !== undefined) parameters.source = query.source;
@@ -156,8 +177,12 @@ export async function readMemoryDetail(
reader: CoreReader,
memoryId: string,
namespace: MemoryNamespace = {},
userId?: string,
): Promise<MemoryDetailDocument> {
const parameters = namespaceParameters(namespace);
const parameters = {
...namespaceParameters(namespace),
...userIdParameters(userId),
};
const outcome = await reader<unknown>(
`/api/v2/memories/${memoryId}`,
Object.keys(parameters).length > 0 ? parameters : undefined,
@@ -236,6 +261,7 @@ export async function addMemory(
operations: CoreOperations,
text: string,
requestId: string = randomUUID(),
userId?: string,
): Promise<MemoryAddDocument> {
const base = {
schema_version: 2 as const,
@@ -259,7 +285,10 @@ export async function addMemory(
}
throw error;
}
const result = await operations.post<unknown>(request.path, request.body);
const result = await operations.post<unknown>(request.path, {
...request.body,
...userIdParameters(userId),
});
if (result.state !== "committed") {
return { ...base, state: result.state, code: result.code };
}
@@ -273,6 +302,7 @@ export async function addMemory(
export async function readMemoryAddStatus(
reader: CoreReader,
requestId: string,
userId?: string,
): Promise<MemoryAddDocument> {
const base = {
schema_version: 2 as const,
@@ -283,7 +313,11 @@ export async function readMemoryAddStatus(
if (!UUID.test(requestId)) {
return { ...base, state: "invalid", code: "invalid_request_id" };
}
const outcome = await reader<unknown>(memoryAddStatusPath(requestId));
const user = userIdParameters(userId);
const outcome =
user.user_id === undefined
? await reader<unknown>(memoryAddStatusPath(requestId))
: await reader<unknown>(memoryAddStatusPath(requestId), user);
if (outcome.state === "not_found") {
return { ...base, state: "not_found", code: "memory_add_not_found" };
}
@@ -338,6 +372,9 @@ export function renderMemoryAddText(document: MemoryAddDocument): string {
export function renderMemoryPageText(document: MemoryPageDocument): string {
const page = document.page;
if (page.state !== "ok") {
if (page.state === "unavailable" && page.code === "embedding_unavailable") {
return "向量服务不可用,无法检索;请检查向量模型配置。\n";
}
if (page.state === "unavailable") {
return "后台正忙,记忆列表这次没查完,不代表没有记忆,请稍后再试\n";
}
+30 -1
View File
@@ -22,6 +22,10 @@ const RESULT_KEYS = new Set([
"stages",
"capabilities",
]);
const OPTIONAL_RESULT_KEYS = [
"embedding_width",
"embedding_switch_required",
] as const;
const STAGE_KEYS = new Set([
"stage",
"ok",
@@ -272,9 +276,34 @@ function validateResult(
inputRevision: number,
): Record<string, unknown> | null {
const result = record(value);
// A Core with the embedding registry adds what the embedding stage measured
// and whether using it needs an embedding switch; older Cores send neither.
const optional = Object.fromEntries(
OPTIONAL_RESULT_KEYS.filter((key) => key in (result ?? {})).map((key) => [
key,
result?.[key],
]),
);
const required = result
? Object.fromEntries(
Object.entries(result).filter(
([key]) => !(OPTIONAL_RESULT_KEYS as readonly string[]).includes(key),
),
)
: null;
if (
!result ||
!exactKeys(result, RESULT_KEYS) ||
!required ||
!exactKeys(required, RESULT_KEYS) ||
!(
optional.embedding_width === undefined ||
optional.embedding_width === null ||
positiveInteger(optional.embedding_width)
) ||
!(
optional.embedding_switch_required === undefined ||
typeof optional.embedding_switch_required === "boolean"
) ||
result.profile_id !== profileId ||
result.input_profile_revision !== inputRevision ||
!positiveInteger(result.profile_revision) ||
+1 -1
View File
@@ -117,7 +117,7 @@ export async function editModelProfile(
return modelProfileResult(profileId, result);
}
function modelProfileResult(
export function modelProfileResult(
operationId: string,
result:
| Awaited<ReturnType<CoreOperations["post"]>>
@@ -0,0 +1,166 @@
import { beforeEach, describe, expect, it, vi } from "vitest";
import { run } from "./cli.js";
const transport = vi.hoisted(() => ({
get: vi.fn(),
post: vi.fn(),
put: vi.fn(),
patch: vi.fn(),
delete: vi.fn(),
}));
vi.mock("axios", () => ({
default: {
create: vi.fn(() => transport),
get: vi.fn(),
post: vi.fn(),
isAxiosError: () => false,
},
}));
const runtime = vi.hoisted(() => ({
inspectRuntimeStatus: vi.fn(),
discoverRuntimeAttachment: vi.fn(),
}));
vi.mock("@one-memory/runtime-control", async (importOriginal) => ({
...(await importOriginal<typeof import("@one-memory/runtime-control")>()),
...runtime,
}));
const VERSIONS = {
user_id: "wife",
request_id: "r-versions",
elapsed_ms: 2,
chains: [
{
memory_id: "l0-head",
content: "The user's name is Fei.",
status: "active",
memory_at: null,
gmt_created: 1_790_000_100,
basic_info: { name: "Fei" },
versions: [
{
memory_id: "l0-head",
content: "The user's name is Fei.",
memory_at: null,
gmt_created: 1_790_000_100,
current: true,
},
{
memory_id: "l0-old",
content: "The user's name is 费天翔.",
memory_at: null,
gmt_created: 1_790_000_000,
current: false,
},
],
},
],
};
beforeEach(() => {
vi.clearAllMocks();
for (const method of Object.values(transport)) method.mockReset();
runtime.inspectRuntimeStatus.mockResolvedValue({
state: "running",
code: "runtime_verified",
instanceId: "instance-1",
host: "127.0.0.1",
port: 8765,
version: "1.2.3",
});
runtime.discoverRuntimeAttachment.mockResolvedValue({
instanceId: "instance-1",
apiToken: "synthetic-runtime-token",
host: "127.0.0.1",
port: 8765,
servicePid: 4242,
health: {
status: "ok",
service: "one-memory-core",
version: "1.2.3",
instance_id: "instance-1",
},
});
transport.get.mockImplementation(async (url: string) =>
url === "/api/v2/user-profile/versions"
? { status: 200, data: VERSIONS }
: { status: 404, data: {} },
);
});
describe("memhub profile --versions", () => {
it("reads the L0 version chain for the named user", async () => {
const result = await capturedRun([
"profile",
"--versions",
"--user-id",
"wife",
"--json",
]);
expect(result.code).toBe(0);
expect(transport.get).toHaveBeenCalledWith(
"/api/v2/user-profile/versions",
{ params: { user_id: "wife" } },
);
const document = JSON.parse(result.stdout);
expect(document.state).toBe("ok");
expect(document.chains[0].versions).toHaveLength(2);
});
it("marks the current version in text and names no user for local", async () => {
const result = await capturedRun(["profile", "--versions"]);
expect(result.code).toBe(0);
const [url, config] = transport.get.mock.calls[0] ?? [];
expect(url).toBe("/api/v2/user-profile/versions");
expect(config?.params?.user_id).toBeUndefined();
expect(result.stdout).toContain("* l0-head");
expect(result.stdout).toContain(" l0-old");
expect(result.stdout).toContain("费天翔");
});
it("refuses an unknown flag and a repeated --versions", async () => {
expect((await capturedRun(["profile", "--version-chain"])).code).toBe(2);
expect(
(await capturedRun(["profile", "--versions", "--versions"])).code,
).toBe(2);
expect(transport.get).not.toHaveBeenCalled();
});
it("reports a malformed answer as invalid", async () => {
transport.get.mockResolvedValue({ status: 200, data: { chains: "x" } });
const result = await capturedRun(["profile", "--versions", "--json"]);
expect(result.code).toBe(1);
expect(JSON.parse(result.stdout).code).toBe("response_invalid");
});
});
async function capturedRun(arguments_: string[]) {
let stdout = "";
let stderr = "";
const stdoutSpy = vi
.spyOn(process.stdout, "write")
.mockImplementation((chunk) => {
stdout += String(chunk);
return true;
});
const stderrSpy = vi
.spyOn(process.stderr, "write")
.mockImplementation((chunk) => {
stderr += String(chunk);
return true;
});
try {
return {
code: await run(arguments_, {
ONE_MEMORY_DATA_DIR: "/synthetic/data",
}),
stdout,
stderr,
};
} finally {
stdoutSpy.mockRestore();
stderrSpy.mockRestore();
}
}
+107 -1
View File
@@ -14,6 +14,11 @@ import {
validateProcessing,
validateSources,
} from "./core-responses.js";
import {
describePipeline,
type PipelineSummary,
pipelineLine,
} from "./status-report.js";
import { PRODUCT } from "./version.js";
type ReadState =
@@ -38,6 +43,21 @@ export interface SourcesDocument {
state: ReadState["state"];
code?: string;
sources: Array<Record<string, unknown>>;
/** Every grant of every user (`GET /api/v1/source-grants`). Absent when
* Core is too old to list them; `sources` is the default user's view. */
grants?: SourceGrant[];
}
/** One source grant. Only Hermes has grants for users other than `local`,
* one per Hermes profile root. */
export interface SourceGrant {
user_id: string;
source_id: string;
root_label: string | null;
root_path: string | null;
authorized: boolean;
revoked_at: string | null;
sync_enabled: boolean;
}
export interface SyncDocument {
@@ -69,6 +89,8 @@ export interface ProgressDocument {
summary_state: ReadState;
summary: Record<string, unknown> | null;
};
/** The SDK mode and last evolve per user; absent on an older Core. */
pipeline?: PipelineSummary | null;
}
export async function readSources(
@@ -86,14 +108,66 @@ export async function readSources(
sources: [],
};
}
const grants = await readSourceGrants(reader);
return {
schema_version: SCHEMA_VERSION,
product: PRODUCT,
state: "ok",
sources: result.data.map(projectSource),
// `/api/v1/sources` is the default user's view of every source.
sources: result.data.map((source) => ({
...projectSource(source),
user_id: "local",
})),
...(grants === null ? {} : { grants }),
};
}
/** The grants of every user, or null when Core cannot list them (an older
* Core answers 404): the source list is then shown as before. */
async function readSourceGrants(
reader: CoreReader,
): Promise<SourceGrant[] | null> {
let outcome: ReadOutcome<unknown>;
try {
outcome = await reader<unknown>("/api/v1/source-grants");
} catch {
return null;
}
if (outcome.state !== "ok") return null;
const value = outcome.data;
if (!value || typeof value !== "object" || Array.isArray(value)) return null;
const items = (value as { grants?: unknown }).grants;
if (!Array.isArray(items)) return null;
const grants: SourceGrant[] = [];
const text = (candidate: unknown): string | null =>
typeof candidate === "string" && candidate.length <= 4096
? candidate
: null;
for (const item of items) {
if (!item || typeof item !== "object" || Array.isArray(item)) return null;
const grant = item as Record<string, unknown>;
if (
typeof grant.user_id !== "string" ||
!/^[A-Za-z0-9._-]{1,64}$/.test(grant.user_id) ||
typeof grant.source_id !== "string" ||
!/^[a-z0-9][a-z0-9-]{0,127}$/.test(grant.source_id) ||
typeof grant.authorized !== "boolean"
) {
return null;
}
grants.push({
user_id: grant.user_id,
source_id: grant.source_id,
root_label: text(grant.root_label),
root_path: text(grant.root_path),
authorized: grant.authorized,
revoked_at: text(grant.revoked_at),
sync_enabled: grant.sync_enabled === true,
});
}
return grants;
}
export async function readSync(reader: CoreReader): Promise<SyncDocument> {
const result = validated(
await reader<unknown>("/api/v1/live-sync"),
@@ -216,6 +290,10 @@ export async function readProgress(
summary:
processing.state === "ok" ? projectProcessing(processing.data) : null,
},
pipeline:
processingRead.state === "ok"
? describePipeline(processingRead.data)
: null,
};
}
@@ -231,6 +309,22 @@ export function renderSourcesText(document: SourcesDocument): string {
);
}
if (document.sources.length === 0) lines.push("(没有来源)");
// The list above is the default user's (`local`). Another user's grant,
// a Hermes profile, is listed with its user so it is not mistaken for it.
const others = (document.grants ?? []).filter(
(grant) => grant.user_id !== "local",
);
if (others.length > 0) {
lines.push("其他用户的连接:");
for (const grant of others) {
const name =
document.sources.find((source) => source.source_id === grant.source_id)
?.display_name ?? grant.source_id;
lines.push(
`- ${name}(用户 ${grant.user_id}):${grant.authorized ? "已连接" : grant.revoked_at ? "已撤销" : "未连接"};处理${grant.sync_enabled ? "已开启" : "已冻结"}${grant.root_path ? `;目录 ${grant.root_path}` : ""}`,
);
}
}
lines.push("");
return lines.join("\n");
}
@@ -270,6 +364,18 @@ export function renderProgressText(document: ProgressDocument): string {
? `记忆整理:待处理 ${document.generation.summary.pending} · 进行中 ${document.generation.summary.running} · 等待模型 ${document.generation.summary.waiting_for_model} · 失败 ${document.generation.summary.failed}`
: "记忆整理:暂时无法读取",
);
if (document.pipeline) {
lines.push(`记忆模式:${pipelineLine(document.pipeline)}`);
for (const [userId, user] of Object.entries(
document.pipeline.evolve_users,
)) {
lines.push(
` 进化 ${userId}:${user.last_run_at ?? "尚未进化"}${
user.last_error ? `(失败:${user.last_error})` : ""
}`,
);
}
}
if (document.imports.output_truncated) {
lines.push("这里只显示汇总;完整记录请使用 --json。");
}
+62 -1
View File
@@ -179,12 +179,72 @@ describe("CLI-07/10 source and synchronization reads", () => {
sync_enabled: true,
sync_completeness: "additions_only",
});
expect(requestedPaths()).toEqual(["/api/v1/sources"]);
// The grants list answers 404 here, as an older Core does: the source
// list is shown as before, every row the default user's.
expect(report.sources[0].user_id).toBe("local");
expect(report.grants).toBeUndefined();
expect(requestedPaths()).toEqual([
"/api/v1/sources",
"/api/v1/source-grants",
]);
expect(requestedPaths().some((path) => path.includes("detect"))).toBe(
false,
);
});
it("shows every user's grant, a Hermes profile with its user", async () => {
const grants = {
schema_version: 1,
grants: [
{
user_id: "local",
source_id: "codex",
root_label: ".codex",
root_path: "/home/synthetic/.codex",
authorized: true,
revoked_at: null,
sync_enabled: true,
},
{
user_id: "wife",
source_id: "hermes",
root_label: "wife",
root_path: "/home/synthetic/.hermes/profiles/wife",
authorized: true,
revoked_at: null,
sync_enabled: true,
},
],
};
transport.get.mockImplementation(async (url: string) =>
url === "/api/v1/source-grants"
? { status: 200, data: grants }
: (RESPONSES.get(url) ?? { status: 404, data: {} }),
);
let streams = captureStreams();
try {
expect(await run(["sources", "list", "--json"], {})).toBe(0);
} finally {
streams.restore();
}
const report = JSON.parse(streams.stdout());
expect(report.grants).toEqual(grants.grants);
expect(report.sources[0]).toMatchObject({
source_id: "codex",
user_id: "local",
});
streams = captureStreams();
try {
expect(await run(["sources", "list"], {})).toBe(0);
} finally {
streams.restore();
}
expect(streams.stdout()).toContain("其他用户的连接:");
expect(streams.stdout()).toContain(
"hermes(用户 wife):已连接;处理已开启;目录 /home/synthetic/.hermes/profiles/wife",
);
});
it("keeps degraded synchronization distinct and prints its error code", async () => {
const streams = captureStreams();
try {
@@ -552,6 +612,7 @@ describe("CLI-19 progress and issue reads", () => {
requestedPaths().every((path) =>
[
"/api/v1/sources",
"/api/v1/source-grants",
"/api/v1/live-sync",
"/api/v1/import-jobs",
"/api/v1/memory-processing",
+161 -1
View File
@@ -18,6 +18,7 @@ import {
validateProcessing,
validateSources,
} from "./core-responses.js";
import { describeEmbeddingSwitchSummary } from "./embedding-switch-flow.js";
import { PRODUCT } from "./version.js";
const MIGRATION_ISSUES_URL =
@@ -57,6 +58,9 @@ export interface StatusDocument {
broker?: BackgroundHealth["broker"];
};
sections: Record<string, Section>;
/** R10: the SDK mode and System 2 state; null when the Runtime is not read
* or is older than the setting. */
pipeline?: PipelineSummary | null;
summary: {
state: "ok" | "partial";
degraded: string[];
@@ -64,6 +68,99 @@ export interface StatusDocument {
};
}
export interface PipelineSummary {
mode: "lite" | "pro" | "ultra";
configured_mode: "lite" | "pro" | "ultra";
forced_by_env: boolean;
last_evolve_at: string | null;
evolve_schedule: { enabled: boolean; time: string } | null;
evolve_running: number;
evolve_users: Record<
string,
{
last_run_at: string | null;
last_error: string | null;
state: string | null;
}
>;
graph_build: string | null;
/** SDK features (`memory_features`) whose value differs from the default. */
features: Record<string, boolean>;
}
const MODES = new Set(["lite", "pro", "ultra"]);
/** Projection of `pipeline` in `/api/v1/memory-processing`; null if absent or malformed. */
export function describePipeline(value: unknown): PipelineSummary | null {
if (!value || typeof value !== "object") return null;
const pipeline = (value as { pipeline?: unknown }).pipeline;
if (!pipeline || typeof pipeline !== "object") return null;
const item = pipeline as Record<string, unknown>;
if (
!MODES.has(String(item.mode)) ||
!MODES.has(String(item.configured_mode))
) {
return null;
}
const evolve =
item.evolve && typeof item.evolve === "object"
? (item.evolve as Record<string, unknown>)
: {};
const schedule =
evolve.schedule && typeof evolve.schedule === "object"
? (evolve.schedule as Record<string, unknown>)
: null;
const users: PipelineSummary["evolve_users"] = {};
if (evolve.users && typeof evolve.users === "object") {
for (const [userId, raw] of Object.entries(
evolve.users as Record<string, unknown>,
)) {
const user =
raw && typeof raw === "object" ? (raw as Record<string, unknown>) : {};
const result =
user.last_result && typeof user.last_result === "object"
? (user.last_result as Record<string, unknown>)
: {};
users[userId] = {
last_run_at:
typeof user.last_run_at === "string" ? user.last_run_at : null,
last_error:
typeof user.last_error === "string" ? user.last_error : null,
state: typeof result.state === "string" ? result.state : null,
};
}
}
const features: Record<string, boolean> = {};
if (item.features && typeof item.features === "object") {
for (const [name, value] of Object.entries(
item.features as Record<string, unknown>,
)) {
if (typeof value === "boolean") features[name] = value;
}
}
const graph =
item.graph_build && typeof item.graph_build === "object"
? (item.graph_build as Record<string, unknown>)
: {};
return {
mode: item.mode as PipelineSummary["mode"],
configured_mode: item.configured_mode as PipelineSummary["mode"],
forced_by_env: item.forced_by_env === true,
last_evolve_at:
typeof item.last_evolve_at === "string" ? item.last_evolve_at : null,
evolve_schedule:
schedule &&
typeof schedule.enabled === "boolean" &&
typeof schedule.time === "string"
? { enabled: schedule.enabled, time: schedule.time }
: null,
evolve_running: Array.isArray(evolve.running) ? evolve.running.length : 0,
evolve_users: users,
graph_build: typeof graph.state === "string" ? graph.state : null,
features,
};
}
const SECTION_NAMES = [
"sources",
"sync",
@@ -97,6 +194,7 @@ export async function composeStatusReport(options: {
attachFailure?: string;
}): Promise<StatusDocument> {
const sections: Record<string, Section> = {};
let pipeline: PipelineSummary | null = null;
if (options.reader === null) {
const state: SectionState =
options.attachFailure === undefined ? "not_checked" : "unavailable";
@@ -118,6 +216,8 @@ export async function composeStatusReport(options: {
validateProcessing,
describeProcessing,
);
pipeline =
processing.state === "ok" ? describePipeline(processing.data) : null;
sections.model = section(model, validateModelProfiles, describeModel);
sections.memories = section(
memories,
@@ -169,6 +269,7 @@ export async function composeStatusReport(options: {
},
bridge,
sections,
pipeline,
summary: {
state:
degraded.length === 0 && notChecked.length === 0 ? "ok" : "partial",
@@ -205,6 +306,11 @@ export function renderStatusText(
`${SECTION_LABELS[name]}:${sectionLine(report.sections[name])}`,
);
}
if (report.pipeline) lines.push(`记忆模式:${pipelineLine(report.pipeline)}`);
const embeddingSwitch = describeEmbeddingSwitchSummary(
report.sections.processing?.embedding_switch,
);
if (embeddingSwitch !== null) lines.push(`向量模型:${embeddingSwitch}`);
if (report.summary.state === "partial") {
const notes = [
report.summary.degraded.length > 0
@@ -220,6 +326,36 @@ export function renderStatusText(
return lines.join("\n");
}
export function pipelineLine(pipeline: PipelineSummary): string {
const parts: string[] = [pipeline.mode];
if (pipeline.forced_by_env)
parts.push("(ONE_MEMORY_GRAPH_STORE 强制 ultra)");
if (pipeline.mode === "ultra") {
const schedule = pipeline.evolve_schedule;
parts.push(
schedule?.enabled
? `,每日 ${schedule.time} 自动进化`
: ",自动进化已关闭",
);
parts.push(
pipeline.last_evolve_at
? `,上次进化 ${pipeline.last_evolve_at}`
: ",尚未进化",
);
if (pipeline.evolve_running > 0)
parts.push(`,进行中 ${pipeline.evolve_running}`);
}
const changed = Object.entries(pipeline.features);
if (changed.length > 0) {
parts.push(
`;SDK 功能:${changed
.map(([name, value]) => `${name}=${value ? "开" : "关"}`)
.join("、")}`,
);
}
return parts.join("");
}
function runtimeLine(report: StatusDocument): string {
if (report.runtime.state !== "running") {
return report.runtime.state === "stopped" ? "未运行" : "暂时无法确认";
@@ -234,7 +370,10 @@ function sectionLine(section: Section | undefined): string {
if (section.state === "not_checked") return "未检查";
if (section.state !== "ok") return "暂时不可用,请稍后重试";
return Object.entries(section)
.filter(([key]) => key !== "state" && key !== "code")
.filter(
([key]) =>
key !== "state" && key !== "code" && key !== "embedding_switch",
)
.map(([key, value]) => `${fieldLabel(key)} ${formatValue(value)}`)
.join(" ");
}
@@ -350,9 +489,30 @@ function describeProcessing(
reason: block.reason,
count: block.count,
})),
...embeddingSwitchField(summary),
};
}
/** The running switch or merge, projected; absent when none runs. */
function embeddingSwitchField(
summary: MemoryProcessingSummary,
): Record<string, unknown> {
const value = (summary as unknown as Record<string, unknown>)
.embedding_switch;
if (!value || typeof value !== "object" || Array.isArray(value)) return {};
const item = value as Record<string, unknown>;
const kind = item.kind === "merge" ? "merge" : "switch";
const state =
typeof item.state === "string" && /^[a-z_]{1,32}$/.test(item.state)
? item.state
: "unknown";
const percent =
typeof item.percent === "number" && Number.isFinite(item.percent)
? Math.min(100, Math.max(0, item.percent))
: 0;
return { embedding_switch: { kind, state, percent } };
}
/** Projection, not passthrough: a Core that leaks a key must not make us leak. */
function describeModel(profiles: ModelProfile[]): Record<string, unknown> {
const active = profiles.find((profile) => profile.is_active);
+405
View File
@@ -0,0 +1,405 @@
import { mkdtemp, writeFile } from "node:fs/promises";
import os, { tmpdir } from "node:os";
import path from "node:path";
import { beforeEach, describe, expect, it, vi } from "vitest";
import { run } from "./cli.js";
const transport = vi.hoisted(() => ({
get: vi.fn(),
post: vi.fn(),
put: vi.fn(),
patch: vi.fn(),
delete: vi.fn(),
}));
const direct = vi.hoisted(() => ({ get: vi.fn(), post: vi.fn() }));
vi.mock("axios", () => ({
default: {
create: vi.fn(() => transport),
get: direct.get,
post: direct.post,
isAxiosError: () => false,
},
}));
const runtime = vi.hoisted(() => ({
inspectRuntimeStatus: vi.fn(),
discoverRuntimeAttachment: vi.fn(),
}));
vi.mock("@one-memory/runtime-control", async (importOriginal) => ({
...(await importOriginal<typeof import("@one-memory/runtime-control")>()),
...runtime,
}));
const REQUEST = "33333333-3333-4333-8333-333333333333";
const HERMES = {
source_id: "hermes",
display_name: "Hermes",
storage_kinds: ["sqlite"],
available: true,
authorized: true,
detected: null,
root_label: ".hermes",
candidate_path: "~/.hermes",
candidate_kind: "default",
candidate_revision: 0,
candidate_root_id: "hermes-root-identity",
imported_messages: 0,
imported_conversations: 0,
capabilities: {
historical_import: "implemented",
incremental_import: "implemented",
live_sync: "implemented",
recall: "implemented",
installer: "not_implemented",
verification: "fixture_only",
},
sync_completeness: "additions_only",
location_state: "found",
};
const CODEX = {
...HERMES,
source_id: "codex",
display_name: "Codex",
candidate_root_id: "codex-root-identity",
};
function addReceipt() {
return {
evidence_id: "44444444-4444-4444-8444-444444444444",
text: "Prefers tea",
scope: "all-apps",
created_at: "2026-10-27T00:00:00Z",
state: "pending",
task_id: null,
memory_ids: [],
error: null,
};
}
beforeEach(() => {
vi.clearAllMocks();
for (const method of [
...Object.values(transport),
...Object.values(direct),
]) {
method.mockReset();
}
runtime.inspectRuntimeStatus.mockResolvedValue({
state: "running",
code: "runtime_verified",
instanceId: "instance-1",
host: "127.0.0.1",
port: 8765,
version: "1.2.3",
});
runtime.discoverRuntimeAttachment.mockResolvedValue({
instanceId: "instance-1",
apiToken: "synthetic-runtime-token",
host: "127.0.0.1",
port: 8765,
servicePid: 4242,
health: {
status: "ok",
service: "one-memory-core",
version: "1.2.3",
instance_id: "instance-1",
},
});
transport.get.mockImplementation(async (url: string) => {
if (url === "/api/v1/sources") {
return { status: 200, data: [HERMES, CODEX] };
}
if (url === "/api/v2/user-profile") {
return {
status: 200,
data: { request_id: "r-1", profile: null, elapsed_ms: 1 },
};
}
return { status: 404, data: {} };
});
transport.post.mockImplementation(
async (url: string, body: Record<string, unknown>) =>
url === "/api/v1/sources/connect"
? {
status: 200,
data: {
request_id: body.request_id,
accepted_source_ids: ["hermes"],
failed_sources: {},
jobs: [],
},
}
: url === "/api/v2/memories"
? { status: 201, data: addReceipt() }
: { status: 404, data: {} },
);
transport.put.mockResolvedValue({ status: 200, data: {} });
transport.delete.mockResolvedValue({ status: 204, data: "" });
direct.post.mockResolvedValue({
status: 202,
data: {
request_id: REQUEST,
state: "accepted",
created_at: "T1",
updated_at: "T1",
},
});
direct.get.mockResolvedValue({
data: {
request_id: REQUEST,
state: "accepted",
created_at: "T1",
updated_at: "T1",
},
});
});
describe("--user-id on the memory commands", () => {
it("passes the user as a query parameter to memory reads and deletes", async () => {
await capturedRun(["memories", "list", "--user-id", "wife", "--json"]);
expect(transport.get).toHaveBeenCalledWith("/api/v2/memory-library", {
params: { offset: 0, limit: 100, user_id: "wife" },
});
// Global: the flag may come before the subcommand too.
await capturedRun(["memories", "--user-id", "wife", "show", "m-1"]);
expect(transport.get).toHaveBeenCalledWith("/api/v2/memories/m-1", {
params: { user_id: "wife" },
});
await capturedRun(["memories", "add-status", REQUEST, "--user-id", "wife"]);
expect(transport.get).toHaveBeenCalledWith(
`/api/v2/memories/additions/${REQUEST}`,
{ params: { user_id: "wife" } },
);
await capturedRun(["profile", "--user-id", "wife", "--json"]);
expect(transport.get).toHaveBeenCalledWith("/api/v2/user-profile", {
params: { user_id: "wife" },
});
await capturedRun([
"memories",
"delete",
"fact-1",
"--kind",
"fact",
"--yes",
"--user-id",
"wife",
]);
expect(transport.delete).toHaveBeenCalledWith(
"/api/v2/memories/fact-1?kind=fact&user_id=wife",
undefined,
);
await capturedRun([
"memories",
"delete",
"m-1",
"--yes",
"--user-id",
"wife",
]);
expect(transport.delete).toHaveBeenCalledWith(
"/api/v2/memories/m-1?user_id=wife",
undefined,
);
await capturedRun(["write", "status", REQUEST, "--user-id", "wife"]);
expect(direct.get.mock.calls[0][0]).toBe(
`http://127.0.0.1:8765/api/v2/memory-writes/${REQUEST}`,
);
expect(direct.get.mock.calls[0][1].params).toEqual({ user_id: "wife" });
});
it("passes the user in the body of memory writes and searches", async () => {
await capturedRun(["memories", "add", "Prefers tea", "--user-id", "wife"]);
expect(transport.post.mock.calls[0][0]).toBe("/api/v2/memories");
expect(transport.post.mock.calls[0][1]).toMatchObject({
text: "Prefers tea",
scope: "all-apps",
user_id: "wife",
});
const directory = await mkdtemp(path.join(tmpdir(), "om-user-id-"));
const content = path.join(directory, "content.txt");
await writeFile(content, "Prefers green tea\n");
await capturedRun([
"memories",
"update",
"m-1",
"--content-file",
content,
"--user-id",
"wife",
]);
expect(transport.put.mock.calls[0][0]).toBe("/api/v2/memories/m-1");
expect(transport.put.mock.calls[0][1]).toMatchObject({ user_id: "wife" });
await capturedRun(["recall", "tea", "--user-id", "wife", "--json"]);
const search = transport.post.mock.calls.find(
([url]) => url === "/api/v2/context/search",
);
expect(search?.[1]).toMatchObject({ query: "tea", user_id: "wife" });
const messages = path.join(directory, "messages.json");
await writeFile(
messages,
JSON.stringify([{ role: "user", content: "I like tea" }]),
);
await capturedRun([
"write",
"submit",
"--messages-file",
messages,
"--session",
"s-1",
"--user-id",
"wife",
]);
expect(direct.post.mock.calls[0][0]).toBe(
"http://127.0.0.1:8765/api/v2/memory-writes",
);
expect(direct.post.mock.calls[0][1]).toMatchObject({
session_id: "s-1",
user_id: "wife",
});
});
it("names no user by default or for --user-id local", async () => {
for (const extra of [[], ["--user-id", "local"]]) {
transport.get.mockClear();
transport.post.mockClear();
await capturedRun(["memories", "list", ...extra, "--json"]);
expect(transport.get).toHaveBeenCalledWith("/api/v2/memory-library", {
params: { offset: 0, limit: 100 },
});
await capturedRun(["profile", ...extra]);
expect(transport.get).toHaveBeenCalledWith(
"/api/v2/user-profile",
undefined,
);
await capturedRun(["memories", "add", "Prefers tea", ...extra]);
expect(transport.post.mock.calls[0][1]).not.toHaveProperty("user_id");
await capturedRun(["recall", "tea", ...extra, "--json"]);
const search = transport.post.mock.calls.find(
([url]) => url === "/api/v2/context/search",
);
expect(search?.[1]).not.toHaveProperty("user_id");
}
});
it("refuses a malformed user id before contacting Core", async () => {
for (const value of ["bad:id", "", "x".repeat(65), "a b", "--json"]) {
const result = await capturedRun([
"memories",
"list",
"--user-id",
value,
]);
expect(result.code).toBe(2);
}
expect(
(await capturedRun(["profile", "--user-id", "a", "--user-id", "b"])).code,
).toBe(2);
expect((await capturedRun(["recall", "tea", "--user-id"])).code).toBe(2);
expect(transport.get).not.toHaveBeenCalled();
expect(transport.post).not.toHaveBeenCalled();
});
});
describe("sources connect for a Hermes profile", () => {
it("connects hermes for another user from its own root", async () => {
const result = await capturedRun([
"sources",
"connect",
"hermes",
"--user-id",
"wife",
"--root",
"~/.hermes/profiles/wife",
"--json",
]);
expect(result.code).toBe(0);
expect(transport.post).toHaveBeenCalledTimes(1);
const [url, body] = transport.post.mock.calls[0];
expect(url).toBe("/api/v1/sources/connect");
expect(body).toMatchObject({
source_ids: ["hermes"],
// Core reads the custom root's identity itself.
source_roots: {},
source_candidate_revisions: {},
onboarding: false,
user_id: "wife",
root: path.join(os.homedir(), ".hermes/profiles/wife"),
});
});
it("keeps a plain connect unchanged and accepts --import with a profile", async () => {
await capturedRun(["sources", "connect", "hermes", "--import", "--json"]);
const plain = transport.post.mock.calls[0][1];
expect(plain).not.toHaveProperty("user_id");
expect(plain).not.toHaveProperty("root");
expect(plain).toMatchObject({
source_roots: { hermes: "hermes-root-identity" },
onboarding: true,
});
transport.post.mockClear();
const result = await capturedRun([
"sources",
"connect",
"hermes",
"--import",
"--user-id",
"wife",
"--root",
"/profiles/wife",
"--request-id",
REQUEST,
"--json",
]);
expect(result.code).toBe(0);
expect(transport.post.mock.calls[0][1]).toMatchObject({
request_id: REQUEST,
user_id: "wife",
root: "/profiles/wife",
});
});
it("refuses --user-id and --root for any source but hermes", async () => {
for (const argv of [
["sources", "connect", "codex", "--user-id", "wife"],
["sources", "connect", "codex", "--root", "/somewhere", "--import"],
["sources", "connect", "hermes", "codex", "--user-id", "wife"],
["sources", "connect", "--all-detected", "--user-id", "wife"],
["sources", "connect", "hermes", "--user-id", "bad:id"],
]) {
const result = await capturedRun(argv);
expect(result.code).toBe(2);
expect(result.stderr).toContain("hermes");
}
expect(transport.post).not.toHaveBeenCalled();
});
});
async function capturedRun(arguments_: string[]) {
let stdout = "";
let stderr = "";
const stdoutSpy = vi
.spyOn(process.stdout, "write")
.mockImplementation((chunk) => {
stdout += String(chunk);
return true;
});
const stderrSpy = vi
.spyOn(process.stderr, "write")
.mockImplementation((chunk) => {
stderr += String(chunk);
return true;
});
try {
return {
code: await run(arguments_, {
ONE_MEMORY_DATA_DIR: "/synthetic/data",
}),
stdout,
stderr,
};
} finally {
stdoutSpy.mockRestore();
stderrSpy.mockRestore();
}
}
+1 -1
View File
@@ -12,7 +12,7 @@ dependencies = [
"fastapi>=0.141.1",
"filelock>=3.32.4",
# Public SDK version validated against MemHub's permission and recall contracts.
"hy-memory==2.0.7",
"hy-memory==2.0.10",
# chromadb arrives through the HY SDK; locked here so an unrelated upgrade
# cannot move the SDK's store. Built-in embedding consumes numpy directly.
"chromadb==1.5.9",
File diff suppressed because it is too large. Load diff
+29 -5
View File
@@ -35,7 +35,7 @@ import re # noqa: E402
import time # noqa: E402
from collections.abc import Iterator, Sequence # noqa: E402
from pathlib import Path # noqa: E402
from typing import TextIO # noqa: E402
from typing import Any, TextIO # noqa: E402
from one_memory.config import load_settings # noqa: E402
from one_memory.runtime_client import RuntimeStatus, inspect_runtime # noqa: E402
@@ -584,6 +584,7 @@ def _sdk_migration_status(options: Sequence[str], stream: TextIO) -> int:
}
else:
from one_memory.hy_runtime_config import HY_MEMORY_NAMESPACE
from one_memory.hy_sdk_runtime import builtin_retrieval_snapshot
from one_memory.sdk_data_plane_migration import (
MemoryAuthority,
SdkDataPlaneMigrationStore,
@@ -595,19 +596,27 @@ def _sdk_migration_status(options: Sequence[str], stream: TextIO) -> int:
authority = SdkDataPlaneMigrationStore(database_path).authority()
if authority is MemoryAuthority.SDK:
profile = read_active_profile(database_path)
profile = read_active_profile(database_path) or builtin_retrieval_snapshot()
leftovers = bool(
legacy_tables_present(database_path)
or abandoned_staging_present(
data_dir=settings.data_dir,
namespace=HY_MEMORY_NAMESPACE,
)
or (profile is not None and legacy_vector_collections(settings.data_dir, profile))
or legacy_vector_collections(settings.data_dir, profile)
)
state = "required" if leftovers else "not_required"
else:
state = "required"
document = {"schema_version": 1, "authority": authority.value, "state": state}
from one_memory.embedding_switch import read_open_job
# Informational. An unfinished embedding switch never makes a
# migration required or blocks it: the upgrade installs, `sdk-migrate`
# leaves the switch's tables alone, and the new Core resumes the job.
switch = read_open_job(database_path)
if switch is not None:
document["embedding_switch"] = switch
from one_memory.sdk_migration_log import write_migration_log
write_migration_log(
@@ -615,7 +624,15 @@ def _sdk_migration_status(options: Sequence[str], stream: TextIO) -> int:
writer="core-sdk-migration-status",
event="status.probed",
outcome="succeeded",
details={"authority": document["authority"], "state": document["state"]},
details={
"authority": document["authority"],
"state": document["state"],
**(
{"embedding_switch": document["embedding_switch"]}
if "embedding_switch" in document
else {}
),
},
)
if as_json:
stream.write(f"{json.dumps(document, ensure_ascii=False)}\n")
@@ -695,6 +712,12 @@ def _refuse_unmigrated_home(data_dir: Path) -> None:
raise SystemExit(SDK_MIGRATION_REQUIRED_EXIT)
def _without_embedding_switch(settings: Any) -> Any:
"""The maintenance Core's settings: it never resumes or runs a switch."""
copy = getattr(settings, "model_copy", None)
return copy(update={"embedding_switch_worker": False}) if callable(copy) else settings
def serve(*, maintenance: bool = False) -> None:
import uvicorn
@@ -739,7 +762,8 @@ def serve(*, maintenance: bool = False) -> None:
if launcher is not None:
ParentWatchdog(launcher).start()
application = create_app(
settings,
# Only the serving Core runs or resumes an embedding switch.
_without_embedding_switch(settings) if maintenance else settings,
manage_runtime=not maintenance,
)
if maintenance:
@@ -45,6 +45,11 @@ class Settings(BaseSettings):
model_request_limit: int | None = Field(default=None, ge=1, le=100_000)
model_request_body_byte_limit: int = Field(default=24_576, ge=1)
model_request_output_token_limit: int = Field(default=16_384, ge=1)
# The embedding switch worker (and the startup merge). The maintenance
# Core turns it off: only the serving Core may resume or run a switch.
embedding_switch_worker: bool = True
# How long a switch waits for model writes already past their claim.
embedding_switch_drain_timeout_seconds: float = Field(default=600, gt=0)
@field_validator("development_origin")
@classmethod
@@ -5,6 +5,9 @@ from typing import Annotated, Literal
from pydantic import BaseModel, ConfigDict, Field, model_validator
from one_memory.sdk_search_options import SdkSearchOptions
from one_memory.user_scope import DEFAULT_USER_ID, UserId
ContextKind = Literal["raw", "memory"]
ContextSearchChannel = Literal["lexical", "semantic"]
ContextSemanticStatus = Literal["not_searched", "completed", "failed", "timed_out"]
@@ -47,17 +50,24 @@ class ContextSearchRequest(BaseModel):
return self
class SdkContextSearchRequest(ContextSearchRequest):
class SdkContextSearchRequest(ContextSearchRequest, SdkSearchOptions):
"""`POST /api/v2/context/search`: the three SDK recall lanes are set apart.
`limit` is the normal lane's top-k and the cap on returned items;
`schema_limit` and `intention_limit` are the SDK's schema (L6) and
proactive intention (L7) lanes. A caller's values are passed through.
The `SdkSearchOptions` fields reach the memory lane's `client.search`;
the raw lane takes the floor and the source, session, agent and time
filters (never wider than the caller's raw authority). Absent, both
lanes make the call they always made.
"""
limit: int = Field(default=10, ge=1, le=20, strict=True)
schema_limit: int = Field(default=3, ge=0, le=20, strict=True)
intention_limit: int = Field(default=1, ge=0, le=20, strict=True)
#: Whose memories to search. The App never sends it and reads `local`.
user_id: UserId = DEFAULT_USER_ID
class ContextMessage(BaseModel):
+105 -25
View File
@@ -19,18 +19,63 @@ from one_memory.context_models import (
SdkMemoryContextItem,
SdkRawContextItem,
)
from one_memory.embedding_health import EMBEDDING_UNAVAILABLE, EmbeddingUnavailableError
from one_memory.external_memory_access import external_access
from one_memory.hy_runtime_config import HY_MEMORY_READER
from one_memory.hy_sdk_runtime import sdk_relevance_floor
from one_memory.hy_sdk_types import (
HySdkMemoryRecord,
parse_sdk_lane_search,
parse_sdk_raw_search,
parse_sdk_search,
)
from one_memory.imports.storage import EvidenceStore
from one_memory.sdk_search_options import SdkSearchFilters, SearchLane
from one_memory.semantic_deadline import SemanticMemoryHit
from one_memory.user_scope import owns
CALL_EVENT_RESPONSE_BUDGET_MS = 250
#: Pass-through filters the raw lane honours. Scene, layers, reader and the
#: schema/working floors describe memory lanes; the raw lane is always the
#: chat store's L1 raw layer.
_RAW_LANE_FILTERS = ("created_after", "session_ids", "agent_ids", "exclude_sources", "rerank")
def raw_lane_sources(
allowed: Sequence[str] | frozenset[str] | None,
requested: Sequence[str] | None,
) -> list[str] | None:
"""The raw lane's source filter: the caller's, never wider than its authority.
`allowed` empty or None means every source (the call Core always made).
None back means nothing is left to search.
"""
allowed_list = list(allowed or ())
if requested is None:
return allowed_list
chosen = [
source for source in dict.fromkeys(requested) if not allowed_list or source in allowed_list
]
return chosen or None
def raw_lane_overrides(value: SdkSearchFilters) -> dict[str, object]:
"""The caller's raw-lane overrides: its floor and its row filters."""
overrides = value.filter_kwargs()
found = {name: overrides[name] for name in _RAW_LANE_FILTERS if name in overrides}
floor = value.lane_floor("raw")
if floor is not None:
found["min_score"] = floor
return found
def hit_namespace(hit: SemanticMemoryHit) -> dict[str, str]:
"""The `get` namespace of a hit: nothing for chat, kind and variant otherwise."""
if hit.kind in {"experience", "fact"} and hit.variant is not None:
return {"kind": hit.kind, "variant": hit.variant}
return {}
def sdk_context_router(dependencies: Sequence[Depends]) -> APIRouter:
"""SDK-native context: no Evidence/Governance ids or fabricated revisions."""
@@ -66,26 +111,36 @@ def sdk_context_router(dependencies: Sequence[Depends]) -> APIRouter:
client = await client_handle.client()
except Exception:
raise HTTPException(503, "sdk_context_unavailable") from None
memory_mode = getattr(request.app.state, "memory_mode", None)
lite = callable(memory_mode) and memory_mode() == "lite"
async def raw_lane() -> tuple[SdkRawContextItem, ...]:
if "raw" not in value.kinds or not authority.raw_allowed:
return ()
sources = raw_lane_sources(authority.raw_sources, value.include_sources)
if sources is None:
return ()
raw = await asyncio.to_thread(
client.search,
value.query,
user_ids=["local"],
agent_ids=[],
include_sources=list(authority.raw_sources or ()),
limit=value.limit,
min_score=0.0,
schema_min_score=0.0,
schema_limit=0,
intention_limit=0,
scene="chat",
reader=HY_MEMORY_READER,
layers=["l1_raw"],
**{
"user_ids": [value.user_id],
"agent_ids": [],
"include_sources": sources,
"limit": value.limit,
"min_score": 0.0,
"schema_min_score": 0.0,
"schema_limit": 0,
"intention_limit": 0,
"scene": "chat",
"reader": HY_MEMORY_READER,
"layers": ["l1_raw"],
**raw_lane_overrides(value),
},
)
hits = tuple(
hit for hit in parse_sdk_raw_search(raw) if value.passes_floor("raw", hit.score)
)
hits = parse_sdk_raw_search(raw)
return tuple(
SdkRawContextItem(
raw_id=hit.raw_id,
@@ -103,27 +158,48 @@ def sdk_context_router(dependencies: Sequence[Depends]) -> APIRouter:
)
async def memory_lane() -> tuple[SdkMemoryContextItem, ...]:
if "memory" not in value.kinds or not authority.memory_allowed:
# lite extracts nothing: its search answers raw turns, which the
# raw lane already reads under the raw authority.
if "memory" not in value.kinds or not authority.memory_allowed or lite:
return ()
overrides = value.search_overrides()
raw = await asyncio.to_thread(
client.search,
value.query,
user_ids=["local"],
agent_ids=[],
limit=value.limit,
min_score=sdk_relevance_floor(),
schema_min_score=sdk_relevance_floor(),
schema_limit=value.schema_limit,
intention_limit=value.intention_limit,
scene="normal",
reader=HY_MEMORY_READER,
**{
"user_ids": [value.user_id],
"agent_ids": [],
"limit": value.limit,
"min_score": sdk_relevance_floor(),
"schema_min_score": sdk_relevance_floor(),
"schema_limit": value.schema_limit,
"intention_limit": value.intention_limit,
"scene": "normal",
"reader": HY_MEMORY_READER,
**overrides,
},
)
hits = (
tuple(
hit
for hit in parse_sdk_lane_search(
raw, error_code="hy_memory_search_shape_unknown"
)
if value.passes_floor(cast(SearchLane, hit.lane), hit.score)
)
if overrides
else parse_sdk_search(raw, error_code="hy_memory_search_shape_unknown")
)
hits = parse_sdk_search(raw, error_code="hy_memory_search_shape_unknown")
items: list[SdkMemoryContextItem] = []
for hit in hits:
record = await asyncio.to_thread(client.get, hit.memory_id)
record = await asyncio.to_thread(client.get, hit.memory_id, **hit_namespace(hit))
memory = None if record is None else HySdkMemoryRecord.from_sdk(record)
if memory is None or memory.status != "active":
# `get` takes no user: a hit is kept only if it is the caller's.
if (
memory is None
or memory.status != "active"
or not owns(memory.user_id, value.user_id)
):
continue
items.append(
SdkMemoryContextItem(
@@ -188,6 +264,10 @@ def sdk_context_router(dependencies: Sequence[Depends]) -> APIRouter:
)
await record(final, timed_out=True)
return final
for lane in (raw_items, memory_items):
if isinstance(lane, EmbeddingUnavailableError):
# No lexical fallback: the query could not be embedded.
raise HTTPException(503, EMBEDDING_UNAVAILABLE)
partial = isinstance(raw_items, Exception) or isinstance(memory_items, Exception)
raw_candidates = (
() if isinstance(raw_items, Exception) else cast(tuple[SdkContextItem, ...], raw_items)
@@ -0,0 +1,180 @@
"""What the embedding switch needs from the SDK's re-embed module, in Core terms.
The switch worker, the startup merge and the retention sweep only talk to a
`ReembedBackend`. The production one (`hy_sdk_migration.SdkReembedBackend`)
is the single place that calls `hy_memory`'s re-embed functions; tests use
an in-memory one. Every space is named explicitly -- collection names and
graph paths -- and nothing here derives a name from a provider or endpoint.
"""
from __future__ import annotations
from collections.abc import Callable, Mapping
from dataclasses import dataclass, field
from pathlib import Path
from typing import Protocol
from one_memory.embedding_spaces import EmbeddingSpace, space_collection_names
from one_memory.model_profiles import ModelProfileSnapshot
class ReembedUnavailableError(RuntimeError):
"""The installed SDK has no re-embed module (before hy-memory 2.0.10)."""
def __init__(self, code: str = "sdk_reembed_unavailable") -> None:
super().__init__(code)
self.code = code
class ReembedCancelledError(RuntimeError):
"""A cancel was seen at a batch boundary; every finished batch is written."""
class EmbeddingWidthError(RuntimeError):
"""The embedding service returned vectors of another width than the target's."""
@dataclass(frozen=True)
class PhysicalSpace:
"""One space's collections, named explicitly."""
collection_base: str
name_dims: int | None
#: The width of the vectors in it, or to be written to it.
dims: int | None
model: str | None = None
#: Explicit role -> collection name; None derives them from the base.
collections: Mapping[str, str] | None = None
def names(self) -> dict[str, str]:
if self.collections is not None:
return dict(self.collections)
return space_collection_names(self.collection_base, self.name_dims)
@classmethod
def of(cls, space: EmbeddingSpace) -> PhysicalSpace:
return cls(
collection_base=space.collection_base,
name_dims=space.name_dims,
dims=space.dims if space.dims is not None else space.name_dims,
model=space.model,
)
@dataclass(frozen=True)
class ScannedSpace:
"""A space found in the vector store, grouped from its collection names."""
collection_base: str
name_dims: int | None
#: The stored width when every non-empty role agrees.
width: int | None
counts: Mapping[str, int]
collections: Mapping[str, str]
per_user: Mapping[str, int] = field(default_factory=dict)
@property
def vectors(self) -> int:
return sum(self.counts.values())
@property
def key(self) -> tuple[str, int]:
return (self.collection_base, self.name_dims or 0)
def physical(self) -> PhysicalSpace:
return PhysicalSpace(
collection_base=self.collection_base,
name_dims=self.name_dims,
dims=self.width if self.width is not None else self.name_dims,
collections=dict(self.collections),
)
@dataclass(frozen=True)
class ReembedStep:
"""Progress of one role (vectors) or table (graph)."""
phase: str
role: str
done: int
total: int
embedded: int = 0
reused: int = 0
kept: int = 0
@dataclass(frozen=True)
class VectorCheck:
per_role: Mapping[str, Mapping[str, int]]
#: Target records per metadata `user_id`.
per_user: Mapping[str, int]
problems: tuple[str, ...]
warnings: tuple[str, ...] = ()
self_recall_hits: int = 0
self_recall_samples: int = 0
@dataclass(frozen=True)
class GraphCheck:
ok: bool
problems: tuple[str, ...] = ()
@dataclass(frozen=True)
class EmbeddingTarget:
"""The embedding that fills a space: the profile to reach it, and its key."""
profile: ModelProfileSnapshot
space: EmbeddingSpace
api_key: str | None
Cancel = Callable[[], bool]
OnStep = Callable[[ReembedStep], None]
class ReembedBackend(Protocol):
def unavailable_reason(self) -> str | None:
"""None when re-embedding works; otherwise the code that says why not."""
async def scan(self, *, per_user: bool = False) -> tuple[ScannedSpace, ...]:
"""Every space with collections in the home's vector store. Read-only."""
async def diff(self, source: PhysicalSpace, target: PhysicalSpace) -> dict[str, int]:
"""Per role, how many ids the source holds that the target does not."""
async def reembed(
self,
source: PhysicalSpace,
target: PhysicalSpace,
*,
embedding: EmbeddingTarget,
cancel: Cancel,
on_step: OnStep,
batch_size: int = 64,
) -> None:
"""Embed every source record the target lacks into the target. Target wins."""
async def verify(
self, source: PhysicalSpace, target: PhysicalSpace, *, embedding: EmbeddingTarget
) -> VectorCheck: ...
async def graph_vectors(self, path: Path) -> int | None:
"""Embeddings a graph holds, or None when there is no graph there."""
async def reembed_graph(
self,
source: Path,
target: Path,
*,
embedding: EmbeddingTarget,
target_dims: int,
cancel: Cancel,
on_step: OnStep,
) -> None: ...
async def verify_graph(self, source: Path, target: Path, *, target_dims: int) -> GraphCheck: ...
async def drop(self, space: PhysicalSpace) -> None: ...
async def drop_graph(self, path: Path) -> None: ...
@@ -0,0 +1,147 @@
"""`embedding_unavailable`: a search that could not embed its query says so.
With a remote embedding model, every search embeds the query through that
service. When the service is down, the SDK swallows the failure and answers
with no hits, which reads as "you have no such memory". There is no lexical
fallback (owner decision): the search fails with the explicit code
`embedding_unavailable`, the API answers 503 with that code, and the App
shows a toast.
The check is only made when a search came back empty or raised, so a working
service costs nothing on searches that find something. It sends one tiny
embedding request to the same endpoint, model and width the read client
uses; the answer is remembered briefly so a burst of searches sends one.
"""
from __future__ import annotations
import threading
import time
from collections.abc import Callable, Mapping
from typing import Any
import httpx
from one_memory.http_client import create_client
from one_memory.hy_sdk_types import HySdkError
from one_memory.model_profiles import ModelProfileSnapshot, ProviderKind
EMBEDDING_UNAVAILABLE = "embedding_unavailable"
_OK_TTL_S = 30.0
_FAILED_TTL_S = 10.0
_TIMEOUT_S = 8.0
def _no_transport() -> httpx.BaseTransport | None:
return None
#: Tests replace this to answer the check without a network.
transport_factory: Callable[[], httpx.BaseTransport | None] = _no_transport
class EmbeddingUnavailableError(HySdkError):
"""The live space's embedding service did not answer; search cannot run."""
def __init__(self) -> None:
super().__init__(EMBEDDING_UNAVAILABLE)
class EmbeddingHealth:
"""Whether a remote embedding answers, remembered for a few seconds."""
def __init__(self) -> None:
self._lock = threading.Lock()
self._known: dict[tuple[str, str, int | None], tuple[bool, float]] = {}
def available(
self,
profile: ModelProfileSnapshot,
*,
endpoint: str | None,
name_dims: int | None,
api_key: str | None,
) -> bool:
provider = profile.embedding_provider_kind or profile.provider_kind
if provider is ProviderKind.LOCAL_ONNX or endpoint is None:
return True
model = profile.embedding_model or ""
key = (endpoint, model, name_dims)
now = time.monotonic()
with self._lock:
known = self._known.get(key)
if known is not None and known[1] > now:
return known[0]
ok = _answers(
endpoint,
model=model,
name_dims=name_dims,
api_key="ollama" if provider is ProviderKind.OLLAMA else api_key,
)
with self._lock:
self._known[key] = (ok, now + (_OK_TTL_S if ok else _FAILED_TTL_S))
return ok
def forget(self) -> None:
with self._lock:
self._known.clear()
def _answers(endpoint: str, *, model: str, name_dims: int | None, api_key: str | None) -> bool:
body: dict[str, Any] = {"model": model, "input": ["ping"]}
if name_dims:
body["dimensions"] = name_dims
headers = {"Authorization": f"Bearer {api_key}"} if api_key else {}
transport = transport_factory()
try:
with create_client(
timeout=_TIMEOUT_S,
**({"transport": transport} if transport is not None else {}),
) as client:
response = client.post(f"{endpoint.rstrip('/')}/embeddings", json=body, headers=headers)
if response.status_code != 200:
return False
data = response.json().get("data")
return bool(isinstance(data, list) and data and data[0].get("embedding"))
except (httpx.HTTPError, ValueError, AttributeError):
return False
def has_hits(result: object) -> bool:
"""Whether an SDK search result holds any hit, whatever its scene's shape."""
if not isinstance(result, Mapping):
return bool(result)
for key in ("memories", "facts", "experiences"):
value = result.get(key)
if isinstance(value, list) and value:
return True
if isinstance(value, Mapping) and any(
isinstance(item, list) and item for item in value.values()
):
return True
return False
class EmbeddingGuardedClient:
"""An SDK client whose `search` reports a dead embedding instead of nothing.
Everything else is the client's own attribute, unchanged.
"""
def __init__(self, client: Any, available: Callable[[], bool]) -> None:
self._client = client
self._available = available
def search(self, *args: Any, **kwargs: Any) -> Any:
try:
result = self._client.search(*args, **kwargs)
except Exception:
if not self._available():
raise EmbeddingUnavailableError() from None
raise
if not has_hits(result) and not self._available():
raise EmbeddingUnavailableError()
return result
def __getattr__(self, name: str) -> Any:
return getattr(self._client, name)
@@ -0,0 +1,554 @@
"""Which embedding space the home reads and writes: an explicit registry.
The SDK names its vector collections after `vector_store.collection_name`
and the configured width, so the space used to follow whatever the active
profile hashed to (`_collection_base`). Any change to that identity -- a
probe recording the measured width, a new endpoint -- silently opened new,
empty collections beside the old ones, and nothing carried the memories over.
The live space is pinned here instead, in the main Core database beside the
model profiles, so a switch commits the new space and the profile that fills
it in one transaction. The identity of a space is its embedding model and
width only: a new endpoint or provider serving the same model at the same
width keeps the space.
Existing homes keep their collections byte-for-byte. The first start registers
the space the active profile (or the built-in model) already opened under its
existing four-attribute name; only spaces created by a switch use the
model-and-width name. A space whose width was never declared ("auto") keeps
its unsuffixed name and only records the width it was measured at.
"""
from __future__ import annotations
import hashlib
import sqlite3
from collections.abc import Iterator, Mapping
from contextlib import contextmanager
from datetime import UTC, datetime, timedelta
from pathlib import Path
from typing import Literal
from uuid import uuid4
from pydantic import BaseModel, ConfigDict
from one_memory.database import take_the_write_lock_first
from one_memory.database_access import main_database
from one_memory.embedding_assets import BUILTIN_EMBEDDING_DIMENSIONS, BUILTIN_EMBEDDING_MODEL
from one_memory.memory_server.migrations import embedding_spaces_ddl
from one_memory.model_profiles import ModelProfileSnapshot, ProviderKind
#: How long a space replaced by a switch or folded in by a merge is kept.
RETIRED_SPACE_RETENTION = timedelta(days=14)
SpaceState = Literal["live", "building", "stray", "retired", "dropped"]
SpaceNaming = Literal["legacy", "v2", "discovered"]
#: Every vector collection one space consists of, as the SDK names them.
VECTOR_ROLES = (
"memory",
"entities",
"coding_keys",
"coding_keys_cv",
"working_facts",
"working_facts_cv",
)
#: The graph directory of the first space; later spaces get their own.
DEFAULT_GRAPH_DIR = "graph"
EMBEDDING_SPACES_SCHEMA = embedding_spaces_ddl()
#: Bumped by every registry write in this process, so cached live rows are
#: reread after a change made through any connection (a probe's, a commit's).
_CHANGES = [0]
def _changed() -> None:
_CHANGES[0] += 1
class EmbeddingSpaceError(RuntimeError):
"""A registry refusal with a stable, product-facing code."""
def __init__(self, code: str) -> None:
super().__init__(code)
self.code = code
class EmbeddingIdentity(BaseModel):
"""What makes two vectors comparable: the model and the width.
`dims=None` is a width nobody declared or measured yet; it matches any
width, because the collection it names was opened without one.
"""
model_config = ConfigDict(frozen=True, extra="forbid")
model: str
dims: int | None = None
def matches(self, other: EmbeddingIdentity) -> bool:
if self.model != other.model:
return False
return self.dims is None or other.dims is None or self.dims == other.dims
class EmbeddingSpace(BaseModel):
"""One embedding space: the physical collections and graph one model fills."""
model_config = ConfigDict(frozen=True, extra="forbid")
space_id: str
state: SpaceState
#: None only for a space found on disk that no registry row named.
model: str | None = None
#: The vector width stored in this space, once known.
dims: int | None = None
#: The width the collection names and embedding requests carry. None is
#: an "auto" space: unsuffixed names and no `dimensions` request field.
name_dims: int | None = None
collection_base: str
naming: SpaceNaming = "legacy"
graph_dir: str | None = DEFAULT_GRAPH_DIR
provider_kind: str | None = None
#: The profile version whose embedding fills this space; None = built-in.
profile_id: str | None = None
profile_revision: int | None = None
#: Vectors counted the last time the space was scanned.
vectors: int | None = None
created_at: str
live_since: str | None = None
retired_at: str | None = None
keep_until: str | None = None
dropped_at: str | None = None
@property
def identity(self) -> EmbeddingIdentity | None:
if self.model is None:
return None
return EmbeddingIdentity(model=self.model, dims=self.dims)
@property
def builtin(self) -> bool:
return self.provider_kind == ProviderKind.LOCAL_ONNX.value and self.profile_id is None
def collection_names(self) -> dict[str, str]:
return space_collection_names(self.collection_base, self.name_dims)
def space_collection_names(base: str, name_dims: int | None) -> dict[str, str]:
"""The SDK's physical collection name per role for one space.
Mirrors `VectorStoreBase` (the `_<dims>` suffix), the entity store and the
coding stores. The SDK's own `vector_space_names` is checked against this
in the migration adapter when the installed SDK provides it.
"""
suffix = f"_{name_dims}" if name_dims else ""
return {
"memory": f"{base}{suffix}",
"entities": f"{base}{suffix}_entities",
"coding_keys": f"{base}_coding_keys{suffix}",
"coding_keys_cv": f"{base}_coding_keys_cv{suffix}",
"working_facts": f"{base}_working_facts{suffix}",
"working_facts_cv": f"{base}_working_facts_cv{suffix}",
}
def embedding_identity(profile: ModelProfileSnapshot) -> EmbeddingIdentity:
"""The embedding identity a profile asks for: model name and width only."""
provider = profile.embedding_provider_kind or profile.provider_kind
local = provider is ProviderKind.LOCAL_ONNX
model = profile.embedding_model or (BUILTIN_EMBEDDING_MODEL if local else None)
if not model:
raise EmbeddingSpaceError("hy_sdk_embedding_model_required")
dims = profile.embedding_dimensions
if local and dims is None:
dims = BUILTIN_EMBEDDING_DIMENSIONS
return EmbeddingIdentity(model=model, dims=dims)
def v2_collection_base(identity: EmbeddingIdentity, generation: int = 0) -> str:
"""The base name a switch gives a new space: model and width, nothing else."""
digest = hashlib.sha256(f"{identity.model}\0{identity.dims or 'auto'}".encode()).hexdigest()
base = f"memhub_e_{digest[:16]}"
return f"{base}_g{generation}" if generation else base
def graph_dir_for(space_id: str) -> str:
return f"graph-{space_id[:8]}"
def derived_space(profile: ModelProfileSnapshot) -> EmbeddingSpace:
"""The space `profile` opened before the registry existed, under its old name.
Never stored as is: bootstrap stores it, and offline tools that run beside
a home with no registry yet read through it.
"""
from one_memory.hy_sdk_runtime import legacy_collection_base
provider = profile.embedding_provider_kind or profile.provider_kind
identity = embedding_identity(profile)
builtin = profile.profile_id == _BUILTIN_PROFILE_ID
return EmbeddingSpace(
space_id="derived",
state="live",
model=identity.model,
dims=identity.dims,
name_dims=identity.dims,
collection_base=legacy_collection_base(profile),
naming="legacy",
graph_dir=DEFAULT_GRAPH_DIR,
provider_kind=provider.value,
profile_id=None if builtin else profile.profile_id,
profile_revision=None if builtin else profile.revision,
created_at=_now(),
)
_BUILTIN_PROFILE_ID = "builtin-retrieval"
def _now() -> str:
return datetime.now(UTC).isoformat()
def _row(row: sqlite3.Row | Mapping[str, object]) -> EmbeddingSpace:
return EmbeddingSpace.model_validate(dict(row))
def ensure_schema(connection: sqlite3.Connection) -> None:
connection.executescript(EMBEDDING_SPACES_SCHEMA)
def registry_present(connection: sqlite3.Connection) -> bool:
return (
connection.execute(
"SELECT 1 FROM sqlite_master WHERE type='table' AND name='embedding_spaces'"
).fetchone()
is not None
)
def live_space_in(connection: sqlite3.Connection) -> EmbeddingSpace | None:
if not registry_present(connection):
return None
previous = connection.row_factory
connection.row_factory = sqlite3.Row
try:
row = connection.execute("SELECT * FROM embedding_spaces WHERE state='live'").fetchone()
finally:
connection.row_factory = previous
return _row(row) if row is not None else None
def read_live_space(database_path: Path) -> EmbeddingSpace | None:
"""The live space, read only, or None for a home with no registry yet."""
if not database_path.is_file():
return None
with main_database(database_path).read(row_factory=sqlite3.Row) as connection:
return live_space_in(connection)
class EmbeddingSpaceStore:
"""The `embedding_spaces` table of the main Core database."""
def __init__(self, database_path: Path) -> None:
self.database_path = database_path
#: The live row, read once per change: every read and write asks for
#: it. Only this process writes the registry while Core serves.
self._live_cache: tuple[int, EmbeddingSpace] | None = None
with self._connect(writing=True) as connection:
ensure_schema(connection)
def invalidate(self) -> None:
_changed()
@contextmanager
def _connect(
self, *, writing: bool = False, lock_first: bool = True
) -> Iterator[sqlite3.Connection]:
database = main_database(self.database_path)
checkout = (
database.write(row_factory=sqlite3.Row)
if writing
else database.read(row_factory=sqlite3.Row)
)
with checkout as connection, connection:
if writing and lock_first:
take_the_write_lock_first(connection)
yield connection
@contextmanager
def transaction(self) -> Iterator[sqlite3.Connection]:
"""One `BEGIN IMMEDIATE` on the main database, committed on success."""
_changed()
try:
with self._connect(writing=True, lock_first=False) as connection:
connection.execute("BEGIN IMMEDIATE")
yield connection
finally:
_changed()
def live(self) -> EmbeddingSpace | None:
generation = _CHANGES[0]
cached = self._live_cache
if cached is not None and cached[0] == generation:
return cached[1]
with self._connect() as connection:
space = live_space_in(connection)
self._live_cache = (generation, space) if space is not None else None
return space
def require_live(self) -> EmbeddingSpace:
space = self.live()
if space is None:
raise EmbeddingSpaceError("embedding_space_not_bootstrapped")
return space
def get(self, space_id: str) -> EmbeddingSpace | None:
with self._connect() as connection:
row = connection.execute(
"SELECT * FROM embedding_spaces WHERE space_id=?", (space_id,)
).fetchone()
return _row(row) if row is not None else None
def list_spaces(self, *, include_dropped: bool = False) -> list[EmbeddingSpace]:
with self._connect() as connection:
rows = connection.execute(
"SELECT * FROM embedding_spaces "
+ ("" if include_dropped else "WHERE state != 'dropped' ")
+ "ORDER BY created_at, space_id"
).fetchall()
return [_row(row) for row in rows]
def bootstrap(self, active: ModelProfileSnapshot | None) -> EmbeddingSpace:
"""Register the space this home already uses, once, under its existing name.
`active` is the active profile, or None for the built-in model. The
collection name is derived exactly as before the registry, so nothing
on disk moves.
"""
from one_memory.hy_sdk_runtime import builtin_retrieval_snapshot
with self.transaction() as connection:
existing = live_space_in(connection)
if existing is not None:
return existing
derived = derived_space(active or builtin_retrieval_snapshot())
space = derived.model_copy(
update={"space_id": str(uuid4()), "live_since": derived.created_at}
)
_insert(connection, space)
return space
def record_width(self, space_id: str, dims: int) -> EmbeddingSpace:
"""A relabel: the space's stored width is now known. Names never change."""
with self.transaction() as connection:
record_width_in(connection, space_id, dims)
space = self.get(space_id)
if space is None: # pragma: no cover - the row was just updated
raise EmbeddingSpaceError("embedding_space_not_found")
return space
def record_vectors(self, counts: Mapping[str, int]) -> None:
"""Cache the vector count per space id from the last scan (display only)."""
if not counts:
return
_changed()
with self._connect(writing=True) as connection:
connection.executemany(
"UPDATE embedding_spaces SET vectors=? WHERE space_id=?",
[(count, space_id) for space_id, count in counts.items()],
)
def register_stray(
self,
*,
collection_base: str,
name_dims: int | None,
dims: int | None,
vectors: int,
) -> EmbeddingSpace:
"""Name a non-empty space the scan found that no row described."""
with self.transaction() as connection:
row = connection.execute(
"SELECT * FROM embedding_spaces WHERE collection_base=? "
"AND COALESCE(name_dims,0)=? AND state != 'dropped'",
(collection_base, name_dims or 0),
).fetchone()
if row is not None:
return _row(row)
space = EmbeddingSpace(
space_id=str(uuid4()),
state="stray",
dims=dims,
name_dims=name_dims,
collection_base=collection_base,
naming="discovered",
graph_dir=None,
vectors=vectors,
created_at=_now(),
)
_insert(connection, space)
return space
def mark_dropped(self, space_id: str) -> None:
_changed()
with self._connect(writing=True) as connection:
connection.execute(
"UPDATE embedding_spaces SET state='dropped', dropped_at=? "
"WHERE space_id=? AND state IN ('retired','building')",
(_now(), space_id),
)
def expired(self, *, now: datetime | None = None) -> list[EmbeddingSpace]:
moment = (now or datetime.now(UTC)).isoformat()
with self._connect() as connection:
rows = connection.execute(
"SELECT * FROM embedding_spaces WHERE state='retired' "
"AND keep_until IS NOT NULL AND keep_until <= ?",
(moment,),
).fetchall()
return [_row(row) for row in rows]
def registry_for(profiles: object) -> EmbeddingSpaceStore | None:
"""The registry beside a profile store, or None for a store with no database."""
path = getattr(profiles, "database_path", None)
return EmbeddingSpaceStore(path) if isinstance(path, Path) else None
def _insert(connection: sqlite3.Connection, space: EmbeddingSpace) -> None:
values = space.model_dump()
columns = ", ".join(values)
marks = ", ".join("?" for _ in values)
connection.execute(
f"INSERT INTO embedding_spaces ({columns}) VALUES ({marks})",
tuple(values.values()),
)
def insert_space_in(connection: sqlite3.Connection, space: EmbeddingSpace) -> None:
_changed()
_insert(connection, space)
def record_width_in(connection: sqlite3.Connection, space_id: str, dims: int) -> None:
_changed()
changed = connection.execute(
"UPDATE embedding_spaces SET dims=? WHERE space_id=? AND (dims IS NULL OR dims=?)",
(dims, space_id, dims),
)
if changed.rowcount != 1:
raise EmbeddingSpaceError("embedding_space_width_conflict")
def space_in(connection: sqlite3.Connection, space_id: str) -> EmbeddingSpace | None:
previous = connection.row_factory
connection.row_factory = sqlite3.Row
try:
row = connection.execute(
"SELECT * FROM embedding_spaces WHERE space_id=?", (space_id,)
).fetchone()
finally:
connection.row_factory = previous
return _row(row) if row is not None else None
def spaces_in(connection: sqlite3.Connection) -> list[EmbeddingSpace]:
previous = connection.row_factory
connection.row_factory = sqlite3.Row
try:
rows = connection.execute("SELECT * FROM embedding_spaces").fetchall()
finally:
connection.row_factory = previous
return [_row(row) for row in rows]
def set_state_in(
connection: sqlite3.Connection,
space_id: str,
*,
state: SpaceState,
expected: tuple[SpaceState, ...],
now: datetime | None = None,
) -> None:
"""Move one space between states, refusing any it was not expected in."""
_changed()
moment = now or datetime.now(UTC)
updates: dict[str, object] = {"state": state}
if state == "live":
updates["live_since"] = moment.isoformat()
elif state == "retired":
updates["retired_at"] = moment.isoformat()
updates["keep_until"] = (moment + RETIRED_SPACE_RETENTION).isoformat()
elif state == "dropped":
updates["dropped_at"] = moment.isoformat()
assignments = ", ".join(f"{column}=?" for column in updates)
marks = ",".join("?" for _ in expected)
changed = connection.execute(
f"UPDATE embedding_spaces SET {assignments} WHERE space_id=? AND state IN ({marks})",
(*updates.values(), space_id, *expected),
)
if changed.rowcount != 1:
raise EmbeddingSpaceError("embedding_space_state_changed")
def next_v2_base(
connection: sqlite3.Connection,
identity: EmbeddingIdentity,
*,
stored_names: frozenset[str],
) -> str:
"""A base for a new space no registry row and no stored collection uses.
Switching back to a model whose retired space is still kept gets a fresh
`_g<n>` base, so stale retired vectors are never mixed with fresh ones.
"""
used = {space.collection_base for space in spaces_in(connection)}
generation = 0
while True:
base = v2_collection_base(identity, generation)
names = space_collection_names(base, identity.dims).values()
if base not in used and not any(name in stored_names for name in names):
return base
generation += 1
def probe_width_allowed(
connection: sqlite3.Connection,
profile_id: str,
width: int,
*,
data_dir: Path,
) -> bool:
"""Whether a probe may record `width` on a profile without moving the space.
A profile that fills the live space (the active one, or the one the space
was built with) may record its measured width only when it is the
space's: equal to the width already known, or -- for an "auto" space
whose width was never recorded -- equal to the width Chroma stored, or
any width when nothing is stored yet. That is a relabel; the names never
change. Any other width needs a switch (B2: it used to silently open a
new, empty collection). Every other profile records what it measured.
"""
live = live_space_in(connection)
if live is None:
return True
active = connection.execute(
"SELECT profile_id FROM model_profiles WHERE is_active=1"
).fetchone()
fills_live = live.profile_id == profile_id or (active is not None and active[0] == profile_id)
if not fills_live:
return True
if live.dims is not None:
return live.dims == width
from one_memory.sdk_vector_reattach import stored_collection_width
stored = stored_collection_width(data_dir, live.collection_names()["memory"])
if stored is not None and stored != width:
return False
record_width_in(connection, live.space_id, width)
return True
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,161 @@
"""HTTP surface of the embedding switch: spaces, plans, jobs, cleanup.
The contract the App and the CLI build on is
`memhub/docs/contracts/embedding-switch-v1.md`. Errors keep Core's shape,
`{"detail": "<code>"}`; a refusal that computed a plan also carries it as
`"plan"`, so a client can show the confirmation without asking again.
"""
from __future__ import annotations
import asyncio
from collections.abc import Sequence
from typing import Any, cast
from fastapi import APIRouter, Request, Response
from fastapi.params import Depends
from fastapi.responses import JSONResponse
from pydantic import BaseModel, ConfigDict, Field
from one_memory.embedding_backend import ReembedUnavailableError
from one_memory.embedding_spaces import EmbeddingSpaceError
from one_memory.embedding_switch import (
EmbeddingSwitchError,
EmbeddingSwitchPlan,
EmbeddingSwitchService,
)
#: HTTP status per refusal code; anything not listed is a 409.
_STATUS = {
"model_profile_not_found": 404,
"embedding_switch_not_found": 404,
"embedding_space_not_found": 404,
"sdk_reembed_unavailable": 503,
"secure_storage_unavailable": 503,
"insufficient_disk_space": 507,
}
class EmbeddingSwitchPlanRequest(BaseModel):
model_config = ConfigDict(extra="forbid")
profile_id: str = Field(min_length=1, max_length=128)
expected_revision: int | None = Field(default=None, ge=1)
class EmbeddingSwitchStartRequest(BaseModel):
model_config = ConfigDict(extra="forbid")
profile_id: str = Field(min_length=1, max_length=128)
expected_revision: int = Field(ge=1)
plan_token: str = Field(min_length=1, max_length=128)
requested_by: str = Field(default="app", pattern=r"^(app|cli)$")
def refusal(code: str, *, plan: EmbeddingSwitchPlan | None = None) -> JSONResponse:
content: dict[str, Any] = {"detail": code}
if plan is not None:
content["plan"] = plan.model_dump(mode="json")
return JSONResponse(status_code=_STATUS.get(code, 409), content=content)
def _service(request: Request) -> EmbeddingSwitchService:
return cast(EmbeddingSwitchService, request.app.state.embedding_switch)
def embedding_switch_router(dependencies: Sequence[Depends]) -> APIRouter:
router = APIRouter(dependencies=dependencies)
@router.get("/api/v1/embedding-spaces")
async def list_embedding_spaces(request: Request) -> dict[str, object]:
"""The registry and what the vector store holds, space by space."""
service = _service(request)
spaces = await asyncio.to_thread(service.spaces.list_spaces)
live = next((space for space in spaces if space.state == "live"), None)
scanned: list[dict[str, object]] | None = None
scan_error = service.backend.unavailable_reason()
if scan_error is None:
try:
found = await service.backend.scan()
except Exception as error: # noqa: BLE001 - the registry still answers
scan_error = getattr(error, "code", None) or "embedding_space_scan_failed"
else:
known = {(space.collection_base, space.name_dims or 0): space for space in spaces}
scanned = [
{
"collection_base": item.collection_base,
"name_dims": item.name_dims,
"width": item.width,
"vectors": item.vectors,
"counts": dict(item.counts),
"space_id": known[item.key].space_id if item.key in known else None,
"state": known[item.key].state if item.key in known else "unregistered",
}
for item in found
]
return {
"schema_version": 1,
"live_space_id": live.space_id if live is not None else None,
"spaces": [space.model_dump(mode="json") for space in spaces],
"scanned": scanned,
"scan_error": scan_error,
}
@router.delete("/api/v1/embedding-spaces/{space_id}", response_model=None)
async def delete_embedding_space(space_id: str, request: Request) -> Any:
"""Clean up a retired space now instead of after its 14 days."""
try:
space = await _service(request).drop_space(space_id)
except (EmbeddingSwitchError, EmbeddingSpaceError, ReembedUnavailableError) as error:
return refusal(error.code)
except RuntimeError as error:
if "held open" in str(error):
return refusal("embedding_space_in_use")
raise
return {"schema_version": 1, "space": space.model_dump(mode="json")}
@router.post("/api/v1/embedding-switches/plan", response_model=None)
async def plan_embedding_switch(value: EmbeddingSwitchPlanRequest, request: Request) -> Any:
try:
plan = await _service(request).plan(
value.profile_id, expected_revision=value.expected_revision
)
except EmbeddingSwitchError as error:
return refusal(error.code, plan=error.plan)
return plan.model_dump(mode="json")
@router.post("/api/v1/embedding-switches", status_code=202, response_model=None)
async def start_embedding_switch(value: EmbeddingSwitchStartRequest, request: Request) -> Any:
try:
job = await _service(request).start_switch(
value.profile_id,
expected_revision=value.expected_revision,
plan_token=value.plan_token,
requested_by=value.requested_by,
)
except EmbeddingSwitchError as error:
return refusal(error.code, plan=error.plan)
return job.model_dump(mode="json")
@router.get("/api/v1/embedding-switches/current")
async def current_embedding_switch(request: Request, response: Response) -> dict[str, object]:
"""Polled by the loading screen, the banner and `memhub models switch status`."""
response.headers["Cache-Control"] = "no-store"
return await asyncio.to_thread(_service(request).current)
@router.get("/api/v1/embedding-switches/{job_id}", response_model=None)
async def get_embedding_switch(job_id: str, request: Request) -> Any:
job = await asyncio.to_thread(_service(request).get_job, job_id)
if job is None:
return refusal("embedding_switch_not_found")
return job.model_dump(mode="json")
@router.post("/api/v1/embedding-switches/{job_id}/cancel", response_model=None)
async def cancel_embedding_switch(job_id: str, request: Request) -> Any:
try:
job = await asyncio.to_thread(_service(request).cancel, job_id)
except EmbeddingSwitchError as error:
return refusal(error.code)
return job.model_dump(mode="json")
return router
@@ -71,6 +71,16 @@ def graph_store_enabled() -> bool:
return raw != "none"
def effective_memory_mode(configured: str = "pro") -> str:
"""The SDK mode clients are built with: the setting, unless dev forces ultra.
`ONE_MEMORY_GRAPH_STORE` (any value but empty or `none`) predates the
setting and keeps working for development: it forces ultra whatever the
configured mode. Without it the configured mode (default pro) applies.
"""
return "ultra" if graph_store_enabled() else configured
def inspect_graph_width_mismatch(config: Any) -> GraphWidthMismatch | None:
"""Return a mismatch without renaming or deleting anything."""
db_path = Path(config.graph_store.db_path)
@@ -83,11 +93,16 @@ def inspect_profile_graph_width(
*,
api_key: str | None,
data_dir: Path,
mode: str | None = None,
graph_dir: str = "graph",
) -> GraphWidthMismatch | None:
"""Inspect the graph path and width selected by the active profile."""
"""Inspect the graph path and width selected by the active profile.
`graph_dir` is the live embedding space's graph (`embedding_spaces`).
"""
del api_key
return _inspect_graph_width(
_profile_graph_path(data_dir),
_profile_graph_path(data_dir, mode, graph_dir),
_profile_embedding_width(profile),
)
@@ -98,11 +113,13 @@ def decide_profile_graph_width(
api_key: str | None,
data_dir: Path,
decision: GraphWidthDecision,
mode: str | None = None,
graph_dir: str = "graph",
) -> GraphWidthOutcome:
"""Apply the user's decision to the active profile's graph."""
del api_key
config = _GraphAdminConfig(
graph_store=_GraphStoreConfig(str(_profile_graph_path(data_dir))),
graph_store=_GraphStoreConfig(str(_profile_graph_path(data_dir, mode, graph_dir))),
embedder=_EmbeddingWidthConfig(_profile_embedding_width(profile)),
)
return apply_graph_width_decision(
@@ -192,6 +209,12 @@ _GRAPH_WIDTH_CACHE: dict[str, int | None] = {}
_GRAPH_WIDTH_LOCK = threading.Lock()
def forget_graph_widths() -> None:
"""Drop every remembered width: an embedding switch moved the graph."""
with _GRAPH_WIDTH_LOCK:
_GRAPH_WIDTH_CACHE.clear()
def _graph_embedding_width(db_path: Path) -> int | None:
"""Read the Memory embedding width directly from Kuzu's schema."""
try:
@@ -240,8 +263,9 @@ def _inspect_graph_width(
)
def _profile_graph_path(data_dir: Path) -> Path:
name = "graph" if graph_store_enabled() else "graph-disabled"
def _profile_graph_path(data_dir: Path, mode: str | None = None, graph_dir: str = "graph") -> Path:
ultra = mode == "ultra" if mode is not None else graph_store_enabled()
name = graph_dir if ultra else "graph-disabled"
return data_dir / "hy-memory" / HY_MEMORY_NAMESPACE / name
@@ -14,7 +14,7 @@ from pathlib import Path
from typing import Literal, cast
HY_MEMORY_DISTRIBUTION = "hy-memory"
HY_MEMORY_VERSION = "2.0.7"
HY_MEMORY_VERSION = "2.0.10"
HY_MEMORY_NAMESPACE = "governed-v2"
DiagnosticLogLevel = Literal["error", "warning", "info", "debug"]
@@ -109,11 +109,30 @@ def apply_diagnostic_log_level(level: DiagnosticLogLevel) -> None:
if any(name == root or name.startswith(f"{root}.") for root in _NOISY_LIBRARY_LOGGERS):
logging.getLogger(name).setLevel(noisy_level)
# HY configures its text log once, on the first client, at the level of
# that moment. Its file handler must follow later changes too, or a live
# switch to debug never reaches hy_memory.log.
for handler in logging.getLogger("hy_memory").handlers:
if isinstance(handler, logging.FileHandler):
handler.setLevel(selected)
os.environ["MEMORY_LOG_LEVEL"] = logging.getLevelName(selected)
os.environ["MEMORY_PIPELINE_TRACE_ENABLED"] = "true" if level == "debug" else "false"
_apply_hy_pipeline_policy()
os.environ["MEMORY_LOG_LLM_CONTENT"] = "true" if level == "debug" else "false"
def _apply_hy_pipeline_policy() -> None:
"""Keep HY's per-step pipeline log and trace off at every level.
Nothing in MemHub reads them: Core's own diagnostics carry the step
timing, and the JSONL/SQLite copies only grew on disk. With both off
the SDK skips the per-step bookkeeping altogether.
"""
os.environ["MEMORY_PIPELINE_LOG_ENABLED"] = "false"
os.environ["MEMORY_PIPELINE_TRACE_ENABLED"] = "false"
def hy_memory_version() -> str:
"""Return the installed SDK version, failing if the pinned dist is absent."""
@@ -131,7 +150,9 @@ def configure_hy_runtime_environment(data_dir: Path) -> None:
os.environ["MEMORY_LOG_DIR"] = str(log_dir)
level = diagnostic_log_level(data_dir)
os.environ["MEMORY_LOG_LEVEL"] = level.upper()
os.environ["MEMORY_PIPELINE_TRACE_ENABLED"] = "true" if level == "debug" else "false"
_apply_hy_pipeline_policy()
# Still meaningful with the pipeline log off: it decides whether HY's
# DEBUG lines in hy_memory.log carry full model prompts and responses.
os.environ["MEMORY_LOG_LLM_CONTENT"] = "true" if level == "debug" else "false"
os.environ["MEMORY_MEMORY_OPERATIONS_ENABLED"] = "false"
os.environ["ANONYMIZED_TELEMETRY"] = "False"
@@ -3,20 +3,26 @@
Runtime product code must use public ``HyMemoryClient`` methods. The startup
migration boundary is the isolated exception: all writers are stopped, so it
may copy, restore and repair SDK-owned local stores.
The embedding switch is the second such boundary: model writes are paused
while it re-embeds one space into another (`SdkReembedBackend`), so it may
read and fill SDK collections through the SDK's re-embed module.
"""
from __future__ import annotations
import asyncio
import json
import sqlite3
from collections.abc import Iterator
from contextlib import closing
from contextlib import closing, contextmanager
from dataclasses import dataclass, field
from datetime import UTC, datetime, timedelta
from pathlib import Path
from typing import Any
from hy_memory import ( # type: ignore[import-untyped]
import hy_memory # type: ignore[import-untyped]
from hy_memory import (
DataPlaneRestore,
DataPlaneSnapshot,
restore_local_data_plane,
@@ -497,7 +503,7 @@ def _ensure_coding_schema(path: Path) -> None:
CodingMemoryStore,
)
# Relies on the pinned SDK's internals (hy-memory==2.0.7 is pinned exactly).
# Relies on the pinned SDK's internals (hy-memory==2.0.10 is pinned exactly).
store = CodingMemoryStore.__new__(CodingMemoryStore)
store._db_path = str(path)
store._conn = None
@@ -511,3 +517,256 @@ def _ensure_coding_schema(path: Path) -> None:
def _batches[T](values: list[T] | tuple[T, ...]) -> Iterator[tuple[T, ...]]:
for start in range(0, len(values), _BATCH):
yield tuple(values[start : start + _BATCH])
# The SDK functions the embedding switch needs. They arrived in hy-memory
# 2.0.10; an older SDK reports `sdk_reembed_unavailable` instead of failing.
_REEMBED_API = (
"VectorSpace",
"scan_vector_spaces",
"diff_spaces",
"reembed_vector_space",
"verify_reembed",
"drop_vector_space",
"reembed_graph",
"verify_graph_reembed",
"drop_graph",
"ReembedCancelled",
)
class SdkReembedBackend:
"""`ReembedBackend` over `hy_memory`'s re-embed module, for one home.
Runs beside live SDK clients: the vector functions open Chroma with the
SDK's own settings, so they share its client system. The graph functions
refuse a graph a live client holds open; the switch closes them first.
"""
def __init__(self, data_dir: Path) -> None:
self.data_dir = data_dir
self.root = data_dir.resolve() / "hy-memory" / HY_MEMORY_NAMESPACE
self.persist_directory = self.root / "vector"
self._chroma_opened = False
def unavailable_reason(self) -> str | None:
missing = [name for name in _REEMBED_API if not hasattr(hy_memory, name)]
return "sdk_reembed_unavailable" if missing else None
def _sdk(self) -> Any:
from one_memory.embedding_backend import ReembedUnavailableError
reason = self.unavailable_reason()
if reason is not None:
raise ReembedUnavailableError(reason)
return hy_memory
def _open_chroma_once(self) -> None:
"""Open Chroma once under the client-construction lock.
Every later open beside the Runtime finds that system. Two first
opens race over Chroma's unlocked cache, and a client under
construction lost its stores that way (see `hy_sdk_runtime`).
"""
if self._chroma_opened or not self.persist_directory.is_dir():
return
import chromadb
from one_memory.hy_sdk_runtime import first_store_open
with first_store_open(self.data_dir):
chromadb.PersistentClient(path=str(self.persist_directory))
self._chroma_opened = True
def _vector_space(self, space: Any) -> Any:
return self._sdk().VectorSpace(
base_name=space.collection_base,
dims=space.dims,
persist_directory=str(self.persist_directory),
model=space.model,
collections=space.names(),
)
async def scan(self, *, per_user: bool = False) -> tuple[Any, ...]:
from one_memory.embedding_backend import ScannedSpace
if not self.persist_directory.is_dir():
return ()
await asyncio.to_thread(self._open_chroma_once)
found = await self._sdk().scan_vector_spaces(str(self.persist_directory), per_user=per_user)
return tuple(
ScannedSpace(
collection_base=str(item.base_name),
name_dims=item.dims,
width=item.width,
counts={role: int(entry.count) for role, entry in item.roles.items()},
collections={role: str(entry.name) for role, entry in item.roles.items()},
per_user=dict(item.per_user) if per_user else {},
)
for item in found
)
async def diff(self, source: Any, target: Any) -> dict[str, int]:
await asyncio.to_thread(self._open_chroma_once)
result = await self._sdk().diff_spaces(
self._vector_space(source), self._vector_space(target)
)
return {role: len(ids) for role, ids in result.missing.items()}
async def reembed(
self,
source: Any,
target: Any,
*,
embedding: Any,
cancel: Any,
on_step: Any,
batch_size: int = 64,
) -> None:
from one_memory.embedding_backend import ReembedStep
await asyncio.to_thread(self._open_chroma_once)
sdk = self._sdk()
def progress(item: Any) -> None:
on_step(
ReembedStep(
phase="vectors",
role=str(item.role),
done=int(item.done),
total=int(item.total),
embedded=int(item.embedded),
reused=int(item.reused),
kept=int(getattr(item, "kept", 0)),
)
)
with self._translated():
await sdk.reembed_vector_space(
self._vector_space(source),
self._vector_space(target),
embedding=self._embedding_service(embedding),
batch_size=batch_size,
cancel=cancel,
on_progress=progress,
# The target wins: an id it already holds is never touched, so
# a resumed run continues and a merge never overwrites live data.
skip_existing=True,
)
async def verify(self, source: Any, target: Any, *, embedding: Any) -> Any:
from one_memory.embedding_backend import VectorCheck
await asyncio.to_thread(self._open_chroma_once)
with self._translated():
result = await self._sdk().verify_reembed(
self._vector_space(source),
self._vector_space(target),
embedding=self._embedding_service(embedding),
)
return VectorCheck(
per_role={role: dict(values) for role, values in result.per_role.items()},
per_user=dict(result.per_user),
problems=tuple(result.problems),
warnings=tuple(result.warnings),
self_recall_hits=int(result.self_recall_hits),
self_recall_samples=int(result.self_recall_samples),
)
async def graph_vectors(self, path: Path) -> int | None:
if not path.exists():
return None
# Not re-exported at the package root; the module is the SDK's own.
plan = await self._sdk().reembed.plan_graph_reembed(path)
return int(plan.vectors)
async def reembed_graph(
self,
source: Path,
target: Path,
*,
embedding: Any,
target_dims: int,
cancel: Any,
on_step: Any,
) -> None:
from one_memory.embedding_backend import ReembedStep
def progress(item: Any) -> None:
on_step(
ReembedStep(
phase="graph",
role=str(item.table),
done=int(item.done),
total=int(item.total),
embedded=int(item.embedded),
)
)
with self._translated():
await self._sdk().reembed_graph(
source,
target,
embedding=self._embedding_service(embedding),
target_dims=target_dims,
cancel=cancel,
on_progress=progress,
# A resumed switch rebuilds the copy from the untouched source.
overwrite=True,
)
async def verify_graph(self, source: Path, target: Path, *, target_dims: int) -> Any:
from one_memory.embedding_backend import GraphCheck
result = await self._sdk().verify_graph_reembed(source, target, target_dims=target_dims)
return GraphCheck(ok=bool(result.ok), problems=tuple(result.problems))
async def drop(self, space: Any) -> None:
await asyncio.to_thread(self._open_chroma_once)
await self._sdk().drop_vector_space(self._vector_space(space))
async def drop_graph(self, path: Path) -> None:
await self._sdk().drop_graph(path)
def _embedding_service(self, embedding: Any) -> Any:
"""The service that computes a target space's vectors.
The built-in model is MemHub's ONNX service, exactly what the Runtime
client embeds with. A remote model gets the SDK's own service, built
from the same configuration the Runtime client would use.
"""
from one_memory.hy_sdk_runtime import (
LocalOnnxEmbeddingService,
build_sdk_config,
retrieval_view,
)
from one_memory.model_profiles import ProviderKind
profile = embedding.profile
provider = profile.embedding_provider_kind or profile.provider_kind
if provider is ProviderKind.LOCAL_ONNX:
return LocalOnnxEmbeddingService()
from hy_memory.core.embed_service import EmbedService # type: ignore[import-untyped]
return EmbedService(
# The retrieval view: re-embedding never needs the generation model.
build_sdk_config(
retrieval_view(profile),
embedding.space,
api_key=embedding.api_key,
data_dir=self.data_dir,
)
)
@contextmanager
def _translated(self) -> Iterator[None]:
"""SDK refusals as the Core errors the switch classifies."""
from one_memory.embedding_backend import EmbeddingWidthError, ReembedCancelledError
sdk = self._sdk()
try:
yield
except sdk.ReembedCancelled:
raise ReembedCancelledError("embedding_switch_cancelled") from None
except hy_memory.EmbeddingDimensionMismatchError as error:
raise EmbeddingWidthError("embedding_dimensions_mismatch") from error
Loaded 100 of 242 files, more files were not shown because too many files have changed in this diff. Show more