Compare commits

..
Author SHA1 Message Date
alvinfeiandClaude Opus 5.5 528d2c9fd7 fix(memhub): Linux lifecycle checks the write queue, not recall without a model
The lifecycle verification imported a conversation on a runner with no
generation model and then expected recall to find it. Under SDK authority
nothing reaches the SDK, not even its raw layer, until a model is active,
so the check could never pass and blocked the Linux release archive.

It now asserts what the runner can prove: the imported and appended turns
reach Core's durable write queue, and survive an idempotent upgrade.
docs/TODO-raw-recall-without-model.md records the product gap.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 20:05:44 +08:00
feitianxiang 8e56bf7a37 Merge pull request #218 from Tencent-Hunyuan/fix/sdk-client-race-and-unknown-jobs
fix(memhub): serialize SDK client startup, log failure frames, settle pre-upgrade unknown writes
2026-10-28 19:49:24 +08:00
alvinfeiandClaude Opus 5.5 352fffb3da fix(memhub): serialize SDK client startup, log failure frames, settle pre-upgrade unknown writes
- hy_sdk_runtime: construct one HyMemoryClient at a time per home. Two
  first-time Chroma opens on a fresh vector dir race Chroma's unlocked
  System cache; on a clean install the drain's and the UI's clients both
  failed and the first write job was lost as unknown.
- sdk_write_processor: a client that fails to construct raises
  hy_memory_client_unavailable before HY is invoked, which parks the job
  (waiting_for_model) instead of leaving it unknown.
- sdk_write_coordinator: a retired profile's handle closes only after its
  last in-flight use (defensive; a profile revision change used to close
  the client under a running write).
- durable_queue: durable_write_job_failed/parked log the exception chain as
  file:line:function frames, never the message.
- sdk_migration_runner: the cutover settles pre-cutover unknown (and
  invoked running) write jobs as cancelled/abandoned_at_cutover and counts
  them as abandoned.unknown_write_jobs; never retried.
- durable_queue/api: retry-unknown and the reconcile helpers no longer
  assume the dropped legacy task tables; the endpoint uses the startup
  queue so it does not recreate memory_write_job_evidence.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 19:39:13 +08:00
feitianxiang 42c0ca1d14 Merge pull request #215 from Tencent-Hunyuan/feat/memhub-sdk-logs-quiet
feat(memhub): HY pipeline log off, SDK text logs only in hy_memory.log (hy-memory 2.0.8)
2026-10-28 15:46:40 +08:00
alvinfeiandClaude Opus 5.5 0ca2e56ccc chore(release): bump MemHub build number to 35
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 15:38:05 +08:00
alvinfeiandClaude Opus 5.5 f328cdbf8e docs: record hy-memory 2.0.8 as published
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 15:38:05 +08:00
alvinfeiandClaude Opus 5.5 3f57a3c978 build(memhub): lock hy-memory 2.0.8 from PyPI
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 15:37:51 +08:00
alvinfeiandClaude Opus 5.5 19a7af0e71 feat(memhub): remove the HY pipeline section from the Debug panel
MemHub no longer writes the HY pipeline JSONL, so the Desktop diagnostics
reader stops scanning hy-memory/governed-v2/logs and the renderer drops the
"hy-memory" service and "pipeline" source. A test covers leftover pipeline
files and hy_memory.log staying out of the panel.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 15:36:50 +08:00
alvinfeiandClaude Opus 5.5 5791c0d1e7 feat(memhub): HY pipeline log off, SDK text logs only in hy_memory.log
- MEMORY_PIPELINE_LOG_ENABLED and MEMORY_PIPELINE_TRACE_ENABLED are false
  at every diagnostic level, both at startup and on a live level change.
  MEMORY_LOG_LLM_CONTENT keeps its debug-only setting: it still decides
  whether HY's DEBUG lines in hy_memory.log carry full prompts/responses.
- After each HyMemoryClient construction (success or failure), Core drops
  the SDK's stdout/stderr StreamHandlers, which under launchd duplicated
  every SDK line into service.log/service-error.log, and forwards SDK
  WARNING+ to Core's core.log handler so the Debug panel still shows SDK
  problems.
- A live level change also sets the level of HY's hy_memory.log handler,
  which the SDK otherwise fixes at the level of the first client.
- log_file.py no longer claims core.log covers the HY SDK at every level.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 15:36:50 +08:00
alvinfeiandClaude Opus 5.5 bf92105210 build(memhub): pin hy-memory 2.0.8
2.0.8 adds MEMORY_PIPELINE_LOG_ENABLED and keeps conversation text out of
INFO logs. apps/core/uv.lock is not updated: 2.0.8 is not on PyPI yet, so
the lock is regenerated (uv lock --directory apps/core) once it is
published.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 15:36:50 +08:00
feitianxiang 3387247842 Merge pull request #214 from Tencent-Hunyuan/feat/sdk-pipeline-log-switch
feat(sdk): switch to turn the pipeline log off, keep conversation text out of INFO logs (2.0.8)
2026-10-28 15:34:41 +08:00
38 changed files with 1097 additions and 452 deletions

No files matched your search

+14
View File
@@ -4,6 +4,18 @@ Hy Memory and MemHub are versioned and released independently.
## Unreleased
- MemHub turns off the HY pipeline log and trace at every diagnostic level
(hy-memory 2.0.8 `MEMORY_PIPELINE_LOG_ENABLED=false`,
`MEMORY_PIPELINE_TRACE_ENABLED=false`), so the per-step JSONL files under
`hy-memory/governed-v2/logs/` and the `pipeline_logs` rows stop growing.
The Debug panel no longer shows the HY pipeline section. Existing files are
left in place.
- HY SDK text logs are written once, to `hy_memory.log`. They are no longer
copied to `service.log` and `service-error.log` through the SDK's
stdout/stderr streams. SDK warnings and errors also go to `core.log`, so the
Debug panel still shows them. A live switch of the diagnostic level now also
changes the level of `hy_memory.log`.
- MemHub shows when working and coding experiences were talked about. After
the SDK upgrade every one of them read "时间未知" (time unknown) because the
SDK never stored their conversation time; hy-memory 2.0.7 stores it and
@@ -78,6 +90,8 @@ Hy Memory and MemHub are versioned and released independently.
## Hy Memory 2.0.8 — 2026-10-28
- Published
[`hy-memory==2.0.8`](https://pypi.org/project/hy-memory/2.0.8/).
- New `MEMORY_PIPELINE_LOG_ENABLED` (default `true`) turns the pipeline JSONL
log (`{MEMORY_LOG_DIR}/<subdir>/<date>.log`) off. With `false` no JSONL
file or directory is written. When `MEMORY_PIPELINE_TRACE_ENABLED` is also
+6 -4
View File
@@ -29,10 +29,12 @@ prompt/response、密钥、访问令牌或完整本地路径。结构化日志
文件。详细合同见
[`docs/contracts/diagnostic-logging.md`](docs/contracts/diagnostic-logging.md)。
用户显式把诊断级别切换为 `debug` 时,Core 会启用 HY pipeline trace,并将
`MEMORY_LOG_LLM_CONTENT` 设为 `true`。该模式会把完整模型 prompt/response 写入本地调试
日志,其中可能包含获授权的对话正文和模型输出;它只应用于有意识的故障排查,不应长期
开启,也不应在未经检查和脱敏时分享相关日志。切回 `info` 会停止后续正文记录,但不会
HY pipeline 日志与 trace 在任何级别下都关闭。HY SDK 的文本日志只写入
`hy-memory/governed-v2/logs/hy_memory.log`,其中的 WARNING 及以上同时写入 `core.log`;
SDK 不在 INFO 及以上级别记录对话或记忆正文。用户显式把诊断级别切换为 `debug` 时,Core 将
`MEMORY_LOG_LLM_CONTENT` 设为 `true`。该模式会把完整模型 prompt/response 写入
`hy_memory.log` 的 DEBUG 行,其中可能包含获授权的对话正文和模型输出;它只应用于有意识的
故障排查,不应长期开启,也不应在未经检查和脱敏时分享相关日志。切回 `info` 会停止后续正文记录,但不会
自动删除此前已写入的调试文件。
具体行为以当前源码、网络请求实现和发布说明为准。发现实现与本说明不一致时,请按
+2 -2
View File
@@ -18,7 +18,7 @@ If you run into a problem, please open an Issue with the version, platform,repro
## 发行与平台合同
源码当前固定使用公开 PyPI 正式版 `hy-memory==2.0.7`。MemHub 源码版本为 `1.0.5`
源码当前固定使用公开 PyPI 正式版 `hy-memory==2.0.8`。MemHub 源码版本为 `1.0.5`
(build 14),GitHub 仓库为
[`Tencent-Hunyuan/Hy-Memory`](https://github.com/Tencent-Hunyuan/Hy-Memory),npm 包名为
`memhub-cli`。当前代码架构见
@@ -118,7 +118,7 @@ MemHub 的 SQLite 数据默认位于 `~/.one-memory`,不在 npm 包或 Runtime
## Hy Memory
SDK 依赖固定为公开 PyPI 正式版 `hy-memory==2.0.7`。SDK 配置、权限边界及 search
SDK 依赖固定为公开 PyPI 正式版 `hy-memory==2.0.8`。SDK 配置、权限边界及 search
返回格式见 [`MEMORY.md`](MEMORY.md)。HY search 返回的 `content` 已包含 SDK 统一生成的
逻辑时间和演化上下文;调用方不应再次转换时间戳或重复拼接 `When`。
+1 -1
View File
@@ -2352,7 +2352,7 @@
{
"SPDXID": "SPDXRef-Package-hy-memory",
"name": "hy-memory",
"versionInfo": "2.0.7",
"versionInfo": "2.0.8",
"downloadLocation": "NOASSERTION",
"filesAnalyzed": false,
"licenseConcluded": "Apache-2.0",
+1 -1
View File
@@ -9,7 +9,7 @@ Per-package license texts are not reproduced here yet; the license
column is filled in as part of release review, and the SBOM records
`NOASSERTION` until then rather than guessing.
## hy-memory 2.0.7
## hy-memory 2.0.8
License: Apache-2.0
+1 -1
View File
@@ -12,7 +12,7 @@ dependencies = [
"fastapi>=0.141.1",
"filelock>=3.32.4",
# Public SDK version validated against MemHub's permission and recall contracts.
"hy-memory==2.0.7",
"hy-memory==2.0.8",
# chromadb arrives through the HY SDK; locked here so an unrelated upgrade
# cannot move the SDK's store. Built-in embedding consumes numpy directly.
"chromadb==1.5.9",
+26 -26
View File
@@ -3150,25 +3150,25 @@ def create_app(
raise HTTPException(status_code=422, detail="memory_task_requires_experience")
namespace = _namespace_kwargs(request.kind, request.variant)
try:
client = await app.state.sdk_write_coordinator.client()
update_kwargs: dict[str, object] = dict(namespace)
if request.task is not None:
update_kwargs["task"] = request.task
result = await asyncio.to_thread(
client.update, memory_id, request.content, **update_kwargs
)
if not isinstance(result, dict):
raise HySdkError("hy_memory_update_rejected")
if result.get("success") is not True:
if "not found" in str(result.get("error_message") or ""):
raise HTTPException(status_code=404, detail="memory_not_found")
raise HySdkError("hy_memory_update_rejected")
# The SDK names the namespace it wrote; read that one back.
if result.get("kind") in {"experience", "fact"}:
namespace = {"kind": result["kind"]}
if result.get("variant") in {"working", "coding"}:
namespace["variant"] = result["variant"]
raw = await asyncio.to_thread(client.get, memory_id, **namespace)
async with app.state.sdk_write_coordinator.use_client() as client:
update_kwargs: dict[str, object] = dict(namespace)
if request.task is not None:
update_kwargs["task"] = request.task
result = await asyncio.to_thread(
client.update, memory_id, request.content, **update_kwargs
)
if not isinstance(result, dict):
raise HySdkError("hy_memory_update_rejected")
if result.get("success") is not True:
if "not found" in str(result.get("error_message") or ""):
raise HTTPException(status_code=404, detail="memory_not_found")
raise HySdkError("hy_memory_update_rejected")
# The SDK names the namespace it wrote; read that one back.
if result.get("kind") in {"experience", "fact"}:
namespace = {"kind": result["kind"]}
if result.get("variant") in {"working", "coding"}:
namespace["variant"] = result["variant"]
raw = await asyncio.to_thread(client.get, memory_id, **namespace)
memory = None if raw is None else HySdkMemoryRecord.from_sdk(raw)
except HTTPException:
raise
@@ -3197,10 +3197,10 @@ def create_app(
) -> Response:
_check_memory_namespace(kind, variant)
try:
client = await app.state.sdk_write_coordinator.client()
result = await asyncio.to_thread(
client.delete, memory_id, **_namespace_kwargs(kind, variant)
)
async with app.state.sdk_write_coordinator.use_client() as client:
result = await asyncio.to_thread(
client.delete, memory_id, **_namespace_kwargs(kind, variant)
)
if not isinstance(result, dict):
raise HySdkError("hy_memory_delete_rejected")
error_code = result.get("error_code")
@@ -3442,9 +3442,9 @@ def create_app(
"""Retry unknown writes only behind the user's explicit confirmation."""
del request
reconciled = await _sdk_memory_runtime(app).reconcile_unknown_outputs()
from one_memory.memory_server.durable_queue import DurableWriteQueue
queue = DurableWriteQueue(app.state.memory_write_queue_path)
# The startup queue knows the authority. A fresh legacy-mode queue
# would recreate the job-evidence table the SDK cutover dropped.
queue = app.state.memory_write_queue
unknown = queue.unknown_jobs(limit=2048)
if app.state.memory_authority is MemoryAuthority.LEGACY:
for job in unknown:
@@ -14,7 +14,7 @@ from pathlib import Path
from typing import Literal, cast
HY_MEMORY_DISTRIBUTION = "hy-memory"
HY_MEMORY_VERSION = "2.0.7"
HY_MEMORY_VERSION = "2.0.8"
HY_MEMORY_NAMESPACE = "governed-v2"
DiagnosticLogLevel = Literal["error", "warning", "info", "debug"]
@@ -109,11 +109,30 @@ def apply_diagnostic_log_level(level: DiagnosticLogLevel) -> None:
if any(name == root or name.startswith(f"{root}.") for root in _NOISY_LIBRARY_LOGGERS):
logging.getLogger(name).setLevel(noisy_level)
# HY configures its text log once, on the first client, at the level of
# that moment. Its file handler must follow later changes too, or a live
# switch to debug never reaches hy_memory.log.
for handler in logging.getLogger("hy_memory").handlers:
if isinstance(handler, logging.FileHandler):
handler.setLevel(selected)
os.environ["MEMORY_LOG_LEVEL"] = logging.getLevelName(selected)
os.environ["MEMORY_PIPELINE_TRACE_ENABLED"] = "true" if level == "debug" else "false"
_apply_hy_pipeline_policy()
os.environ["MEMORY_LOG_LLM_CONTENT"] = "true" if level == "debug" else "false"
def _apply_hy_pipeline_policy() -> None:
"""Keep HY's per-step pipeline log and trace off at every level.
Nothing in MemHub reads them: Core's own diagnostics carry the step
timing, and the JSONL/SQLite copies only grew on disk. With both off
the SDK skips the per-step bookkeeping altogether.
"""
os.environ["MEMORY_PIPELINE_LOG_ENABLED"] = "false"
os.environ["MEMORY_PIPELINE_TRACE_ENABLED"] = "false"
def hy_memory_version() -> str:
"""Return the installed SDK version, failing if the pinned dist is absent."""
@@ -131,7 +150,9 @@ def configure_hy_runtime_environment(data_dir: Path) -> None:
os.environ["MEMORY_LOG_DIR"] = str(log_dir)
level = diagnostic_log_level(data_dir)
os.environ["MEMORY_LOG_LEVEL"] = level.upper()
os.environ["MEMORY_PIPELINE_TRACE_ENABLED"] = "true" if level == "debug" else "false"
_apply_hy_pipeline_policy()
# Still meaningful with the pipeline log off: it decides whether HY's
# DEBUG lines in hy_memory.log carry full model prompts and responses.
os.environ["MEMORY_LOG_LLM_CONTENT"] = "true" if level == "debug" else "false"
os.environ["MEMORY_MEMORY_OPERATIONS_ENABLED"] = "false"
os.environ["ANONYMIZED_TELEMETRY"] = "False"
@@ -497,7 +497,7 @@ def _ensure_coding_schema(path: Path) -> None:
CodingMemoryStore,
)
# Relies on the pinned SDK's internals (hy-memory==2.0.7 is pinned exactly).
# Relies on the pinned SDK's internals (hy-memory==2.0.8 is pinned exactly).
store = CodingMemoryStore.__new__(CodingMemoryStore)
store._db_path = str(path)
store._conn = None
@@ -36,6 +36,7 @@ from one_memory.hy_graph_admin import (
)
from one_memory.hy_sdk_types import HySdkRuntimeError
from one_memory.local_embedding import LocalEmbedder
from one_memory.log_file import confine_hy_sdk_logging
from one_memory.model_profiles import (
ModelProfileSnapshot,
ProviderKind,
@@ -66,6 +67,16 @@ _STORE_GATE = threading.Condition()
_OFFLINE_STORES: set[Path] = set()
_OPENING: dict[Path, int] = {}
# One client constructs at a time per home, its first-time store setup
# included. The constructor opens Chroma's `PersistentClient`, and two first
# opens on one directory race over that same unlocked System cache. On a
# clean install the write drain's client and a UI read client started in the
# same second; neither reached `VectorStore ready`, and the first write job
# was lost as unknown. Taken inside `acquire_client`, after the offline gate
# and never while `_STORE_GATE` is held, so it cannot deadlock with
# `hold_stores_offline`, which refuses while `_OPENING` counts a client.
_CONSTRUCTING: dict[Path, threading.Lock] = {}
class LocalOnnxEmbeddingService:
"""Public ``EmbeddingService`` implementation backed by MemHub's ONNX model."""
@@ -381,16 +392,23 @@ def _acquire_client(
embedding_service: LocalOnnxEmbeddingService | None = None
if (profile.embedding_provider_kind or profile.provider_kind) is ProviderKind.LOCAL_ONNX:
embedding_service = LocalOnnxEmbeddingService()
with _STORE_GATE:
constructing = _CONSTRUCTING.setdefault(key, threading.Lock())
try:
client = HyMemoryClient(
config=config,
mode="ultra" if graph_store_enabled() else "pro",
runtime=runtime,
embedding_service=embedding_service,
)
with constructing:
client = HyMemoryClient(
config=config,
mode="ultra" if graph_store_enabled() else "pro",
runtime=runtime,
embedding_service=embedding_service,
)
except BaseException:
_release_runtime(key)
raise
finally:
# The constructor's first step configures HY's text log (once per
# process), so by here it exists whether or not construction succeeded.
confine_hy_sdk_logging()
return ClientLease(client, key)
+55 -2
View File
@@ -20,8 +20,11 @@ def start_logging_to_file(data_dir: Path) -> Path:
nowhere. That includes the loop watchdog: the one report that names what
stalled the app was unreadable in exactly the situation it exists for.
The active diagnostic collection level controls Core, HY SDK, and provider
client records. Uvicorn keeps its request log on its own non-propagating
The active diagnostic collection level controls Core and provider client
records. The HY SDK keeps its own text log, hy_memory.log under the SDK's
log directory, and stops propagating here once its first client is built;
`confine_hy_sdk_logging` then forwards only its warnings and errors to
this file. Uvicorn keeps its request log on its own non-propagating
loggers.
"""
destination = log_path(data_dir)
@@ -49,3 +52,53 @@ def start_logging_to_file(data_dir: Path) -> Path:
if root.level > logging.INFO or root.level == logging.NOTSET:
root.setLevel(logging.INFO)
return destination
HY_SDK_LOGGER = "hy_memory"
class _CoreLogForwarder(logging.Handler):
"""Hands HY warnings to Core's own core.log handler, rotation and format."""
def __init__(self, target: logging.Handler) -> None:
super().__init__(level=logging.WARNING)
self.target = target
def emit(self, record: logging.LogRecord) -> None:
self.target.handle(record)
def confine_hy_sdk_logging() -> None:
"""Keep HY's text log in hy_memory.log, with only its problems in core.log.
HY's first client sets up hy_memory.log plus a stdout and a stderr stream
on the "hy_memory" logger, and stops it propagating to the root. Under
launchd those streams are service.log and service-error.log, so every SDK
line was written twice, and none reached core.log. Drop the streams and
forward WARNING and above to core.log, which the Debug panel reads; the
SDK keeps conversation text out of those levels.
Idempotent. Does nothing until HY has configured its logger, because a
propagating logger already reaches core.log through the root.
"""
sdk = logging.getLogger(HY_SDK_LOGGER)
if sdk.propagate:
return
for handler in list(sdk.handlers):
if isinstance(handler, logging.StreamHandler) and not isinstance(
handler, logging.FileHandler
):
sdk.removeHandler(handler)
if any(isinstance(handler, _CoreLogForwarder) for handler in sdk.handlers):
return
core = next(
(
handler
for handler in logging.getLogger().handlers
if isinstance(handler, RotatingFileHandler)
and Path(handler.baseFilename).name == "core.log"
),
None,
)
if core is not None:
sdk.addHandler(_CoreLogForwarder(core))
@@ -52,6 +52,7 @@ from one_memory.memory_tasks import (
from one_memory.memory_tasks import (
POLICY_STOPPED_CODES as _TASK_POLICY_STOPPED_CODES,
)
from one_memory.sdk_migration_log import error_chain_text
from one_memory.service_levels import ServiceLevel, service_level_context
logger = logging.getLogger(__name__)
@@ -241,14 +242,16 @@ PERMISSION_REQUIRED_CODES = frozenset(
)
#: Model-side codes the SDK write coordinator raises before it can lease a
#: client (no active profile, missing credential, secure storage down). They prove
#: nothing was sent, so the write parks instead of being lost. Transport codes
#: that can arrive after a request was in flight stay `unknown`.
#: client (no active profile, missing credential, secure storage down, or the
#: client failed to construct). They prove nothing was sent, so the write
#: parks instead of being lost. Transport codes that can arrive after a
#: request was in flight stay `unknown`.
MODEL_NOT_READY_CODES = frozenset(
{
"model_profile_not_active",
"model_secret_unavailable",
"secure_storage_unavailable",
"hy_memory_client_unavailable",
}
)
@@ -292,6 +295,23 @@ _VALID_JOB_STATES = frozenset(
#: A row in one of these states can never be moved back to a live one.
_TERMINAL_WRITE_STATES = frozenset({COMPLETED, FAILED, DEAD, CANCELLED})
#: The legacy task rows a write job can be linked to. The SDK cutover drops
#: them (`target_schema.LEGACY_TABLES_IN_DROP_ORDER`, inputs before tasks), so
#: the job-only paths must not assume any of them.
_LEGACY_TASK_LINK_TABLES = ("memory_tasks", "memory_task_inputs", "memory_write_job_evidence")
def _legacy_task_links_present(connection: sqlite3.Connection) -> bool:
present = {
str(row[0])
for row in connection.execute(
"SELECT name FROM sqlite_master WHERE type='table' AND name IN (?,?,?)",
_LEGACY_TASK_LINK_TABLES,
)
}
return present == set(_LEGACY_TASK_LINK_TABLES)
#: Columns a current schema declares but an older `memory_write_jobs` may lack.
#: `CREATE TABLE IF NOT EXISTS` is a no-op for an existing table, so without
#: this an upgraded install would keep an older queue that every claim/lease
@@ -1982,21 +2002,24 @@ class DurableWriteQueue:
or str(row["last_error"]) in NON_RETRYABLE_POST_WRITE_CODES
):
return 0
connection.execute(
"""
UPDATE memory_tasks
SET status='pending', runner_token=NULL, attempts=0,
last_error_code='model_content_manual_retry_requested', updated_at=?
WHERE status='unknown' AND task_id IN (
SELECT inputs.task_id
FROM memory_task_inputs AS inputs, json_each(inputs.refs_json) AS ref
JOIN memory_write_job_evidence AS linked
ON linked.evidence_id=json_extract(ref.value, '$.evidence_id')
WHERE linked.job_id=?
# After the SDK cutover the legacy task tables are gone and the
# job is the only row to reset.
if not self.sdk_authority and _legacy_task_links_present(connection):
connection.execute(
"""
UPDATE memory_tasks
SET status='pending', runner_token=NULL, attempts=0,
last_error_code='model_content_manual_retry_requested', updated_at=?
WHERE status='unknown' AND task_id IN (
SELECT inputs.task_id
FROM memory_task_inputs AS inputs, json_each(inputs.refs_json) AS ref
JOIN memory_write_job_evidence AS linked
ON linked.evidence_id=json_extract(ref.value, '$.evidence_id')
WHERE linked.job_id=?
)
""",
(now, job_id),
)
""",
(now, job_id),
)
changed = connection.execute(
"""
UPDATE memory_write_jobs
@@ -2098,9 +2121,7 @@ class DurableWriteQueue:
f"WHERE state=? AND job_id IN ({placeholders})",
(PENDING, now, UNKNOWN, *job_ids),
)
if connection.execute(
"SELECT 1 FROM sqlite_master WHERE type='table' AND name='memory_tasks'"
).fetchone():
if _legacy_task_links_present(connection):
connection.execute(
f"UPDATE memory_tasks SET status=?, runner_token=NULL, "
f"last_error_code='model_content_reconciled_absent_retry', updated_at=? "
@@ -2158,9 +2179,7 @@ class DurableWriteQueue:
f"WHERE state=? AND attempts=1 AND job_id IN ({placeholders})",
(PENDING, now, UNKNOWN, *job_ids),
)
if connection.execute(
"SELECT 1 FROM sqlite_master WHERE type='table' AND name='memory_tasks'"
).fetchone():
if _legacy_task_links_present(connection):
connection.execute(
f"UPDATE memory_tasks SET status=?, runner_token=NULL, "
f"last_error_code='model_content_unknown_auto_retry', updated_at=? "
@@ -2177,6 +2196,14 @@ class DurableWriteQueue:
def resolve_unknown_from_linked_output(self, job_id: str) -> bool:
"""Close an unknown durable row when local governance output proves success."""
with main_database(self.database_path).read(row_factory=sqlite3.Row) as connection:
if not (
_legacy_task_links_present(connection)
and connection.execute(
"SELECT 1 FROM sqlite_master WHERE type='table' AND name='memory_task_outputs'"
).fetchone()
):
# No legacy governance output survives the SDK cutover.
return False
row = connection.execute(
"""
SELECT 1
@@ -2559,9 +2586,22 @@ async def execute_job(
if code == "model_profile_not_active":
logger.debug("durable_write_job_parked job_id=%s error_code=%s", job_id, code)
elif state in PARKED_STATES:
logger.info("durable_write_job_parked job_id=%s error_code=%s", job_id, code)
logger.info(
"durable_write_job_parked job_id=%s error_code=%s chain=%s",
job_id,
code,
error_chain_text(error),
)
else:
logger.error("durable_write_job_failed job_id=%s error_code=%s", job_id, code)
# Where it failed, as `file:line:function` frames of the exception
# chain, never its message: `error_code=AttributeError` alone
# named no location.
logger.error(
"durable_write_job_failed job_id=%s error_code=%s chain=%s",
job_id,
code,
error_chain_text(error),
)
return
await in_the_background(
queue.settle,
@@ -5,7 +5,8 @@ from __future__ import annotations
import asyncio
import logging
import re
from contextlib import suppress
from collections.abc import AsyncIterator
from contextlib import AsyncExitStack, asynccontextmanager, suppress
from datetime import UTC, datetime, timedelta
from pathlib import Path
from typing import Any
@@ -164,44 +165,45 @@ class SdkMemoryRuntime:
if str(error) == "model_profile_not_active":
return HyGraphSnapshot(consulted=False)
raise
try:
client = await self.coordinator.client()
except Exception as error:
raise HySdkError("hy_memory_graph_read_failed") from error
return await asyncio.to_thread(
read_graph_snapshot,
client,
"local",
"*",
limit,
)
async with AsyncExitStack() as stack:
try:
client = await stack.enter_async_context(self.coordinator.use_client())
except Exception as error:
raise HySdkError("hy_memory_graph_read_failed") from error
return await asyncio.to_thread(
read_graph_snapshot,
client,
"local",
"*",
limit,
)
async def digest(self) -> dict[str, object]:
client = await self.coordinator.client()
try:
raw = await asyncio.to_thread(
client.list_memories,
user_id="local",
limit=1_000,
offset=0,
)
agent_ids = _agent_ids(raw)
digests = [
{
"agent_id": agent_id,
"schemas_recorded": 0,
**dict(
await asyncio.to_thread(
client.evolve,
user_id="local",
agent_id=agent_id,
)
),
}
for agent_id in agent_ids
]
except Exception as error:
raise HySdkError("hy_memory_digest_failed") from error
async with self.coordinator.use_client() as client:
try:
raw = await asyncio.to_thread(
client.list_memories,
user_id="local",
limit=1_000,
offset=0,
)
agent_ids = _agent_ids(raw)
digests = [
{
"agent_id": agent_id,
"schemas_recorded": 0,
**dict(
await asyncio.to_thread(
client.evolve,
user_id="local",
agent_id=agent_id,
)
),
}
for agent_id in agent_ids
]
except Exception as error:
raise HySdkError("hy_memory_digest_failed") from error
return {
"success": all(result.get("success") is not False for result in digests),
"scope_count": len(digests),
@@ -240,36 +242,39 @@ class SdkMemoryRuntime:
decision=decision,
)
async def _mutation_client(self) -> Any:
@asynccontextmanager
async def _mutation_client(self) -> AsyncIterator[Any]:
"""The write client, with no-model and keychain refusals as SDK codes.
Callers record these against the edit or deletion instead of failing
the request that already committed it.
"""
try:
return await self.coordinator.client()
except RuntimeError as error:
code = str(error)
if not _STABLE_CODE.fullmatch(code):
code = "hy_memory_client_unavailable"
raise HySdkError(code) from error
async with AsyncExitStack() as stack:
try:
client = await stack.enter_async_context(self.coordinator.use_client())
except RuntimeError as error:
code = str(error)
if not _STABLE_CODE.fullmatch(code):
code = "hy_memory_client_unavailable"
raise HySdkError(code) from error
yield client
async def update_hy_memory(self, memory_id: str, content: str) -> dict[str, object]:
client = await self._mutation_client()
try:
result = await asyncio.to_thread(client.update, memory_id, content)
except Exception as error:
raise HySdkError("hy_memory_update_failed") from error
async with self._mutation_client() as client:
try:
result = await asyncio.to_thread(client.update, memory_id, content)
except Exception as error:
raise HySdkError("hy_memory_update_failed") from error
if not isinstance(result, dict) or result.get("success") is not True:
raise HySdkError("hy_memory_update_rejected")
return result
async def finish_deletion(self, memory_id: str) -> None:
client = await self._mutation_client()
try:
result = await asyncio.to_thread(client.delete, memory_id)
except Exception as error:
raise HySdkError("hy_memory_delete_failed") from error
async with self._mutation_client() as client:
try:
result = await asyncio.to_thread(client.delete, memory_id)
except Exception as error:
raise HySdkError("hy_memory_delete_failed") from error
if not isinstance(result, dict):
raise HySdkError("hy_memory_delete_rejected")
error_code = result.get("error_code")
@@ -35,6 +35,8 @@ from one_memory.memory_server.durable_queue import (
COMPLETED,
FAILED,
REAPED_PAYLOAD_MARKER,
RUNNING,
UNKNOWN,
DurableWriteQueue,
)
from one_memory.memory_server.unified_entry import UnifiedWriteIntake
@@ -317,11 +319,45 @@ def move_pending_dispatches(database_path: Path) -> int:
return len(conversations)
#: Why a pre-cutover write job that may have reached HY was closed.
ABANDONED_AT_CUTOVER = "abandoned_at_cutover"
def settle_legacy_unknown_write_jobs(database_path: Path) -> int:
"""Close write jobs the legacy pipeline left possibly-sent. Safe to repeat.
They ran under the legacy pipeline, so the SDK's `get_write_outcome` knows
none of them and the unknown reconcile can never settle them: a migrated
1.0.4 home showed 1000+ unknown forever. They are not retried, since they
may already have written and the SDK has no idempotency key. `cancelled`,
not `failed`: the upgrade ended them, nothing broke, and a `failed` count
would put the home in the failed state for good. `hy_invoked` is kept, so
no readmission path can pick one up again.
"""
with main_database(database_path).read(row_factory=sqlite3.Row) as database:
if "memory_write_jobs" not in _tables(database):
return 0
now = datetime.now(UTC).isoformat()
with main_database(database_path).write(autocommit=True) as database:
# A `running` row already marked invoked would be reclaimed to
# `unknown` on the next open, so it is the same case.
return int(
database.execute(
"UPDATE memory_write_jobs SET state=?,lease_owner=NULL,"
"lease_expires_at=NULL,last_error=?,updated_at=? "
"WHERE state=? OR (state=? AND hy_invoked=1)",
(CANCELLED, ABANDONED_AT_CUTOVER, now, UNKNOWN, RUNNING),
).rowcount
)
def abandoned_legacy_work(database_path: Path) -> dict[str, int]:
"""Count legacy work that ends with the cutover instead of blocking it.
A task that may have reached the SDK is not replayed: the SDK already holds
whatever it committed. Pending user deletions are not carried over.
Settled write jobs are counted by their code, so a repeated preparation
reports the same number.
"""
abandoned: dict[str, int] = {}
with main_database(database_path).read(row_factory=sqlite3.Row) as database:
@@ -332,6 +368,11 @@ def abandoned_legacy_work(database_path: Path) -> dict[str, int]:
"memory_tasks",
"status='unknown' OR (status='running' AND hy_invocation_count > 0)",
),
(
"unknown_write_jobs",
"memory_write_jobs",
f"state='{CANCELLED}' AND last_error='{ABANDONED_AT_CUTOVER}'",
),
("deletions", "memory_deletions", "status IN ('pending','running')"),
)
for name, table, where in checks:
@@ -354,6 +395,10 @@ def prepare_legacy_work_for_cutover(
running, so a failure names it.
"""
trace = {} if trace is None else trace
# First: a possibly-sent job never runs again, so its input is not
# frozen, and input released since must not block the cutover.
trace["sub_step"] = "unknown_write_jobs"
settle_legacy_unknown_write_jobs(database_path)
trace["sub_step"] = "write_queue"
frozen, moved = prepare_legacy_write_queue_for_cutover(database_path)
trace["sub_step"] = "pending_dispatches"
@@ -3,7 +3,8 @@
from __future__ import annotations
import asyncio
from collections.abc import Awaitable, Callable, Mapping, Sequence
from collections.abc import AsyncIterator, Awaitable, Callable, Mapping, Sequence
from contextlib import asynccontextmanager
from dataclasses import dataclass
from datetime import datetime
from typing import Any
@@ -56,7 +57,12 @@ class SdkWriteProcessor:
scope = _write_scope(records)
metadata = _write_metadata(records, scope)
client = await self._client_handle.client()
try:
client = await self._client_handle.client()
except Exception as error:
# Before `before_hy_invocation`: nothing was sent, so the job
# waits and retries instead of being left `unknown`.
raise RuntimeError("hy_memory_client_unavailable") from error
if extracted:
contents = [sanitize_model_input(record.verbatim) for record in records]
@@ -125,7 +131,14 @@ class SdkWriteProcessor:
class SdkWriteCoordinator:
"""Resolve the active model profile, then execute one public SDK write."""
"""Resolve the active model profile, then execute one public SDK write.
A model profile change retires the previous profile's handle, but only
closes it once nothing uses it. Closing it at once tore the client out
from under a write still running on it: the SDK cleared its stores and
the SharedRuntime closed mid-`add`, the job failed with AttributeError
after HY was invoked, and it was left `unknown`.
"""
def __init__(
self,
@@ -142,6 +155,9 @@ class SdkWriteCoordinator:
self.transport_factory = transport_factory
self.sedimentation_enabled = sedimentation_enabled
self._handles: dict[tuple[str, int], SdkClientHandle] = {}
#: Uses in flight per handle; a retired handle closes at zero.
self._uses: dict[SdkClientHandle, int] = {}
self._retired: set[SdkClientHandle] = set()
self._lock = asyncio.Lock()
async def process_sdk_job(
@@ -153,28 +169,30 @@ class SdkWriteCoordinator:
) -> SdkWriteReceipt:
if not self.sedimentation_enabled():
raise RuntimeError("model_sedimentation_disabled")
handle = await self._active_handle()
return await SdkWriteProcessor(handle).process_sdk_job(
records,
request_id=request_id,
before_hy_invocation=before_hy_invocation,
)
async with self._active_handle() as handle:
return await SdkWriteProcessor(handle).process_sdk_job(
records,
request_id=request_id,
before_hy_invocation=before_hy_invocation,
)
async def inquire_write_outcome(
self, request_id: str
) -> SdkWriteReceipt | None:
return await SdkWriteProcessor(
await self._active_handle()
).inquire_write_outcome(request_id)
async with self._active_handle() as handle:
return await SdkWriteProcessor(handle).inquire_write_outcome(request_id)
async def close(self) -> None:
async with self._lock:
handles, self._handles = tuple(self._handles.values()), {}
handles = (*self._handles.values(), *self._retired)
self._handles, self._retired, self._uses = {}, set(), {}
await asyncio.gather(*(handle.close() for handle in handles))
async def client(self) -> Any:
"""Return the public SDK client selected by the active write profile."""
return await (await self._active_handle()).client()
@asynccontextmanager
async def use_client(self) -> AsyncIterator[Any]:
"""The public SDK client of the active write profile, kept open while used."""
async with self._active_handle() as handle:
yield await handle.client()
async def active_profile_credentials(
self,
@@ -183,7 +201,9 @@ class SdkWriteCoordinator:
_identity, profile, secret = await asyncio.to_thread(self._read_active_credentials)
return profile, secret
async def _active_handle(self) -> SdkClientHandle:
@asynccontextmanager
async def _active_handle(self) -> AsyncIterator[SdkClientHandle]:
"""Use the active profile's handle; retire the others, closing the unused."""
identity, profile, secret = await asyncio.to_thread(
self._read_active_credentials
)
@@ -197,14 +217,25 @@ class SdkWriteCoordinator:
http_transport_factory=self.transport_factory(profile),
)
self._handles[identity] = handle
stale = [
value
for key, value in self._handles.items()
if key != identity
]
self._retired.update(value for key, value in self._handles.items() if key != identity)
self._handles = {identity: handle}
await asyncio.gather(*(value.close() for value in stale))
return handle
self._uses[handle] = self._uses.get(handle, 0) + 1
unused = {value for value in self._retired if not self._uses.get(value)}
self._retired -= unused
await asyncio.gather(*(value.close() for value in unused))
try:
yield handle
finally:
# Bookkeeping without an await, so a cancelled caller still
# releases its use.
remaining = self._uses.get(handle, 0) - 1
if remaining > 0:
self._uses[handle] = remaining
else:
self._uses.pop(handle, None)
if remaining <= 0 and handle in self._retired:
self._retired.discard(handle)
await handle.close()
def _read_active_credentials(
self,
+4 -2
View File
@@ -79,7 +79,7 @@
| `test_database_file_permissions.py` | `contract` | 覆盖 a new main database is readable by its owner alone、the wal sidecars are not left world readable、a database from an older build is tightened on open 场景。 |
| `test_db_session_utc.py` | `contract` | The MySQL session must be pinned to UTC. |
| `test_derived_echo.py` | `contract` | A host that saves our injection back must not mint a new user decision. |
| `test_detailed_logging.py` | `contract` | 覆盖 detailed pipeline logging defaults off and survives restart、live log level applies to full hy and provider call chain、info collects hy info without enabling debug pipeline content 等 7 个场景。 |
| `test_detailed_logging.py` | `contract` | 覆盖 detailed logging defaults off and survives restart、hy pipeline log and trace stay off at every level、live level change reaches the hy text log file 等 7 个场景。 |
| `test_diagnostic_log.py` | `contract` | 覆盖 high frequency success events are summarized but warnings are not、structured log is private correlated and owner only、arbitrary detail and absolute stack are rejected 等 14 个场景。 |
| `test_diagnostic_log_durability.py` | `contract` | 写线程报告的「已落盘」必须是真的。 |
| `test_diagnostic_log_off_loop.py` | `contract` | 诊断日志不得在调用线程上做磁盘 I/O。 |
@@ -112,6 +112,7 @@
| `test_hy_log_privacy.py` | `contract` | HY SDK records are not rewritten by One Memory's logging setup. |
| `test_hy_network_egress.py` | `contract` | Where a real HY client is allowed to send bytes, and where it is not. |
| `test_hy_runtime_config.py` | `contract` | MEMORY.md is the audited record of what we send HY; keep it answerable. |
| `test_hy_sdk_log_routing.py` | `contract` | HY SDK text logs go to hy_memory.log only, with warnings also in core.log. |
| `test_hy_step_trace_switch.py` | `contract` | Rule 15: the per-step HY trace is off unless the debug tier asks for it. |
| `test_import_api.py` | `integration` | 覆盖 resume during cleanup returns conflict without internal error、authorize detect import and revoke api flow、protected api rejects missing token and foreign origin 等 5 个场景。 |
| `test_import_control_races_review.py` | `contract` | Read-only review repro: real runner/store, synthetic delayed adapter cleanup. |
@@ -245,6 +246,7 @@
| `test_schema_gate.py` | `contract` | 覆盖 fresh database records the build schema、future schema refuses to open、legacy database without a ledger is left untouched 场景。 |
| `test_schema_migrations.py` | `contract` | 覆盖 evidence store adds source ordinal to previous schema、import store migrates previous job and error schema、old active job index lets a sync sit beside paused history 等 4 个场景。 |
| `test_schema_ownership_gate.py` | `contract` | A1-03/A1-04: schema ownership, and the hard part of telling DDL from reads. |
| `test_sdk_client_construction.py` | `contract` | SDK clients of one home are constructed one at a time. |
| `test_sdk_data_plane_migration.py` | `contract` | SDK-authority migration ledger, aliases and atomic cutover marker. |
| `test_sdk_memory_qa.py` | `contract` | 覆盖 sdk ask propagates sdk search failure without legacy fallback、sdk ask revalidates memory before returning extract、sdk ask uses native sdk memory without governance fields 场景。 |
| `test_sdk_migration_bundle.py` | `contract` | SDK-authority cutover requires a verified management + SDK data-plane bundle. |
@@ -307,4 +309,4 @@
| `test_workspace_recall.py` | `contract` | Workspace memories were unreachable from every host; these pin the way in. |
| `test_write_queue_lease_renewal.py` | `contract` | Rule 8: a lease must outlive the work it covers, and be renewed while it runs. |
共 302 个测试文件。
共 304 个测试文件。
@@ -14,8 +14,8 @@ from pathlib import Path
import pytest
PRIMARY_MARKERS = ("unit", "contract", "integration", "performance", "live")
EXPECTED_TEST_FILE_COUNT = 302
EXPECTED_TEST_FILES_SHA256 = "07707528025ac134e46c93ebef12335e852df1ed0f52f8beb62abb3b4d1b0e97"
EXPECTED_TEST_FILE_COUNT = 304
EXPECTED_TEST_FILES_SHA256 = "b28f022b9688c96ca563c59de63e95d344d2f01ba3b6020b422bf4dfb27520d6"
# Pure transformations, validation models, parity checks and state machines.
# This list is intentionally small: uncertain files default to contract.
+40 -43
View File
@@ -1,6 +1,5 @@
import json
import logging
from importlib import import_module
from pathlib import Path
from one_memory.hy_runtime_config import (
@@ -12,20 +11,50 @@ from one_memory.hy_runtime_config import (
)
def test_detailed_pipeline_logging_defaults_off_and_survives_restart(
tmp_path: Path, monkeypatch
) -> None:
monkeypatch.delenv("MEMORY_PIPELINE_TRACE_ENABLED", raising=False)
def test_detailed_logging_defaults_off_and_survives_restart(tmp_path: Path, monkeypatch) -> None:
monkeypatch.delenv("MEMORY_LOG_LEVEL", raising=False)
assert detailed_logging_enabled(tmp_path) is False
configure_hy_runtime_environment(tmp_path)
assert __import__("os").environ["MEMORY_PIPELINE_TRACE_ENABLED"] == "false"
assert __import__("os").environ["MEMORY_LOG_LEVEL"] == "INFO"
assert diagnostic_log_level(tmp_path) == "info"
assert set_detailed_logging(tmp_path, True) is True
assert detailed_logging_enabled(tmp_path) is True
monkeypatch.delenv("MEMORY_PIPELINE_TRACE_ENABLED", raising=False)
monkeypatch.delenv("MEMORY_LOG_LEVEL", raising=False)
configure_hy_runtime_environment(tmp_path)
assert __import__("os").environ["MEMORY_PIPELINE_TRACE_ENABLED"] == "true"
assert __import__("os").environ["MEMORY_LOG_LEVEL"] == "DEBUG"
def test_hy_pipeline_log_and_trace_stay_off_at_every_level(tmp_path: Path, monkeypatch) -> None:
environment = __import__("os").environ
for level in ("error", "warning", "info", "debug"):
for name in ("MEMORY_PIPELINE_LOG_ENABLED", "MEMORY_PIPELINE_TRACE_ENABLED"):
monkeypatch.setenv(name, "true")
set_diagnostic_log_level(tmp_path, level)
assert environment["MEMORY_PIPELINE_LOG_ENABLED"] == "false"
assert environment["MEMORY_PIPELINE_TRACE_ENABLED"] == "false"
for name in ("MEMORY_PIPELINE_LOG_ENABLED", "MEMORY_PIPELINE_TRACE_ENABLED"):
monkeypatch.setenv(name, "true")
configure_hy_runtime_environment(tmp_path)
assert environment["MEMORY_PIPELINE_LOG_ENABLED"] == "false"
assert environment["MEMORY_PIPELINE_TRACE_ENABLED"] == "false"
def test_live_level_change_reaches_the_hy_text_log_file(tmp_path: Path) -> None:
sdk = logging.getLogger("hy_memory")
text_log = logging.FileHandler(tmp_path / "hy_memory.log", encoding="utf-8")
text_log.setLevel(logging.INFO)
sdk.addHandler(text_log)
try:
set_diagnostic_log_level(tmp_path, "debug")
assert text_log.level == logging.DEBUG
set_diagnostic_log_level(tmp_path, "warning")
assert text_log.level == logging.WARNING
finally:
sdk.removeHandler(text_log)
text_log.close()
set_diagnostic_log_level(tmp_path, "info")
def test_live_log_level_applies_to_full_hy_and_provider_call_chain(
@@ -60,43 +89,9 @@ def test_info_collects_hy_info_without_enabling_debug_pipeline_content(
assert logging.getLogger("openai._base_client").isEnabledFor(logging.INFO)
assert not logging.getLogger("openai._base_client").isEnabledFor(logging.DEBUG)
assert __import__("os").environ["MEMORY_LOG_LEVEL"] == "INFO"
assert __import__("os").environ["MEMORY_PIPELINE_TRACE_ENABLED"] == "false"
assert __import__("os").environ["MEMORY_LOG_LLM_CONTENT"] == "false"
def test_live_debug_switch_makes_sdk_pipeline_writer_keep_original_content(
tmp_path: Path,
) -> None:
writer_type = import_module("hy_memory.utils.pipeline_log_writer").PipelineLogWriter
writer = writer_type(str(tmp_path / "pipeline"))
set_diagnostic_log_level(tmp_path, "info")
writer.write_step(
request_id="info",
step="EXTRACT",
prompt="info prompt",
response="info response",
)
set_diagnostic_log_level(tmp_path, "debug")
writer.write_step(
request_id="debug",
step="EXTRACT",
prompt="debug prompt",
response="debug response",
)
lines = [
json.loads(line)
for target in (tmp_path / "pipeline").rglob("*.log")
for line in target.read_text(encoding="utf-8").splitlines()
]
info, debug = lines
assert info["prompt"].startswith("<redacted len=")
assert info["content"].startswith("<redacted len=")
assert debug["prompt"] == "debug prompt"
assert debug["content"] == "debug response"
def test_log_level_document_migrates_the_old_boolean_shape(tmp_path: Path) -> None:
target = tmp_path / "settings" / "diagnostic-logging.json"
target.parent.mkdir(parents=True)
@@ -117,6 +112,7 @@ def test_configure_environment_passes_debug_content_settings_to_sdk(
"MEMORY_DATA_DIR",
"MEMORY_LOG_DIR",
"MEMORY_LOG_LEVEL",
"MEMORY_PIPELINE_LOG_ENABLED",
"MEMORY_PIPELINE_TRACE_ENABLED",
"MEMORY_LOG_LLM_CONTENT",
):
@@ -129,5 +125,6 @@ def test_configure_environment_passes_debug_content_settings_to_sdk(
assert environment["MEMORY_DATA_DIR"] == str(memory_root)
assert environment["MEMORY_LOG_DIR"] == str(memory_root / "logs")
assert environment["MEMORY_LOG_LEVEL"] == "DEBUG"
assert environment["MEMORY_PIPELINE_TRACE_ENABLED"] == "true"
assert environment["MEMORY_PIPELINE_LOG_ENABLED"] == "false"
assert environment["MEMORY_PIPELINE_TRACE_ENABLED"] == "false"
assert environment["MEMORY_LOG_LLM_CONTENT"] == "true"
@@ -0,0 +1,146 @@
"""HY SDK text logs go to hy_memory.log only, with warnings also in core.log.
The SDK's first client adds a stdout and a stderr stream to the "hy_memory"
logger and stops it propagating. Under launchd those streams are
service.log and service-error.log, so each SDK line used to be written
twice and none reached core.log.
"""
import logging
import sys
from logging.handlers import RotatingFileHandler
from pathlib import Path
from types import SimpleNamespace
import pytest
from one_memory import hy_sdk_runtime
from one_memory.log_file import (
HY_SDK_LOGGER,
confine_hy_sdk_logging,
log_path,
start_logging_to_file,
)
@pytest.fixture
def sdk_logger():
"""The "hy_memory" logger as HY's setup_logging leaves it, restored after."""
sdk = logging.getLogger(HY_SDK_LOGGER)
root = logging.getLogger()
saved = (list(sdk.handlers), sdk.propagate, sdk.level, list(root.handlers), root.level)
yield sdk
for handler in sdk.handlers:
if handler not in saved[0]:
handler.close()
for handler in root.handlers:
if handler not in saved[3]:
handler.close()
sdk.handlers, sdk.propagate, root.handlers = saved[0], saved[1], saved[3]
sdk.setLevel(saved[2])
root.setLevel(saved[4])
def _configure_like_hy(sdk: logging.Logger, log_dir: Path) -> logging.FileHandler:
log_dir.mkdir(parents=True, exist_ok=True)
text_log = logging.FileHandler(log_dir / "hy_memory.log", encoding="utf-8")
stdout = logging.StreamHandler(sys.stdout)
stderr = logging.StreamHandler(sys.stderr)
stderr.setLevel(logging.WARNING)
sdk.handlers = [text_log, stdout, stderr]
sdk.propagate = False
sdk.setLevel(logging.INFO)
return text_log
def test_sdk_lines_stay_in_hy_memory_log_and_warnings_reach_core_log(
tmp_path: Path, sdk_logger: logging.Logger
) -> None:
start_logging_to_file(tmp_path)
text_log = _configure_like_hy(sdk_logger, tmp_path / "hy")
confine_hy_sdk_logging()
confine_hy_sdk_logging()
assert not [
handler
for handler in sdk_logger.handlers
if isinstance(handler, logging.StreamHandler)
and not isinstance(handler, logging.FileHandler)
]
assert text_log in sdk_logger.handlers
assert len(sdk_logger.handlers) == 2, "the core.log forwarder is added once"
child = logging.getLogger("hy_memory.client")
child.info("[add] routine sdk line")
child.warning("[add] sdk problem")
for handler in logging.getLogger().handlers:
handler.flush()
text_log.flush()
hy_text = (tmp_path / "hy" / "hy_memory.log").read_text(encoding="utf-8")
core_text = log_path(tmp_path).read_text(encoding="utf-8")
assert "routine sdk line" in hy_text and "sdk problem" in hy_text
assert "sdk problem" in core_text
assert "routine sdk line" not in core_text
assert core_text.count("sdk problem") == 1
def test_nothing_changes_before_hy_has_configured_its_logger(
tmp_path: Path, sdk_logger: logging.Logger
) -> None:
start_logging_to_file(tmp_path)
sdk_logger.handlers = []
sdk_logger.propagate = True
confine_hy_sdk_logging()
assert sdk_logger.handlers == []
def test_without_core_log_the_streams_are_still_dropped(
tmp_path: Path, sdk_logger: logging.Logger
) -> None:
logging.getLogger().handlers = [
handler
for handler in logging.getLogger().handlers
if not isinstance(handler, RotatingFileHandler)
]
text_log = _configure_like_hy(sdk_logger, tmp_path / "hy")
confine_hy_sdk_logging()
assert sdk_logger.handlers == [text_log]
@pytest.mark.parametrize("fails", [False, True])
def test_every_client_construction_confines_the_sdk_log(
tmp_path: Path, monkeypatch, fails: bool
) -> None:
order: list[str] = []
class Client:
def __init__(self, **_options):
order.append("client")
if fails:
raise RuntimeError("construction failed")
monkeypatch.setattr(hy_sdk_runtime, "HyMemoryClient", Client)
monkeypatch.setattr(hy_sdk_runtime, "build_sdk_config", lambda *_a, **_k: SimpleNamespace())
monkeypatch.setattr(hy_sdk_runtime, "inspect_graph_width_mismatch", lambda _c: None)
monkeypatch.setattr(hy_sdk_runtime, "_acquire_runtime", lambda _c, _k: object())
monkeypatch.setattr(hy_sdk_runtime, "_release_runtime", lambda _k: None)
monkeypatch.setattr(hy_sdk_runtime, "confine_hy_sdk_logging", lambda: order.append("confine"))
profile = SimpleNamespace(embedding_provider_kind=None, provider_kind=None)
if fails:
with pytest.raises(RuntimeError):
hy_sdk_runtime._acquire_client(
profile, api_key=None, data_dir=tmp_path, http_transport_factory=None
)
else:
hy_sdk_runtime._acquire_client(
profile, api_key=None, data_dir=tmp_path, http_transport_factory=None
)
assert order == ["client", "confine"]
@@ -6,6 +6,7 @@ and delete pass the memory's kind/variant through to the SDK.
"""
import asyncio
from contextlib import asynccontextmanager
import pytest
from httpx import ASGITransport, AsyncClient
@@ -310,12 +311,13 @@ def run_with_sdk(tmp_path, monkeypatch, scenario):
async def close(self):
return None
@asynccontextmanager
async def write_client():
return sdk
yield sdk
async with app.router.lifespan_context(app):
app.state.sdk_client_handle = Handle()
monkeypatch.setattr(app.state.sdk_write_coordinator, "client", write_client)
monkeypatch.setattr(app.state.sdk_write_coordinator, "use_client", write_client)
async with client_for(app) as client:
await scenario(app, sdk, client)
@@ -1187,6 +1187,167 @@ def test_the_drain_tick_writes_a_quiet_live_conversation(tmp_path, monkeypatch)
assert flushed
class _SwitchableProfiles:
"""Active profile identity that a test can change mid-write."""
def __init__(self) -> None:
self.identity = ("profile-a", 1)
def active_identity(self):
return self.identity
def snapshot(self, profile_id, revision):
return type(
"Snapshot", (), {"profile_id": profile_id, "revision": revision, "has_api_key": False}
)()
def resolve_snapshot_secret(self, *_args):
return None
class _TrackedHandle:
def __init__(self, profile, **_kwargs) -> None:
self.profile_id = profile.profile_id
self.closed = False
async def client(self):
return self
async def close(self):
self.closed = True
def _coordinator(monkeypatch, opened: list[_TrackedHandle]):
from one_memory import sdk_write_processor
def handle(profile, **kwargs):
value = _TrackedHandle(profile, **kwargs)
opened.append(value)
return value
monkeypatch.setattr(sdk_write_processor, "SdkClientHandle", handle)
profiles = _SwitchableProfiles()
coordinator = sdk_write_processor.SdkWriteCoordinator(
profiles=profiles, # type: ignore[arg-type]
secrets=None, # type: ignore[arg-type]
data_dir=None,
transport_factory=lambda _profile: None,
sedimentation_enabled=lambda: True,
)
return coordinator, profiles
def test_a_profile_change_does_not_close_the_client_of_a_write_in_flight(monkeypatch) -> None:
"""Build 35 lost a clean install's first write as unknown (AttributeError).
Saving a model profile changed the active identity while the write ran on
the old profile's client. The next coordinator call closed that client at
once: the SDK cleared its stores and the SharedRuntime closed mid-`add`.
"""
from one_memory import sdk_write_processor
opened: list[_TrackedHandle] = []
coordinator, profiles = _coordinator(monkeypatch, opened)
seen: list[tuple[str, bool]] = []
writing = asyncio.Event()
async def process(self, records, *, request_id, before_hy_invocation):
client = await self._client_handle.client()
await before_hy_invocation()
seen.append((client.profile_id, client.closed))
if client.profile_id == "profile-a":
await writing.wait()
# Still open while its own write runs.
seen.append((client.profile_id, client.closed))
return sdk_write_processor.SdkWriteReceipt()
monkeypatch.setattr(sdk_write_processor.SdkWriteProcessor, "process_sdk_job", process)
async def noop():
return None
async def scenario():
first = asyncio.create_task(
coordinator.process_sdk_job((), request_id="a", before_hy_invocation=noop)
)
while not seen:
await asyncio.sleep(0)
profiles.identity = ("profile-b", 2)
# A second job and an inquiry under the new profile.
await coordinator.process_sdk_job((), request_id="b", before_hy_invocation=noop)
assert [handle.closed for handle in opened] == [False, False]
writing.set()
await first
old, new = opened
# Retired after its last use, and new work used the new profile.
assert old.closed is True and new.closed is False
await coordinator.close()
assert new.closed is True
asyncio.run(scenario())
assert seen == [
("profile-a", False),
("profile-b", False),
("profile-b", False),
("profile-a", False),
]
def test_an_unused_stale_handle_still_closes_at_once(monkeypatch) -> None:
opened: list[_TrackedHandle] = []
coordinator, profiles = _coordinator(monkeypatch, opened)
async def scenario():
async with coordinator.use_client() as client:
assert client.profile_id == "profile-a"
assert opened[0].closed is False
profiles.identity = ("profile-b", 2)
async with coordinator.use_client() as client:
assert client.profile_id == "profile-b"
assert opened[0].closed is True
assert opened[1].closed is False
await coordinator.close()
assert opened[1].closed is True
asyncio.run(scenario())
def test_a_client_that_fails_to_construct_parks_the_job_instead_of_unknown(tmp_path) -> None:
"""Nothing was sent when the client could not be built, so it retries on its own."""
from one_memory.sdk_write_processor import SdkWriteProcessor
class _BrokenHandle:
async def client(self):
raise AttributeError("'RustBindingsAPI' object has no attribute 'bindings'")
path = tmp_path / "one-memory.sqlite3"
queue = DurableWriteQueue(path)
record = queue.submit(_request())
_claim_through_drain(queue, record.job_id)
class _Supervisor:
async def process_sdk_job(self, _records, **kwargs):
return await SdkWriteProcessor(_BrokenHandle()).process_sdk_job( # type: ignore[arg-type]
(_sdk_evidence("e1"),), **kwargs
)
asyncio.run(
execute_job(
record.job_id,
queue=queue,
supervisor=_Supervisor(),
evidence=_EvidenceStore({"e1"}),
)
)
[row] = _rows(path)
assert (row["state"], row["hy_invoked"]) == (WAITING_FOR_MODEL, 0)
assert row["last_error"] == "hy_memory_client_unavailable"
later = datetime.now(UTC) + timedelta(minutes=5)
assert queue.readmit_model_waiters(now=later) == 1
assert _state(path, record.job_id) == PENDING
def test_sdk_unknown_outcome_inquiry_uses_only_public_receipt() -> None:
from one_memory.sdk_write_processor import SdkWriteProcessor, SdkWriteReceipt
@@ -1699,6 +1860,46 @@ def test_a_raw_provider_message_is_never_stored_as_the_error(tmp_path) -> None:
assert stored == "RuntimeError"
def test_a_failed_job_logs_where_it_failed_but_never_the_message(tmp_path, caplog) -> None:
"""`error_code=AttributeError` alone named no location; frames, never the message."""
path = tmp_path / "one-memory.sqlite3"
queue = DurableWriteQueue(path)
record = queue.submit(_request())
_claim_through_drain(queue, record.job_id)
def closed_store(client):
return client.private_store_name.write
class _Failing:
async def process_sdk_job(self, _records, **kwargs):
await kwargs["before_hy_invocation"]()
try:
closed_store(type("Client", (), {"private_store_name": None})())
except AttributeError as error:
raise RuntimeError("quoting 'my secret diary'") from error
with caplog.at_level(logging.ERROR, logger="one_memory.memory_server.durable_queue"):
asyncio.run(
execute_job(
record.job_id,
queue=queue,
supervisor=_Failing(),
evidence=_EvidenceStore({"e1"}),
)
)
[line] = [
message.getMessage()
for message in caplog.records
if "durable_write_job_failed" in message.getMessage()
]
assert f"job_id={record.job_id} error_code=RuntimeError chain=RuntimeError@" in line
assert ";AttributeError@" in line
assert "test_memory_server_durable_queue.py:" in line and ":closed_store" in line
assert "secret diary" not in line
assert "private_store_name" not in line
def test_a_machine_error_code_is_preserved(tmp_path) -> None:
"""Recognised codes survive; only free-form text is dropped."""
from one_memory.memory_server.durable_queue import safe_error_code
@@ -0,0 +1,61 @@
"""SDK clients of one home are constructed one at a time.
Build 35, clean install: right after the model was activated, the write
drain's client and a UI read client started in the same second. Each opened
Chroma on the same empty vector directory, Chroma's shared System cache has
no lock around that first open, and neither client reached `VectorStore
ready`. The first write job was lost as unknown.
"""
import threading
import time
from one_memory import hy_sdk_runtime
from one_memory.hy_sdk_runtime import acquire_client, builtin_retrieval_snapshot
def test_two_clients_on_a_fresh_home_are_constructed_one_at_a_time(tmp_path, monkeypatch):
real = hy_sdk_runtime.HyMemoryClient
state = {"inside": 0, "most": 0}
count = threading.Lock()
def recording_client(**options):
with count:
state["inside"] += 1
state["most"] = max(state["most"], state["inside"])
try:
# Widen the window two unserialized constructors would share.
time.sleep(0.2)
return real(**options)
finally:
with count:
state["inside"] -= 1
monkeypatch.setattr(hy_sdk_runtime, "HyMemoryClient", recording_client)
start = threading.Barrier(2)
leases: list[hy_sdk_runtime.ClientLease] = []
failures: list[BaseException] = []
def open_client():
start.wait()
try:
leases.append(
acquire_client(builtin_retrieval_snapshot(), api_key=None, data_dir=tmp_path)
)
except BaseException as error: # noqa: BLE001 - reported below
failures.append(error)
threads = [threading.Thread(target=open_client) for _ in range(2)]
for thread in threads:
thread.start()
for thread in threads:
thread.join(120)
try:
assert failures == []
assert len(leases) == 2
assert state["most"] == 1
for lease in leases:
assert lease.client.memory_stats(user_id="local") is not None
finally:
for lease in leases:
lease.close()
@@ -37,6 +37,7 @@ from one_memory.sdk_migration_runner import (
abandoned_legacy_work,
commit_sdk_cutover,
move_pending_dispatches,
prepare_legacy_work_for_cutover,
prepare_legacy_write_queue_for_cutover,
recover_abandoned_staging,
resume_legacy_cleanup,
@@ -896,6 +897,116 @@ def test_legacy_work_that_may_have_reached_the_sdk_does_not_block_cutover(tmp_pa
assert store.authority() is MemoryAuthority.SDK
def test_pre_upgrade_unknown_writes_are_settled_at_cutover_not_retried(store):
"""A migrated 1.0.4 home kept 1000+ jobs unknown forever.
The legacy pipeline invoked them, so the SDK's write-outcome inquiry knows
none of them and the automatic reconcile could never settle one.
"""
from one_memory.memory_server.durable_queue import (
CANCELLED,
PENDING,
RUNNING,
UNKNOWN,
)
queue = DurableWriteQueue(store.database_path)
def job(key: str, state: str, *, invoked: bool) -> str:
identifier = queue.submit(
WriteRequest(
internal_session_id="session-1",
source_id="cursor",
batch_identity=BatchIdentity(
internal_session_id="session-1", first_round=1, last_round=1
),
evidence_ids=["gone"],
idempotency_key=key,
)
).job_id
with main_database(store.database_path).write() as database, database:
database.execute(
"UPDATE memory_write_jobs SET state=?,hy_invoked=?,"
"last_error='model_content_outcome_unknown' WHERE job_id=?",
(state, int(invoked), identifier),
)
return identifier
# Their Evidence is gone too, which must not block the cutover.
unknown = [job(f"unknown-{index}", UNKNOWN, invoked=True) for index in range(3)]
sent_running = job("sent-running", RUNNING, invoked=True)
finished = job("finished", "completed", invoked=True)
first = prepare_legacy_work_for_cutover(store.database_path)
assert first.abandoned == {"unknown_write_jobs": 4}
# Resumed after an interruption: nothing changes, the count is the same.
again = prepare_legacy_work_for_cutover(store.database_path)
assert again.as_document() == first.as_document()
with main_database(store.database_path).read() as database:
rows = dict(
(str(row[0]), (str(row[1]), str(row[2]), int(row[3])))
for row in database.execute(
"SELECT job_id,state,last_error,hy_invoked FROM memory_write_jobs"
)
)
for identifier in (*unknown, sent_running):
assert rows[identifier] == (CANCELLED, "abandoned_at_cutover", 1)
assert rows[finished][0] == "completed"
surviving = DurableWriteQueue(store.database_path, sdk_authority=True)
assert surviving.unknown_jobs() == ()
# Never requeued: they may already have written.
assert surviving.retry_unknown_confirmed(unknown[0]) == 0
assert surviving.readmit_parked() == 0
assert PENDING not in {state for state, _, _ in rows.values()}
def test_retrying_unknown_writes_works_after_the_legacy_tables_are_dropped(store):
"""The retry button raised `no such table: memory_tasks` after an upgrade."""
from one_memory.memory_server.durable_queue import PENDING, UNKNOWN
from one_memory.target_schema import drop_legacy_tables
queue = DurableWriteQueue(store.database_path)
identifier = queue.submit(
WriteRequest(
internal_session_id="session-1",
source_id="cursor",
batch_identity=BatchIdentity(
internal_session_id="session-1", first_round=1, last_round=1
),
evidence_ids=["e1"],
idempotency_key="after-upgrade",
)
).job_id
drop_legacy_tables(store.database_path)
assert legacy_tables_present(store.database_path) == ()
old = datetime(2000, 1, 1, tzinfo=UTC).isoformat()
def make_unknown(code: str, attempts: int) -> None:
with main_database(store.database_path).write() as database, database:
database.execute(
"UPDATE memory_write_jobs SET state=?,hy_invoked=1,last_error=?,"
"attempts=?,updated_at=? WHERE job_id=?",
(UNKNOWN, code, attempts, old, identifier),
)
for sdk_authority in (True, False):
surviving = DurableWriteQueue(store.database_path, sdk_authority=sdk_authority)
make_unknown("model_content_outcome_unknown", 1)
assert surviving.resolve_unknown_from_linked_output(identifier) is False
assert surviving.retry_unknown_confirmed(identifier) == 1
assert surviving.get("after-upgrade").state == PENDING
make_unknown("hy_memory_timeout", 1)
since = datetime.now(UTC)
assert surviving.retry_timed_out_unknown_after_reconcile(older_than=since) == 1
make_unknown("model_content_outcome_unknown", 1)
assert surviving.retry_first_unknown_after_reconcile(older_than=since) == 1
if sdk_authority:
# The SDK-authority queue never brings a dropped table back.
assert legacy_tables_present(store.database_path) == ()
def _protective(home: Path) -> Path:
return home / "hy-memory" / ".governed-v2.pre-migration-0123456789abcdef"
@@ -91,7 +91,7 @@ def test_bundle_contains_management_sdk_config_and_verified_manifest(tmp_path):
assert manifest["bundle_id"] == "migration-1"
assert manifest["sdk_namespace"] == "governed-v2"
assert manifest["runtime_build"] == "build-13"
assert manifest["snapshot_sdk_version"] == "2.0.7"
assert manifest["snapshot_sdk_version"] == "2.0.8"
paths = {item["path"]: item for item in manifest["files"]}
assert {
"management/one-memory.sqlite3",
+4 -4
View File
@@ -991,7 +991,7 @@ wheels = [
[[package]]
name = "hy-memory"
version = "2.0.7"
version = "2.0.8"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "chromadb" },
@@ -1008,9 +1008,9 @@ dependencies = [
{ name = "scikit-learn" },
{ name = "tiktoken" },
]
sdist = { url = "https://files.pythonhosted.org/packages/17/b5/87a43c8d8b77fdfbbe7974d1e93af50ddf82317c5d56e180b865fb19e181/hy_memory-2.0.7.tar.gz", hash = "sha256:85636487e5bd4b6595a8c15301499efb365b6010c236a45c51c144f6986a4222", size = 854913, upload-time = "2026-10-28T03:35:38.127Z" }
sdist = { url = "https://files.pythonhosted.org/packages/21/15/83ca83b649ead238f186fdb33efdc73615fbba279efbbdca3857c1a86beb/hy_memory-2.0.8.tar.gz", hash = "sha256:0fa0ed1a1160b389a3d7c13a9fb6528b140d7ec23cf49e65f1be5974820a576b", size = 856025, upload-time = "2026-10-28T07:35:47.783Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/7c/b8/a6e0bfd17aeb432b0470f4dc707271e6476f5ac34f3b301ab3ad2e1c0d83/hy_memory-2.0.7-py3-none-any.whl", hash = "sha256:80a501e17987202ee1150a91b071313e890300620f353f9e89c967286dc1a645", size = 919841, upload-time = "2026-10-28T03:35:35.399Z" },
{ url = "https://files.pythonhosted.org/packages/11/6a/c699cac0ff06bd089fceeff90446130763d8187e649dbdb0313014491a10/hy_memory-2.0.8-py3-none-any.whl", hash = "sha256:68bd789df24e51e64bb231e97a8d24ee89b4aa5f9ba14c3e0824589e9213bf7c", size = 921119, upload-time = "2026-10-28T07:35:45.571Z" },
]
[[package]]
@@ -1837,7 +1837,7 @@ requires-dist = [
{ name = "fastapi", specifier = ">=0.141.1" },
{ name = "filelock", specifier = ">=3.32.4" },
{ name = "httpx", specifier = ">=0.28.1" },
{ name = "hy-memory", specifier = "==2.0.7" },
{ name = "hy-memory", specifier = "==2.0.8" },
{ name = "keyring", specifier = ">=25.7.0" },
{ name = "numpy", specifier = "==2.5.2" },
{ name = "onnxruntime", specifier = "==1.23.2" },
+17 -53
View File
@@ -230,39 +230,21 @@ it("filters raw Core INFO and DEBUG while preserving warnings and errors", async
).toBe(true);
});
it("shows HY pipeline metrics without exposing model content", async () => {
const dataDir = await mkdtemp(
path.join(os.tmpdir(), "om-debug-hy-pipeline-"),
);
it("does not read HY's own log directory", async () => {
// MemHub turns the HY pipeline log off; files left by older builds and
// HY's text log stay on disk but are not part of the Debug panel.
const dataDir = await mkdtemp(path.join(os.tmpdir(), "om-debug-hy-logs-"));
roots.push(dataDir);
const pipelineDir = path.join(
dataDir,
"hy-memory",
"governed-v2",
"logs",
"default",
);
await mkdir(pipelineDir, { recursive: true });
const target = path.join(pipelineDir, "2026-10-14.log");
await writeFile(target, "");
const hyLogs = path.join(dataDir, "hy-memory", "governed-v2", "logs");
await mkdir(path.join(hyLogs, "default"), { recursive: true });
const cursor = await captureDebugLogCursor(dataDir);
const pipelineEntry = {
request_id: "request-safe-1",
user_id: "private-user",
agent_id: "private-agent",
session_id: "private-session",
step: "EXTRACT",
prompt: "private prompt",
content: "private response",
parsed: "private parsed output",
tokens: 42,
elapsed_ms: 123.5,
timestamp: "2026-10-14T10:01:00.000",
};
await appendFile(target, `${JSON.stringify(pipelineEntry)}\n`);
await writeFile(
path.join(dataDir, "hy-memory", "governed-v2", "logs", "hy_memory.log"),
`${JSON.stringify({ ...pipelineEntry, request_id: "must-not-appear" })}\n`,
path.join(hyLogs, "default", "2026-10-14.log"),
`${JSON.stringify({ request_id: "old-pipeline", step: "EXTRACT" })}\n`,
);
await writeFile(
path.join(hyLogs, "hy_memory.log"),
"2026-10-14 10:01:00 [WARNING] [-] hy_memory.client: hy text log\n",
);
const snapshot = await readDebugLogSnapshot(
@@ -271,29 +253,11 @@ it("shows HY pipeline metrics without exposing model content", async () => {
cursor,
);
expect(snapshot.entries).toEqual([
expect.objectContaining({
service: "hy-memory",
component: "hy.pipeline",
event: "hy.pipeline.extract",
message: "EXTRACT · 42 tokens · 124 ms",
duration_ms: 123.5,
trace_id: "request-safe-1",
details: { phase: "EXTRACT", count: 42 },
source: "pipeline",
}),
]);
const serialized = JSON.stringify(snapshot);
for (const secret of [
"private prompt",
"private response",
"private parsed output",
"private-user",
"private-agent",
"private-session",
]) {
expect(serialized).not.toContain(secret);
}
expect(snapshot.entries).toEqual([]);
expect(cursor).toEqual({
diagnosticLogs: [],
serviceLogs: expect.any(Array),
});
});
it("logs a renderer ui-action with action, page, phase and the shared trace id", async () => {
+4 -146
View File
@@ -18,7 +18,6 @@ const UI_TRACE = "one-memory:log-ui-trace";
const DEBUG_LOGS = "one-memory:get-debug-logs";
const DEBUG_LOG_NAME =
/^diagnostic-(runtime|core|host-bridge|desktop|cli)\.jsonl(?:\.\d+)?$/;
const PIPELINE_LOG_NAME = /^\d{4}-\d{2}-\d{2}\.log$/;
const SERVICE_LOG_NAMES = [
"service.log",
"service-error.log",
@@ -30,24 +29,16 @@ interface ServiceLogCursor {
files: Array<{ device: number; inode: number; offset: number }>;
}
interface PipelineLogCursor {
relativePath: string;
device: number;
inode: number;
offset: number;
}
export interface DebugLogCursor {
diagnosticLogs: Array<{ device: number; inode: number; offset: number }>;
serviceLogs: ServiceLogCursor[];
pipelineLogs: PipelineLogCursor[];
}
export interface DebugLogEntry {
id: string;
timestamp: string;
level: "debug" | "info" | "warn" | "error";
service: "runtime" | "core" | "host-bridge" | "desktop" | "cli" | "hy-memory";
service: "runtime" | "core" | "host-bridge" | "desktop" | "cli";
component: string;
event: string;
outcome: string;
@@ -57,7 +48,7 @@ export interface DebugLogEntry {
trace_id?: string;
details?: Record<string, string | number | boolean | null>;
message?: string;
source: "diagnostic" | "stdout" | "stderr" | "pipeline";
source: "diagnostic" | "stdout" | "stderr";
}
export interface DebugLogSnapshot {
@@ -259,7 +250,6 @@ export async function readDebugLogSnapshot(
cursor: DebugLogCursor = {
diagnosticLogs: [],
serviceLogs: [],
pipelineLogs: [],
},
): Promise<DebugLogSnapshot> {
const started = Number.isFinite(startedAt) ? startedAt : Date.now();
@@ -300,7 +290,6 @@ export async function readDebugLogSnapshot(
);
}
}
entries.push(...(await readPipelineLogEntries(dataDir, cursor, started)));
entries.sort((left, right) =>
left.timestamp === right.timestamp
? left.id.localeCompare(right.id)
@@ -319,7 +308,6 @@ export async function captureDebugLogCursor(
): Promise<DebugLogCursor> {
const directory = path.join(dataDir, "logs");
const names = await readdir(directory).catch(() => []);
const pipelineRoot = path.join(dataDir, "hy-memory", "governed-v2", "logs");
return {
diagnosticLogs: (
await Promise.all(
@@ -355,7 +343,6 @@ export async function captureDebugLogCursor(
};
}),
),
pipelineLogs: await pipelineLogFiles(pipelineRoot),
};
}
@@ -462,8 +449,8 @@ async function readServiceLogEntries(
if (!message) continue;
const level = serviceLogLevel(message, cursor.name);
// Raw stdout is an implementation trace, not a product event. Core's
// structured diagnostics and HY pipeline logs already carry meaningful
// successful work with request/task identity. Showing raw INFO/DEBUG
// structured diagnostics already carry meaningful successful work with
// request/task identity. Showing raw INFO/DEBUG
// here duplicated those events and turned polling access logs, file
// watchers and library initialization into thousands of fake "useful"
// rows. Keep raw output only when it needs attention.
@@ -543,135 +530,6 @@ function logLineTimestamp(
return new Date(Math.max(startedAt, fallback)).toISOString();
}
async function pipelineLogFiles(root: string): Promise<PipelineLogCursor[]> {
const rootInfo = await lstat(root).catch(() => null);
if (!rootInfo?.isDirectory() || rootInfo.isSymbolicLink()) return [];
const files: PipelineLogCursor[] = [];
const visit = async (directory: string): Promise<void> => {
for (const entry of await readdir(directory, { withFileTypes: true })) {
const target = path.join(directory, entry.name);
if (entry.isSymbolicLink()) continue;
if (entry.isDirectory()) {
await visit(target);
continue;
}
// The same root also contains HY's human-readable hy_memory.log.
// Pipeline files are date-named JSONL; accepting every .log file makes
// the Debug poller repeatedly parse unrelated text and still display no
// HY entries.
if (!entry.isFile() || !PIPELINE_LOG_NAME.test(entry.name)) continue;
const info = await safeRegularFile(target);
if (!info) continue;
files.push({
relativePath: path.relative(root, target),
device: info.dev,
inode: info.ino,
offset: info.size,
});
}
};
await visit(root);
return files;
}
async function readPipelineLogEntries(
dataDir: string,
cursor: DebugLogCursor,
startedAt: number,
): Promise<DebugLogEntry[]> {
const root = path.join(dataDir, "hy-memory", "governed-v2", "logs");
const current = await pipelineLogFiles(root);
const baseline = new Map(
cursor.pipelineLogs.map((file) => [
`${file.device}:${file.inode}`,
file.offset,
]),
);
const entries: DebugLogEntry[] = [];
for (const file of current) {
const target = path.join(root, file.relativePath);
const offset = baseline.get(`${file.device}:${file.inode}`) ?? 0;
const info = await safeRegularFile(target);
if (!info || info.size <= offset) continue;
const content = await readFileFromOffset(target, offset).catch(() => "");
let lineNumber = 0;
for (const line of content.split("\n")) {
lineNumber += 1;
if (!line.trim()) continue;
const entry = pipelineLogEntry(
line,
file,
lineNumber,
offset,
startedAt,
info.mtimeMs,
);
if (entry) entries.push(entry);
}
}
return entries;
}
function pipelineLogEntry(
line: string,
file: PipelineLogCursor,
lineNumber: number,
offset: number,
startedAt: number,
fallbackTime: number,
): DebugLogEntry | null {
let value: Record<string, unknown>;
try {
value = JSON.parse(line) as Record<string, unknown>;
} catch {
return null;
}
const step = typeof value.step === "string" ? value.step.trim() : "";
if (!step || !/^[A-Za-z0-9_-]{1,80}$/.test(step)) return null;
const rawTimestamp =
typeof value.timestamp === "string" ? value.timestamp : "";
const parsedTimestamp = Date.parse(rawTimestamp);
const timestamp = new Date(
Math.max(
startedAt,
Number.isFinite(parsedTimestamp) ? parsedTimestamp : fallbackTime,
),
).toISOString();
const elapsed = safeFiniteNumber(value.elapsed_ms);
const tokens = safeFiniteNumber(value.tokens);
const requestId =
typeof value.request_id === "string" && value.request_id.length <= 256
? value.request_id
: undefined;
return {
id: `hy-pipeline:${file.device}:${file.inode}:${offset}:${lineNumber}`,
timestamp,
level: "info",
service: "hy-memory",
component: "hy.pipeline",
event: `hy.pipeline.${step.toLowerCase().replaceAll("_", "-")}`,
outcome: "observed",
...(elapsed === undefined ? {} : { duration_ms: elapsed }),
...(requestId ? { trace_id: requestId } : {}),
details: {
phase: step,
...(tokens === undefined ? {} : { count: tokens }),
},
message: [
step,
...(tokens === undefined ? [] : [`${Math.round(tokens)} tokens`]),
...(elapsed === undefined ? [] : [`${Math.round(elapsed)} ms`]),
].join(" · "),
source: "pipeline",
};
}
function safeFiniteNumber(value: unknown): number | undefined {
return typeof value === "number" && Number.isFinite(value)
? value
: undefined;
}
export function installDiagnosticsIpc(
options: DiagnosticIpcOptions,
): () => void {
-1
View File
@@ -50,7 +50,6 @@ const SERVICE_LABELS: Record<DebugLogEntry["service"], string> = {
core: "Core",
"host-bridge": "Host Bridge",
cli: "CLI",
"hy-memory": "HY Memory",
};
export function DebugLogPanel() {
+2 -9
View File
@@ -4,14 +4,7 @@ export interface DebugLogEntry {
id: string;
timestamp: string;
level: DebugLogLevel;
service:
| "runtime"
| "core"
| "host-bridge"
| "desktop"
| "cli"
| "hy-memory"
| "renderer";
service: "runtime" | "core" | "host-bridge" | "desktop" | "cli" | "renderer";
component: string;
event: string;
outcome: string;
@@ -21,7 +14,7 @@ export interface DebugLogEntry {
trace_id?: string;
details?: Record<string, string | number | boolean | null>;
message?: string;
source: "diagnostic" | "stdout" | "stderr" | "console" | "pipeline";
source: "diagnostic" | "stdout" | "stderr" | "console";
}
export interface DebugLogSnapshot {
+1 -1
View File
@@ -282,7 +282,7 @@ Support is declared per immutable GitHub Release manifest:
- Linux x64/arm64 Runtime on native GHA runners.
- macOS arm64/x64 App/Runtime: Developer ID, notarization, native acceptance.
- Windows is not supported.
- `hy-memory==2.0.7` is the current stable SDK dependency; the release
- `hy-memory==2.0.8` is the current stable SDK dependency; the release
lock and [`../MEMORY.md`](../MEMORY.md) are authoritative when the pin changes.
See [`INSTALLATION.md`](INSTALLATION.md).
+6 -6
View File
@@ -7,12 +7,12 @@
3. 如果停住,停在导入、模型调用还是最终写入。
默认 `info` 下的官方结构化事件不应写入会话正文、模型输入输出或 API Key;但本地原始日志允许
新增受限标量标签,仍应按私人数据处理,不能直接对外发送。`debug` 会额外启用 HY pipeline
trace,其中可能包含完整 prompt/response 和获授权的对话正文。
新增受限标量标签,仍应按私人数据处理,不能直接对外发送。`debug` 会让 HY 的
`hy_memory.log` 记录 DEBUG 行,其中可能包含完整 prompt/response 和获授权的对话正文。
排查时只通过 App 的“导出日志”提供文件:导出过程会按安全白名单重新生成诊断事件,不会打包
原始日志。不要直接发送 `diagnostic-*.jsonl`、整个 `~/.one-memory` 数据目录或 debug pipeline
日志;后者如确需分享,必须由开发人员另行确认并脱敏。
原始日志。不要直接发送 `diagnostic-*.jsonl`、整个 `~/.one-memory` 数据目录或 debug 级别的
`hy_memory.log`;后者如确需分享,必须由开发人员另行确认并脱敏。
## 1. Log level 设什么
@@ -181,8 +181,8 @@ memhub logs --since 24h --errors-only --tail 500 --json
memhub status --json
```
以下 `jq` 命令仅用于 `diagnostic-core.jsonl`,不要对 `core.log`、`service.log` 或 debug
pipeline 日志直接运行后把完整输出发给他人。
以下 `jq` 命令仅用于 `diagnostic-core.jsonl`,不要对 `core.log`、`service.log` 或
`hy_memory.log` 直接运行后把完整输出发给他人。
### 查看最近的失败
@@ -0,0 +1,66 @@
# TODO: recall imported conversations before a model is configured
Status: open. Found 2026-10-28 by the Linux lifecycle run on build 35.
## Problem
A new user who has imported conversations but not configured a generation
model gets nothing from `MemHub_recall` or `memhub recall`.
`recall_with_fallback` (`apps/core/src/one_memory/recall_fallback.py`) is
meant to cover this case: without a model it searches the SDK raw layer
(`l1_raw`), the imported conversations themselves. But under SDK authority
nothing writes that layer until a model is active:
- File import and live capture only stage conversations into
`memory_write_jobs` (assembling, then pending).
- The durable write drain claims a job only when
`_sedimentation_claim_enabled` holds, which requires an active model
profile (`api.py`).
- The SDK writes `l1_raw` only inside `client.add`, which the drain calls.
So until a model is configured, the raw layer is empty and the fallback has
nothing to search. Before the SDK cutover, imported Evidence lived in
MemHub's own database and was searchable without a model.
The Linux lifecycle verification used to assert "import, then recall finds
it" on a machine with no model. It now asserts that the import reached the
write queue instead (`scripts/platform/linux/verify-linux-lifecycle.mjs`,
`waitForQueued`). Restore the recall assertion once this is fixed.
## Direction
Write imported conversations to the SDK raw layer without a generation
model, and leave extraction for when a model is configured.
- The raw layer needs embeddings only. The bundled local ONNX model
(`builtin_retrieval_snapshot`) can produce them, so no network model is
needed.
- SDK: a public raw-only write, for example `add(..., infer=False)` or
`add_raw(messages, ...)`, that stores `l1_raw` with source, session,
workspace and `memory_at`, and returns its raw ids. It must be idempotent
by `request_id`, like `add`.
- MemHub: split the claim gate. Without a model, the drain may claim a
pending job for a raw-only write. Once a model is active, the same job
still needs extraction, and that write must not store `l1_raw` again.
Either the job records that its raw part is done, or extraction runs
through `add_extracted` / a mode that skips the raw store.
- The source sync switch and grant still apply, exactly as for extraction.
## Open questions
- Should raw-only ingestion run at all when the user has turned
sedimentation off, or does that switch mean "store nothing"?
- Storage cost: raw ingestion of a full history import without a model
embeds every turn up front. Check the time and disk cost on a large
history.
- When the user configures a model later, extract from the stored raw rows,
or re-read the jobs' frozen inputs? The second keeps one write path.
## Done when
- On a fresh install with no model, `memhub recall "<phrase from an
imported conversation>"` returns that conversation with `mode: "raw"`.
- Configuring a model afterwards extracts memories from the same
conversations without duplicating their raw rows.
- `verify-linux-lifecycle.mjs` asserts recall again.
+1 -1
View File
@@ -2,7 +2,7 @@
"name": "one-memory",
"version": "1.0.5",
"release": {
"build_number": 34,
"build_number": 35,
"macos_min_version": "13.0",
"app_update": "manual",
"runtime_manifest_schema": 1,
+1 -1
View File
@@ -36,7 +36,7 @@ MEMMY_LICENSE = (
MEMMY_PROVENANCE = MEMMY_LICENSE.with_name("PROVENANCE.md")
MEMMY_PIN = "25acd5f29fc7df73b943713abe61f1b2f9839887"
HY_MEMORY_NAME = "hy-memory"
HY_MEMORY_VERSION = "2.0.7"
HY_MEMORY_VERSION = "2.0.8"
class SbomError(RuntimeError):
+1 -1
View File
@@ -12,7 +12,7 @@ test("installation docs disclose publication and lifecycle boundaries", async ()
"utf8",
);
assert.match(readme, /`hy-memory==2\.0\.7`/);
assert.match(readme, /`hy-memory==2\.0\.8`/);
assert.match(readme, /`v1\.0\.5` 的发行目标是 Linux x64\/arm64 Runtime/);
assert.match(readme, /Linux 制品由 GitHub Actions/);
assert.match(readme, /macOS.*Developer ID 签名、公证/s);
@@ -66,7 +66,6 @@ describe("Linux runtime scripts", () => {
'"stop"',
'"init"',
'"scan"',
'"recall"',
'"uninstall"',
]) {
assert.match(script, new RegExp(command));
@@ -88,9 +87,11 @@ describe("Linux runtime scripts", () => {
script,
/Exit 1 is a valid transient receipt[\s\S]{0,300}\[0, 1, 3\]/,
);
assert.match(script, /append_recalled/);
assert.match(script, /initial_queued/);
assert.match(script, /append_queued/);
assert.match(script, /FROM memory_write_jobs/);
assert.match(script, /idempotent_upgrade_instance_id_changed/);
assert.match(script, /post_upgrade_recalled/);
assert.match(script, /post_upgrade_queued/);
assert.match(script, /uninstall_preserved_data/);
assert.match(script, /is-enabled/);
assert.match(script, /is-active/);
@@ -10,6 +10,7 @@ import {
} from "node:fs";
import os from "node:os";
import path from "node:path";
import { DatabaseSync } from "node:sqlite";
function fail(message, receipt) {
if (receipt) process.stdout.write(`${JSON.stringify(receipt, null, 2)}\n`);
@@ -146,18 +147,35 @@ function doctorCheck(document, id) {
return check;
}
async function waitForRecall(prefix, phrase) {
/**
* Whether an imported phrase reached Core's durable write queue.
*
* Recall cannot prove an import here: this runner has no generation model,
* and with SDK authority nothing reaches the SDK (not even its raw layer)
* until a model is active. What the lifecycle can prove is that the import
* was read and queued for the SDK, durably, in the Runtime's own database.
*/
async function waitForQueued(phrase) {
const deadline = Date.now() + 90_000;
let attempt = 0;
const database = path.join(dataDir, "one-memory.sqlite3");
while (Date.now() < deadline) {
attempt += 1;
const result = run(
`${prefix}-${attempt}`,
["recall", phrase, "--json"],
[0, 1, 3],
15_000,
);
if (result.stdout.includes(phrase)) return true;
if (existsSync(database)) {
let connection;
try {
connection = new DatabaseSync(database, { readOnly: true });
const row = connection
.prepare(
"SELECT COUNT(*) AS count FROM memory_write_jobs " +
"WHERE instr(payload_json, ?) > 0",
)
.get(phrase);
if (Number(row?.count) > 0) return true;
} catch {
// The queue table appears once Core has migrated its schema.
} finally {
connection?.close();
}
}
await new Promise((resolve) => setTimeout(resolve, 2_000));
}
return false;
@@ -320,12 +338,11 @@ try {
"--import",
"--json",
]);
observations.initial_recalled = await waitForRecall(
"recall-initial",
observations.initial_queued = await waitForQueued(
"Linux lifecycle initial phrase",
);
if (!observations.initial_recalled)
throw new Error("initial import was not recalled");
if (!observations.initial_queued)
throw new Error("initial import was not queued for the SDK");
status.import = "passed";
writeFileSync(
@@ -338,12 +355,11 @@ try {
[0],
30_000,
);
observations.append_recalled = await waitForRecall(
"recall-appended",
observations.append_queued = await waitForQueued(
"Linux lifecycle appended phrase",
);
if (!observations.append_recalled)
throw new Error("appended turn was not recalled");
if (!observations.append_queued)
throw new Error("appended turn was not queued for the SDK");
status.append_sync = "passed";
const beforeUpgrade = await waitForRunningStatus("status-before-upgrade");
@@ -378,12 +394,11 @@ try {
if (!observations.idempotent_upgrade_instance_id_changed) {
throw new Error("idempotent upgrade did not restart the Runtime");
}
observations.post_upgrade_recalled = await waitForRecall(
"recall-after-upgrade",
observations.post_upgrade_queued = await waitForQueued(
"Linux lifecycle appended phrase",
);
if (!observations.post_upgrade_recalled) {
throw new Error("idempotent upgrade lost previously recalled data");
if (!observations.post_upgrade_queued) {
throw new Error("idempotent upgrade lost previously queued data");
}
status.upgrade_idempotent = "passed";
+3 -3
View File
@@ -13,7 +13,7 @@ spec.loader.exec_module(module)
def test_hy_memory_notice_comes_from_the_installed_dist_info(tmp_path: Path) -> None:
dist_info = tmp_path / "hy_memory-2.0.7.dist-info"
dist_info = tmp_path / "hy_memory-2.0.8.dist-info"
licenses = dist_info / "licenses"
licenses.mkdir(parents=True)
(dist_info / "METADATA").write_text(
@@ -21,7 +21,7 @@ def test_hy_memory_notice_comes_from_the_installed_dist_info(tmp_path: Path) ->
(
"Metadata-Version: 2.4",
"Name: hy-memory",
"Version: 2.0.7",
"Version: 2.0.8",
"License-Expression: Apache-2.0",
"",
)
@@ -38,7 +38,7 @@ def test_hy_memory_notice_comes_from_the_installed_dist_info(tmp_path: Path) ->
notice = module.hy_memory_notice(dist_info)
assert notice["name"] == "hy-memory"
assert notice["version"] == "2.0.7"
assert notice["version"] == "2.0.8"
assert notice["license"] == "Apache-2.0"
assert notice["copyright"] == "Copyright (C) 2026 Tencent. All rights reserved."
assert "Licensed under the Apache License" in notice["text"]