Compare commits
32
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
0ca2e56ccc | ||
|
|
f328cdbf8e | ||
|
|
3f57a3c978 | ||
|
|
19a7af0e71 | ||
|
|
5791c0d1e7 | ||
|
|
bf92105210 | ||
|
|
3387247842 | ||
|
|
791285f4c9 | ||
|
|
dc762df78b | ||
|
|
8277a72aea | ||
|
|
5eed0f81fb | ||
|
|
59ca5afca6 | ||
|
|
2648012ad5 | ||
|
|
63a80092c2 | ||
|
|
6ba49bf112 | ||
|
|
cf410aa9cf | ||
|
|
30c0b1d5ff | ||
|
|
1086f4d582 | ||
|
|
32539e7a82 | ||
|
|
cfae020a31 | ||
|
|
5d51513ec8 | ||
|
|
9db5b49ccb | ||
|
|
cb7df2a022 | ||
|
|
812d1c3846 | ||
|
|
c8b660651e | ||
|
|
bdd2d9cf84 | ||
|
|
fc2e51a8cc | ||
|
|
d4fca0639e | ||
|
|
a17230dadb | ||
|
|
d76bfa6474 | ||
|
|
e9698a293c | ||
|
|
6718ba8bdf |
No files matched your search
@@ -4,6 +4,26 @@ Hy Memory and MemHub are versioned and released independently.
|
||||
|
||||
## Unreleased
|
||||
|
||||
- MemHub turns off the HY pipeline log and trace at every diagnostic level
|
||||
(hy-memory 2.0.8 `MEMORY_PIPELINE_LOG_ENABLED=false`,
|
||||
`MEMORY_PIPELINE_TRACE_ENABLED=false`), so the per-step JSONL files under
|
||||
`hy-memory/governed-v2/logs/` and the `pipeline_logs` rows stop growing.
|
||||
The Debug panel no longer shows the HY pipeline section. Existing files are
|
||||
left in place.
|
||||
- HY SDK text logs are written once, to `hy_memory.log`. They are no longer
|
||||
copied to `service.log` and `service-error.log` through the SDK's
|
||||
stdout/stderr streams. SDK warnings and errors also go to `core.log`, so the
|
||||
Debug panel still shows them. A live switch of the diagnostic level now also
|
||||
changes the level of `hy_memory.log`.
|
||||
|
||||
- MemHub shows when working and coding experiences were talked about. After
|
||||
the SDK upgrade every one of them read "时间未知" (time unknown) because the
|
||||
SDK never stored their conversation time; hy-memory 2.0.7 stores it and
|
||||
Core now passes it through the memory list, detail, search and home
|
||||
summary. An experience stored before 2.0.7 shows its write time, labelled
|
||||
"写入于" / "写入时间", instead of "时间未知". Metadata repair also writes
|
||||
the conversation time it recovers from the migration bundle into the SDK's
|
||||
coding/working stores when they already have the column.
|
||||
- The SDK migration no longer fails its first run on a home with legacy
|
||||
vectors: reattaching them left a Chroma instance open with its own settings,
|
||||
and the readback that followed could not open the same store
|
||||
@@ -68,6 +88,44 @@ Hy Memory and MemHub are versioned and released independently.
|
||||
`hy_node_permissions` proof table. Evidence, governed memories, model
|
||||
profiles, SDK stores, and source/host grants are preserved.
|
||||
|
||||
## Hy Memory 2.0.8 — 2026-10-28
|
||||
|
||||
- Published
|
||||
[`hy-memory==2.0.8`](https://pypi.org/project/hy-memory/2.0.8/).
|
||||
- New `MEMORY_PIPELINE_LOG_ENABLED` (default `true`) turns the pipeline JSONL
|
||||
log (`{MEMORY_LOG_DIR}/<subdir>/<date>.log`) off. With `false` no JSONL
|
||||
file or directory is written. When `MEMORY_PIPELINE_TRACE_ENABLED` is also
|
||||
`false`, `store_pipeline_log` becomes a no-op that returns `True` at once:
|
||||
no timing, redaction, background task or database write. The init log line
|
||||
now states whether the JSONL log is on or off.
|
||||
- Conversation and memory text no longer appears in INFO or WARNING logs. The
|
||||
`[add]` input, search queries and rewrites, coding experience tasks, fact
|
||||
statements and subjects, reconcile op details, extracted info, embedding
|
||||
inputs and unparsable model output moved to DEBUG. INFO lines keep ids,
|
||||
counts and character lengths; parse-failure warnings remain but report only
|
||||
the length of the output, e.g. `failed to parse output (812 chars)`. The
|
||||
`[embed]` INFO line now carries `request.input_chars` instead of
|
||||
`request.input`. The `JSON_PARSE_FAILED` error message from extraction no
|
||||
longer embeds the model output either.
|
||||
|
||||
## Hy Memory 2.0.7 — 2026-10-28
|
||||
|
||||
- Published
|
||||
[`hy-memory==2.0.7`](https://pypi.org/project/hy-memory/2.0.7/).
|
||||
- Coding and working experiences and facts now keep their conversation time.
|
||||
`add(..., memory_at=...)` reached only the chat path before; the
|
||||
productivity path dropped it, so every experience had only its write time
|
||||
and MemHub showed them as "时间未知" (time unknown). `coding_memory_meta`
|
||||
and `working_fact` gain a nullable `memory_at` column (epoch seconds) in
|
||||
both the working and coding stores; existing databases get it on open, and
|
||||
their old rows stay `None`. Without `memory_at` a write uses the time the
|
||||
request arrived, as chat does. When an experience is updated by a later
|
||||
conversation it keeps the later of the two times.
|
||||
- `list_memories` (`scene="coding"`, `"working"` and `"all"`), `get` and
|
||||
`search` return `memory_at` for experiences and facts in the same field and
|
||||
format as chat (epoch seconds, `None` when unknown). Experience search
|
||||
results now also carry `created_at`.
|
||||
|
||||
## Hy Memory 2.0.6 — 2026-10-27
|
||||
|
||||
- Published
|
||||
|
||||
@@ -191,6 +191,7 @@ SDK 不设置 Rerank Search 并发上限。开启后模型在 `HyMemoryClient`
|
||||
| `MEMORY_LOG_FORMAT` | `text` | `text` / `json` |
|
||||
| `MEMORY_LOG_DIR` | — | 统一日志根目录 |
|
||||
| `MEMORY_TRACE_LOG_DIR` | — | JSONL 日志目录的兼容旧名;优先使用 `MEMORY_LOG_DIR` |
|
||||
| `MEMORY_PIPELINE_LOG_ENABLED` | `true` | 是否写 pipeline JSONL 日志(`{MEMORY_LOG_DIR}/<subdir>/<date>.log`);`false` 完全不写 |
|
||||
| `MEMORY_PIPELINE_TRACE_ENABLED` | `true` | 是否写入 cache 的 pipeline trace 表 |
|
||||
| `MEMORY_MEMORY_OPERATIONS_ENABLED` | `false` | 是否写入 memory operation 审计记录 |
|
||||
| `MEMORY_HISTORY_ENABLE` | `true` | 是否启用历史记录 |
|
||||
|
||||
@@ -1177,7 +1177,7 @@ class Extractor:
|
||||
if not tc_id:
|
||||
logger.error(
|
||||
f"[extractor] tool_call_id is empty! "
|
||||
f"raw tc: {json.dumps(tc_raw, ensure_ascii=False, default=str)[:500]}"
|
||||
f"raw_keys={list(tc_raw.keys())}"
|
||||
)
|
||||
try:
|
||||
t_res = await tool_registry.dispatch(
|
||||
@@ -1311,11 +1311,12 @@ class Extractor:
|
||||
if parsed is None:
|
||||
if not all_tool_calls:
|
||||
logger.warning(
|
||||
f"[extractor] JSON parse failed: {raw_text[:200]}"
|
||||
f"[extractor] JSON parse failed ({len(raw_text)} chars)"
|
||||
)
|
||||
logger.debug(f"[extractor] unparsable output: {raw_text[:200]!r}")
|
||||
return ExtractResult(
|
||||
success=False,
|
||||
error=f"JSON parse failed: {raw_text[:200]}",
|
||||
error=f"JSON parse failed ({len(raw_text)} chars)",
|
||||
error_code="JSON_PARSE_FAILED",
|
||||
raw_response=raw_text,
|
||||
tokens_used=total_tokens,
|
||||
|
||||
@@ -129,8 +129,7 @@ def _parse_tool_calls_list(raw_calls: list, _json) -> List[Dict[str, Any]]:
|
||||
tc_id = tc.get("id") or ""
|
||||
if not tc_id:
|
||||
logger.error(
|
||||
f"[llm] tool_call missing 'id'! raw: "
|
||||
f"{_json.dumps(tc, ensure_ascii=False, default=str)[:500]}"
|
||||
f"[llm] tool_call missing 'id'! keys={list(tc.keys())}"
|
||||
)
|
||||
|
||||
# 保留原始 tc 完整结构,只规范化 arguments
|
||||
|
||||
@@ -234,7 +234,7 @@ class MemAgent:
|
||||
f"identity={len(identity)} facts={len(facts)} "
|
||||
f"basic_info={basic_field_count}fields"
|
||||
)
|
||||
logger.info(f"MemAgent: extracted_info={extract_result.info}")
|
||||
logger.debug(f"MemAgent: extracted_info={extract_result.info}")
|
||||
|
||||
# 3. 冲突检测(如果有已有记忆)
|
||||
if mode == ProcessMode.FULL and existing_memories:
|
||||
|
||||
@@ -50,7 +50,7 @@ def is_canonical_state_node(node: Any) -> bool:
|
||||
|
||||
|
||||
def _log_ops_detail(ops: List["ReconcileOp"]) -> None:
|
||||
"""以单条 INFO(JSON list)输出 reconcile 的逐条结果,便于线上 debug。"""
|
||||
"""以单条 DEBUG(JSON list)输出 reconcile 的逐条结果;含记忆正文,不进 INFO。"""
|
||||
if not ops:
|
||||
return
|
||||
view = [
|
||||
@@ -68,7 +68,7 @@ def _log_ops_detail(ops: List["ReconcileOp"]) -> None:
|
||||
}
|
||||
for o in ops
|
||||
]
|
||||
logger.info(
|
||||
logger.debug(
|
||||
f"[reconciler] ops detail: {json.dumps(view, ensure_ascii=False, default=str)}"
|
||||
)
|
||||
|
||||
@@ -1514,9 +1514,10 @@ class MemoryReconciler:
|
||||
logger.warning(
|
||||
f"[reconciler] RECONCILE JSON parse failed "
|
||||
f"(attempt {_attempt+1}/{_MAX_PARSE_ATTEMPTS}), "
|
||||
f"retrying: raw="
|
||||
f"{raw_response[:200] if _log_llm_content_enabled() else '<redacted>'}"
|
||||
f"retrying: raw({len(raw_response or '')} chars)"
|
||||
)
|
||||
if _log_llm_content_enabled():
|
||||
logger.debug(f"[reconciler] unparsable raw={raw_response[:200]!r}")
|
||||
|
||||
recon_elapsed = (_time.perf_counter() - t_recon) * 1000
|
||||
|
||||
@@ -1528,8 +1529,8 @@ class MemoryReconciler:
|
||||
error_code = "LLM_UNAVAILABLE"
|
||||
elif raw_response.strip():
|
||||
error_msg = (
|
||||
f"RECONCILE JSON parse failed after {_n_attempts} attempts: "
|
||||
f"{raw_response[:200] if _log_llm_content_enabled() else '<redacted>'}"
|
||||
f"RECONCILE JSON parse failed after {_n_attempts} attempts "
|
||||
f"({len(raw_response)} chars)"
|
||||
)
|
||||
error_code = "JSON_PARSE_FAILED"
|
||||
else:
|
||||
@@ -1806,7 +1807,8 @@ class MemoryReconciler:
|
||||
except Exception as _re:
|
||||
logger.debug(f"[reconciler] json_repair failed: {_re}")
|
||||
if data is None:
|
||||
logger.warning(f"[reconciler] failed to parse ops JSON: {text[:200]}")
|
||||
logger.warning(f"[reconciler] failed to parse ops JSON ({len(text)} chars)")
|
||||
logger.debug(f"[reconciler] unparsable ops output: {text[:200]!r}")
|
||||
return []
|
||||
|
||||
if not isinstance(data, list):
|
||||
@@ -2018,11 +2020,11 @@ class MemoryReconciler:
|
||||
if clash is not None:
|
||||
logger.warning(
|
||||
f"[reconciler] memory_id {clash} already touched by '{touched[clash]}', "
|
||||
f"current op '{op.op} ({op.reason[:60]})' dropped"
|
||||
f"current op '{op.op}' dropped"
|
||||
)
|
||||
continue
|
||||
for t in target_ids:
|
||||
touched[t] = f"{op.op}({op.reason[:40]})"
|
||||
touched[t] = op.op
|
||||
validated.append(op)
|
||||
|
||||
return validated
|
||||
@@ -211,7 +211,9 @@ def parse_tool_calls_from_json(raw: Any) -> List[ToolCall]:
|
||||
try:
|
||||
args = json.loads(args_raw) if args_raw.strip() else {}
|
||||
except json.JSONDecodeError:
|
||||
logger.warning(f"[tools] failed to parse arguments as JSON: {args_raw[:120]}")
|
||||
logger.warning(
|
||||
f"[tools] failed to parse arguments as JSON ({len(args_raw)} chars)"
|
||||
)
|
||||
args = {}
|
||||
elif isinstance(args_raw, dict):
|
||||
args = args_raw
|
||||
|
||||
+27
-14
@@ -744,7 +744,7 @@ class HyMemoryClient:
|
||||
def _install_pipeline_observability_hooks(self) -> None:
|
||||
"""
|
||||
包装 cache.store_pipeline_log:
|
||||
- Log:JSONL 文件,始终写入(环节级,不依赖 Trace 开关)
|
||||
- Log:JSONL 文件,由 MEMORY_PIPELINE_LOG_ENABLED 控制(默认开启)
|
||||
- Trace:SQLite pipeline_logs,由 MEMORY_PIPELINE_TRACE_ENABLED 控制(Inspector)
|
||||
- 落盘 fire-and-forget,不阻塞 search/write 热路径
|
||||
"""
|
||||
@@ -958,7 +958,7 @@ class HyMemoryClient:
|
||||
parsed_tool_calls = json.loads(raw_tool_calls) if raw_tool_calls else []
|
||||
except Exception:
|
||||
logger.warning(
|
||||
"tool_calls 是非法 JSON 字符串,已忽略: %r", raw_tool_calls[:200]
|
||||
"tool_calls 是非法 JSON 字符串,已忽略 (%d chars)", len(raw_tool_calls)
|
||||
)
|
||||
parsed_tool_calls = []
|
||||
raw_tool_calls = parsed_tool_calls if isinstance(parsed_tool_calls, list) else []
|
||||
@@ -1544,6 +1544,7 @@ class HyMemoryClient:
|
||||
# LLM 写的"这个结论一路怎么改成现在这样",直接进提示词;
|
||||
# 没被覆盖过就是空串。不参与检索——召回认的是 statement 的当前值。
|
||||
"fact_chain_summary": fact.fact_chain_summary,
|
||||
"memory_at": _epoch_seconds(fact.memory_at),
|
||||
"created_at": fact.created_at.isoformat() if fact.created_at else None,
|
||||
"score": round(float(score), 4),
|
||||
"source": fact.source,
|
||||
@@ -1652,6 +1653,8 @@ class HyMemoryClient:
|
||||
"score": (round(float(score_by_id[d.experience_id]), 4)
|
||||
if d.experience_id in score_by_id else None),
|
||||
"confidence": d.confidence,
|
||||
"memory_at": _epoch_seconds(d.memory_at),
|
||||
"created_at": d.created_at.isoformat() if d.created_at else None,
|
||||
"updated_at": d.updated_at.isoformat() if d.updated_at else None,
|
||||
"source": d.source,
|
||||
"extraction_source": d.extraction_source,
|
||||
@@ -1953,6 +1956,7 @@ class HyMemoryClient:
|
||||
"content" chat 正文;经验为 "task\nsolution";事实为 statement
|
||||
"score" RRF 融合分;"lane_score" 为该路原始分;"lane_rank" 为该路名次
|
||||
"source", "created_at", "updated_at"(秒级 epoch,取不到为 None)
|
||||
"memory_at" 对话时间(秒级 epoch),各路原有字段,三路同名同格式
|
||||
chat 路内部名次:normal → schema → proactive 依次排;working/coding 路
|
||||
内部名次:事实与经验各按原顺序交替(事实在前),两层谁也不压谁。
|
||||
"""
|
||||
@@ -2144,7 +2148,8 @@ class HyMemoryClient:
|
||||
"limit", "offset", "request_id", "elapsed_ms"}
|
||||
每条带 "scene"、"kind"(chat | experience)、"memory_id"、"content"
|
||||
(经验为 "task\nsolution")、"source"、"status"、"created_at"、
|
||||
"updated_at"(秒级 epoch),以及该路原有字段。
|
||||
"updated_at"(秒级 epoch),以及该路原有字段(含对话时间
|
||||
"memory_at",秒级 epoch,三路同名同格式)。
|
||||
"""
|
||||
return self._loop_thread.run(
|
||||
self.async_list_memories(
|
||||
@@ -2967,6 +2972,7 @@ class HyMemoryClient:
|
||||
session_id=session_id,
|
||||
request_id=request_id,
|
||||
source=source,
|
||||
memory_at=memory_at,
|
||||
)
|
||||
# Public receipt always identifies the productivity namespace.
|
||||
resp["variant"] = scene_variant
|
||||
@@ -3002,13 +3008,13 @@ class HyMemoryClient:
|
||||
f"[add] writing {preview} user={uid} "
|
||||
f"agent_id={agent_id} session_id={session_id} mode={self._mode}"
|
||||
)
|
||||
logger.info(f"[add] messages detail: {[m.to_dict() for m in chat_messages]}")
|
||||
logger.debug(f"[add] messages detail: {[m.to_dict() for m in chat_messages]}")
|
||||
else:
|
||||
logger.info(
|
||||
f"[add] writing text({len(content)} chars) user={uid} "
|
||||
f"agent_id={agent_id} session_id={session_id} mode={self._mode} "
|
||||
f"content={content}"
|
||||
f"agent_id={agent_id} session_id={session_id} mode={self._mode}"
|
||||
)
|
||||
logger.debug(f"[add] text content={content!r}")
|
||||
|
||||
# mode → agent_mode 映射
|
||||
# lite: 不调 LLM agent
|
||||
@@ -4028,14 +4034,14 @@ class HyMemoryClient:
|
||||
decided_variant = HyMemoryClient._scene_variant(scene)
|
||||
logger.info(
|
||||
f"[search] scene explicit={decided_scene} (no LLM judge); "
|
||||
f"target={effective_target_query!r}"
|
||||
f"target({len(effective_target_query or '')} chars)"
|
||||
)
|
||||
elif not agentic_enabled and not queries_supplied:
|
||||
# 默认 light search 仅指 query="...":固定 normal,绝不调用 LLM。
|
||||
decided_scene = "normal"
|
||||
logger.info(
|
||||
f"[search] search_mode=default single-query scene=normal (no LLM); "
|
||||
f"target={target_query!r}"
|
||||
f"target({len(target_query or '')} chars)"
|
||||
)
|
||||
elif agentic_enabled:
|
||||
# agentic:单/多 query 都强制一次 LLM 完成三分类 + rewrite + 时间解析。
|
||||
@@ -4070,14 +4076,18 @@ class HyMemoryClient:
|
||||
rewrite_created_after = ar.get("created_after")
|
||||
logger.info(
|
||||
f"[search] agentic scene={decided_scene} via rewrite-LLM "
|
||||
f"(classify={want_classify}); orig={target_query!r} "
|
||||
f"effective={effective_target_query!r} rewrite_used={rewrite_used} "
|
||||
f"(classify={want_classify}); rewrite_used={rewrite_used} "
|
||||
f"created_after={rewrite_created_after}"
|
||||
)
|
||||
logger.debug(
|
||||
f"[search] agentic rewrite: orig={target_query!r} "
|
||||
f"effective={effective_target_query!r}"
|
||||
)
|
||||
else:
|
||||
decided_scene = "normal"
|
||||
logger.info(
|
||||
f"[search] agentic rewrite-LLM fallback=normal; target={target_query!r}"
|
||||
f"[search] agentic rewrite-LLM fallback=normal; "
|
||||
f"target({len(target_query or '')} chars)"
|
||||
)
|
||||
elif rewrite_enabled and not _too_long:
|
||||
# default + queries= 的原逻辑:配置开启时一次 LLM 做 rewrite/time,
|
||||
@@ -4156,7 +4166,7 @@ class HyMemoryClient:
|
||||
decided_scene = "normal"
|
||||
logger.info(
|
||||
f"[search] default multi-query features disabled; no LLM; "
|
||||
f"target={target_query!r}"
|
||||
f"target({len(target_query or '')} chars)"
|
||||
)
|
||||
|
||||
# ────────────────────────────────────────────────────────────
|
||||
@@ -4209,7 +4219,7 @@ class HyMemoryClient:
|
||||
else:
|
||||
logger.info(
|
||||
f"[search] routing to PRODUCTIVITY path variant={decided_variant}: "
|
||||
f"target={effective_target_query!r} "
|
||||
f"target({len(effective_target_query or '')} chars) "
|
||||
f"workspace_id={workspace_id!r} branch={branch!r}"
|
||||
)
|
||||
return await self._search_coding(
|
||||
@@ -4237,9 +4247,11 @@ class HyMemoryClient:
|
||||
# Chat 召回链路(原有逻辑)
|
||||
# ────────────────────────────────────────────────────────────
|
||||
logger.info(
|
||||
f"[search] query='{effective_target_query}' user_ids={effective_user_ids} "
|
||||
f"[search] query({len(effective_target_query or '')} chars) "
|
||||
f"user_ids={effective_user_ids} "
|
||||
f"agent_ids={effective_agent_ids} session_ids={effective_session_ids} limit={limit}"
|
||||
)
|
||||
logger.debug(f"[search] query={effective_target_query!r}")
|
||||
|
||||
# rewrite 抽出的时间下界与用户显式 created_after 取更晚者(更严格)
|
||||
effective_created_after = created_after
|
||||
@@ -5920,6 +5932,7 @@ class HyMemoryClient:
|
||||
"source_fact_ids": list(d.source_fact_ids),
|
||||
"confidence": d.confidence,
|
||||
"status": MemoryStatus.ACTIVE.value,
|
||||
"memory_at": _epoch_seconds(d.memory_at),
|
||||
"created_at": d.created_at.isoformat() if d.created_at else None,
|
||||
"updated_at": d.updated_at.isoformat() if d.updated_at else None,
|
||||
"source": d.source,
|
||||
|
||||
@@ -1166,7 +1166,10 @@ class CodingMemoryExtractor:
|
||||
obj = None
|
||||
|
||||
if not isinstance(obj, dict):
|
||||
logger.warning(f"[coding-extract] failed to parse output: {text!r}")
|
||||
logger.warning(
|
||||
f"[coding-extract] failed to parse output ({len(text or '')} chars)"
|
||||
)
|
||||
logger.debug(f"[coding-extract] unparsable output: {text!r}")
|
||||
return [], []
|
||||
|
||||
# "memories" 是双层化之前的 key,老 prompt 缓存 / 旧模型可能还这么回
|
||||
|
||||
@@ -454,9 +454,10 @@ class WorkingFactReconciler:
|
||||
workspace_id: Optional[str] = None,
|
||||
branch: Optional[str] = None,
|
||||
source: str = "default",
|
||||
memory_at: Optional[datetime] = None,
|
||||
) -> Tuple[List[WorkingFact], List[FactReconcileOp]]:
|
||||
"""
|
||||
执行 reconcile + 持久化。
|
||||
执行 reconcile + 持久化。memory_at 是这段对话的时间,记到新事实上。
|
||||
|
||||
Returns:
|
||||
(真正落库的新事实, 执行过的 ops)。LLM 挂掉时退化为全 INSERT 不覆盖,
|
||||
@@ -493,6 +494,7 @@ class WorkingFactReconciler:
|
||||
workspace_id=workspace_id,
|
||||
branch=branch,
|
||||
source=source,
|
||||
memory_at=memory_at,
|
||||
)
|
||||
logger.info(
|
||||
f"[fact-reconcile] drafts={len(drafts)} inserted={len(facts)} "
|
||||
@@ -568,9 +570,12 @@ class WorkingFactReconciler:
|
||||
kept.append(fact)
|
||||
if blocked:
|
||||
logger.info(
|
||||
f"[fact-reconcile] subject gate blocked {len(blocked)} candidate(s) "
|
||||
f"for subject={draft.subject!r}: "
|
||||
+ ", ".join(f"{f.fact_id}({f.subject!r})" for f in blocked)
|
||||
f"[fact-reconcile] subject gate blocked {len(blocked)} candidate(s): "
|
||||
+ ", ".join(f.fact_id for f in blocked)
|
||||
)
|
||||
logger.debug(
|
||||
f"[fact-reconcile] subject gate: draft subject={draft.subject!r} "
|
||||
f"blocked subjects={[f.subject for f in blocked]!r}"
|
||||
)
|
||||
result.append(kept)
|
||||
return result
|
||||
@@ -743,7 +748,10 @@ class WorkingFactReconciler:
|
||||
obj = None
|
||||
|
||||
if not isinstance(obj, dict):
|
||||
logger.warning(f"[fact-reconcile] failed to parse ops output: {text!r}")
|
||||
logger.warning(
|
||||
f"[fact-reconcile] failed to parse ops output ({len(text or '')} chars)"
|
||||
)
|
||||
logger.debug(f"[fact-reconcile] unparsable ops output: {text!r}")
|
||||
return {}
|
||||
ops_raw = obj.get("ops")
|
||||
if not isinstance(ops_raw, list):
|
||||
@@ -781,6 +789,7 @@ class WorkingFactReconciler:
|
||||
workspace_id: Optional[str],
|
||||
branch: Optional[str] = None,
|
||||
source: str = "default",
|
||||
memory_at: Optional[datetime] = None,
|
||||
) -> Tuple[List[WorkingFact], List[FactReconcileOp]]:
|
||||
inserted: List[WorkingFact] = []
|
||||
ops: List[FactReconcileOp] = []
|
||||
@@ -801,6 +810,7 @@ class WorkingFactReconciler:
|
||||
subject=draft.subject,
|
||||
details=draft.details,
|
||||
status="active",
|
||||
memory_at=memory_at,
|
||||
)
|
||||
|
||||
allowed = {c.fact_id for c in (candidates[idx] if idx < len(candidates) else [])}
|
||||
@@ -831,8 +841,11 @@ class WorkingFactReconciler:
|
||||
# 别的主体的状态不是。闸门关掉时这道复查一并跳过,否则闸门就没关干净。
|
||||
if self.subject_gate and _subject_conflict(draft.subject, old.subject):
|
||||
logger.warning(
|
||||
f"[fact-reconcile] refused cross-subject SUPERSEDE: "
|
||||
f"{old.fact_id}({old.subject!r}) by draft subject {draft.subject!r}"
|
||||
f"[fact-reconcile] refused cross-subject SUPERSEDE: {old.fact_id}"
|
||||
)
|
||||
logger.debug(
|
||||
f"[fact-reconcile] cross-subject: old subject={old.subject!r} "
|
||||
f"draft subject={draft.subject!r}"
|
||||
)
|
||||
continue
|
||||
try:
|
||||
|
||||
+21
-10
@@ -195,8 +195,10 @@ async def classify_messages_scene(
|
||||
result = _parse_scene_json(resp.content)
|
||||
if result is None:
|
||||
logger.warning(
|
||||
f"[coding-judge] failed to parse classify_messages output: {resp.content!r}; defaulting to chat"
|
||||
f"[coding-judge] failed to parse classify_messages output "
|
||||
f"({len(resp.content or '')} chars); defaulting to chat"
|
||||
)
|
||||
logger.debug(f"[coding-judge] unparsable classify_messages output: {resp.content!r}")
|
||||
await _store_coding_log(
|
||||
cache, request_id=request_id, user_id=user_id, agent_id=agent_id,
|
||||
session_id=session_id, step="CODING_JUDGE", prompt=prompt,
|
||||
@@ -207,9 +209,8 @@ async def classify_messages_scene(
|
||||
return "chat"
|
||||
scene = result["scene"]
|
||||
parsed = {**result, "is_coding": scene != "chat"}
|
||||
logger.info(
|
||||
f"[coding-judge] scene={scene} reason={result.get('reason', '')!r}"
|
||||
)
|
||||
logger.info(f"[coding-judge] scene={scene}")
|
||||
logger.debug(f"[coding-judge] scene={scene} reason={result.get('reason', '')!r}")
|
||||
await _store_coding_log(
|
||||
cache, request_id=request_id, user_id=user_id, agent_id=agent_id,
|
||||
session_id=session_id, step="CODING_JUDGE", prompt=prompt,
|
||||
@@ -295,8 +296,10 @@ async def classify_queries_scene(
|
||||
result = _parse_scene_json(resp.content)
|
||||
if result is None:
|
||||
logger.warning(
|
||||
f"[coding-judge] failed to parse classify_queries output: {resp.content!r}; defaulting to chat"
|
||||
f"[coding-judge] failed to parse classify_queries output "
|
||||
f"({len(resp.content or '')} chars); defaulting to chat"
|
||||
)
|
||||
logger.debug(f"[coding-judge] unparsable classify_queries output: {resp.content!r}")
|
||||
return "chat"
|
||||
scene = result["scene"]
|
||||
logger.info(f"[coding-judge] queries scene={scene}")
|
||||
@@ -422,15 +425,18 @@ async def classify_and_rewrite_queries(
|
||||
scene = _parse_scene_label(head)
|
||||
if scene is None:
|
||||
logger.warning(
|
||||
f"[coding-judge] classify_and_rewrite unrecognized head={head!r}; falling back to chat"
|
||||
f"[coding-judge] classify_and_rewrite unrecognized head "
|
||||
f"({len(head)} chars); falling back to chat"
|
||||
)
|
||||
logger.debug(f"[coding-judge] classify_and_rewrite unrecognized head={head!r}")
|
||||
return {"scene": "chat", "is_coding": False, "rewrite_query": rewrite, "ok": False}
|
||||
is_coding = scene != "chat"
|
||||
|
||||
logger.info(
|
||||
f"[coding-judge] classify_and_rewrite scene={scene} "
|
||||
f"rewrite={rewrite!r}"
|
||||
f"rewrite({len(rewrite)} chars)"
|
||||
)
|
||||
logger.debug(f"[coding-judge] classify_and_rewrite rewrite={rewrite!r}")
|
||||
return {"scene": scene, "is_coding": is_coding, "rewrite_query": rewrite, "ok": True}
|
||||
|
||||
|
||||
@@ -569,8 +575,9 @@ async def analyze_queries(
|
||||
parsed_scene = _parse_scene_label(head)
|
||||
if parsed_scene is None:
|
||||
logger.warning(
|
||||
f"[coding-judge] analyze_queries unrecognized head={head!r}; ok=False"
|
||||
f"[coding-judge] analyze_queries unrecognized head ({len(head)} chars); ok=False"
|
||||
)
|
||||
logger.debug(f"[coding-judge] analyze_queries unrecognized head={head!r}")
|
||||
return {**fallback, "since_raw": ""}
|
||||
scene = parsed_scene
|
||||
idx = 1
|
||||
@@ -589,9 +596,10 @@ async def analyze_queries(
|
||||
created_after = _since_to_created_after(since_raw, current_time)
|
||||
|
||||
logger.info(
|
||||
f"[coding-judge] analyze scene={scene} rewrite={rewrite!r} "
|
||||
f"[coding-judge] analyze scene={scene} rewrite({len(rewrite)} chars) "
|
||||
f"since={since_raw!r} created_after={created_after}"
|
||||
)
|
||||
logger.debug(f"[coding-judge] analyze rewrite={rewrite!r}")
|
||||
return {
|
||||
"scene": scene, "is_coding": scene != "chat", "rewrite_query": rewrite,
|
||||
"created_after": created_after, "since_raw": since_raw, "ok": True,
|
||||
@@ -614,7 +622,10 @@ def _since_to_created_after(since_raw: str, current_time: str) -> Optional[float
|
||||
return dt.timestamp()
|
||||
except (ValueError, TypeError):
|
||||
continue
|
||||
logger.warning(f"[coding-judge] analyze_queries unparsable SINCE={since_raw!r}")
|
||||
logger.warning(
|
||||
f"[coding-judge] analyze_queries unparsable SINCE ({len(since_raw)} chars)"
|
||||
)
|
||||
logger.debug(f"[coding-judge] analyze_queries unparsable SINCE={since_raw!r}")
|
||||
return None
|
||||
|
||||
|
||||
|
||||
@@ -388,7 +388,10 @@ class CodingMemoryReconciler:
|
||||
obj = None
|
||||
|
||||
if not obj or not isinstance(obj, dict):
|
||||
logger.warning(f"[coding-reconcile] failed to parse ops output: {text!r}")
|
||||
logger.warning(
|
||||
f"[coding-reconcile] failed to parse ops output ({len(text or '')} chars)"
|
||||
)
|
||||
logger.debug(f"[coding-reconcile] unparsable ops output: {text!r}")
|
||||
return []
|
||||
ops_raw = obj.get("ops") or []
|
||||
if not isinstance(ops_raw, list):
|
||||
@@ -441,10 +444,10 @@ class CodingMemoryReconciler:
|
||||
for op in ops:
|
||||
if op.action == "ADD":
|
||||
if op.draft_idx is None or not (0 <= op.draft_idx < len(drafts)):
|
||||
logger.warning(f"[coding-reconcile] drop invalid ADD: {op.to_dict()}")
|
||||
logger.warning(f"[coding-reconcile] drop invalid ADD: {_op_ref(op)}")
|
||||
continue
|
||||
if op.draft_idx in consumed:
|
||||
logger.warning(f"[coding-reconcile] drop ADD on consumed draft: {op.to_dict()}")
|
||||
logger.warning(f"[coding-reconcile] drop ADD on consumed draft: {_op_ref(op)}")
|
||||
continue
|
||||
d = drafts[op.draft_idx]
|
||||
memory = self._draft_to_new_memory(d)
|
||||
@@ -458,18 +461,18 @@ class CodingMemoryReconciler:
|
||||
|
||||
elif op.action == "UPDATE":
|
||||
if op.draft_idx is None or not (0 <= op.draft_idx < len(drafts)):
|
||||
logger.warning(f"[coding-reconcile] drop invalid UPDATE: {op.to_dict()}")
|
||||
logger.warning(f"[coding-reconcile] drop invalid UPDATE: {_op_ref(op)}")
|
||||
continue
|
||||
if not op.target_experience_id:
|
||||
logger.warning(f"[coding-reconcile] drop UPDATE missing target: {op.to_dict()}")
|
||||
logger.warning(f"[coding-reconcile] drop UPDATE missing target: {_op_ref(op)}")
|
||||
continue
|
||||
if not self._target_was_offered(op, candidates_per_draft):
|
||||
logger.warning(
|
||||
f"[coding-reconcile] drop UPDATE outside authority scope: {op.to_dict()}"
|
||||
f"[coding-reconcile] drop UPDATE outside authority scope: {_op_ref(op)}"
|
||||
)
|
||||
continue
|
||||
if op.draft_idx in consumed:
|
||||
logger.warning(f"[coding-reconcile] drop UPDATE on consumed draft: {op.to_dict()}")
|
||||
logger.warning(f"[coding-reconcile] drop UPDATE on consumed draft: {_op_ref(op)}")
|
||||
continue
|
||||
d = drafts[op.draft_idx]
|
||||
memory = self._draft_to_new_memory(d, experience_id=op.target_experience_id)
|
||||
@@ -488,17 +491,17 @@ class CodingMemoryReconciler:
|
||||
|
||||
elif op.action == "DELETE":
|
||||
if not op.target_experience_id:
|
||||
logger.warning(f"[coding-reconcile] drop DELETE missing target: {op.to_dict()}")
|
||||
logger.warning(f"[coding-reconcile] drop DELETE missing target: {_op_ref(op)}")
|
||||
continue
|
||||
if not self._target_was_offered(op, candidates_per_draft):
|
||||
logger.warning(
|
||||
f"[coding-reconcile] drop DELETE outside authority scope: {op.to_dict()}"
|
||||
f"[coding-reconcile] drop DELETE outside authority scope: {_op_ref(op)}"
|
||||
)
|
||||
continue
|
||||
# DELETE 需要知道 user_id;通过 draft_idx(如有)或第一个 draft 推断
|
||||
user_id = self._infer_user_id(op, drafts)
|
||||
if not user_id:
|
||||
logger.warning(f"[coding-reconcile] drop DELETE no user_id context: {op.to_dict()}")
|
||||
logger.warning(f"[coding-reconcile] drop DELETE no user_id context: {_op_ref(op)}")
|
||||
continue
|
||||
try:
|
||||
ok = await self.store.delete(op.target_experience_id, user_id=user_id)
|
||||
@@ -577,6 +580,7 @@ class CodingMemoryReconciler:
|
||||
confidence=d.confidence,
|
||||
source=d.source,
|
||||
extraction_source=d.extraction_source,
|
||||
memory_at=d.memory_at,
|
||||
)
|
||||
|
||||
|
||||
@@ -584,6 +588,14 @@ class CodingMemoryReconciler:
|
||||
# Helpers
|
||||
# ================================================================
|
||||
|
||||
def _op_ref(op: ReconcileOp) -> str:
|
||||
"""op 的日志引用:只带动作与编号,不带 LLM 写的 reason。"""
|
||||
return (
|
||||
f"action={op.action} draft_idx={op.draft_idx} "
|
||||
f"target={op.target_experience_id}"
|
||||
)
|
||||
|
||||
|
||||
def _truncate(s: str, n: int) -> str:
|
||||
if not s:
|
||||
return ""
|
||||
|
||||
@@ -86,6 +86,7 @@ CREATE TABLE IF NOT EXISTS coding_memory_meta (
|
||||
confidence REAL NOT NULL DEFAULT 0.7,
|
||||
source TEXT NOT NULL DEFAULT 'default',
|
||||
extraction_source TEXT NOT NULL DEFAULT 'auto_extract',
|
||||
memory_at INTEGER,
|
||||
created_at TEXT NOT NULL,
|
||||
updated_at TEXT NOT NULL
|
||||
);
|
||||
@@ -109,6 +110,7 @@ CREATE TABLE IF NOT EXISTS working_fact (
|
||||
derived_experience_ids TEXT NOT NULL DEFAULT '[]',
|
||||
status TEXT NOT NULL DEFAULT 'active',
|
||||
fact_chain_summary TEXT NOT NULL DEFAULT '',
|
||||
memory_at INTEGER,
|
||||
created_at TEXT NOT NULL
|
||||
);
|
||||
-- 反查前驱(谁的 fact_link 指向我)走这个索引,写演变总结时要顺着链往回走
|
||||
@@ -164,6 +166,8 @@ _ADDED_COLUMNS = {
|
||||
"coding_memory_meta": {
|
||||
"source_fact_ids": "TEXT NOT NULL DEFAULT '[]'",
|
||||
"extraction_source": "TEXT NOT NULL DEFAULT 'auto_extract'",
|
||||
# 对话时间,秒级 epoch(同 chat)。老库里的行是 NULL:那时没存过。
|
||||
"memory_at": "INTEGER",
|
||||
},
|
||||
# 早期版本存过一列 fact_chain(机械累积的历代说法),已废弃:同样的信息顺着
|
||||
# fact_link 反查就有。老库里那一列留着不管,有默认值不影响插入。
|
||||
@@ -177,6 +181,7 @@ _ADDED_COLUMNS = {
|
||||
"subject": "TEXT NOT NULL DEFAULT ''",
|
||||
"details": "TEXT NOT NULL DEFAULT ''",
|
||||
"source": "TEXT NOT NULL DEFAULT 'default'",
|
||||
"memory_at": "INTEGER",
|
||||
},
|
||||
}
|
||||
|
||||
@@ -290,6 +295,29 @@ def _parse_dt(s: Optional[str]) -> Optional[datetime]:
|
||||
return None
|
||||
|
||||
|
||||
def _from_epoch(value: Any) -> Optional[datetime]:
|
||||
"""memory_at 列(秒级 epoch)→ datetime;NULL 或坏值返回 None。"""
|
||||
if value is None:
|
||||
return None
|
||||
try:
|
||||
return datetime.fromtimestamp(int(value))
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
|
||||
def _to_epoch(value: Optional[datetime]) -> Optional[int]:
|
||||
return int(value.timestamp()) if isinstance(value, datetime) else None
|
||||
|
||||
|
||||
def _later(a: Optional[datetime], b: Optional[datetime]) -> Optional[datetime]:
|
||||
"""两个对话时间取较晚的;一边缺就取另一边。"""
|
||||
if a is None:
|
||||
return b
|
||||
if b is None:
|
||||
return a
|
||||
return a if _to_epoch(a) >= _to_epoch(b) else b
|
||||
|
||||
|
||||
def _row_to_memory(row: sqlite3.Row) -> CodingMemory:
|
||||
"""SQLite row → CodingMemory"""
|
||||
return CodingMemory(
|
||||
@@ -308,6 +336,7 @@ def _row_to_memory(row: sqlite3.Row) -> CodingMemory:
|
||||
confidence=row["confidence"],
|
||||
source=row["source"],
|
||||
extraction_source=row["extraction_source"],
|
||||
memory_at=_from_epoch(row["memory_at"]),
|
||||
created_at=_parse_dt(row["created_at"]),
|
||||
updated_at=_parse_dt(row["updated_at"]),
|
||||
)
|
||||
@@ -334,6 +363,7 @@ def _row_to_fact(row: sqlite3.Row) -> WorkingFact:
|
||||
# 老库刚 ADD COLUMN 时可能取不到这一列,用 keys() 兜一下
|
||||
fact_chain_summary=(row["fact_chain_summary"] or ""
|
||||
if "fact_chain_summary" in row.keys() else ""),
|
||||
memory_at=_from_epoch(row["memory_at"]),
|
||||
created_at=_parse_dt(row["created_at"]),
|
||||
)
|
||||
|
||||
@@ -575,7 +605,11 @@ class CodingMemoryStore:
|
||||
await self._upsert_vec_keys(memory)
|
||||
logger.info(
|
||||
f"[coding-store] inserted experience_id={memory.experience_id} "
|
||||
f"task={memory.task!r} keys={1 + len(memory.search_keys)}"
|
||||
f"keys={1 + len(memory.search_keys)}"
|
||||
)
|
||||
logger.debug(
|
||||
f"[coding-store] inserted experience_id={memory.experience_id} "
|
||||
f"task={memory.task!r}"
|
||||
)
|
||||
|
||||
async def update(self, memory: CodingMemory) -> None:
|
||||
@@ -597,6 +631,8 @@ class CodingMemoryStore:
|
||||
memory.updated_at = datetime.utcnow()
|
||||
if memory.created_at is None:
|
||||
memory.created_at = prev.created_at
|
||||
# 合并后的经验讲的是两段对话,对话时间取较晚的那段
|
||||
memory.memory_at = _later(prev.memory_at, memory.memory_at)
|
||||
await _run_in_pool(self._upsert_sql_sync, memory)
|
||||
# 删旧 VDB
|
||||
await self._delete_vec_keys_by_count(memory.experience_id, prev_key_count)
|
||||
@@ -970,7 +1006,8 @@ class CodingMemoryStore:
|
||||
fact.created_at = datetime.utcnow()
|
||||
await _run_in_pool(self._upsert_fact_sql_sync, fact)
|
||||
await self._upsert_fact_vec(fact)
|
||||
logger.info(
|
||||
logger.info(f"[coding-store] inserted fact_id={fact.fact_id}")
|
||||
logger.debug(
|
||||
f"[coding-store] inserted fact_id={fact.fact_id} "
|
||||
f"statement={fact.statement!r}"
|
||||
)
|
||||
@@ -1199,8 +1236,8 @@ class CodingMemoryStore:
|
||||
experience_id, user_id, agent_id, task, search_keys, solution,
|
||||
boundary_envs, workspace_id, branch, session_id,
|
||||
files, source_fact_ids, confidence, source, extraction_source,
|
||||
created_at, updated_at
|
||||
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
memory_at, created_at, updated_at
|
||||
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
ON CONFLICT(experience_id) DO UPDATE SET
|
||||
user_id = excluded.user_id,
|
||||
agent_id = excluded.agent_id,
|
||||
@@ -1216,6 +1253,7 @@ class CodingMemoryStore:
|
||||
confidence = excluded.confidence,
|
||||
source = excluded.source,
|
||||
extraction_source = excluded.extraction_source,
|
||||
memory_at = excluded.memory_at,
|
||||
updated_at = excluded.updated_at
|
||||
""",
|
||||
(
|
||||
@@ -1226,6 +1264,7 @@ class CodingMemoryStore:
|
||||
json.dumps(memory.files, ensure_ascii=False),
|
||||
json.dumps(memory.source_fact_ids, ensure_ascii=False),
|
||||
memory.confidence, memory.source, memory.extraction_source,
|
||||
_to_epoch(memory.memory_at),
|
||||
(memory.created_at or datetime.utcnow()).isoformat(),
|
||||
(memory.updated_at or datetime.utcnow()).isoformat(),
|
||||
),
|
||||
@@ -1366,8 +1405,8 @@ class CodingMemoryStore:
|
||||
fact_id, user_id, agent_id, session_id, workspace_id, branch,
|
||||
source, statement, files, subject, details,
|
||||
fact_link, derived_experience_ids,
|
||||
status, fact_chain_summary, created_at
|
||||
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
status, fact_chain_summary, memory_at, created_at
|
||||
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
ON CONFLICT(fact_id) DO UPDATE SET
|
||||
user_id = excluded.user_id,
|
||||
agent_id = excluded.agent_id,
|
||||
@@ -1382,7 +1421,8 @@ class CodingMemoryStore:
|
||||
fact_link = excluded.fact_link,
|
||||
derived_experience_ids = excluded.derived_experience_ids,
|
||||
status = excluded.status,
|
||||
fact_chain_summary = excluded.fact_chain_summary
|
||||
fact_chain_summary = excluded.fact_chain_summary,
|
||||
memory_at = excluded.memory_at
|
||||
""",
|
||||
(
|
||||
fact.fact_id, fact.user_id, fact.agent_id,
|
||||
@@ -1395,6 +1435,7 @@ class CodingMemoryStore:
|
||||
json.dumps(fact.derived_experience_ids, ensure_ascii=False),
|
||||
fact.status,
|
||||
fact.fact_chain_summary or "",
|
||||
_to_epoch(fact.memory_at),
|
||||
(fact.created_at or datetime.utcnow()).isoformat(),
|
||||
),
|
||||
)
|
||||
|
||||
@@ -25,6 +25,11 @@ FactStatus = Literal["active", "superseded"]
|
||||
MAX_FACT_CHAIN_ENTRIES = 8
|
||||
|
||||
|
||||
def _epoch(value: Optional[datetime]) -> Optional[int]:
|
||||
"""memory_at 对外是秒级 epoch,与 chat 一致。"""
|
||||
return int(value.timestamp()) if isinstance(value, datetime) else None
|
||||
|
||||
|
||||
# ================================================================
|
||||
# 草稿(LLM extract 阶段输出)
|
||||
# ================================================================
|
||||
@@ -53,6 +58,8 @@ class CodingMemoryDraft:
|
||||
files: List[str] = field(default_factory=list)
|
||||
source: str = "default"
|
||||
extraction_source: str = "auto_extract"
|
||||
# 对话时间(SDK 注入,同 chat 的 memory_at)
|
||||
memory_at: Optional[datetime] = None
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
from ..utils.source_filter import normalize_source
|
||||
@@ -73,6 +80,7 @@ class CodingMemoryDraft:
|
||||
"files": list(self.files),
|
||||
"source": self.source,
|
||||
"extraction_source": self.extraction_source,
|
||||
"memory_at": _epoch(self.memory_at),
|
||||
}
|
||||
|
||||
|
||||
@@ -111,6 +119,9 @@ class CodingMemory:
|
||||
extraction_source: str = "auto_extract"
|
||||
|
||||
# 时间
|
||||
# memory_at 是对话时间(同 chat),created_at / updated_at 是写库时间。
|
||||
# UPDATE 合并时取两者中较晚的那个对话时间,见 store.update。
|
||||
memory_at: Optional[datetime] = None
|
||||
created_at: Optional[datetime] = None
|
||||
updated_at: Optional[datetime] = None
|
||||
|
||||
@@ -135,6 +146,7 @@ class CodingMemory:
|
||||
"confidence": self.confidence,
|
||||
"source": self.source,
|
||||
"extraction_source": self.extraction_source,
|
||||
"memory_at": _epoch(self.memory_at),
|
||||
"created_at": self.created_at.isoformat() if self.created_at else None,
|
||||
"updated_at": self.updated_at.isoformat() if self.updated_at else None,
|
||||
}
|
||||
@@ -249,6 +261,7 @@ class WorkingFact:
|
||||
fact_chain_summary: str = ""
|
||||
|
||||
# 时间
|
||||
memory_at: Optional[datetime] = None # 对话时间(同 chat),没传时是收到请求的时间
|
||||
created_at: Optional[datetime] = None # 产生时间,新的优先
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
@@ -272,6 +285,7 @@ class WorkingFact:
|
||||
"derived_experience_ids": list(self.derived_experience_ids),
|
||||
"status": self.status,
|
||||
"fact_chain_summary": self.fact_chain_summary,
|
||||
"memory_at": _epoch(self.memory_at),
|
||||
"created_at": self.created_at.isoformat() if self.created_at else None,
|
||||
}
|
||||
|
||||
|
||||
@@ -18,6 +18,7 @@ CodingWriter - 写入端协调器
|
||||
import logging
|
||||
import time
|
||||
import uuid
|
||||
from datetime import datetime
|
||||
from typing import Any, Dict, List, Optional
|
||||
|
||||
from ..agent.llm_provider import LLMProvider
|
||||
@@ -79,9 +80,11 @@ class CodingWriter:
|
||||
session_id: Optional[str] = None,
|
||||
request_id: Optional[str] = None,
|
||||
source: str = "default",
|
||||
memory_at: Optional[datetime] = None,
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
working/coding 段写入端到端。返回响应 dict(兼容 client.add 返回结构):
|
||||
working/coding 段写入端到端。memory_at 是对话时间,记到这次写出的经验和
|
||||
事实上;不传则用收到请求的时间(与 chat 链路一致)。返回响应 dict(兼容 client.add 返回结构):
|
||||
{
|
||||
"success": True,
|
||||
"scene": "productivity",
|
||||
@@ -101,6 +104,8 @@ class CodingWriter:
|
||||
"""
|
||||
request_id = request_id or str(uuid.uuid4())
|
||||
t0 = time.perf_counter()
|
||||
if memory_at is None:
|
||||
memory_at = datetime.now()
|
||||
|
||||
if not messages:
|
||||
return self._empty_response(request_id, t0)
|
||||
@@ -172,6 +177,7 @@ class CodingWriter:
|
||||
# extractor 只负责内容;调用来源由 SDK 入口注入,不能交给 LLM 生成。
|
||||
for draft in drafts:
|
||||
draft.source = source
|
||||
draft.memory_at = memory_at
|
||||
|
||||
# 3. 经验层 reconcile + 持久化
|
||||
ops: List[ReconcileOp] = (
|
||||
@@ -197,6 +203,7 @@ class CodingWriter:
|
||||
workspace_id=workspace_id,
|
||||
branch=branch,
|
||||
source=source,
|
||||
memory_at=memory_at,
|
||||
)
|
||||
|
||||
experience_ids = [op.target_experience_id for op in ops
|
||||
|
||||
@@ -80,7 +80,7 @@ def _count_tokens(text: str) -> int:
|
||||
try:
|
||||
import tiktoken
|
||||
|
||||
return len(tiktoken.get_encoding("cl100k_base").encode(text))
|
||||
return len(tiktoken.get_encoding("cl100k_base").encode(text, disallowed_special=()))
|
||||
except Exception:
|
||||
return len(text) // _CHARS_PER_TOKEN_FALLBACK + 1
|
||||
|
||||
@@ -98,7 +98,8 @@ def _truncate_for_embedding(text: str, limit: int = _EMBED_MAX_TOKENS) -> str:
|
||||
import tiktoken
|
||||
|
||||
enc = tiktoken.get_encoding("cl100k_base")
|
||||
ids = enc.encode(text)
|
||||
# Input is ordinary user text, not a tokenizer control-token stream.
|
||||
ids = enc.encode(text, disallowed_special=())
|
||||
if len(ids) <= limit:
|
||||
return text
|
||||
dropped = len(ids) - limit
|
||||
@@ -753,16 +754,22 @@ class EmbedService:
|
||||
"""
|
||||
打一条 INFO 级 embedding 请求日志(含 request_id / provider / model / api / 入参 / 出参)。
|
||||
|
||||
出参只含 vector 维度,不含向量数值。request_id 同时写入 message,
|
||||
出参只含 vector 维度,不含向量数值。入参的 input 是待向量化的原文
|
||||
(对话 / 记忆 / 查询),INFO 只记每条的字符数(input_chars),原文仅在
|
||||
DEBUG 单独输出。request_id 同时写入 message,
|
||||
并依赖 log formatter 的 %(request_id)s(setup_logging 后生效)。
|
||||
"""
|
||||
texts = request_payload.get("input")
|
||||
info_request = {k: v for k, v in request_payload.items() if k != "input"}
|
||||
if isinstance(texts, list):
|
||||
info_request["input_chars"] = [len(str(t)) for t in texts]
|
||||
payload = {
|
||||
"request_id": get_request_id(),
|
||||
"status": status,
|
||||
"provider": self._embed_config.provider,
|
||||
"model": self._embed_config.model,
|
||||
"api": api_url,
|
||||
"request": request_payload,
|
||||
"request": info_request,
|
||||
"response": response_payload,
|
||||
}
|
||||
try:
|
||||
@@ -770,6 +777,10 @@ class EmbedService:
|
||||
except Exception:
|
||||
body = str(payload)
|
||||
logger.info(f"[embed] {body}")
|
||||
if texts is not None and logger.isEnabledFor(logging.DEBUG):
|
||||
logger.debug(
|
||||
f"[embed] input: {json.dumps(texts, ensure_ascii=False, default=str)}"
|
||||
)
|
||||
|
||||
async def _embed_openai(
|
||||
self, texts: List[str]
|
||||
|
||||
@@ -115,11 +115,16 @@ class ReconcileHybridRetriever:
|
||||
fused_max = max(fused_all) if fused_all else 0.0
|
||||
|
||||
base = (
|
||||
f"[reconcile-hybrid][{req}] pool={pool_size} q_terms={q_terms!r} "
|
||||
f"[reconcile-hybrid][{req}] pool={pool_size} q_terms={len(q_terms)} "
|
||||
f"raw_bm25 max={raw_max:.4f} nonzero={raw_nonzero}/{len(raw_bm25)} "
|
||||
f"bm25_norm max={norm_max:.4f} nonzero={norm_nonzero}/{len(bm25_norms)} "
|
||||
f"sem_max={sem_max:.4f} fused_max={fused_max:.4f} "
|
||||
f"kept={kept}/{len(fused_all)} min_score={self._cfg.min_score} "
|
||||
f"query({len(query_text or '')} chars)"
|
||||
)
|
||||
# query 是待对账的新记忆原文,词项与预览只进 DEBUG
|
||||
logger.debug(
|
||||
f"[reconcile-hybrid][{req}] q_terms={q_terms!r} "
|
||||
f"query={q_preview!r} lemmatized={lem_preview!r}"
|
||||
)
|
||||
|
||||
|
||||
@@ -692,9 +692,10 @@ class CrossDomainSweeper:
|
||||
)
|
||||
|
||||
logger.info(
|
||||
f"[sweeper] Created L6 core {core_id[:12]}: '{core_content[:60]}' "
|
||||
f"[sweeper] Created L6 core {core_id[:12]} "
|
||||
f"from {len(cluster_ids)} basics, evidence_inherited={len(inherited_evidence_ids)}"
|
||||
)
|
||||
logger.debug(f"[sweeper] L6 core {core_id[:12]} content={core_content[:60]!r}")
|
||||
return 1, 0, [core_id]
|
||||
|
||||
except Exception as e:
|
||||
|
||||
@@ -3587,8 +3587,9 @@ class BaseReadPipeline(ReadPipeline):
|
||||
f"hidden_l0={len(hidden_l0_hits)} fwd_l6={len(forward_l6)} rev_l6={len(reverse_l6)} "
|
||||
f"fused_l6={len(fused_l6)} graph_intent={len(graph_intention_hits)} "
|
||||
f"returned={len(final_results)} "
|
||||
f"query='{request.query[:80]}'"
|
||||
f"query({len(request.query or '')} chars)"
|
||||
)
|
||||
logger.debug(f"[read/{self.VERSION}] query={request.query[:80]!r}")
|
||||
|
||||
response.success = True
|
||||
|
||||
|
||||
@@ -2516,7 +2516,8 @@ def _parse_operations_json(text: str) -> Optional[List[Dict[str, Any]]]:
|
||||
return result
|
||||
return None
|
||||
except json.JSONDecodeError:
|
||||
logger.warning(f"[S2-agent] JSON parse failed: {json_str[:200]}")
|
||||
logger.warning(f"[S2-agent] JSON parse failed ({len(json_str)} chars)")
|
||||
logger.debug(f"[S2-agent] unparsable output: {json_str[:200]!r}")
|
||||
return None
|
||||
|
||||
|
||||
|
||||
@@ -4,6 +4,7 @@ from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import concurrent.futures
|
||||
import contextvars
|
||||
import functools
|
||||
from concurrent.futures import Executor
|
||||
from typing import Any, Callable, Optional
|
||||
@@ -39,7 +40,9 @@ async def run_in_executor_guarded(
|
||||
|
||||
async def to_thread_guarded(func: Callable[..., Any], *args: Any, **kwargs: Any) -> Any:
|
||||
"""Guarded equivalent of ``asyncio.to_thread`` using the default pool."""
|
||||
return await run_in_executor_guarded(None, func, *args, **kwargs)
|
||||
context = contextvars.copy_context()
|
||||
call = functools.partial(func, *args, **kwargs)
|
||||
return await run_in_executor_guarded(None, context.run, call)
|
||||
|
||||
|
||||
def concurrent_result_guarded(
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""
|
||||
Pipeline Log Writer — 环节级 JSONL 日志(始终开启,与 SQLite Trace 分离)。
|
||||
Pipeline Log Writer — 环节级 JSONL 日志(默认开启,MEMORY_PIPELINE_LOG_ENABLED=false 关闭;
|
||||
与 SQLite Trace 分离)。
|
||||
|
||||
目录: {log_dir}/{subdir}/{date}.log
|
||||
每行: request_id, namespace, user_id, agent_id, step, prompt, content, parsed,
|
||||
|
||||
@@ -1,16 +1,20 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""
|
||||
Pipeline 可观测性:Log(文件 JSONL,始终开启)与 Trace(SQLite pipeline_logs,可关闭)。
|
||||
Pipeline 可观测性:Log(文件 JSONL,默认开启,可关闭)与 Trace(SQLite pipeline_logs,可关闭)。
|
||||
|
||||
- Log:运维 tail/grep,每个环节即时一行,不依赖后续步骤成功与否。
|
||||
- Trace:Inspector / 调试,落库 pipeline_logs(需 MEMORY_CACHE_BACKEND=sqlite 等持久化后端)。
|
||||
|
||||
环境变量:
|
||||
MEMORY_PIPELINE_LOG_ENABLED — Log 写 JSONL 文件,默认 true;false 时完全不写
|
||||
MEMORY_PIPELINE_TRACE_ENABLED — Trace 落库,默认 true;OpenClaw C 端可设 false
|
||||
MEMORY_MEMORY_OPERATIONS_ENABLED — memory_operations 审计表落库,默认 false
|
||||
MEMORY_LOG_DIR — Log 目录(优先)
|
||||
MEMORY_TRACE_LOG_DIR — Log 目录(兼容旧名)
|
||||
|
||||
Log 与 Trace 都关闭时不包装 store_pipeline_log,而是换成立即返回 True 的
|
||||
空操作:热路径不做任何计时、脱敏或后台任务,也不写库。
|
||||
|
||||
子目录({log_dir}/{subdir}/{date}.log)由 PipelineLogWriter 从每条记录的
|
||||
user_id 前缀(第一个 "__" 左侧)切分;无前缀回落 "default"。该前缀是
|
||||
通用 namespace,可用于项目、部署或数据分桶。
|
||||
@@ -94,6 +98,16 @@ def is_pipeline_trace_enabled() -> bool:
|
||||
)
|
||||
|
||||
|
||||
def is_pipeline_log_enabled() -> bool:
|
||||
"""是否把 pipeline step 写成 JSONL 文件({log_dir}/<subdir>/<date>.log)。默认开启。"""
|
||||
return os.getenv("MEMORY_PIPELINE_LOG_ENABLED", "true").strip().lower() in (
|
||||
"true",
|
||||
"1",
|
||||
"yes",
|
||||
"on",
|
||||
)
|
||||
|
||||
|
||||
def is_memory_operations_enabled() -> bool:
|
||||
"""是否将知识库变动写入 memory_operations 审计表。默认关闭(false)。"""
|
||||
return os.getenv("MEMORY_MEMORY_OPERATIONS_ENABLED", "false").lower() in (
|
||||
@@ -125,7 +139,7 @@ def redact_content(value: str) -> str:
|
||||
|
||||
def resolve_pipeline_log_dir() -> str:
|
||||
"""
|
||||
JSONL pipeline log 根目录(始终使用,有默认值)。
|
||||
JSONL pipeline log 根目录(MEMORY_PIPELINE_LOG_ENABLED 开启时使用,有默认值)。
|
||||
|
||||
优先级: MEMORY_LOG_DIR > MEMORY_TRACE_LOG_DIR > {MEMORY_DATA_DIR}/logs/pipeline
|
||||
"""
|
||||
@@ -228,6 +242,7 @@ def wrap_store_pipeline_log(
|
||||
) -> Callable[..., Any]:
|
||||
"""包装 cache.store_pipeline_log:计时同步、JSONL/DB 异步落盘,不堵主请求。
|
||||
|
||||
- log_writer 为 None 时不写 JSONL(MEMORY_PIPELINE_LOG_ENABLED=false)
|
||||
- step-to-step elapsed_ms / event_at_ms 在调用瞬间打点(与业务墙钟对齐)
|
||||
- event_at_ms:调用方传入非 0 值则直接复用(App 已在自己被调用的那一刻
|
||||
同步打点,比这里更接近真实事件时刻);传 0 / 不传才用本地 time.time()
|
||||
@@ -410,7 +425,8 @@ def wrap_store_pipeline_log(
|
||||
try:
|
||||
loop = asyncio.get_running_loop()
|
||||
# JSONL 用线程池,与 loop 无亲和,留在当前 loop。
|
||||
_track_task(loop.create_task(_persist_jsonl()))
|
||||
if log_writer is not None:
|
||||
_track_task(loop.create_task(_persist_jsonl()))
|
||||
# DB trace 用 cache pool(可能 aiomysql,绑定 owner_loop)。若当前不在
|
||||
# owner_loop(如 search 走独立 loop),派发回 owner_loop 执行,避免跨 loop。
|
||||
if trace_enabled:
|
||||
@@ -446,25 +462,26 @@ def wrap_store_pipeline_log(
|
||||
)
|
||||
except RuntimeError:
|
||||
# 无 running loop(极少见):退化为同步写,保证不丢日志
|
||||
try:
|
||||
log_writer.write_step(
|
||||
request_id=request_id,
|
||||
user_id=user_id,
|
||||
agent_id=agent_id,
|
||||
session_id=session_id,
|
||||
step=step,
|
||||
prompt=prompt,
|
||||
response=response,
|
||||
parsed=parsed,
|
||||
memory_ids=mem_ids,
|
||||
elapsed_ms=step_elapsed_ms,
|
||||
prompt_tokens=prompt_tokens,
|
||||
completion_tokens=completion_tokens,
|
||||
total_tokens=total_tokens,
|
||||
event_at_ms=event_at_ms,
|
||||
)
|
||||
except Exception:
|
||||
pass
|
||||
if log_writer is not None:
|
||||
try:
|
||||
log_writer.write_step(
|
||||
request_id=request_id,
|
||||
user_id=user_id,
|
||||
agent_id=agent_id,
|
||||
session_id=session_id,
|
||||
step=step,
|
||||
prompt=prompt,
|
||||
response=response,
|
||||
parsed=parsed,
|
||||
memory_ids=mem_ids,
|
||||
elapsed_ms=step_elapsed_ms,
|
||||
prompt_tokens=prompt_tokens,
|
||||
completion_tokens=completion_tokens,
|
||||
total_tokens=total_tokens,
|
||||
event_at_ms=event_at_ms,
|
||||
)
|
||||
except Exception:
|
||||
pass
|
||||
if trace_enabled:
|
||||
try:
|
||||
result = original_store(
|
||||
@@ -496,6 +513,11 @@ def wrap_store_pipeline_log(
|
||||
return _store_pipeline_observability
|
||||
|
||||
|
||||
async def _noop_store_pipeline_log(*args: Any, **kwargs: Any) -> bool:
|
||||
"""Log 与 Trace 都关闭时的 store_pipeline_log:不计时、不写文件、不写库。"""
|
||||
return True
|
||||
|
||||
|
||||
def install_pipeline_observability(
|
||||
cache: Any,
|
||||
*,
|
||||
@@ -503,6 +525,9 @@ def install_pipeline_observability(
|
||||
) -> None:
|
||||
"""给 cache 安装 JSONL + Trace 包装(fire-and-forget)。
|
||||
|
||||
MEMORY_PIPELINE_LOG_ENABLED=false 时不写 JSONL;若 Trace 也关闭,
|
||||
store_pipeline_log 被换成空操作(不包装、不写库)。
|
||||
|
||||
在 cache 所属 loop(write loop / _LoopThread)上调用,本函数会捕获该 loop
|
||||
作为 owner_loop 传给包装器:search 走独立 loop 后,从 search loop 触发的 DB
|
||||
落库会被派发回这个 owner_loop,避免跨 loop 用 aiomysql 污染连接池。
|
||||
@@ -514,15 +539,31 @@ def install_pipeline_observability(
|
||||
except RuntimeError:
|
||||
owner_loop = None
|
||||
|
||||
log_dir = resolve_pipeline_log_dir()
|
||||
log_writer = PipelineLogWriter(log_dir)
|
||||
log_enabled = is_pipeline_log_enabled()
|
||||
trace_enabled = is_pipeline_trace_enabled()
|
||||
logger.info(
|
||||
f"[{log_label}] Pipeline log (JSONL) enabled: {log_dir}/<subdir>/<date>.log "
|
||||
f"(subdir from user_id prefix before '__', else default) ; "
|
||||
f"pipeline trace (DB)={'on' if trace_enabled else 'off'} ; "
|
||||
f"persist=async(fire-and-forget)"
|
||||
)
|
||||
if not log_enabled and not trace_enabled:
|
||||
logger.info(
|
||||
f"[{log_label}] Pipeline log (JSONL) disabled ; pipeline trace (DB)=off ; "
|
||||
f"store_pipeline_log=no-op"
|
||||
)
|
||||
cache.store_pipeline_log = _noop_store_pipeline_log
|
||||
return
|
||||
|
||||
log_writer = None
|
||||
if log_enabled:
|
||||
log_dir = resolve_pipeline_log_dir()
|
||||
log_writer = PipelineLogWriter(log_dir)
|
||||
logger.info(
|
||||
f"[{log_label}] Pipeline log (JSONL) enabled: {log_dir}/<subdir>/<date>.log "
|
||||
f"(subdir from user_id prefix before '__', else default) ; "
|
||||
f"pipeline trace (DB)={'on' if trace_enabled else 'off'} ; "
|
||||
f"persist=async(fire-and-forget)"
|
||||
)
|
||||
else:
|
||||
logger.info(
|
||||
f"[{log_label}] Pipeline log (JSONL) disabled ; "
|
||||
f"pipeline trace (DB)=on ; persist=async(fire-and-forget)"
|
||||
)
|
||||
cache.store_pipeline_log = wrap_store_pipeline_log(
|
||||
cache.store_pipeline_log,
|
||||
log_writer=log_writer,
|
||||
|
||||
@@ -36,7 +36,8 @@ def count_tokens(text: str) -> int:
|
||||
enc = _get_encoder()
|
||||
if enc:
|
||||
try:
|
||||
return len(enc.encode(text))
|
||||
# Quoted control-token spellings are ordinary input text.
|
||||
return len(enc.encode(text, disallowed_special=()))
|
||||
except Exception as e:
|
||||
logger.debug(f"[token_count] encode failed, char fallback: {e}")
|
||||
return max(1, len(text) // 4)
|
||||
|
||||
+6
-4
@@ -29,10 +29,12 @@ prompt/response、密钥、访问令牌或完整本地路径。结构化日志
|
||||
文件。详细合同见
|
||||
[`docs/contracts/diagnostic-logging.md`](docs/contracts/diagnostic-logging.md)。
|
||||
|
||||
用户显式把诊断级别切换为 `debug` 时,Core 会启用 HY pipeline trace,并将
|
||||
`MEMORY_LOG_LLM_CONTENT` 设为 `true`。该模式会把完整模型 prompt/response 写入本地调试
|
||||
日志,其中可能包含获授权的对话正文和模型输出;它只应用于有意识的故障排查,不应长期
|
||||
开启,也不应在未经检查和脱敏时分享相关日志。切回 `info` 会停止后续正文记录,但不会
|
||||
HY pipeline 日志与 trace 在任何级别下都关闭。HY SDK 的文本日志只写入
|
||||
`hy-memory/governed-v2/logs/hy_memory.log`,其中的 WARNING 及以上同时写入 `core.log`;
|
||||
SDK 不在 INFO 及以上级别记录对话或记忆正文。用户显式把诊断级别切换为 `debug` 时,Core 将
|
||||
`MEMORY_LOG_LLM_CONTENT` 设为 `true`。该模式会把完整模型 prompt/response 写入
|
||||
`hy_memory.log` 的 DEBUG 行,其中可能包含获授权的对话正文和模型输出;它只应用于有意识的
|
||||
故障排查,不应长期开启,也不应在未经检查和脱敏时分享相关日志。切回 `info` 会停止后续正文记录,但不会
|
||||
自动删除此前已写入的调试文件。
|
||||
|
||||
具体行为以当前源码、网络请求实现和发布说明为准。发现实现与本说明不一致时,请按
|
||||
|
||||
+2
-2
@@ -18,7 +18,7 @@ If you run into a problem, please open an Issue with the version, platform,repro
|
||||
|
||||
## 发行与平台合同
|
||||
|
||||
源码当前固定使用公开 PyPI 正式版 `hy-memory==2.0.6`。MemHub 源码版本为 `1.0.5`
|
||||
源码当前固定使用公开 PyPI 正式版 `hy-memory==2.0.8`。MemHub 源码版本为 `1.0.5`
|
||||
(build 14),GitHub 仓库为
|
||||
[`Tencent-Hunyuan/Hy-Memory`](https://github.com/Tencent-Hunyuan/Hy-Memory),npm 包名为
|
||||
`memhub-cli`。当前代码架构见
|
||||
@@ -118,7 +118,7 @@ MemHub 的 SQLite 数据默认位于 `~/.one-memory`,不在 npm 包或 Runtime
|
||||
|
||||
## Hy Memory
|
||||
|
||||
SDK 依赖固定为公开 PyPI 正式版 `hy-memory==2.0.6`。SDK 配置、权限边界及 search
|
||||
SDK 依赖固定为公开 PyPI 正式版 `hy-memory==2.0.8`。SDK 配置、权限边界及 search
|
||||
返回格式见 [`MEMORY.md`](MEMORY.md)。HY search 返回的 `content` 已包含 SDK 统一生成的
|
||||
逻辑时间和演化上下文;调用方不应再次转换时间戳或重复拼接 `When`。
|
||||
|
||||
|
||||
@@ -2352,7 +2352,7 @@
|
||||
{
|
||||
"SPDXID": "SPDXRef-Package-hy-memory",
|
||||
"name": "hy-memory",
|
||||
"versionInfo": "2.0.6",
|
||||
"versionInfo": "2.0.8",
|
||||
"downloadLocation": "NOASSERTION",
|
||||
"filesAnalyzed": false,
|
||||
"licenseConcluded": "Apache-2.0",
|
||||
|
||||
@@ -9,7 +9,7 @@ Per-package license texts are not reproduced here yet; the license
|
||||
column is filled in as part of release review, and the SBOM records
|
||||
`NOASSERTION` until then rather than guessing.
|
||||
|
||||
## hy-memory 2.0.6
|
||||
## hy-memory 2.0.8
|
||||
|
||||
License: Apache-2.0
|
||||
|
||||
|
||||
@@ -12,7 +12,7 @@ dependencies = [
|
||||
"fastapi>=0.141.1",
|
||||
"filelock>=3.32.4",
|
||||
# Public SDK version validated against MemHub's permission and recall contracts.
|
||||
"hy-memory==2.0.6",
|
||||
"hy-memory==2.0.8",
|
||||
# chromadb arrives through the HY SDK; locked here so an unrelated upgrade
|
||||
# cannot move the SDK's store. Built-in embedding consumes numpy directly.
|
||||
"chromadb==1.5.9",
|
||||
|
||||
@@ -14,7 +14,7 @@ from pathlib import Path
|
||||
from typing import Literal, cast
|
||||
|
||||
HY_MEMORY_DISTRIBUTION = "hy-memory"
|
||||
HY_MEMORY_VERSION = "2.0.6"
|
||||
HY_MEMORY_VERSION = "2.0.8"
|
||||
HY_MEMORY_NAMESPACE = "governed-v2"
|
||||
DiagnosticLogLevel = Literal["error", "warning", "info", "debug"]
|
||||
|
||||
@@ -109,11 +109,30 @@ def apply_diagnostic_log_level(level: DiagnosticLogLevel) -> None:
|
||||
if any(name == root or name.startswith(f"{root}.") for root in _NOISY_LIBRARY_LOGGERS):
|
||||
logging.getLogger(name).setLevel(noisy_level)
|
||||
|
||||
# HY configures its text log once, on the first client, at the level of
|
||||
# that moment. Its file handler must follow later changes too, or a live
|
||||
# switch to debug never reaches hy_memory.log.
|
||||
for handler in logging.getLogger("hy_memory").handlers:
|
||||
if isinstance(handler, logging.FileHandler):
|
||||
handler.setLevel(selected)
|
||||
|
||||
os.environ["MEMORY_LOG_LEVEL"] = logging.getLevelName(selected)
|
||||
os.environ["MEMORY_PIPELINE_TRACE_ENABLED"] = "true" if level == "debug" else "false"
|
||||
_apply_hy_pipeline_policy()
|
||||
os.environ["MEMORY_LOG_LLM_CONTENT"] = "true" if level == "debug" else "false"
|
||||
|
||||
|
||||
def _apply_hy_pipeline_policy() -> None:
|
||||
"""Keep HY's per-step pipeline log and trace off at every level.
|
||||
|
||||
Nothing in MemHub reads them: Core's own diagnostics carry the step
|
||||
timing, and the JSONL/SQLite copies only grew on disk. With both off
|
||||
the SDK skips the per-step bookkeeping altogether.
|
||||
"""
|
||||
|
||||
os.environ["MEMORY_PIPELINE_LOG_ENABLED"] = "false"
|
||||
os.environ["MEMORY_PIPELINE_TRACE_ENABLED"] = "false"
|
||||
|
||||
|
||||
def hy_memory_version() -> str:
|
||||
"""Return the installed SDK version, failing if the pinned dist is absent."""
|
||||
|
||||
@@ -131,7 +150,9 @@ def configure_hy_runtime_environment(data_dir: Path) -> None:
|
||||
os.environ["MEMORY_LOG_DIR"] = str(log_dir)
|
||||
level = diagnostic_log_level(data_dir)
|
||||
os.environ["MEMORY_LOG_LEVEL"] = level.upper()
|
||||
os.environ["MEMORY_PIPELINE_TRACE_ENABLED"] = "true" if level == "debug" else "false"
|
||||
_apply_hy_pipeline_policy()
|
||||
# Still meaningful with the pipeline log off: it decides whether HY's
|
||||
# DEBUG lines in hy_memory.log carry full model prompts and responses.
|
||||
os.environ["MEMORY_LOG_LLM_CONTENT"] = "true" if level == "debug" else "false"
|
||||
os.environ["MEMORY_MEMORY_OPERATIONS_ENABLED"] = "false"
|
||||
os.environ["ANONYMIZED_TELEMETRY"] = "False"
|
||||
|
||||
@@ -426,6 +426,7 @@ def _repair_coding(
|
||||
path = root / name
|
||||
if not path.is_file():
|
||||
continue
|
||||
_ensure_coding_schema(path)
|
||||
with closing(sqlite3.connect(path)) as database:
|
||||
tables = {
|
||||
str(row[0])
|
||||
@@ -437,10 +438,14 @@ def _repair_coding(
|
||||
):
|
||||
if table not in tables:
|
||||
continue
|
||||
# _ensure_coding_schema added it; an SDK without the column
|
||||
# (before 2.0.7) leaves the time to the vector repair alone.
|
||||
has_memory_at = "memory_at" in _columns(database, table)
|
||||
rows = database.execute(
|
||||
f"SELECT {identifier},source,created_at FROM {table}"
|
||||
f"SELECT {identifier},source,created_at,"
|
||||
f"{'memory_at' if has_memory_at else 'NULL'} FROM {table}"
|
||||
).fetchall()
|
||||
for memory_id, source, created_at in rows:
|
||||
for memory_id, source, created_at, memory_at in rows:
|
||||
desired = mapping.get(str(memory_id))
|
||||
if desired is None:
|
||||
continue
|
||||
@@ -449,6 +454,13 @@ def _repair_coding(
|
||||
if desired.source and source in {None, "", "default"}:
|
||||
updates.append("source=?")
|
||||
values.append(desired.source)
|
||||
if (
|
||||
has_memory_at
|
||||
and desired.memory_at is not None
|
||||
and memory_at != desired.memory_at
|
||||
):
|
||||
updates.append("memory_at=?")
|
||||
values.append(desired.memory_at)
|
||||
if desired.created_at is not None:
|
||||
desired_created_at = (
|
||||
datetime.fromtimestamp(desired.created_at, tz=UTC)
|
||||
@@ -470,6 +482,32 @@ def _repair_coding(
|
||||
return repaired
|
||||
|
||||
|
||||
def _ensure_coding_schema(path: Path) -> None:
|
||||
"""Bring a Coding/Working store to the installed SDK's schema.
|
||||
|
||||
A migrated home's stores were written by an older SDK and may lack columns
|
||||
the repair writes (``memory_at`` since hy-memory 2.0.7). The repair runs
|
||||
before any SDK client opens them, so the SDK's own schema step is run here
|
||||
rather than repeating its DDL. ``CodingMemoryStore`` has no public
|
||||
schema-only entry point: ``initialize()`` also opens its Chroma
|
||||
collections, which must stay closed during the offline repair. So only its
|
||||
SQLite step runs, on a store object that carries nothing but that path.
|
||||
"""
|
||||
from hy_memory.coding.store import ( # type: ignore[import-untyped]
|
||||
CodingMemoryStore,
|
||||
)
|
||||
|
||||
# Relies on the pinned SDK's internals (hy-memory==2.0.8 is pinned exactly).
|
||||
store = CodingMemoryStore.__new__(CodingMemoryStore)
|
||||
store._db_path = str(path)
|
||||
store._conn = None
|
||||
try:
|
||||
store._init_sqlite_sync()
|
||||
finally:
|
||||
if store._conn is not None:
|
||||
store._conn.close()
|
||||
|
||||
|
||||
def _batches[T](values: list[T] | tuple[T, ...]) -> Iterator[tuple[T, ...]]:
|
||||
for start in range(0, len(values), _BATCH):
|
||||
yield tuple(values[start : start + _BATCH])
|
||||
@@ -36,6 +36,7 @@ from one_memory.hy_graph_admin import (
|
||||
)
|
||||
from one_memory.hy_sdk_types import HySdkRuntimeError
|
||||
from one_memory.local_embedding import LocalEmbedder
|
||||
from one_memory.log_file import confine_hy_sdk_logging
|
||||
from one_memory.model_profiles import (
|
||||
ModelProfileSnapshot,
|
||||
ProviderKind,
|
||||
@@ -391,6 +392,10 @@ def _acquire_client(
|
||||
except BaseException:
|
||||
_release_runtime(key)
|
||||
raise
|
||||
finally:
|
||||
# The constructor's first step configures HY's text log (once per
|
||||
# process), so by here it exists whether or not construction succeeded.
|
||||
confine_hy_sdk_logging()
|
||||
return ClientLease(client, key)
|
||||
|
||||
|
||||
|
||||
@@ -151,6 +151,7 @@ class HySdkMemoryRecord(BaseModel):
|
||||
if not content.strip():
|
||||
raise HySdkReadError("hy_memory_record_shape_unknown")
|
||||
status = str(raw.get("status") or "active")
|
||||
memory_at = raw.get("memory_at")
|
||||
created_at = _epoch(raw.get("created_at"))
|
||||
updated_at = _epoch(raw.get("updated_at"))
|
||||
try:
|
||||
@@ -167,6 +168,9 @@ class HySdkMemoryRecord(BaseModel):
|
||||
str(raw["session_id"]) if isinstance(raw.get("session_id"), str) else None
|
||||
),
|
||||
source=str(raw["source"]) if isinstance(raw.get("source"), str) else None,
|
||||
# Conversation time; hy-memory 2.0.7 stores it for experiences
|
||||
# and facts. Older stores leave it unknown.
|
||||
memory_at=memory_at if isinstance(memory_at, int) else None,
|
||||
gmt_created=created_at,
|
||||
scene=cast(MemoryScene, variant),
|
||||
kind=cast(MemoryKindName, kind),
|
||||
|
||||
@@ -20,8 +20,11 @@ def start_logging_to_file(data_dir: Path) -> Path:
|
||||
nowhere. That includes the loop watchdog: the one report that names what
|
||||
stalled the app was unreadable in exactly the situation it exists for.
|
||||
|
||||
The active diagnostic collection level controls Core, HY SDK, and provider
|
||||
client records. Uvicorn keeps its request log on its own non-propagating
|
||||
The active diagnostic collection level controls Core and provider client
|
||||
records. The HY SDK keeps its own text log, hy_memory.log under the SDK's
|
||||
log directory, and stops propagating here once its first client is built;
|
||||
`confine_hy_sdk_logging` then forwards only its warnings and errors to
|
||||
this file. Uvicorn keeps its request log on its own non-propagating
|
||||
loggers.
|
||||
"""
|
||||
destination = log_path(data_dir)
|
||||
@@ -49,3 +52,53 @@ def start_logging_to_file(data_dir: Path) -> Path:
|
||||
if root.level > logging.INFO or root.level == logging.NOTSET:
|
||||
root.setLevel(logging.INFO)
|
||||
return destination
|
||||
|
||||
|
||||
HY_SDK_LOGGER = "hy_memory"
|
||||
|
||||
|
||||
class _CoreLogForwarder(logging.Handler):
|
||||
"""Hands HY warnings to Core's own core.log handler, rotation and format."""
|
||||
|
||||
def __init__(self, target: logging.Handler) -> None:
|
||||
super().__init__(level=logging.WARNING)
|
||||
self.target = target
|
||||
|
||||
def emit(self, record: logging.LogRecord) -> None:
|
||||
self.target.handle(record)
|
||||
|
||||
|
||||
def confine_hy_sdk_logging() -> None:
|
||||
"""Keep HY's text log in hy_memory.log, with only its problems in core.log.
|
||||
|
||||
HY's first client sets up hy_memory.log plus a stdout and a stderr stream
|
||||
on the "hy_memory" logger, and stops it propagating to the root. Under
|
||||
launchd those streams are service.log and service-error.log, so every SDK
|
||||
line was written twice, and none reached core.log. Drop the streams and
|
||||
forward WARNING and above to core.log, which the Debug panel reads; the
|
||||
SDK keeps conversation text out of those levels.
|
||||
|
||||
Idempotent. Does nothing until HY has configured its logger, because a
|
||||
propagating logger already reaches core.log through the root.
|
||||
"""
|
||||
sdk = logging.getLogger(HY_SDK_LOGGER)
|
||||
if sdk.propagate:
|
||||
return
|
||||
for handler in list(sdk.handlers):
|
||||
if isinstance(handler, logging.StreamHandler) and not isinstance(
|
||||
handler, logging.FileHandler
|
||||
):
|
||||
sdk.removeHandler(handler)
|
||||
if any(isinstance(handler, _CoreLogForwarder) for handler in sdk.handlers):
|
||||
return
|
||||
core = next(
|
||||
(
|
||||
handler
|
||||
for handler in logging.getLogger().handlers
|
||||
if isinstance(handler, RotatingFileHandler)
|
||||
and Path(handler.baseFilename).name == "core.log"
|
||||
),
|
||||
None,
|
||||
)
|
||||
if core is not None:
|
||||
sdk.addHandler(_CoreLogForwarder(core))
|
||||
@@ -79,7 +79,7 @@
|
||||
| `test_database_file_permissions.py` | `contract` | 覆盖 a new main database is readable by its owner alone、the wal sidecars are not left world readable、a database from an older build is tightened on open 场景。 |
|
||||
| `test_db_session_utc.py` | `contract` | The MySQL session must be pinned to UTC. |
|
||||
| `test_derived_echo.py` | `contract` | A host that saves our injection back must not mint a new user decision. |
|
||||
| `test_detailed_logging.py` | `contract` | 覆盖 detailed pipeline logging defaults off and survives restart、live log level applies to full hy and provider call chain、info collects hy info without enabling debug pipeline content 等 7 个场景。 |
|
||||
| `test_detailed_logging.py` | `contract` | 覆盖 detailed logging defaults off and survives restart、hy pipeline log and trace stay off at every level、live level change reaches the hy text log file 等 7 个场景。 |
|
||||
| `test_diagnostic_log.py` | `contract` | 覆盖 high frequency success events are summarized but warnings are not、structured log is private correlated and owner only、arbitrary detail and absolute stack are rejected 等 14 个场景。 |
|
||||
| `test_diagnostic_log_durability.py` | `contract` | 写线程报告的「已落盘」必须是真的。 |
|
||||
| `test_diagnostic_log_off_loop.py` | `contract` | 诊断日志不得在调用线程上做磁盘 I/O。 |
|
||||
@@ -112,6 +112,7 @@
|
||||
| `test_hy_log_privacy.py` | `contract` | HY SDK records are not rewritten by One Memory's logging setup. |
|
||||
| `test_hy_network_egress.py` | `contract` | Where a real HY client is allowed to send bytes, and where it is not. |
|
||||
| `test_hy_runtime_config.py` | `contract` | MEMORY.md is the audited record of what we send HY; keep it answerable. |
|
||||
| `test_hy_sdk_log_routing.py` | `contract` | HY SDK text logs go to hy_memory.log only, with warnings also in core.log. |
|
||||
| `test_hy_step_trace_switch.py` | `contract` | Rule 15: the per-step HY trace is off unless the debug tier asks for it. |
|
||||
| `test_import_api.py` | `integration` | 覆盖 resume during cleanup returns conflict without internal error、authorize detect import and revoke api flow、protected api rejects missing token and foreign origin 等 5 个场景。 |
|
||||
| `test_import_control_races_review.py` | `contract` | Read-only review repro: real runner/store, synthetic delayed adapter cleanup. |
|
||||
@@ -307,4 +308,4 @@
|
||||
| `test_workspace_recall.py` | `contract` | Workspace memories were unreachable from every host; these pin the way in. |
|
||||
| `test_write_queue_lease_renewal.py` | `contract` | Rule 8: a lease must outlive the work it covers, and be renewed while it runs. |
|
||||
|
||||
共 302 个测试文件。
|
||||
共 303 个测试文件。
|
||||
@@ -14,8 +14,8 @@ from pathlib import Path
|
||||
import pytest
|
||||
|
||||
PRIMARY_MARKERS = ("unit", "contract", "integration", "performance", "live")
|
||||
EXPECTED_TEST_FILE_COUNT = 302
|
||||
EXPECTED_TEST_FILES_SHA256 = "07707528025ac134e46c93ebef12335e852df1ed0f52f8beb62abb3b4d1b0e97"
|
||||
EXPECTED_TEST_FILE_COUNT = 303
|
||||
EXPECTED_TEST_FILES_SHA256 = "7993203411697fc0bf42b92fa316f222a66e6826c5b4a0132dbb0a1ca6da4e32"
|
||||
|
||||
# Pure transformations, validation models, parity checks and state machines.
|
||||
# This list is intentionally small: uncertain files default to contract.
|
||||
|
||||
@@ -1,6 +1,5 @@
|
||||
import json
|
||||
import logging
|
||||
from importlib import import_module
|
||||
from pathlib import Path
|
||||
|
||||
from one_memory.hy_runtime_config import (
|
||||
@@ -12,20 +11,50 @@ from one_memory.hy_runtime_config import (
|
||||
)
|
||||
|
||||
|
||||
def test_detailed_pipeline_logging_defaults_off_and_survives_restart(
|
||||
tmp_path: Path, monkeypatch
|
||||
) -> None:
|
||||
monkeypatch.delenv("MEMORY_PIPELINE_TRACE_ENABLED", raising=False)
|
||||
def test_detailed_logging_defaults_off_and_survives_restart(tmp_path: Path, monkeypatch) -> None:
|
||||
monkeypatch.delenv("MEMORY_LOG_LEVEL", raising=False)
|
||||
assert detailed_logging_enabled(tmp_path) is False
|
||||
configure_hy_runtime_environment(tmp_path)
|
||||
assert __import__("os").environ["MEMORY_PIPELINE_TRACE_ENABLED"] == "false"
|
||||
assert __import__("os").environ["MEMORY_LOG_LEVEL"] == "INFO"
|
||||
assert diagnostic_log_level(tmp_path) == "info"
|
||||
|
||||
assert set_detailed_logging(tmp_path, True) is True
|
||||
assert detailed_logging_enabled(tmp_path) is True
|
||||
monkeypatch.delenv("MEMORY_PIPELINE_TRACE_ENABLED", raising=False)
|
||||
monkeypatch.delenv("MEMORY_LOG_LEVEL", raising=False)
|
||||
configure_hy_runtime_environment(tmp_path)
|
||||
assert __import__("os").environ["MEMORY_PIPELINE_TRACE_ENABLED"] == "true"
|
||||
assert __import__("os").environ["MEMORY_LOG_LEVEL"] == "DEBUG"
|
||||
|
||||
|
||||
def test_hy_pipeline_log_and_trace_stay_off_at_every_level(tmp_path: Path, monkeypatch) -> None:
|
||||
environment = __import__("os").environ
|
||||
for level in ("error", "warning", "info", "debug"):
|
||||
for name in ("MEMORY_PIPELINE_LOG_ENABLED", "MEMORY_PIPELINE_TRACE_ENABLED"):
|
||||
monkeypatch.setenv(name, "true")
|
||||
set_diagnostic_log_level(tmp_path, level)
|
||||
assert environment["MEMORY_PIPELINE_LOG_ENABLED"] == "false"
|
||||
assert environment["MEMORY_PIPELINE_TRACE_ENABLED"] == "false"
|
||||
|
||||
for name in ("MEMORY_PIPELINE_LOG_ENABLED", "MEMORY_PIPELINE_TRACE_ENABLED"):
|
||||
monkeypatch.setenv(name, "true")
|
||||
configure_hy_runtime_environment(tmp_path)
|
||||
assert environment["MEMORY_PIPELINE_LOG_ENABLED"] == "false"
|
||||
assert environment["MEMORY_PIPELINE_TRACE_ENABLED"] == "false"
|
||||
|
||||
|
||||
def test_live_level_change_reaches_the_hy_text_log_file(tmp_path: Path) -> None:
|
||||
sdk = logging.getLogger("hy_memory")
|
||||
text_log = logging.FileHandler(tmp_path / "hy_memory.log", encoding="utf-8")
|
||||
text_log.setLevel(logging.INFO)
|
||||
sdk.addHandler(text_log)
|
||||
try:
|
||||
set_diagnostic_log_level(tmp_path, "debug")
|
||||
assert text_log.level == logging.DEBUG
|
||||
set_diagnostic_log_level(tmp_path, "warning")
|
||||
assert text_log.level == logging.WARNING
|
||||
finally:
|
||||
sdk.removeHandler(text_log)
|
||||
text_log.close()
|
||||
set_diagnostic_log_level(tmp_path, "info")
|
||||
|
||||
|
||||
def test_live_log_level_applies_to_full_hy_and_provider_call_chain(
|
||||
@@ -60,43 +89,9 @@ def test_info_collects_hy_info_without_enabling_debug_pipeline_content(
|
||||
assert logging.getLogger("openai._base_client").isEnabledFor(logging.INFO)
|
||||
assert not logging.getLogger("openai._base_client").isEnabledFor(logging.DEBUG)
|
||||
assert __import__("os").environ["MEMORY_LOG_LEVEL"] == "INFO"
|
||||
assert __import__("os").environ["MEMORY_PIPELINE_TRACE_ENABLED"] == "false"
|
||||
assert __import__("os").environ["MEMORY_LOG_LLM_CONTENT"] == "false"
|
||||
|
||||
|
||||
def test_live_debug_switch_makes_sdk_pipeline_writer_keep_original_content(
|
||||
tmp_path: Path,
|
||||
) -> None:
|
||||
writer_type = import_module("hy_memory.utils.pipeline_log_writer").PipelineLogWriter
|
||||
writer = writer_type(str(tmp_path / "pipeline"))
|
||||
|
||||
set_diagnostic_log_level(tmp_path, "info")
|
||||
writer.write_step(
|
||||
request_id="info",
|
||||
step="EXTRACT",
|
||||
prompt="info prompt",
|
||||
response="info response",
|
||||
)
|
||||
set_diagnostic_log_level(tmp_path, "debug")
|
||||
writer.write_step(
|
||||
request_id="debug",
|
||||
step="EXTRACT",
|
||||
prompt="debug prompt",
|
||||
response="debug response",
|
||||
)
|
||||
|
||||
lines = [
|
||||
json.loads(line)
|
||||
for target in (tmp_path / "pipeline").rglob("*.log")
|
||||
for line in target.read_text(encoding="utf-8").splitlines()
|
||||
]
|
||||
info, debug = lines
|
||||
assert info["prompt"].startswith("<redacted len=")
|
||||
assert info["content"].startswith("<redacted len=")
|
||||
assert debug["prompt"] == "debug prompt"
|
||||
assert debug["content"] == "debug response"
|
||||
|
||||
|
||||
def test_log_level_document_migrates_the_old_boolean_shape(tmp_path: Path) -> None:
|
||||
target = tmp_path / "settings" / "diagnostic-logging.json"
|
||||
target.parent.mkdir(parents=True)
|
||||
@@ -117,6 +112,7 @@ def test_configure_environment_passes_debug_content_settings_to_sdk(
|
||||
"MEMORY_DATA_DIR",
|
||||
"MEMORY_LOG_DIR",
|
||||
"MEMORY_LOG_LEVEL",
|
||||
"MEMORY_PIPELINE_LOG_ENABLED",
|
||||
"MEMORY_PIPELINE_TRACE_ENABLED",
|
||||
"MEMORY_LOG_LLM_CONTENT",
|
||||
):
|
||||
@@ -129,5 +125,6 @@ def test_configure_environment_passes_debug_content_settings_to_sdk(
|
||||
assert environment["MEMORY_DATA_DIR"] == str(memory_root)
|
||||
assert environment["MEMORY_LOG_DIR"] == str(memory_root / "logs")
|
||||
assert environment["MEMORY_LOG_LEVEL"] == "DEBUG"
|
||||
assert environment["MEMORY_PIPELINE_TRACE_ENABLED"] == "true"
|
||||
assert environment["MEMORY_PIPELINE_LOG_ENABLED"] == "false"
|
||||
assert environment["MEMORY_PIPELINE_TRACE_ENABLED"] == "false"
|
||||
assert environment["MEMORY_LOG_LLM_CONTENT"] == "true"
|
||||
@@ -0,0 +1,146 @@
|
||||
"""HY SDK text logs go to hy_memory.log only, with warnings also in core.log.
|
||||
|
||||
The SDK's first client adds a stdout and a stderr stream to the "hy_memory"
|
||||
logger and stops it propagating. Under launchd those streams are
|
||||
service.log and service-error.log, so each SDK line used to be written
|
||||
twice and none reached core.log.
|
||||
"""
|
||||
|
||||
import logging
|
||||
import sys
|
||||
from logging.handlers import RotatingFileHandler
|
||||
from pathlib import Path
|
||||
from types import SimpleNamespace
|
||||
|
||||
import pytest
|
||||
|
||||
from one_memory import hy_sdk_runtime
|
||||
from one_memory.log_file import (
|
||||
HY_SDK_LOGGER,
|
||||
confine_hy_sdk_logging,
|
||||
log_path,
|
||||
start_logging_to_file,
|
||||
)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def sdk_logger():
|
||||
"""The "hy_memory" logger as HY's setup_logging leaves it, restored after."""
|
||||
sdk = logging.getLogger(HY_SDK_LOGGER)
|
||||
root = logging.getLogger()
|
||||
saved = (list(sdk.handlers), sdk.propagate, sdk.level, list(root.handlers), root.level)
|
||||
yield sdk
|
||||
for handler in sdk.handlers:
|
||||
if handler not in saved[0]:
|
||||
handler.close()
|
||||
for handler in root.handlers:
|
||||
if handler not in saved[3]:
|
||||
handler.close()
|
||||
sdk.handlers, sdk.propagate, root.handlers = saved[0], saved[1], saved[3]
|
||||
sdk.setLevel(saved[2])
|
||||
root.setLevel(saved[4])
|
||||
|
||||
|
||||
def _configure_like_hy(sdk: logging.Logger, log_dir: Path) -> logging.FileHandler:
|
||||
log_dir.mkdir(parents=True, exist_ok=True)
|
||||
text_log = logging.FileHandler(log_dir / "hy_memory.log", encoding="utf-8")
|
||||
stdout = logging.StreamHandler(sys.stdout)
|
||||
stderr = logging.StreamHandler(sys.stderr)
|
||||
stderr.setLevel(logging.WARNING)
|
||||
sdk.handlers = [text_log, stdout, stderr]
|
||||
sdk.propagate = False
|
||||
sdk.setLevel(logging.INFO)
|
||||
return text_log
|
||||
|
||||
|
||||
def test_sdk_lines_stay_in_hy_memory_log_and_warnings_reach_core_log(
|
||||
tmp_path: Path, sdk_logger: logging.Logger
|
||||
) -> None:
|
||||
start_logging_to_file(tmp_path)
|
||||
text_log = _configure_like_hy(sdk_logger, tmp_path / "hy")
|
||||
|
||||
confine_hy_sdk_logging()
|
||||
confine_hy_sdk_logging()
|
||||
|
||||
assert not [
|
||||
handler
|
||||
for handler in sdk_logger.handlers
|
||||
if isinstance(handler, logging.StreamHandler)
|
||||
and not isinstance(handler, logging.FileHandler)
|
||||
]
|
||||
assert text_log in sdk_logger.handlers
|
||||
assert len(sdk_logger.handlers) == 2, "the core.log forwarder is added once"
|
||||
|
||||
child = logging.getLogger("hy_memory.client")
|
||||
child.info("[add] routine sdk line")
|
||||
child.warning("[add] sdk problem")
|
||||
for handler in logging.getLogger().handlers:
|
||||
handler.flush()
|
||||
text_log.flush()
|
||||
|
||||
hy_text = (tmp_path / "hy" / "hy_memory.log").read_text(encoding="utf-8")
|
||||
core_text = log_path(tmp_path).read_text(encoding="utf-8")
|
||||
assert "routine sdk line" in hy_text and "sdk problem" in hy_text
|
||||
assert "sdk problem" in core_text
|
||||
assert "routine sdk line" not in core_text
|
||||
assert core_text.count("sdk problem") == 1
|
||||
|
||||
|
||||
def test_nothing_changes_before_hy_has_configured_its_logger(
|
||||
tmp_path: Path, sdk_logger: logging.Logger
|
||||
) -> None:
|
||||
start_logging_to_file(tmp_path)
|
||||
sdk_logger.handlers = []
|
||||
sdk_logger.propagate = True
|
||||
|
||||
confine_hy_sdk_logging()
|
||||
|
||||
assert sdk_logger.handlers == []
|
||||
|
||||
|
||||
def test_without_core_log_the_streams_are_still_dropped(
|
||||
tmp_path: Path, sdk_logger: logging.Logger
|
||||
) -> None:
|
||||
logging.getLogger().handlers = [
|
||||
handler
|
||||
for handler in logging.getLogger().handlers
|
||||
if not isinstance(handler, RotatingFileHandler)
|
||||
]
|
||||
text_log = _configure_like_hy(sdk_logger, tmp_path / "hy")
|
||||
|
||||
confine_hy_sdk_logging()
|
||||
|
||||
assert sdk_logger.handlers == [text_log]
|
||||
|
||||
|
||||
@pytest.mark.parametrize("fails", [False, True])
|
||||
def test_every_client_construction_confines_the_sdk_log(
|
||||
tmp_path: Path, monkeypatch, fails: bool
|
||||
) -> None:
|
||||
order: list[str] = []
|
||||
|
||||
class Client:
|
||||
def __init__(self, **_options):
|
||||
order.append("client")
|
||||
if fails:
|
||||
raise RuntimeError("construction failed")
|
||||
|
||||
monkeypatch.setattr(hy_sdk_runtime, "HyMemoryClient", Client)
|
||||
monkeypatch.setattr(hy_sdk_runtime, "build_sdk_config", lambda *_a, **_k: SimpleNamespace())
|
||||
monkeypatch.setattr(hy_sdk_runtime, "inspect_graph_width_mismatch", lambda _c: None)
|
||||
monkeypatch.setattr(hy_sdk_runtime, "_acquire_runtime", lambda _c, _k: object())
|
||||
monkeypatch.setattr(hy_sdk_runtime, "_release_runtime", lambda _k: None)
|
||||
monkeypatch.setattr(hy_sdk_runtime, "confine_hy_sdk_logging", lambda: order.append("confine"))
|
||||
profile = SimpleNamespace(embedding_provider_kind=None, provider_kind=None)
|
||||
|
||||
if fails:
|
||||
with pytest.raises(RuntimeError):
|
||||
hy_sdk_runtime._acquire_client(
|
||||
profile, api_key=None, data_dir=tmp_path, http_transport_factory=None
|
||||
)
|
||||
else:
|
||||
hy_sdk_runtime._acquire_client(
|
||||
profile, api_key=None, data_dir=tmp_path, http_transport_factory=None
|
||||
)
|
||||
|
||||
assert order == ["client", "confine"]
|
||||
@@ -55,6 +55,9 @@ def chat_row(memory_id, content, *, source="cursor"):
|
||||
}
|
||||
|
||||
|
||||
EXPERIENCE_MEMORY_AT = 1_789_990_000
|
||||
|
||||
|
||||
def experience_row(memory_id, task, solution, *, variant="working", source="codebuddy"):
|
||||
return {
|
||||
"experience_id": memory_id,
|
||||
@@ -63,6 +66,8 @@ def experience_row(memory_id, task, solution, *, variant="working", source="code
|
||||
"task": task,
|
||||
"solution": solution,
|
||||
"source": source,
|
||||
# Conversation time, epoch seconds like chat (hy-memory 2.0.7).
|
||||
"memory_at": EXPERIENCE_MEMORY_AT,
|
||||
"created_at": "2026-10-01T08:00:00",
|
||||
"updated_at": "2026-10-02T08:00:00",
|
||||
"_variant": variant,
|
||||
@@ -446,6 +451,7 @@ def test_keyword_filter_reads_experience_and_fact_text(tmp_path, monkeypatch):
|
||||
assert experience["content"] == "Fix flaky tests\nPin the pytest seed"
|
||||
assert experience["status"] == "active"
|
||||
assert experience["updated_at"] > experience["created_at"]
|
||||
assert experience["memory_at"] == EXPERIENCE_MEMORY_AT
|
||||
fact = by_id["fact-1"]
|
||||
assert (fact["scene"], fact["kind"], fact["variant"]) == ("coding", "fact", "coding")
|
||||
assert fact["content"] == "The pytest suite runs in CI"
|
||||
@@ -488,6 +494,10 @@ def test_empty_query_lists_all_scenes_with_source_filter_and_totals(tmp_path, mo
|
||||
"chat",
|
||||
"chat",
|
||||
]
|
||||
assert [item["memory_at"] for item in body["memories"][:2]] == [
|
||||
EXPERIENCE_MEMORY_AT,
|
||||
EXPERIENCE_MEMORY_AT,
|
||||
]
|
||||
call = sdk.list_calls[0]
|
||||
assert call["scene"] == "all"
|
||||
assert call["include_sources"] is None
|
||||
@@ -527,6 +537,7 @@ def test_detail_edit_delete_pass_kind_and_variant(tmp_path, monkeypatch):
|
||||
assert record["task"] == "Old task"
|
||||
assert record["solution"] == "Old solution"
|
||||
assert record["content"] == "Old task\nOld solution"
|
||||
assert record["memory_at"] == EXPERIENCE_MEMORY_AT
|
||||
assert sdk.get_calls[-1] == ("exp-1", "experience", "working")
|
||||
|
||||
# Chat detail with no namespace keeps the plain SDK call.
|
||||
@@ -692,5 +703,6 @@ def test_home_summary_counts_what_the_memory_page_lists(tmp_path, monkeypatch):
|
||||
assert (home_list["offset"], home_list["limit"], home_list["order"]) == (0, 1, "desc")
|
||||
assert body["latest"]["memory_id"] == "exp-w"
|
||||
assert body["latest"]["scene"] == "working"
|
||||
assert body["latest"]["memory_at"] == EXPERIENCE_MEMORY_AT
|
||||
|
||||
run_with_sdk(tmp_path, monkeypatch, scenario)
|
||||
@@ -137,19 +137,38 @@ def test_post_cutover_metadata_repair_restores_source_and_original_times(store):
|
||||
)
|
||||
coding = data_dir / "hy-memory" / "governed-v2" / "coding_cv.sqlite3"
|
||||
with sqlite3.connect(coding) as database:
|
||||
database.execute(
|
||||
"CREATE TABLE coding_memory_meta ("
|
||||
"experience_id TEXT,user_id TEXT,agent_id TEXT,solution TEXT,"
|
||||
"source TEXT,created_at TEXT,updated_at TEXT)"
|
||||
# A store written by hy-memory 2.0.6, before the conversation time
|
||||
# column: the repair must add it through the SDK and then fill it.
|
||||
database.executescript(
|
||||
"""
|
||||
CREATE TABLE coding_memory_meta (
|
||||
experience_id TEXT PRIMARY KEY, user_id TEXT NOT NULL,
|
||||
agent_id TEXT NOT NULL DEFAULT 'default_agent', task TEXT NOT NULL,
|
||||
search_keys TEXT NOT NULL DEFAULT '[]', solution TEXT NOT NULL,
|
||||
boundary_envs TEXT NOT NULL DEFAULT '', workspace_id TEXT,
|
||||
branch TEXT, session_id TEXT, files TEXT NOT NULL DEFAULT '[]',
|
||||
source_fact_ids TEXT NOT NULL DEFAULT '[]',
|
||||
confidence REAL NOT NULL DEFAULT 0.7,
|
||||
source TEXT NOT NULL DEFAULT 'default',
|
||||
extraction_source TEXT NOT NULL DEFAULT 'auto_extract',
|
||||
created_at TEXT NOT NULL, updated_at TEXT NOT NULL
|
||||
);
|
||||
CREATE TABLE working_fact (
|
||||
fact_id TEXT PRIMARY KEY, user_id TEXT NOT NULL,
|
||||
agent_id TEXT NOT NULL DEFAULT 'default_agent', session_id TEXT,
|
||||
workspace_id TEXT, branch TEXT, source TEXT NOT NULL DEFAULT 'default',
|
||||
statement TEXT NOT NULL, files TEXT NOT NULL DEFAULT '[]',
|
||||
subject TEXT NOT NULL DEFAULT '', details TEXT NOT NULL DEFAULT '',
|
||||
fact_link TEXT, derived_experience_ids TEXT NOT NULL DEFAULT '[]',
|
||||
status TEXT NOT NULL DEFAULT 'active',
|
||||
fact_chain_summary TEXT NOT NULL DEFAULT '', created_at TEXT NOT NULL
|
||||
);
|
||||
"""
|
||||
)
|
||||
database.execute(
|
||||
"CREATE TABLE working_fact ("
|
||||
"fact_id TEXT,user_id TEXT,agent_id TEXT,statement TEXT,"
|
||||
"source TEXT,created_at TEXT)"
|
||||
)
|
||||
database.execute(
|
||||
"INSERT INTO coding_memory_meta VALUES "
|
||||
"('sdk-1','local','scope','remember','default','now','now')"
|
||||
"INSERT INTO coding_memory_meta "
|
||||
"(experience_id,user_id,agent_id,task,solution,source,created_at,updated_at) "
|
||||
"VALUES ('sdk-1','local','scope','remember','remember','default','now','now')"
|
||||
)
|
||||
|
||||
result = repair_sdk_metadata_from_bundle(
|
||||
@@ -168,10 +187,11 @@ def test_post_cutover_metadata_repair_restores_source_and_original_times(store):
|
||||
2026, 10, 18, tzinfo=UTC
|
||||
)
|
||||
with sqlite3.connect(coding) as database:
|
||||
source, created_at = database.execute(
|
||||
"SELECT source,created_at FROM coding_memory_meta WHERE experience_id='sdk-1'"
|
||||
source, memory_at, created_at = database.execute(
|
||||
"SELECT source,memory_at,created_at FROM coding_memory_meta WHERE experience_id='sdk-1'"
|
||||
).fetchone()
|
||||
assert source == "workbuddy"
|
||||
assert datetime.fromtimestamp(memory_at, tz=UTC) == datetime(2026, 10, 17, tzinfo=UTC)
|
||||
assert created_at.startswith("2026-10-18T00:00:00")
|
||||
|
||||
repeated = repair_sdk_metadata_from_bundle(
|
||||
|
||||
@@ -91,7 +91,7 @@ def test_bundle_contains_management_sdk_config_and_verified_manifest(tmp_path):
|
||||
assert manifest["bundle_id"] == "migration-1"
|
||||
assert manifest["sdk_namespace"] == "governed-v2"
|
||||
assert manifest["runtime_build"] == "build-13"
|
||||
assert manifest["snapshot_sdk_version"] == "2.0.6"
|
||||
assert manifest["snapshot_sdk_version"] == "2.0.8"
|
||||
paths = {item["path"]: item for item in manifest["files"]}
|
||||
assert {
|
||||
"management/one-memory.sqlite3",
|
||||
|
||||
Generated
+4
-4
@@ -991,7 +991,7 @@ wheels = [
|
||||
|
||||
[[package]]
|
||||
name = "hy-memory"
|
||||
version = "2.0.6"
|
||||
version = "2.0.8"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "chromadb" },
|
||||
@@ -1008,9 +1008,9 @@ dependencies = [
|
||||
{ name = "scikit-learn" },
|
||||
{ name = "tiktoken" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/18/98/312ac8bcbe41e63200a93d296300b2e6748e1a532dad01f009dad57ba408/hy_memory-2.0.6.tar.gz", hash = "sha256:d0f54d54867ee7e722a8dd1a88a00c0576904435f15ae7899fe640840bff74bc", size = 855200, upload-time = "2026-10-27T07:53:10.022Z" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/21/15/83ca83b649ead238f186fdb33efdc73615fbba279efbbdca3857c1a86beb/hy_memory-2.0.8.tar.gz", hash = "sha256:0fa0ed1a1160b389a3d7c13a9fb6528b140d7ec23cf49e65f1be5974820a576b", size = 856025, upload-time = "2026-10-28T07:35:47.783Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/27/f1/d4866cffe426d77b857168cc694b15df866f3d9f45152c17c148e1595db4/hy_memory-2.0.6-py3-none-any.whl", hash = "sha256:b0e666f8e89d1763bb31c0e67502df74434134a638ff6fd1912de9e936be2483", size = 918955, upload-time = "2026-10-27T07:53:07.686Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/11/6a/c699cac0ff06bd089fceeff90446130763d8187e649dbdb0313014491a10/hy_memory-2.0.8-py3-none-any.whl", hash = "sha256:68bd789df24e51e64bb231e97a8d24ee89b4aa5f9ba14c3e0824589e9213bf7c", size = 921119, upload-time = "2026-10-28T07:35:45.571Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -1837,7 +1837,7 @@ requires-dist = [
|
||||
{ name = "fastapi", specifier = ">=0.141.1" },
|
||||
{ name = "filelock", specifier = ">=3.32.4" },
|
||||
{ name = "httpx", specifier = ">=0.28.1" },
|
||||
{ name = "hy-memory", specifier = "==2.0.6" },
|
||||
{ name = "hy-memory", specifier = "==2.0.8" },
|
||||
{ name = "keyring", specifier = ">=25.7.0" },
|
||||
{ name = "numpy", specifier = "==2.5.2" },
|
||||
{ name = "onnxruntime", specifier = "==1.23.2" },
|
||||
|
||||
@@ -230,39 +230,21 @@ it("filters raw Core INFO and DEBUG while preserving warnings and errors", async
|
||||
).toBe(true);
|
||||
});
|
||||
|
||||
it("shows HY pipeline metrics without exposing model content", async () => {
|
||||
const dataDir = await mkdtemp(
|
||||
path.join(os.tmpdir(), "om-debug-hy-pipeline-"),
|
||||
);
|
||||
it("does not read HY's own log directory", async () => {
|
||||
// MemHub turns the HY pipeline log off; files left by older builds and
|
||||
// HY's text log stay on disk but are not part of the Debug panel.
|
||||
const dataDir = await mkdtemp(path.join(os.tmpdir(), "om-debug-hy-logs-"));
|
||||
roots.push(dataDir);
|
||||
const pipelineDir = path.join(
|
||||
dataDir,
|
||||
"hy-memory",
|
||||
"governed-v2",
|
||||
"logs",
|
||||
"default",
|
||||
);
|
||||
await mkdir(pipelineDir, { recursive: true });
|
||||
const target = path.join(pipelineDir, "2026-10-14.log");
|
||||
await writeFile(target, "");
|
||||
const hyLogs = path.join(dataDir, "hy-memory", "governed-v2", "logs");
|
||||
await mkdir(path.join(hyLogs, "default"), { recursive: true });
|
||||
const cursor = await captureDebugLogCursor(dataDir);
|
||||
const pipelineEntry = {
|
||||
request_id: "request-safe-1",
|
||||
user_id: "private-user",
|
||||
agent_id: "private-agent",
|
||||
session_id: "private-session",
|
||||
step: "EXTRACT",
|
||||
prompt: "private prompt",
|
||||
content: "private response",
|
||||
parsed: "private parsed output",
|
||||
tokens: 42,
|
||||
elapsed_ms: 123.5,
|
||||
timestamp: "2026-10-14T10:01:00.000",
|
||||
};
|
||||
await appendFile(target, `${JSON.stringify(pipelineEntry)}\n`);
|
||||
await writeFile(
|
||||
path.join(dataDir, "hy-memory", "governed-v2", "logs", "hy_memory.log"),
|
||||
`${JSON.stringify({ ...pipelineEntry, request_id: "must-not-appear" })}\n`,
|
||||
path.join(hyLogs, "default", "2026-10-14.log"),
|
||||
`${JSON.stringify({ request_id: "old-pipeline", step: "EXTRACT" })}\n`,
|
||||
);
|
||||
await writeFile(
|
||||
path.join(hyLogs, "hy_memory.log"),
|
||||
"2026-10-14 10:01:00 [WARNING] [-] hy_memory.client: hy text log\n",
|
||||
);
|
||||
|
||||
const snapshot = await readDebugLogSnapshot(
|
||||
@@ -271,29 +253,11 @@ it("shows HY pipeline metrics without exposing model content", async () => {
|
||||
cursor,
|
||||
);
|
||||
|
||||
expect(snapshot.entries).toEqual([
|
||||
expect.objectContaining({
|
||||
service: "hy-memory",
|
||||
component: "hy.pipeline",
|
||||
event: "hy.pipeline.extract",
|
||||
message: "EXTRACT · 42 tokens · 124 ms",
|
||||
duration_ms: 123.5,
|
||||
trace_id: "request-safe-1",
|
||||
details: { phase: "EXTRACT", count: 42 },
|
||||
source: "pipeline",
|
||||
}),
|
||||
]);
|
||||
const serialized = JSON.stringify(snapshot);
|
||||
for (const secret of [
|
||||
"private prompt",
|
||||
"private response",
|
||||
"private parsed output",
|
||||
"private-user",
|
||||
"private-agent",
|
||||
"private-session",
|
||||
]) {
|
||||
expect(serialized).not.toContain(secret);
|
||||
}
|
||||
expect(snapshot.entries).toEqual([]);
|
||||
expect(cursor).toEqual({
|
||||
diagnosticLogs: [],
|
||||
serviceLogs: expect.any(Array),
|
||||
});
|
||||
});
|
||||
|
||||
it("logs a renderer ui-action with action, page, phase and the shared trace id", async () => {
|
||||
|
||||
@@ -18,7 +18,6 @@ const UI_TRACE = "one-memory:log-ui-trace";
|
||||
const DEBUG_LOGS = "one-memory:get-debug-logs";
|
||||
const DEBUG_LOG_NAME =
|
||||
/^diagnostic-(runtime|core|host-bridge|desktop|cli)\.jsonl(?:\.\d+)?$/;
|
||||
const PIPELINE_LOG_NAME = /^\d{4}-\d{2}-\d{2}\.log$/;
|
||||
const SERVICE_LOG_NAMES = [
|
||||
"service.log",
|
||||
"service-error.log",
|
||||
@@ -30,24 +29,16 @@ interface ServiceLogCursor {
|
||||
files: Array<{ device: number; inode: number; offset: number }>;
|
||||
}
|
||||
|
||||
interface PipelineLogCursor {
|
||||
relativePath: string;
|
||||
device: number;
|
||||
inode: number;
|
||||
offset: number;
|
||||
}
|
||||
|
||||
export interface DebugLogCursor {
|
||||
diagnosticLogs: Array<{ device: number; inode: number; offset: number }>;
|
||||
serviceLogs: ServiceLogCursor[];
|
||||
pipelineLogs: PipelineLogCursor[];
|
||||
}
|
||||
|
||||
export interface DebugLogEntry {
|
||||
id: string;
|
||||
timestamp: string;
|
||||
level: "debug" | "info" | "warn" | "error";
|
||||
service: "runtime" | "core" | "host-bridge" | "desktop" | "cli" | "hy-memory";
|
||||
service: "runtime" | "core" | "host-bridge" | "desktop" | "cli";
|
||||
component: string;
|
||||
event: string;
|
||||
outcome: string;
|
||||
@@ -57,7 +48,7 @@ export interface DebugLogEntry {
|
||||
trace_id?: string;
|
||||
details?: Record<string, string | number | boolean | null>;
|
||||
message?: string;
|
||||
source: "diagnostic" | "stdout" | "stderr" | "pipeline";
|
||||
source: "diagnostic" | "stdout" | "stderr";
|
||||
}
|
||||
|
||||
export interface DebugLogSnapshot {
|
||||
@@ -259,7 +250,6 @@ export async function readDebugLogSnapshot(
|
||||
cursor: DebugLogCursor = {
|
||||
diagnosticLogs: [],
|
||||
serviceLogs: [],
|
||||
pipelineLogs: [],
|
||||
},
|
||||
): Promise<DebugLogSnapshot> {
|
||||
const started = Number.isFinite(startedAt) ? startedAt : Date.now();
|
||||
@@ -300,7 +290,6 @@ export async function readDebugLogSnapshot(
|
||||
);
|
||||
}
|
||||
}
|
||||
entries.push(...(await readPipelineLogEntries(dataDir, cursor, started)));
|
||||
entries.sort((left, right) =>
|
||||
left.timestamp === right.timestamp
|
||||
? left.id.localeCompare(right.id)
|
||||
@@ -319,7 +308,6 @@ export async function captureDebugLogCursor(
|
||||
): Promise<DebugLogCursor> {
|
||||
const directory = path.join(dataDir, "logs");
|
||||
const names = await readdir(directory).catch(() => []);
|
||||
const pipelineRoot = path.join(dataDir, "hy-memory", "governed-v2", "logs");
|
||||
return {
|
||||
diagnosticLogs: (
|
||||
await Promise.all(
|
||||
@@ -355,7 +343,6 @@ export async function captureDebugLogCursor(
|
||||
};
|
||||
}),
|
||||
),
|
||||
pipelineLogs: await pipelineLogFiles(pipelineRoot),
|
||||
};
|
||||
}
|
||||
|
||||
@@ -462,8 +449,8 @@ async function readServiceLogEntries(
|
||||
if (!message) continue;
|
||||
const level = serviceLogLevel(message, cursor.name);
|
||||
// Raw stdout is an implementation trace, not a product event. Core's
|
||||
// structured diagnostics and HY pipeline logs already carry meaningful
|
||||
// successful work with request/task identity. Showing raw INFO/DEBUG
|
||||
// structured diagnostics already carry meaningful successful work with
|
||||
// request/task identity. Showing raw INFO/DEBUG
|
||||
// here duplicated those events and turned polling access logs, file
|
||||
// watchers and library initialization into thousands of fake "useful"
|
||||
// rows. Keep raw output only when it needs attention.
|
||||
@@ -543,135 +530,6 @@ function logLineTimestamp(
|
||||
return new Date(Math.max(startedAt, fallback)).toISOString();
|
||||
}
|
||||
|
||||
async function pipelineLogFiles(root: string): Promise<PipelineLogCursor[]> {
|
||||
const rootInfo = await lstat(root).catch(() => null);
|
||||
if (!rootInfo?.isDirectory() || rootInfo.isSymbolicLink()) return [];
|
||||
const files: PipelineLogCursor[] = [];
|
||||
const visit = async (directory: string): Promise<void> => {
|
||||
for (const entry of await readdir(directory, { withFileTypes: true })) {
|
||||
const target = path.join(directory, entry.name);
|
||||
if (entry.isSymbolicLink()) continue;
|
||||
if (entry.isDirectory()) {
|
||||
await visit(target);
|
||||
continue;
|
||||
}
|
||||
// The same root also contains HY's human-readable hy_memory.log.
|
||||
// Pipeline files are date-named JSONL; accepting every .log file makes
|
||||
// the Debug poller repeatedly parse unrelated text and still display no
|
||||
// HY entries.
|
||||
if (!entry.isFile() || !PIPELINE_LOG_NAME.test(entry.name)) continue;
|
||||
const info = await safeRegularFile(target);
|
||||
if (!info) continue;
|
||||
files.push({
|
||||
relativePath: path.relative(root, target),
|
||||
device: info.dev,
|
||||
inode: info.ino,
|
||||
offset: info.size,
|
||||
});
|
||||
}
|
||||
};
|
||||
await visit(root);
|
||||
return files;
|
||||
}
|
||||
|
||||
async function readPipelineLogEntries(
|
||||
dataDir: string,
|
||||
cursor: DebugLogCursor,
|
||||
startedAt: number,
|
||||
): Promise<DebugLogEntry[]> {
|
||||
const root = path.join(dataDir, "hy-memory", "governed-v2", "logs");
|
||||
const current = await pipelineLogFiles(root);
|
||||
const baseline = new Map(
|
||||
cursor.pipelineLogs.map((file) => [
|
||||
`${file.device}:${file.inode}`,
|
||||
file.offset,
|
||||
]),
|
||||
);
|
||||
const entries: DebugLogEntry[] = [];
|
||||
for (const file of current) {
|
||||
const target = path.join(root, file.relativePath);
|
||||
const offset = baseline.get(`${file.device}:${file.inode}`) ?? 0;
|
||||
const info = await safeRegularFile(target);
|
||||
if (!info || info.size <= offset) continue;
|
||||
const content = await readFileFromOffset(target, offset).catch(() => "");
|
||||
let lineNumber = 0;
|
||||
for (const line of content.split("\n")) {
|
||||
lineNumber += 1;
|
||||
if (!line.trim()) continue;
|
||||
const entry = pipelineLogEntry(
|
||||
line,
|
||||
file,
|
||||
lineNumber,
|
||||
offset,
|
||||
startedAt,
|
||||
info.mtimeMs,
|
||||
);
|
||||
if (entry) entries.push(entry);
|
||||
}
|
||||
}
|
||||
return entries;
|
||||
}
|
||||
|
||||
function pipelineLogEntry(
|
||||
line: string,
|
||||
file: PipelineLogCursor,
|
||||
lineNumber: number,
|
||||
offset: number,
|
||||
startedAt: number,
|
||||
fallbackTime: number,
|
||||
): DebugLogEntry | null {
|
||||
let value: Record<string, unknown>;
|
||||
try {
|
||||
value = JSON.parse(line) as Record<string, unknown>;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
const step = typeof value.step === "string" ? value.step.trim() : "";
|
||||
if (!step || !/^[A-Za-z0-9_-]{1,80}$/.test(step)) return null;
|
||||
const rawTimestamp =
|
||||
typeof value.timestamp === "string" ? value.timestamp : "";
|
||||
const parsedTimestamp = Date.parse(rawTimestamp);
|
||||
const timestamp = new Date(
|
||||
Math.max(
|
||||
startedAt,
|
||||
Number.isFinite(parsedTimestamp) ? parsedTimestamp : fallbackTime,
|
||||
),
|
||||
).toISOString();
|
||||
const elapsed = safeFiniteNumber(value.elapsed_ms);
|
||||
const tokens = safeFiniteNumber(value.tokens);
|
||||
const requestId =
|
||||
typeof value.request_id === "string" && value.request_id.length <= 256
|
||||
? value.request_id
|
||||
: undefined;
|
||||
return {
|
||||
id: `hy-pipeline:${file.device}:${file.inode}:${offset}:${lineNumber}`,
|
||||
timestamp,
|
||||
level: "info",
|
||||
service: "hy-memory",
|
||||
component: "hy.pipeline",
|
||||
event: `hy.pipeline.${step.toLowerCase().replaceAll("_", "-")}`,
|
||||
outcome: "observed",
|
||||
...(elapsed === undefined ? {} : { duration_ms: elapsed }),
|
||||
...(requestId ? { trace_id: requestId } : {}),
|
||||
details: {
|
||||
phase: step,
|
||||
...(tokens === undefined ? {} : { count: tokens }),
|
||||
},
|
||||
message: [
|
||||
step,
|
||||
...(tokens === undefined ? [] : [`${Math.round(tokens)} tokens`]),
|
||||
...(elapsed === undefined ? [] : [`${Math.round(elapsed)} ms`]),
|
||||
].join(" · "),
|
||||
source: "pipeline",
|
||||
};
|
||||
}
|
||||
|
||||
function safeFiniteNumber(value: unknown): number | undefined {
|
||||
return typeof value === "number" && Number.isFinite(value)
|
||||
? value
|
||||
: undefined;
|
||||
}
|
||||
|
||||
export function installDiagnosticsIpc(
|
||||
options: DiagnosticIpcOptions,
|
||||
): () => void {
|
||||
|
||||
@@ -15,6 +15,7 @@ const fake = vi.hoisted(() => ({
|
||||
missingBroker: false,
|
||||
hostCapability: true,
|
||||
open: vi.fn(),
|
||||
refreshHosts: vi.fn(async (_layouts: unknown, _options: unknown) => ({})),
|
||||
host: vi.fn<
|
||||
(input: {
|
||||
hostId: string;
|
||||
@@ -274,7 +275,9 @@ beforeEach(async () => {
|
||||
fake.open.mockResolvedValue({
|
||||
hostGrantStore: {},
|
||||
repairInstalledCommand: vi.fn(async () => []),
|
||||
refreshInstalledHosts: fake.refreshHosts,
|
||||
});
|
||||
fake.refreshHosts.mockClear();
|
||||
fake.host.mockResolvedValue({ state: "synthetic-remote-result" });
|
||||
Object.defineProperty(process, "resourcesPath", {
|
||||
configurable: true,
|
||||
@@ -590,5 +593,11 @@ describe("packaged Desktop host-control ownership", () => {
|
||||
expect(fake.ensureRuntime).not.toHaveBeenCalled();
|
||||
expect(fake.createClient).not.toHaveBeenCalled();
|
||||
expect(fake.windows).toHaveLength(1);
|
||||
// The embedded Runtime is this process, so Desktop refreshes the hosts
|
||||
// it installed once the Runtime is up, as a standalone one would itself.
|
||||
expect(fake.refreshHosts).toHaveBeenCalledTimes(1);
|
||||
expect(fake.refreshHosts.mock.calls[0]?.[1]).toMatchObject({
|
||||
diagnostics: expect.anything(),
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -21,6 +21,8 @@ import {
|
||||
claudeCodeLayout,
|
||||
codexLayout,
|
||||
cursorLayout,
|
||||
MCP_TOOLSET_KEY,
|
||||
MCP_TOOLSET_VERSION,
|
||||
mcpConfigPath,
|
||||
piLayout,
|
||||
planRecallInstall,
|
||||
@@ -29,6 +31,7 @@ import {
|
||||
workbuddyLayout,
|
||||
workbuddyScatteredLayout,
|
||||
} from "@one-memory/host-bridge/installer";
|
||||
import type { DiagnosticInput } from "@one-memory/runtime-control";
|
||||
import { afterEach, describe, expect, it, vi } from "vitest";
|
||||
import { probeRecallInstall } from "../../../packages/host-bridge/src/recall-probe.js";
|
||||
import { skillFilePath } from "../../../packages/host-bridge/src/skill-drift.js";
|
||||
@@ -1384,3 +1387,197 @@ describe("upgrading installs left by an earlier release", () => {
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe("refreshing installed hosts after an upgrade", () => {
|
||||
function recorder() {
|
||||
const events: DiagnosticInput[] = [];
|
||||
return {
|
||||
events,
|
||||
sink: {
|
||||
log: async (event: DiagnosticInput) => {
|
||||
events.push(event);
|
||||
},
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
async function linkedWorkbuddy(context: Awaited<ReturnType<typeof fixture>>) {
|
||||
const layout = workbuddyLayout(path.join(context.root, "workbuddy"));
|
||||
const preview = await context.permissions.previewInstall(layout);
|
||||
const result = await context.permissions.install(
|
||||
layout,
|
||||
preview.fingerprint,
|
||||
);
|
||||
await context.permissions.setPermission(
|
||||
layout,
|
||||
approval(required(result.permission.receipt)),
|
||||
);
|
||||
return layout;
|
||||
}
|
||||
|
||||
/** Put back the skill an earlier release wrote, journaled as ours. */
|
||||
async function ageSkill(layout: ReturnType<typeof workbuddyLayout>) {
|
||||
const skillPath = required(skillFilePath(layout));
|
||||
const aged = (await readFile(skillPath, "utf8"))
|
||||
.replace(/skill-version=\d+/, "skill-version=15")
|
||||
.replace("`MemHub_recall`", "`MemHub_ask`");
|
||||
await writeFile(skillPath, aged);
|
||||
const journal = JSON.parse(await readFile(layout.journalPath, "utf8"));
|
||||
journal.installed_skill_hash = createHash("sha256")
|
||||
.update(aged)
|
||||
.digest("hex");
|
||||
await writeFile(layout.journalPath, JSON.stringify(journal));
|
||||
return aged;
|
||||
}
|
||||
|
||||
it("rewrites an old skill on the first start of a build, and only once per build", async () => {
|
||||
const context = await fixture();
|
||||
const layout = await linkedWorkbuddy(context);
|
||||
const current = await readFile(required(skillFilePath(layout)), "utf8");
|
||||
await ageSkill(layout);
|
||||
const { events, sink } = recorder();
|
||||
|
||||
const first = await context.permissions.refreshInstalledHosts([layout], {
|
||||
build: "34",
|
||||
diagnostics: sink,
|
||||
});
|
||||
expect(first).toMatchObject({
|
||||
state: "refreshed",
|
||||
synced: ["workbuddy"],
|
||||
failed: [],
|
||||
});
|
||||
expect(await readFile(required(skillFilePath(layout)), "utf8")).toBe(
|
||||
current,
|
||||
);
|
||||
|
||||
// Same build again: one file read, nothing rewritten, even if stale.
|
||||
const aged = await ageSkill(layout);
|
||||
expect(
|
||||
await context.permissions.refreshInstalledHosts([layout], {
|
||||
build: "34",
|
||||
diagnostics: sink,
|
||||
}),
|
||||
).toMatchObject({ state: "already_refreshed", synced: [] });
|
||||
expect(await readFile(required(skillFilePath(layout)), "utf8")).toBe(aged);
|
||||
|
||||
// The next build refreshes again.
|
||||
expect(
|
||||
await context.permissions.refreshInstalledHosts([layout], {
|
||||
build: "35",
|
||||
}),
|
||||
).toMatchObject({ state: "refreshed", synced: ["workbuddy"] });
|
||||
expect(await readFile(required(skillFilePath(layout)), "utf8")).toBe(
|
||||
current,
|
||||
);
|
||||
|
||||
// Diagnostics carry host ids and counts only, never file content.
|
||||
expect(events.map((event) => event.event)).toEqual([
|
||||
"host_refresh.host_updated",
|
||||
"host_refresh.completed",
|
||||
"host_refresh.skipped",
|
||||
]);
|
||||
expect(events[0].details).toEqual({ source_id: "workbuddy" });
|
||||
expect(events[1].details).toEqual({
|
||||
count: 1,
|
||||
updated_count: 1,
|
||||
error_count: 0,
|
||||
});
|
||||
expect(JSON.stringify(events)).not.toContain("MemHub_");
|
||||
expect(JSON.stringify(events)).not.toContain(context.root);
|
||||
});
|
||||
|
||||
it("leaves a skill the user edited untouched", async () => {
|
||||
const context = await fixture();
|
||||
const layout = await linkedWorkbuddy(context);
|
||||
const edited = `${await ageSkill(layout)}\nUser addition\n`;
|
||||
await writeFile(required(skillFilePath(layout)), edited);
|
||||
|
||||
expect(
|
||||
await context.permissions.refreshInstalledHosts([layout], {
|
||||
build: "34",
|
||||
}),
|
||||
).toMatchObject({ state: "refreshed", synced: [], failed: [] });
|
||||
expect(await readFile(required(skillFilePath(layout)), "utf8")).toBe(
|
||||
edited,
|
||||
);
|
||||
});
|
||||
|
||||
it("rewrites an MCP entry without the tool set stamp, in the plugin manifest too", async () => {
|
||||
const context = await fixture();
|
||||
const layout = await linkedWorkbuddy(context);
|
||||
const plugin = required(layout.plugin);
|
||||
const manifests = [
|
||||
path.join(plugin.installRoot, ".codebuddy-plugin", "plugin.json"),
|
||||
path.join(
|
||||
plugin.marketplaceRoot,
|
||||
"plugins",
|
||||
plugin.pluginName,
|
||||
".codebuddy-plugin",
|
||||
"plugin.json",
|
||||
),
|
||||
];
|
||||
// What an earlier release wrote: the same entry with no stamp, journaled.
|
||||
const journal = JSON.parse(await readFile(layout.journalPath, "utf8"));
|
||||
for (const manifestPath of manifests) {
|
||||
const manifest = JSON.parse(await readFile(manifestPath, "utf8"));
|
||||
expect(manifest.mcpServers["one-memory"].env[MCP_TOOLSET_KEY]).toBe(
|
||||
MCP_TOOLSET_VERSION,
|
||||
);
|
||||
delete manifest.mcpServers["one-memory"].env[MCP_TOOLSET_KEY];
|
||||
const text = `${JSON.stringify(manifest, null, 2)}\n`;
|
||||
await writeFile(manifestPath, text);
|
||||
const entry = journal.plugin_files.find(
|
||||
(file: { path: string }) => file.path === manifestPath,
|
||||
);
|
||||
entry.hash = createHash("sha256").update(text).digest("hex");
|
||||
journal.installed_server_hash = createHash("sha256")
|
||||
.update(JSON.stringify(manifest.mcpServers["one-memory"]))
|
||||
.digest("hex");
|
||||
}
|
||||
await writeFile(layout.journalPath, JSON.stringify(journal));
|
||||
const granted = await context.permissions.getPermission(layout);
|
||||
expect(granted.state).toBe("approved");
|
||||
expect(
|
||||
(await readInstalledRecallCommand(layout))?.env[MCP_TOOLSET_KEY],
|
||||
).toBeUndefined();
|
||||
|
||||
expect(
|
||||
await context.permissions.refreshInstalledHosts([layout], {
|
||||
build: "34",
|
||||
}),
|
||||
).toMatchObject({ synced: ["workbuddy"], failed: [] });
|
||||
for (const manifestPath of manifests) {
|
||||
const manifest = JSON.parse(await readFile(manifestPath, "utf8"));
|
||||
expect(manifest.mcpServers["one-memory"].env[MCP_TOOLSET_KEY]).toBe(
|
||||
MCP_TOOLSET_VERSION,
|
||||
);
|
||||
}
|
||||
// Rewriting our own entry keeps the user's authorization.
|
||||
expect(await context.permissions.getPermission(layout)).toEqual(granted);
|
||||
});
|
||||
|
||||
it("rewrites a config-file MCP entry without the stamp", async () => {
|
||||
const context = await fixture();
|
||||
await install(context);
|
||||
const configPath = mcpConfigPath(context.layout);
|
||||
const config = JSON.parse(await readFile(configPath, "utf8"));
|
||||
delete config.mcpServers["one-memory"].env[MCP_TOOLSET_KEY];
|
||||
await writeFile(configPath, `${JSON.stringify(config, null, 2)}\n`);
|
||||
const journal = JSON.parse(
|
||||
await readFile(context.layout.journalPath, "utf8"),
|
||||
);
|
||||
journal.installed_server_hash = createHash("sha256")
|
||||
.update(JSON.stringify(config.mcpServers["one-memory"]))
|
||||
.digest("hex");
|
||||
await writeFile(context.layout.journalPath, JSON.stringify(journal));
|
||||
|
||||
expect(
|
||||
await context.permissions.refreshInstalledHosts([context.layout]),
|
||||
).toMatchObject({ state: "refreshed", synced: ["cursor"], failed: [] });
|
||||
const refreshed = JSON.parse(await readFile(configPath, "utf8"));
|
||||
expect(refreshed.mcpServers["one-memory"].env[MCP_TOOLSET_KEY]).toBe(
|
||||
MCP_TOOLSET_VERSION,
|
||||
);
|
||||
expect(refreshed.mcpServers.neighbor).toEqual({ command: "keep" });
|
||||
});
|
||||
});
|
||||
@@ -1369,6 +1369,24 @@ async function initialize(isQuitting: () => boolean): Promise<void> {
|
||||
});
|
||||
await runtimeSupervisor.start();
|
||||
if (isQuitting()) return;
|
||||
// A standalone Runtime refreshes the hosts it installed as it starts, so
|
||||
// an upgrade through the App or `memhub upgrade` is covered there. An
|
||||
// embedded one is this process: do the same here, now it is up. Only a
|
||||
// packaged build, so a source checkout never repoints real hosts at itself.
|
||||
if (localPermissions && app.isPackaged) {
|
||||
void localPermissions
|
||||
.refreshInstalledHosts(
|
||||
hosts.map((host) => host.layout),
|
||||
{
|
||||
build:
|
||||
packagedBuildNumber === undefined
|
||||
? undefined
|
||||
: String(packagedBuildNumber),
|
||||
diagnostics: hostDiagnostics,
|
||||
},
|
||||
)
|
||||
.catch(() => undefined);
|
||||
}
|
||||
if (
|
||||
app.isPackaged &&
|
||||
process.platform === "darwin" &&
|
||||
|
||||
@@ -50,7 +50,6 @@ const SERVICE_LABELS: Record<DebugLogEntry["service"], string> = {
|
||||
core: "Core",
|
||||
"host-bridge": "Host Bridge",
|
||||
cli: "CLI",
|
||||
"hy-memory": "HY Memory",
|
||||
};
|
||||
|
||||
export function DebugLogPanel() {
|
||||
|
||||
@@ -510,7 +510,7 @@ it("D96: the memory page has no raw-dialogue tab and never loads the dialogue li
|
||||
it("D96: lists a memory by when its conversation happened, not when it was generated", () => {
|
||||
const today = new Date().toISOString();
|
||||
const thisYear = new Date().getFullYear();
|
||||
const items = ["第一条", "第二条", "第三条"].map((content, i) => ({
|
||||
const items = ["第一条", "第二条", "第三条", "第四条"].map((content, i) => ({
|
||||
...memories[0],
|
||||
memory_id: `t${i}`,
|
||||
content,
|
||||
@@ -525,10 +525,11 @@ it("D96: lists a memory by when its conversation happened, not when it was gener
|
||||
total: 519,
|
||||
offset: 100,
|
||||
limit: 100,
|
||||
source_ids_by_memory: { t0: [], t1: [], t2: [] },
|
||||
source_ids_by_memory: { t0: [], t1: [], t2: [], t3: [] },
|
||||
source_time_by_memory: {
|
||||
t0: { at: `${thisYear}-10-02T10:54:00`, basis: "conversation" },
|
||||
t1: { at: `${thisYear}-09-30T08:00:00`, basis: "import" },
|
||||
t3: { at: `${thisYear}-09-29T08:00:00`, basis: "write" },
|
||||
},
|
||||
})}
|
||||
/>,
|
||||
@@ -540,6 +541,10 @@ it("D96: lists a memory by when its conversation happened, not when it was gener
|
||||
expect(
|
||||
within(list).getByRole("button", { name: /第二条/ }).textContent,
|
||||
).toContain("导入于 9月30日");
|
||||
// An experience stored without a conversation time names its write time.
|
||||
expect(
|
||||
within(list).getByRole("button", { name: /第四条/ }).textContent,
|
||||
).toContain("写入于 9月29日");
|
||||
// D99: the generated day keeps its own line, so a memory made today from an
|
||||
// old conversation still reads as new rather than looking misplaced.
|
||||
expect(
|
||||
|
||||
@@ -140,7 +140,7 @@ function monthDay(value: string, locale: GuiLocale): string | null {
|
||||
|
||||
/** D96: when the cited conversation happened, not when the memory was generated. */
|
||||
function sourceStamp(
|
||||
time: { at: string; basis: "conversation" | "import" } | undefined,
|
||||
time: { at: string; basis: "conversation" | "import" | "write" } | undefined,
|
||||
fallback: string | null | undefined,
|
||||
locale: GuiLocale = "zh-CN",
|
||||
): string {
|
||||
@@ -148,7 +148,11 @@ function sourceStamp(
|
||||
if (time && day)
|
||||
return memoryText(
|
||||
locale,
|
||||
time.basis === "conversation" ? "对话于 {day}" : "导入于 {day}",
|
||||
time.basis === "conversation"
|
||||
? "对话于 {day}"
|
||||
: time.basis === "write"
|
||||
? "写入于 {day}"
|
||||
: "导入于 {day}",
|
||||
{ day },
|
||||
);
|
||||
return dayStamp(fallback, locale);
|
||||
|
||||
@@ -176,3 +176,41 @@ it("stamps today's source and conversation time without an evidence page", () =>
|
||||
expect(screen.getByText(/对话时间 今天 /)).toBeTruthy();
|
||||
expect(screen.queryByRole("button", { name: "查看原文" })).toBeNull();
|
||||
});
|
||||
it("shows an experience's conversation time when the SDK stored one", () => {
|
||||
const today = new Date().toISOString();
|
||||
render(
|
||||
<SelectedMemoryDetail
|
||||
detail={{
|
||||
...detail,
|
||||
evidence: [],
|
||||
sdk_status: "active",
|
||||
source_id: "codebuddy",
|
||||
sdk_memory_at: today,
|
||||
sdk_created_at: today,
|
||||
sdk_ref: { scene: "coding", kind: "experience", variant: "coding" },
|
||||
}}
|
||||
sources={[]}
|
||||
/>,
|
||||
);
|
||||
expect(screen.getByText(/对话时间 今天 /)).toBeTruthy();
|
||||
expect(screen.queryByText(/时间未知/)).toBeNull();
|
||||
});
|
||||
it("labels an experience without a conversation time by its write time", () => {
|
||||
render(
|
||||
<SelectedMemoryDetail
|
||||
detail={{
|
||||
...detail,
|
||||
evidence: [],
|
||||
sdk_status: "active",
|
||||
source_id: "codebuddy",
|
||||
sdk_memory_at: null,
|
||||
sdk_created_at: new Date().toISOString(),
|
||||
sdk_ref: { scene: "working", kind: "experience", variant: "working" },
|
||||
}}
|
||||
sources={[]}
|
||||
/>,
|
||||
);
|
||||
expect(screen.getByText(/来源 codebuddy · 写入时间 今天 /)).toBeTruthy();
|
||||
expect(screen.queryByText(/对话时间/)).toBeNull();
|
||||
expect(screen.queryByText(/时间未知/)).toBeNull();
|
||||
});
|
||||
@@ -19,7 +19,10 @@ export function SelectedMemoryDetail({
|
||||
detail: MemoryDetail;
|
||||
sources: SourceResponse[];
|
||||
/** D99: when the cited conversation happened, in the list's own terms. */
|
||||
conversationAt?: { at: string; basis: "conversation" | "import" } | null;
|
||||
conversationAt?: {
|
||||
at: string;
|
||||
basis: "conversation" | "import" | "write";
|
||||
} | null;
|
||||
}) {
|
||||
const { locale, t } = useGuiLanguage();
|
||||
const memory = detail.memory;
|
||||
@@ -42,6 +45,17 @@ export function SelectedMemoryDetail({
|
||||
? t("selectedMemory.status.superseded")
|
||||
: t("selectedMemory.status.unknown");
|
||||
const sourceId = detail.source_id ?? evidence?.source_id;
|
||||
const created = clockStamp(
|
||||
detail.sdk_created_at ?? memory.updated_at,
|
||||
locale,
|
||||
);
|
||||
// An SDK experience or fact stored before hy-memory 2.0.7 has no
|
||||
// conversation time. It shows when it was written, named as such, instead
|
||||
// of "时间未知".
|
||||
const writtenOnly =
|
||||
detail.sdk_ref !== undefined &&
|
||||
detail.sdk_ref.kind !== "chat" &&
|
||||
!detail.sdk_memory_at;
|
||||
const source =
|
||||
detail.sdk_status && sourceId === UNATTRIBUTED_SOURCE_ID
|
||||
? t("selectedMemory.sourceUnattributed")
|
||||
@@ -77,25 +91,28 @@ export function SelectedMemoryDetail({
|
||||
<p className="selected-memory-source">
|
||||
{t("selectedMemory.sourceLine", {
|
||||
source,
|
||||
time: clockStamp(
|
||||
detail.sdk_memory_at ?? evidence?.created_at,
|
||||
locale,
|
||||
),
|
||||
time: writtenOnly
|
||||
? t("selectedMemory.writeTime", { created })
|
||||
: clockStamp(
|
||||
detail.sdk_memory_at ?? evidence?.created_at,
|
||||
locale,
|
||||
),
|
||||
})}
|
||||
</p>
|
||||
) : null}
|
||||
<p className="selected-memory-times">
|
||||
{t("selectedMemory.timestamps", {
|
||||
conversation: clockStamp(
|
||||
conversationAt?.at ?? detail.sdk_memory_at ?? evidence?.created_at,
|
||||
locale,
|
||||
),
|
||||
created: clockStamp(
|
||||
detail.sdk_created_at ?? memory.updated_at,
|
||||
locale,
|
||||
),
|
||||
})}
|
||||
</p>
|
||||
{writtenOnly ? null : (
|
||||
<p className="selected-memory-times">
|
||||
{t("selectedMemory.timestamps", {
|
||||
conversation: clockStamp(
|
||||
conversationAt?.at ??
|
||||
detail.sdk_memory_at ??
|
||||
evidence?.created_at,
|
||||
locale,
|
||||
),
|
||||
created,
|
||||
})}
|
||||
</p>
|
||||
)}
|
||||
</article>
|
||||
);
|
||||
}
|
||||
@@ -4,14 +4,7 @@ export interface DebugLogEntry {
|
||||
id: string;
|
||||
timestamp: string;
|
||||
level: DebugLogLevel;
|
||||
service:
|
||||
| "runtime"
|
||||
| "core"
|
||||
| "host-bridge"
|
||||
| "desktop"
|
||||
| "cli"
|
||||
| "hy-memory"
|
||||
| "renderer";
|
||||
service: "runtime" | "core" | "host-bridge" | "desktop" | "cli" | "renderer";
|
||||
component: string;
|
||||
event: string;
|
||||
outcome: string;
|
||||
@@ -21,7 +14,7 @@ export interface DebugLogEntry {
|
||||
trace_id?: string;
|
||||
details?: Record<string, string | number | boolean | null>;
|
||||
message?: string;
|
||||
source: "diagnostic" | "stdout" | "stderr" | "console" | "pipeline";
|
||||
source: "diagnostic" | "stdout" | "stderr" | "console";
|
||||
}
|
||||
|
||||
export interface DebugLogSnapshot {
|
||||
|
||||
@@ -119,8 +119,12 @@ it("carries each memory's scene and namespace, and edits an experience's solutio
|
||||
});
|
||||
expect(page.window_id).toBe("w1");
|
||||
expect(page.window_reset).toBe(true);
|
||||
// An experience has no conversation time to show as "imported".
|
||||
expect(page.source_time_by_memory?.e).toBeUndefined();
|
||||
// Stored without a conversation time: it shows when it was written, not
|
||||
// an "import".
|
||||
expect(page.source_time_by_memory?.e).toEqual({
|
||||
at: new Date((NOW - 86_400) * 1_000).toISOString(),
|
||||
basis: "write",
|
||||
});
|
||||
expect(page.items[1]).toMatchObject({
|
||||
kind: "working",
|
||||
updated_at: new Date((NOW - 3_600) * 1_000).toISOString(),
|
||||
@@ -140,3 +144,29 @@ it("carries each memory's scene and namespace, and edits an experience's solutio
|
||||
expect(sdkEditableText(experience)).toBe("Pin seed");
|
||||
expect(sdkEditableText(record({}))).toBe("memory");
|
||||
});
|
||||
|
||||
it("shows an experience's conversation time when the SDK stored one", () => {
|
||||
const at = NOW - 2 * 86_400;
|
||||
const experience = record({
|
||||
memory_id: "e",
|
||||
layer: "experience",
|
||||
scene: "coding",
|
||||
kind: "experience",
|
||||
variant: "coding",
|
||||
memory_at: at,
|
||||
created_at: NOW - 3_600,
|
||||
});
|
||||
const page = sdkLibraryPage({
|
||||
memories: [experience],
|
||||
total: 1,
|
||||
limit: 20,
|
||||
offset: 0,
|
||||
});
|
||||
expect(page.source_time_by_memory?.e).toEqual({
|
||||
at: new Date(at * 1_000).toISOString(),
|
||||
basis: "conversation",
|
||||
});
|
||||
expect(sdkDetail(experience).sdk_memory_at).toBe(
|
||||
new Date(at * 1_000).toISOString(),
|
||||
);
|
||||
});
|
||||
@@ -154,22 +154,24 @@ export function sdkDetail(record: SdkMemoryRecord): MemoryDetail {
|
||||
export function sdkLibraryPage(page: SdkMemoryPage): MemoryLibraryPage {
|
||||
const sourceTime: Record<
|
||||
string,
|
||||
{ at: string; basis: "conversation" | "import" }
|
||||
{ at: string; basis: "conversation" | "import" | "write" }
|
||||
> = {};
|
||||
const sourceIds: Record<string, string[]> = {};
|
||||
const refs: Record<string, SdkMemoryRef> = {};
|
||||
for (const record of page.memories) {
|
||||
sourceIds[record.memory_id] = [sourceOf(record)];
|
||||
refs[record.memory_id] = sdkMemoryRef(record);
|
||||
// An experience or fact has no conversation time; the list shows when it
|
||||
// was last updated instead of calling its creation an import.
|
||||
if (refs[record.memory_id].kind !== "chat") continue;
|
||||
// An experience or fact stored before hy-memory 2.0.7 has no
|
||||
// conversation time. It shows when it was written, labelled as such,
|
||||
// rather than calling its creation an import.
|
||||
const chat = refs[record.memory_id].kind === "chat";
|
||||
const conversation = plausible(record.memory_at);
|
||||
const at = conversation ?? plausible(record.gmt_created);
|
||||
const at =
|
||||
conversation ?? plausible(chat ? record.gmt_created : record.created_at);
|
||||
if (at === null) continue;
|
||||
sourceTime[record.memory_id] = {
|
||||
at: new Date(at * 1_000).toISOString(),
|
||||
basis: conversation !== null ? "conversation" : "import",
|
||||
basis: conversation !== null ? "conversation" : chat ? "import" : "write",
|
||||
};
|
||||
}
|
||||
return {
|
||||
|
||||
@@ -282,7 +282,7 @@ Support is declared per immutable GitHub Release manifest:
|
||||
- Linux x64/arm64 Runtime on native GHA runners.
|
||||
- macOS arm64/x64 App/Runtime: Developer ID, notarization, native acceptance.
|
||||
- Windows is not supported.
|
||||
- `hy-memory==2.0.6` is the current stable SDK dependency; the release
|
||||
- `hy-memory==2.0.8` is the current stable SDK dependency; the release
|
||||
lock and [`../MEMORY.md`](../MEMORY.md) are authoritative when the pin changes.
|
||||
|
||||
See [`INSTALLATION.md`](INSTALLATION.md).
|
||||
|
||||
@@ -7,12 +7,12 @@
|
||||
3. 如果停住,停在导入、模型调用还是最终写入。
|
||||
|
||||
默认 `info` 下的官方结构化事件不应写入会话正文、模型输入输出或 API Key;但本地原始日志允许
|
||||
新增受限标量标签,仍应按私人数据处理,不能直接对外发送。`debug` 会额外启用 HY pipeline
|
||||
trace,其中可能包含完整 prompt/response 和获授权的对话正文。
|
||||
新增受限标量标签,仍应按私人数据处理,不能直接对外发送。`debug` 会让 HY 的
|
||||
`hy_memory.log` 记录 DEBUG 行,其中可能包含完整 prompt/response 和获授权的对话正文。
|
||||
|
||||
排查时只通过 App 的“导出日志”提供文件:导出过程会按安全白名单重新生成诊断事件,不会打包
|
||||
原始日志。不要直接发送 `diagnostic-*.jsonl`、整个 `~/.one-memory` 数据目录或 debug pipeline
|
||||
日志;后者如确需分享,必须由开发人员另行确认并脱敏。
|
||||
原始日志。不要直接发送 `diagnostic-*.jsonl`、整个 `~/.one-memory` 数据目录或 debug 级别的
|
||||
`hy_memory.log`;后者如确需分享,必须由开发人员另行确认并脱敏。
|
||||
|
||||
## 1. Log level 设什么
|
||||
|
||||
@@ -181,8 +181,8 @@ memhub logs --since 24h --errors-only --tail 500 --json
|
||||
memhub status --json
|
||||
```
|
||||
|
||||
以下 `jq` 命令仅用于 `diagnostic-core.jsonl`,不要对 `core.log`、`service.log` 或 debug
|
||||
pipeline 日志直接运行后把完整输出发给他人。
|
||||
以下 `jq` 命令仅用于 `diagnostic-core.jsonl`,不要对 `core.log`、`service.log` 或
|
||||
`hy_memory.log` 直接运行后把完整输出发给他人。
|
||||
|
||||
### 查看最近的失败
|
||||
|
||||
|
||||
@@ -39,10 +39,12 @@ which talks to a broker on the same machine.
|
||||
| `ONE_MEMORY_BROKER_DISCOVERY` | absolute path to the discovery file | How the server finds the running app. Wrong value means recall returns nothing, with no error. |
|
||||
| `ONE_MEMORY_MANAGED_BY` | `one-memory` | Marks the entry as ours, so our installer will update or remove it later instead of treating it as a stranger's and leaving it alone. |
|
||||
| `ELECTRON_RUN_AS_NODE` | `1` | The MCP entry ships inside the app bundle and is started through the app's own binary. Without this the binary opens the window instead of running the script. |
|
||||
| `ONE_MEMORY_MCP_TOOLSET` | tool set version, e.g. `3` | Names the MemHub tool set the entry was written for. MemHub changes it only when tools are added, removed or renamed, so hosts that cache a server's tool list by its launch config restart the server and drop removed tools. The server itself ignores it. |
|
||||
|
||||
`ONE_MEMORY_MANAGED_BY` is optional for a hand-made entry. Set it if you want
|
||||
MemHub to manage the entry for you afterwards; leave it out if you would
|
||||
rather own the entry yourself.
|
||||
rather own the entry yourself. `ONE_MEMORY_MCP_TOOLSET` is optional too; a
|
||||
managed entry without the current value is rewritten on the next upgrade.
|
||||
|
||||
## 3. MCP hosts
|
||||
|
||||
@@ -57,7 +59,8 @@ Register a stdio server named `one-memory`. The product name is MemHub; the conf
|
||||
"env": {
|
||||
"ELECTRON_RUN_AS_NODE": "1",
|
||||
"ONE_MEMORY_BROKER_DISCOVERY": "<absolute path to the discovery file>",
|
||||
"ONE_MEMORY_MANAGED_BY": "one-memory"
|
||||
"ONE_MEMORY_MANAGED_BY": "one-memory",
|
||||
"ONE_MEMORY_MCP_TOOLSET": "3"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
+1
-1
@@ -2,7 +2,7 @@
|
||||
"name": "one-memory",
|
||||
"version": "1.0.5",
|
||||
"release": {
|
||||
"build_number": 33,
|
||||
"build_number": 35,
|
||||
"macos_min_version": "13.0",
|
||||
"app_update": "manual",
|
||||
"runtime_manifest_schema": 1,
|
||||
|
||||
@@ -468,10 +468,12 @@ export interface MemoryLibraryPage {
|
||||
limit: number;
|
||||
/** Active evidence sources for each returned memory, from the same authorized page snapshot. */
|
||||
source_ids_by_memory?: Record<string, string[]>;
|
||||
/** When each returned memory's cited conversation happened (D96). `import` when only the import time is known. */
|
||||
/** When each returned memory's cited conversation happened (D96). `import`
|
||||
* when only the import time is known; `write` for an SDK experience or fact
|
||||
* stored without a conversation time, which shows when it was written. */
|
||||
source_time_by_memory?: Record<
|
||||
string,
|
||||
{ at: string; basis: "conversation" | "import" }
|
||||
{ at: string; basis: "conversation" | "import" | "write" }
|
||||
>;
|
||||
/** True when the read budget ran out mid-page (typically during a history
|
||||
* import): a partial snapshot, not an authoritative view. Show "importing"
|
||||
|
||||
@@ -5,6 +5,8 @@ import { describe, expect, it } from "vitest";
|
||||
import type { RecallRuntime } from "./recall-installer.js";
|
||||
import {
|
||||
DISCOVERY_KEY,
|
||||
MCP_TOOLSET_KEY,
|
||||
MCP_TOOLSET_VERSION,
|
||||
OWNER_KEY,
|
||||
OWNER_VALUE,
|
||||
renderConnectionMaterial,
|
||||
@@ -60,6 +62,7 @@ describe("mcp install snippet", () => {
|
||||
const env = parsed.mcpServers[SERVER_NAME].env;
|
||||
expect(env[DISCOVERY_KEY]).toBe(runtime().discoveryPath);
|
||||
expect(env[OWNER_KEY]).toBe(OWNER_VALUE);
|
||||
expect(env[MCP_TOOLSET_KEY]).toBe(MCP_TOOLSET_VERSION);
|
||||
expect(env.ELECTRON_RUN_AS_NODE).toBe("1");
|
||||
});
|
||||
|
||||
@@ -174,9 +177,10 @@ describe("public contract document", () => {
|
||||
// reports it to us. Same class of problem R-33 treated for skill text,
|
||||
// and treated the same way: one source, checked by a test.
|
||||
const doc = await fs.readFile(CONTRACT_DOC, "utf8");
|
||||
for (const key of [DISCOVERY_KEY, OWNER_KEY]) {
|
||||
for (const key of [DISCOVERY_KEY, OWNER_KEY, MCP_TOOLSET_KEY]) {
|
||||
expect(doc).toContain(key);
|
||||
}
|
||||
expect(doc).toContain(`"${MCP_TOOLSET_KEY}": "${MCP_TOOLSET_VERSION}"`);
|
||||
expect(doc).toContain(OWNER_VALUE);
|
||||
expect(doc).toContain(SERVER_NAME);
|
||||
});
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
|
||||
import { type GrantedTools, McpSession } from "./mcp-session.js";
|
||||
import { MCP_TOOL_NAMES, MCP_TOOLSET_VERSION } from "./mcp-toolset.js";
|
||||
|
||||
function sessionWith(granted: GrantedTools): McpSession {
|
||||
return new McpSession(
|
||||
@@ -71,3 +72,14 @@ describe("BUG-15: the listed tools follow what the grant allows", () => {
|
||||
expect(names).toHaveLength(3);
|
||||
});
|
||||
});
|
||||
|
||||
describe("the tool set stamped on installed entries", () => {
|
||||
it("names exactly the tools the server lists", async () => {
|
||||
// Change a tool without moving MCP_TOOLSET_VERSION and hosts that cache
|
||||
// tool lists by config fingerprint keep showing the old set. Update both,
|
||||
// and this pinned version, together.
|
||||
const names = await listedNames(sessionWith(async () => null));
|
||||
expect(names).toEqual([...MCP_TOOL_NAMES]);
|
||||
expect(MCP_TOOLSET_VERSION).toBe("3");
|
||||
});
|
||||
});
|
||||
@@ -127,6 +127,8 @@ export class McpSession {
|
||||
}
|
||||
|
||||
private async listedTools() {
|
||||
// Any change to this set moves MCP_TOOL_NAMES and MCP_TOOLSET_VERSION in
|
||||
// `mcp-toolset.ts`, so installed entries change and hosts relist.
|
||||
const definitions = [
|
||||
recallToolDefinition(),
|
||||
writeToolDefinition(),
|
||||
|
||||
@@ -0,0 +1,29 @@
|
||||
/** The tool contract of MemHub's MCP server, and the stamp that names it.
|
||||
*
|
||||
* Kept apart from `mcp-session.ts` so the installer can stamp an entry without
|
||||
* pulling in the broker client, and listed here rather than inferred so a test
|
||||
* can hold `tools/list` to it: change a tool there and that test fails until
|
||||
* this list, and with it the version, moves too.
|
||||
*/
|
||||
|
||||
/** The tools `tools/list` returns, in order. */
|
||||
export const MCP_TOOL_NAMES = [
|
||||
"MemHub_recall",
|
||||
"MemHub_write",
|
||||
"MemHub_diagnostic_trace",
|
||||
] as const;
|
||||
|
||||
/** Bumped whenever a tool is added, removed or renamed.
|
||||
*
|
||||
* Written into every managed MCP entry under `MCP_TOOLSET_KEY`. Hosts such as
|
||||
* WorkBuddy cache a server's tool list under a fingerprint of its launch
|
||||
* config, and our entry is otherwise byte-identical across upgrades, since it
|
||||
* runs through the `current` symlink. Moving this value is what changes that
|
||||
* fingerprint, so those hosts restart the server and drop the tools it no
|
||||
* longer has. Deliberately not the build number: that would rewrite every
|
||||
* host's config on every upgrade, for nothing.
|
||||
*/
|
||||
export const MCP_TOOLSET_VERSION = "3";
|
||||
|
||||
/** The env key the version is written under. */
|
||||
export const MCP_TOOLSET_KEY = "ONE_MEMORY_MCP_TOOLSET";
|
||||
@@ -5,6 +5,7 @@ import { describe, expect, it } from "vitest";
|
||||
|
||||
import {
|
||||
applyRecallInstall,
|
||||
MCP_TOOLSET_VERSION,
|
||||
mcpConfigPath,
|
||||
opencodeLayout,
|
||||
planRecallInstall,
|
||||
@@ -46,6 +47,7 @@ describe("OpenCode recall installer", () => {
|
||||
ELECTRON_RUN_AS_NODE: "1",
|
||||
ONE_MEMORY_MANAGED_BY: "one-memory",
|
||||
ONE_MEMORY_BROKER_DISCOVERY: "/opt/one-memory/runtime.json",
|
||||
ONE_MEMORY_MCP_TOOLSET: MCP_TOOLSET_VERSION,
|
||||
},
|
||||
enabled: true,
|
||||
});
|
||||
|
||||
@@ -58,6 +58,7 @@ import {
|
||||
readDshServer,
|
||||
removeDshServer,
|
||||
} from "./dsh-patch.js";
|
||||
import { MCP_TOOLSET_KEY, MCP_TOOLSET_VERSION } from "./mcp-toolset.js";
|
||||
import {
|
||||
RECALL_PRESENTATION_GUIDANCE,
|
||||
RECALL_PRESENTATION_GUIDANCE_ZH,
|
||||
@@ -88,8 +89,12 @@ export {
|
||||
type RecallLinkState,
|
||||
type RecallProbe,
|
||||
} from "./recall-probe.js";
|
||||
|
||||
export { planRecallRepoint, type RecallRepoint };
|
||||
export {
|
||||
MCP_TOOLSET_KEY,
|
||||
MCP_TOOLSET_VERSION,
|
||||
planRecallRepoint,
|
||||
type RecallRepoint,
|
||||
};
|
||||
|
||||
/** Rewrite hosts whose installed command is not the one we would write now.
|
||||
*
|
||||
@@ -2844,6 +2849,9 @@ function managedServer(runtime: RecallRuntime): ManagedServer {
|
||||
ELECTRON_RUN_AS_NODE: "1",
|
||||
[DISCOVERY_KEY]: runtime.discoveryPath,
|
||||
[OWNER_KEY]: OWNER_VALUE,
|
||||
// Moves the host's fingerprint of this entry when the tool set changes;
|
||||
// see `MCP_TOOLSET_VERSION`. An entry without it reads as stale.
|
||||
[MCP_TOOLSET_KEY]: MCP_TOOLSET_VERSION,
|
||||
...(runtime.credentialFile === undefined
|
||||
? {}
|
||||
: { ONE_MEMORY_HOST_CREDENTIAL_FILE: runtime.credentialFile }),
|
||||
|
||||
@@ -5,6 +5,7 @@ import path from "node:path";
|
||||
import { describe, expect, it } from "vitest";
|
||||
|
||||
import { applyCursorInstall, planCursorInstall } from "./cursor-installer.js";
|
||||
import { MCP_TOOLSET_KEY, MCP_TOOLSET_VERSION } from "./mcp-toolset.js";
|
||||
import {
|
||||
applyRecallInstall,
|
||||
applyRecallUninstall,
|
||||
@@ -15,6 +16,7 @@ import {
|
||||
planRecallUninstall,
|
||||
SKILL_MARKER,
|
||||
SKILL_VERSION,
|
||||
verifyRecallInstall,
|
||||
} from "./recall-installer.js";
|
||||
import {
|
||||
checkSkillDrift,
|
||||
@@ -387,6 +389,59 @@ describe("skill drift (R-33)", () => {
|
||||
});
|
||||
}, 30_000);
|
||||
|
||||
it("marks stale an MCP entry written before the tool set was stamped, and sync stamps it", async () => {
|
||||
// Our entry is otherwise identical across upgrades, so hosts that cache a
|
||||
// tool list by config fingerprint keep the removed tools until it changes.
|
||||
const cursorDir = await fixtureDir();
|
||||
const discovery = path.join(cursorDir, "broker.json");
|
||||
await writeFile(discovery, "{}\n");
|
||||
await applyCursorInstall(
|
||||
await planCursorInstall(options(cursorDir, discovery)),
|
||||
);
|
||||
const layout = cursorLayout(cursorDir);
|
||||
const configPath = layout.configPath as string;
|
||||
const config = JSON.parse(await readFile(configPath, "utf8")) as {
|
||||
mcpServers: Record<string, { env: Record<string, string> }>;
|
||||
};
|
||||
expect(config.mcpServers["one-memory"].env[MCP_TOOLSET_KEY]).toBe(
|
||||
MCP_TOOLSET_VERSION,
|
||||
);
|
||||
// What an earlier release wrote and journaled: the same entry, no stamp.
|
||||
delete config.mcpServers["one-memory"].env[MCP_TOOLSET_KEY];
|
||||
await writeFile(configPath, `${JSON.stringify(config, null, 2)}\n`);
|
||||
const journal = JSON.parse(await readFile(layout.journalPath, "utf8"));
|
||||
journal.installed_server_hash = createHash("sha256")
|
||||
.update(JSON.stringify(config.mcpServers["one-memory"]))
|
||||
.digest("hex");
|
||||
await writeFile(layout.journalPath, `${JSON.stringify(journal)}\n`);
|
||||
|
||||
// Still intact as installed, so the link is not reported broken...
|
||||
await expect(verifyRecallInstall(layout)).resolves.toMatchObject({
|
||||
hostId: "cursor",
|
||||
});
|
||||
// ...but stale against this release, and safe to sync without asking.
|
||||
await expect(
|
||||
checkSkillDrift(layout, runtime(discovery)),
|
||||
).resolves.toMatchObject({
|
||||
state: "stale",
|
||||
detail: "the MCP entry predates the tool set this MemHub serves",
|
||||
});
|
||||
const sync = await planSkillSync(layout, runtime(discovery));
|
||||
expect(sync?.requiresConfirmation).toBe(false);
|
||||
if (sync === null) throw new Error("expected a stale entry sync");
|
||||
await applyRecallInstall(sync.plan);
|
||||
|
||||
const synced = JSON.parse(await readFile(configPath, "utf8")) as {
|
||||
mcpServers: Record<string, { env: Record<string, string> }>;
|
||||
};
|
||||
expect(synced.mcpServers["one-memory"].env[MCP_TOOLSET_KEY]).toBe(
|
||||
MCP_TOOLSET_VERSION,
|
||||
);
|
||||
await expect(
|
||||
checkSkillDrift(layout, runtime(discovery)),
|
||||
).resolves.toMatchObject({ state: "in_sync" });
|
||||
}, 30_000);
|
||||
|
||||
it("requires recall for dependent answers, not on every unrelated turn", async () => {
|
||||
const cursorDir = await fixtureDir();
|
||||
const discovery = path.join(cursorDir, "broker.json");
|
||||
|
||||
@@ -2,6 +2,7 @@ import { createHash } from "node:crypto";
|
||||
import path from "node:path";
|
||||
import { captureFilePath, namesCapture } from "./capture-hooks.js";
|
||||
import { PLUGIN_HOOKS } from "./codebuddy-plugin.js";
|
||||
import { MCP_TOOLSET_KEY, MCP_TOOLSET_VERSION } from "./mcp-toolset.js";
|
||||
import {
|
||||
asInstalled,
|
||||
expectedPluginHooks,
|
||||
@@ -9,6 +10,7 @@ import {
|
||||
type RecallHostLayout,
|
||||
type RecallInstallPlan,
|
||||
type RecallRuntime,
|
||||
readInstalledRecallCommand,
|
||||
renderSkillContent,
|
||||
SKILL_VERSION,
|
||||
} from "./recall-installer.js";
|
||||
@@ -99,7 +101,23 @@ export async function checkSkillDrift(
|
||||
runtime: RecallRuntime,
|
||||
): Promise<SkillDrift | null> {
|
||||
const skill = await checkSkillFileDrift(layout, runtime);
|
||||
if (skill === null || skill.state === "modified") return skill;
|
||||
if (skill === null) {
|
||||
// A host with no skill mechanism still has an entry whose tool set can
|
||||
// age. One whose skill is merely missing is left alone: syncing would put
|
||||
// back a file somebody removed.
|
||||
if (skillFilePath(layout) !== null) return null;
|
||||
return (await serverToolsetDrift(layout)) === "stale"
|
||||
? {
|
||||
hostId: layout.hostId,
|
||||
state: "stale",
|
||||
installedVersion: null,
|
||||
currentVersion: SKILL_VERSION,
|
||||
skillPath: null,
|
||||
detail: TOOLSET_STALE_DETAIL,
|
||||
}
|
||||
: null;
|
||||
}
|
||||
if (skill.state === "modified") return skill;
|
||||
const hooks = await pluginHooksDrift(layout, runtime);
|
||||
if (hooks === "modified") {
|
||||
return {
|
||||
@@ -108,15 +126,19 @@ export async function checkSkillDrift(
|
||||
detail: "the plugin hook file was edited after install",
|
||||
};
|
||||
}
|
||||
if (hooks === "stale" && skill.state === "in_sync") {
|
||||
if (skill.state === "stale") return skill;
|
||||
if (hooks === "stale") {
|
||||
return {
|
||||
...skill,
|
||||
state: "stale",
|
||||
detail: "plugin hooks are still ours, but have moved since install",
|
||||
};
|
||||
}
|
||||
if ((await serverToolsetDrift(layout)) === "stale") {
|
||||
return { ...skill, state: "stale", detail: TOOLSET_STALE_DETAIL };
|
||||
}
|
||||
const capture = await captureHooksDrift(layout);
|
||||
if (capture === "stale" && skill.state === "in_sync") {
|
||||
if (capture === "stale") {
|
||||
return {
|
||||
...skill,
|
||||
state: "stale",
|
||||
@@ -126,6 +148,29 @@ export async function checkSkillDrift(
|
||||
return skill;
|
||||
}
|
||||
|
||||
const TOOLSET_STALE_DETAIL =
|
||||
"the MCP entry predates the tool set this MemHub serves";
|
||||
|
||||
/** Whether our MCP entry names the tool set this release serves.
|
||||
*
|
||||
* The entry is otherwise identical across upgrades, so a host that caches a
|
||||
* server's tool list under a fingerprint of its config (WorkBuddy does) keeps
|
||||
* listing tools that are gone until the entry itself changes. Only an entry
|
||||
* marked as ours is read; a CLI-channel host has no entry at all.
|
||||
*/
|
||||
async function serverToolsetDrift(
|
||||
layout: RecallHostLayout,
|
||||
): Promise<"in_sync" | "stale" | null> {
|
||||
if (layout.channel !== "mcp") return null;
|
||||
// A config the host cannot parse either is not ours to judge here; the
|
||||
// install and link checks report it.
|
||||
const installed = await readInstalledRecallCommand(layout).catch(() => null);
|
||||
if (installed === null) return null;
|
||||
return installed.env[MCP_TOOLSET_KEY] === MCP_TOOLSET_VERSION
|
||||
? "in_sync"
|
||||
: "stale";
|
||||
}
|
||||
|
||||
/** Detect duplicated command hooks left by earlier installs.
|
||||
*
|
||||
* `checkSkillDrift` historically compared only the Skill and plugin hooks;
|
||||
|
||||
@@ -108,6 +108,7 @@ export const currentChineseMessages = {
|
||||
"selectedMemory.version": "第 {revision} 版",
|
||||
"selectedMemory.sourceLine": "来源 {source} · {time}",
|
||||
"selectedMemory.timestamps": "对话时间 {conversation} · 沉淀时间 {created}",
|
||||
"selectedMemory.writeTime": "写入时间 {created}",
|
||||
"selectedMemory.edit": "编辑",
|
||||
"selectedMemory.evidence": "原始依据",
|
||||
"selectedMemory.evidenceLine": "{author} · 原对话 {time}:",
|
||||
@@ -258,6 +259,7 @@ export const currentEnglishMessages: Record<
|
||||
"selectedMemory.sourceLine": "Source {source} · {time}",
|
||||
"selectedMemory.timestamps":
|
||||
"Conversation time {conversation} · Created {created}",
|
||||
"selectedMemory.writeTime": "Written {created}",
|
||||
"selectedMemory.edit": "Edit",
|
||||
"selectedMemory.evidence": "Original evidence",
|
||||
"selectedMemory.evidenceLine": "{author} · Original conversation {time}:",
|
||||
@@ -418,6 +420,7 @@ export const currentSpanishMessages: Record<
|
||||
"selectedMemory.sourceLine": "Origen {source} · {time}",
|
||||
"selectedMemory.timestamps":
|
||||
"Conversación {conversation} · Guardada {created}",
|
||||
"selectedMemory.writeTime": "Escrita {created}",
|
||||
"selectedMemory.edit": "Editar",
|
||||
"selectedMemory.evidence": "Evidencia original",
|
||||
"selectedMemory.evidenceLine": "{author} · Conversación original {time}:",
|
||||
|
||||
@@ -1774,6 +1774,7 @@ const baseEnglishMessages = {
|
||||
" · Import count is still updating; refresh to check",
|
||||
"memory:对话于 {day}": "Conversation on {day}",
|
||||
"memory:导入于 {day}": "Imported on {day}",
|
||||
"memory:写入于 {day}": "Written on {day}",
|
||||
"memory:已开始处理所选对话": "Processing selected conversations",
|
||||
"memory:处理请求已受理": "Processing request received",
|
||||
"memory:模型连接已通过。首批记忆生成后,会逐步出现在记忆列表里。":
|
||||
|
||||
@@ -336,6 +336,7 @@ export const memoryChineseMessages = {
|
||||
"memory: · 导入数量仍在更新,可刷新核对": " · 导入数量仍在更新,可刷新核对",
|
||||
"memory:对话于 {day}": "对话于 {day}",
|
||||
"memory:导入于 {day}": "导入于 {day}",
|
||||
"memory:写入于 {day}": "写入于 {day}",
|
||||
"memory:已开始处理所选对话": "已开始处理所选对话",
|
||||
"memory:处理请求已受理": "处理请求已受理",
|
||||
"memory:模型连接已通过。首批记忆生成后,会逐步出现在记忆列表里。":
|
||||
@@ -750,6 +751,7 @@ export const memorySpanishMessages: Record<
|
||||
" · La importación continúa; actualiza para verificar",
|
||||
"memory:对话于 {day}": "Conversación del {day}",
|
||||
"memory:导入于 {day}": "Importada el {day}",
|
||||
"memory:写入于 {day}": "Escrita el {day}",
|
||||
"memory:已开始处理所选对话": "Procesamiento de conversaciones iniciado",
|
||||
"memory:处理请求已受理": "Solicitud de procesamiento aceptada",
|
||||
"memory:模型连接已通过。首批记忆生成后,会逐步出现在记忆列表里。":
|
||||
|
||||
@@ -41,6 +41,7 @@ import {
|
||||
repairInstalledRecallCommands,
|
||||
verifyRecallInstall,
|
||||
} from "@one-memory/host-bridge/installer";
|
||||
import type { DiagnosticInput, DiagnosticSink } from "./diagnostic-log.js";
|
||||
import {
|
||||
hasSyncHookArtifacts,
|
||||
loadSyncHookAssets,
|
||||
@@ -96,8 +97,22 @@ interface InstallationBinding {
|
||||
}
|
||||
|
||||
const CREDENTIAL_ENV = "ONE_MEMORY_HOST_CREDENTIAL_FILE";
|
||||
/** The build whose host refresh last completed, beside the grants. */
|
||||
const HOST_REFRESH_FILE = "host-refresh.json";
|
||||
const locks = new Map<string, Promise<void>>();
|
||||
|
||||
/** What one pass of `refreshInstalledHosts` did, by host id. */
|
||||
export interface HostRefreshReport {
|
||||
/** `already_refreshed` when this build had already completed a pass. */
|
||||
state: "refreshed" | "already_refreshed";
|
||||
checked: number;
|
||||
/** Hosts whose installed command or discovery path was moved. */
|
||||
repointed: string[];
|
||||
/** Hosts whose stale skill, hooks, plugin files or MCP entry were rewritten. */
|
||||
synced: string[];
|
||||
failed: string[];
|
||||
}
|
||||
|
||||
export const LOCAL_EXPERIENCE_UNAVAILABLE = "体验版暂不支持";
|
||||
|
||||
/** Trusted local Runtime control plane. Never exposed through MCP or the Core API.
|
||||
@@ -201,11 +216,20 @@ export class HostPermissions {
|
||||
* touching authorization. A skill the user edited is left for the detail page.
|
||||
*/
|
||||
async syncStaleInstalls(layouts: RecallHostLayout[]): Promise<string[]> {
|
||||
return (await this.syncStale(layouts)).synced;
|
||||
}
|
||||
|
||||
private async syncStale(
|
||||
layouts: RecallHostLayout[],
|
||||
): Promise<{ synced: string[]; failed: string[] }> {
|
||||
const synced: string[] = [];
|
||||
const failed: string[] = [];
|
||||
for (const layout of layouts) {
|
||||
try {
|
||||
await this.withHostLock(layout, async () => {
|
||||
await this.requireLinked(layout);
|
||||
// Not linked from this data directory is not ours to sync, and not
|
||||
// a failure either.
|
||||
if (!(await this.isLinked(layout))) return;
|
||||
const sync = await planSkillSync(layout, this.runtimeFor(layout));
|
||||
if (sync === null || sync.requiresConfirmation) return;
|
||||
const preview = await previewRecallInstall(sync.plan);
|
||||
@@ -215,9 +239,128 @@ export class HostPermissions {
|
||||
});
|
||||
} catch {
|
||||
// One host that cannot be synced must not stop the runtime starting.
|
||||
failed.push(layout.hostId);
|
||||
}
|
||||
}
|
||||
return synced;
|
||||
return { synced, failed };
|
||||
}
|
||||
|
||||
/** Refresh every host this directory installed, once per build.
|
||||
*
|
||||
* An upgrade changes the call rules, hooks and MCP entry we would write, but
|
||||
* nothing rewrote the copies hosts already had, so they kept telling the
|
||||
* model to call tools that were gone. This runs the same repoint and stale
|
||||
* sync the detail page offers, without asking, for files that are still
|
||||
* exactly ours; an edited skill stays for the user to confirm.
|
||||
*
|
||||
* The build it last completed for is recorded beside the grants, so later
|
||||
* launches of the same build cost one file read. A run in which any host
|
||||
* failed is not recorded, so the next launch tries again. With no build to
|
||||
* name (a source checkout) it runs every time and records nothing.
|
||||
*/
|
||||
async refreshInstalledHosts(
|
||||
layouts: RecallHostLayout[],
|
||||
options: { build?: string; diagnostics?: DiagnosticSink } = {},
|
||||
): Promise<HostRefreshReport> {
|
||||
const log = (input: DiagnosticInput) =>
|
||||
options.diagnostics?.log(input).catch(() => undefined);
|
||||
const build = options.build?.trim() || undefined;
|
||||
if (build !== undefined && (await this.readRefreshedBuild()) === build) {
|
||||
await log({
|
||||
level: "info",
|
||||
component: "host.refresh",
|
||||
event: "host_refresh.skipped",
|
||||
outcome: "skipped",
|
||||
details: { reason: "already_refreshed", count: layouts.length },
|
||||
});
|
||||
return {
|
||||
state: "already_refreshed",
|
||||
checked: layouts.length,
|
||||
repointed: [],
|
||||
synced: [],
|
||||
failed: [],
|
||||
};
|
||||
}
|
||||
const started = Date.now();
|
||||
const repointed: string[] = [];
|
||||
const failed: string[] = [];
|
||||
for (const layout of layouts) {
|
||||
try {
|
||||
repointed.push(...(await this.repairInstalledCommands([layout])));
|
||||
} catch {
|
||||
failed.push(layout.hostId);
|
||||
}
|
||||
}
|
||||
const stale = await this.syncStale(layouts);
|
||||
failed.push(...stale.failed.filter((hostId) => !failed.includes(hostId)));
|
||||
for (const hostId of new Set([...repointed, ...stale.synced])) {
|
||||
await log({
|
||||
level: "info",
|
||||
component: "host.refresh",
|
||||
event: "host_refresh.host_updated",
|
||||
outcome: "succeeded",
|
||||
details: { source_id: hostId },
|
||||
});
|
||||
}
|
||||
for (const hostId of failed) {
|
||||
await log({
|
||||
level: "warn",
|
||||
component: "host.refresh",
|
||||
event: "host_refresh.host_failed",
|
||||
outcome: "failed",
|
||||
error_code: "host_refresh_failed",
|
||||
details: { source_id: hostId },
|
||||
});
|
||||
}
|
||||
if (build !== undefined && failed.length === 0) {
|
||||
await this.writeRefreshedBuild(build).catch(() => undefined);
|
||||
}
|
||||
await log({
|
||||
level: failed.length === 0 ? "info" : "warn",
|
||||
component: "host.refresh",
|
||||
event: "host_refresh.completed",
|
||||
outcome: failed.length === 0 ? "succeeded" : "degraded",
|
||||
duration_ms: Date.now() - started,
|
||||
details: {
|
||||
count: layouts.length,
|
||||
updated_count: new Set([...repointed, ...stale.synced]).size,
|
||||
error_count: failed.length,
|
||||
},
|
||||
});
|
||||
return {
|
||||
state: "refreshed",
|
||||
checked: layouts.length,
|
||||
repointed,
|
||||
synced: stale.synced,
|
||||
failed,
|
||||
};
|
||||
}
|
||||
|
||||
private async readRefreshedBuild(): Promise<string | null> {
|
||||
try {
|
||||
return await storageGuard(async () => {
|
||||
const file = path.join(this.directory, HOST_REFRESH_FILE);
|
||||
if ((await optionalStat(file)) === null) return null;
|
||||
const value: unknown = JSON.parse(await privateRead(file));
|
||||
return isObject(value) &&
|
||||
value.schemaVersion === 1 &&
|
||||
typeof value.build === "string"
|
||||
? value.build
|
||||
: null;
|
||||
});
|
||||
} catch {
|
||||
// Unreadable reads as never refreshed: the refresh is idempotent.
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
private async writeRefreshedBuild(build: string): Promise<void> {
|
||||
await storageGuard(() =>
|
||||
privateWrite(
|
||||
path.join(this.directory, HOST_REFRESH_FILE),
|
||||
`${JSON.stringify({ schemaVersion: 1, build })}\n`,
|
||||
),
|
||||
);
|
||||
}
|
||||
|
||||
/** Same repair for one host. Does not revoke or reissue its authorization. */
|
||||
@@ -942,6 +1085,16 @@ export class HostPermissions {
|
||||
throw new Error("host_permission_installation_incomplete");
|
||||
}
|
||||
|
||||
/** `requireLinked` as a question, for sweeps that skip hosts not ours. */
|
||||
private async isLinked(layout: RecallHostLayout): Promise<boolean> {
|
||||
if (!supported(layout)) return false;
|
||||
const current = await this.inspect(layout, false);
|
||||
return (
|
||||
(current.state === "approved" || current.state === "install-only") &&
|
||||
current.reason === null
|
||||
);
|
||||
}
|
||||
|
||||
/** Turning off a host with nothing of ours on it is an empty removal, not a
|
||||
* failure; one whose state cannot be settled is an empty removal carrying
|
||||
* what to look at, which uninstall refuses. Only what is ours for certain
|
||||
|
||||
@@ -79,6 +79,7 @@ export {
|
||||
} from "./host-control-client.js";
|
||||
export {
|
||||
HostPermissions,
|
||||
type HostRefreshReport,
|
||||
LOCAL_EXPERIENCE_UNAVAILABLE,
|
||||
type RecallPermissionStatus,
|
||||
type RevokeRecallPermissionInput,
|
||||
|
||||
@@ -226,6 +226,7 @@ describe("host runtime attestation", () => {
|
||||
"repairInstalledCommand",
|
||||
"repairInstalledCommands",
|
||||
"syncStaleInstalls",
|
||||
"refreshInstalledHosts",
|
||||
"previewInstall",
|
||||
"install",
|
||||
"setPermission",
|
||||
|
||||
@@ -600,6 +600,7 @@ export function guardedPermissions(
|
||||
"previewSyncHook",
|
||||
"applySyncHook",
|
||||
"syncStaleInstalls",
|
||||
"refreshInstalledHosts",
|
||||
]);
|
||||
return new Proxy(owner, {
|
||||
get(target, property, receiver) {
|
||||
@@ -739,10 +740,17 @@ export async function createRuntimeService(
|
||||
).filter((host) => host.hostId !== "openclaw");
|
||||
await verifyHostRuntime().then(
|
||||
async () => {
|
||||
await permissions.repairInstalledCommands(
|
||||
// Every Runtime start after an upgrade, the App's or `memhub
|
||||
// upgrade`'s, comes through here first; once per build after that.
|
||||
await permissions.refreshInstalledHosts(
|
||||
hosts.map((host) => host.layout),
|
||||
{
|
||||
build:
|
||||
config.buildIdentity?.buildNumber ??
|
||||
config.buildIdentity?.version,
|
||||
diagnostics: hostDiagnostics,
|
||||
},
|
||||
);
|
||||
await permissions.syncStaleInstalls(hosts.map((host) => host.layout));
|
||||
},
|
||||
() => undefined,
|
||||
);
|
||||
|
||||
@@ -2,9 +2,11 @@ import {
|
||||
chmod,
|
||||
mkdir,
|
||||
mkdtemp,
|
||||
readdir,
|
||||
readFile,
|
||||
rm,
|
||||
stat,
|
||||
symlink,
|
||||
writeFile,
|
||||
} from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
@@ -38,6 +40,38 @@ function supervisor() {
|
||||
};
|
||||
}
|
||||
|
||||
const PREVIOUS_BOOT = "00000000-0000-4000-8000-00000000000a";
|
||||
const CURRENT_BOOT = "00000000-0000-4000-8000-00000000000b";
|
||||
|
||||
/** Seed a lock as an existing owner would leave it; no owner = legacy debris. */
|
||||
async function seedLock(dataDir: string, owner?: object): Promise<string> {
|
||||
const lock = path.join(dataDir, "runtime-service.lock");
|
||||
await mkdir(lock, { mode: 0o700 });
|
||||
if (owner) {
|
||||
await writeFile(path.join(lock, "owner.json"), JSON.stringify(owner), {
|
||||
mode: 0o600,
|
||||
});
|
||||
}
|
||||
await chmod(lock, 0o700);
|
||||
return lock;
|
||||
}
|
||||
|
||||
async function ownerOf(dataDir: string): Promise<Record<string, unknown>> {
|
||||
return JSON.parse(
|
||||
await readFile(
|
||||
path.join(dataDir, "runtime-service.lock", "owner.json"),
|
||||
"utf8",
|
||||
),
|
||||
);
|
||||
}
|
||||
|
||||
/** Every lock, staging and cleared-lease entry in the data dir. */
|
||||
async function lockEntries(dataDir: string): Promise<string[]> {
|
||||
return (await readdir(dataDir))
|
||||
.filter((name) => name.startsWith("runtime-service.lock"))
|
||||
.sort();
|
||||
}
|
||||
|
||||
describe("RuntimeService", () => {
|
||||
it("publishes and removes an outer Runtime receipt without secrets", async () => {
|
||||
const dataDir = await directory("runtime-service-");
|
||||
@@ -71,18 +105,7 @@ describe("RuntimeService", () => {
|
||||
|
||||
it("refuses a live owner of the same Runtime home", async () => {
|
||||
const dataDir = await directory("runtime-service-live-");
|
||||
const lock = path.join(dataDir, "runtime-service.lock");
|
||||
await import("node:fs/promises").then(({ mkdir }) =>
|
||||
mkdir(lock, { mode: 0o700 }),
|
||||
);
|
||||
await writeFile(
|
||||
path.join(lock, "owner.json"),
|
||||
JSON.stringify({ pid: 777 }),
|
||||
{
|
||||
mode: 0o600,
|
||||
},
|
||||
);
|
||||
await chmod(lock, 0o700);
|
||||
await seedLock(dataDir, { pid: 777 });
|
||||
const service = new RuntimeService({
|
||||
dataDir,
|
||||
createSupervisor: supervisor,
|
||||
@@ -94,27 +117,228 @@ describe("RuntimeService", () => {
|
||||
);
|
||||
});
|
||||
|
||||
it("reclaims a stale validated lease", async () => {
|
||||
const dataDir = await directory("runtime-service-stale-");
|
||||
const lock = path.join(dataDir, "runtime-service.lock");
|
||||
await import("node:fs/promises").then(({ mkdir }) =>
|
||||
mkdir(lock, { mode: 0o700 }),
|
||||
);
|
||||
await writeFile(
|
||||
path.join(lock, "owner.json"),
|
||||
JSON.stringify({ pid: 777 }),
|
||||
{
|
||||
mode: 0o600,
|
||||
},
|
||||
);
|
||||
await chmod(lock, 0o700);
|
||||
it("reclaims a previous-boot lease even when a recycled PID answers the liveness probe", async () => {
|
||||
// Issue #184: a forced power-off leaves runtime-service.lock behind;
|
||||
// after the reboot the owner's PID belongs to another process, so a
|
||||
// liveness-only check reports "already running" forever and launchd's
|
||||
// KeepAlive loops the wrapper. A lease from another boot can never have
|
||||
// a living owner, whatever the PID table says.
|
||||
const dataDir = await directory("runtime-service-prevboot-");
|
||||
await seedLock(dataDir, { pid: 777, boot_id: PREVIOUS_BOOT });
|
||||
const service = new RuntimeService({
|
||||
dataDir,
|
||||
createSupervisor: supervisor,
|
||||
pid: 4321,
|
||||
processIsAlive: () => true,
|
||||
bootId: () => CURRENT_BOOT,
|
||||
});
|
||||
|
||||
await service.start();
|
||||
expect(await ownerOf(dataDir)).toMatchObject({
|
||||
pid: 4321,
|
||||
boot_id: CURRENT_BOOT,
|
||||
});
|
||||
await service.stop();
|
||||
expect(await lockEntries(dataDir)).toEqual([]);
|
||||
});
|
||||
|
||||
it("refuses a live owner from the current boot", async () => {
|
||||
const dataDir = await directory("runtime-service-sameboot-live-");
|
||||
await seedLock(dataDir, { pid: 777, boot_id: CURRENT_BOOT });
|
||||
const service = new RuntimeService({
|
||||
dataDir,
|
||||
createSupervisor: supervisor,
|
||||
pid: 4321,
|
||||
processIsAlive: (pid) => pid === 777,
|
||||
bootId: () => CURRENT_BOOT,
|
||||
});
|
||||
|
||||
await expect(service.start()).rejects.toThrow(
|
||||
"runtime_service_already_running",
|
||||
);
|
||||
expect(await ownerOf(dataDir)).toMatchObject({ pid: 777 });
|
||||
});
|
||||
|
||||
it("reclaims a dead owner from the current boot", async () => {
|
||||
const dataDir = await directory("runtime-service-sameboot-dead-");
|
||||
await seedLock(dataDir, { pid: 777, boot_id: CURRENT_BOOT });
|
||||
const service = new RuntimeService({
|
||||
dataDir,
|
||||
createSupervisor: supervisor,
|
||||
pid: 4321,
|
||||
processIsAlive: () => false,
|
||||
bootId: () => CURRENT_BOOT,
|
||||
});
|
||||
|
||||
await service.start();
|
||||
expect(await ownerOf(dataDir)).toMatchObject({ pid: 4321 });
|
||||
await service.stop();
|
||||
});
|
||||
|
||||
it.each([
|
||||
["the lease predates boot ids", { pid: 777 }, CURRENT_BOOT],
|
||||
["the lease has an empty boot id", { pid: 777, boot_id: "" }, CURRENT_BOOT],
|
||||
[
|
||||
"the current boot id is unknown",
|
||||
{ pid: 777, boot_id: PREVIOUS_BOOT },
|
||||
null,
|
||||
],
|
||||
])("falls back to the PID probe when %s", async (_case, owner, current) => {
|
||||
const dataDir = await directory("runtime-service-unknown-boot-");
|
||||
await seedLock(dataDir, owner);
|
||||
const live = new RuntimeService({
|
||||
dataDir,
|
||||
createSupervisor: supervisor,
|
||||
pid: 4321,
|
||||
processIsAlive: (pid) => pid === 777,
|
||||
bootId: () => current,
|
||||
});
|
||||
await expect(live.start()).rejects.toThrow(
|
||||
"runtime_service_already_running",
|
||||
);
|
||||
expect(await ownerOf(dataDir)).toMatchObject({ pid: 777 });
|
||||
|
||||
const dead = new RuntimeService({
|
||||
dataDir,
|
||||
createSupervisor: supervisor,
|
||||
pid: 4321,
|
||||
processIsAlive: () => false,
|
||||
bootId: () => current,
|
||||
});
|
||||
await dead.start();
|
||||
expect(await ownerOf(dataDir)).toMatchObject({ pid: 4321 });
|
||||
await dead.stop();
|
||||
});
|
||||
|
||||
it("records the platform boot id in owner.json", async () => {
|
||||
const dataDir = await directory("runtime-service-boot-id-");
|
||||
const service = new RuntimeService({
|
||||
dataDir,
|
||||
createSupervisor: supervisor,
|
||||
});
|
||||
|
||||
await service.start();
|
||||
const owner = await ownerOf(dataDir);
|
||||
if (process.platform === "darwin" || process.platform === "linux") {
|
||||
expect(owner.boot_id).toMatch(/^[0-9A-Za-z-]{8,64}$/);
|
||||
} else {
|
||||
expect(owner.boot_id).toBeNull();
|
||||
}
|
||||
expect(owner.pid).toBe(process.pid);
|
||||
await service.stop();
|
||||
});
|
||||
|
||||
it("leaves no lock behind when publishing fails before the rename", async () => {
|
||||
// The lock only appears through a rename of a fully written staging
|
||||
// directory, so a failure (or power cut) while staging cannot leave a
|
||||
// lock without owner.json.
|
||||
const dataDir = await directory("runtime-service-publish-failure-");
|
||||
const broken = new RuntimeService({
|
||||
dataDir,
|
||||
createSupervisor: supervisor,
|
||||
bootId: () => {
|
||||
throw new Error("simulated crash while staging");
|
||||
},
|
||||
});
|
||||
|
||||
await expect(broken.start()).rejects.toThrow(
|
||||
"runtime_service_lock_unavailable",
|
||||
);
|
||||
expect(await lockEntries(dataDir)).toEqual([]);
|
||||
|
||||
const next = new RuntimeService({
|
||||
dataDir,
|
||||
createSupervisor: supervisor,
|
||||
pid: 4321,
|
||||
bootId: () => CURRENT_BOOT,
|
||||
});
|
||||
await next.start();
|
||||
expect(await ownerOf(dataDir)).toMatchObject({ pid: 4321 });
|
||||
await next.stop();
|
||||
});
|
||||
|
||||
it("clears an owner-less legacy lock created before the current boot", async () => {
|
||||
// Older versions created the lock with mkdir and wrote owner.json
|
||||
// afterwards; a power cut in between left an empty lock that used to
|
||||
// fail as runtime_service_lock_invalid forever.
|
||||
const dataDir = await directory("runtime-service-legacy-prevboot-");
|
||||
const lock = await seedLock(dataDir);
|
||||
const createdAt = (await stat(lock)).mtimeMs;
|
||||
const service = new RuntimeService({
|
||||
dataDir,
|
||||
createSupervisor: supervisor,
|
||||
pid: 4321,
|
||||
bootId: () => CURRENT_BOOT,
|
||||
legacyBootTime: () => createdAt + 60_000,
|
||||
});
|
||||
|
||||
await service.start();
|
||||
expect(await ownerOf(dataDir)).toMatchObject({ pid: 4321 });
|
||||
await service.stop();
|
||||
});
|
||||
|
||||
it.each([
|
||||
["it may be from the current boot", (createdAt: number) => createdAt - 1],
|
||||
["the boot time is unknown", () => null],
|
||||
])(
|
||||
"keeps refusing an owner-less legacy lock when %s",
|
||||
async (_case, boot) => {
|
||||
const dataDir = await directory("runtime-service-legacy-sameboot-");
|
||||
const lock = await seedLock(dataDir);
|
||||
const createdAt = (await stat(lock)).mtimeMs;
|
||||
const service = new RuntimeService({
|
||||
dataDir,
|
||||
createSupervisor: supervisor,
|
||||
bootId: () => CURRENT_BOOT,
|
||||
legacyBootTime: () => boot(createdAt),
|
||||
});
|
||||
|
||||
await expect(service.start()).rejects.toThrow(
|
||||
"runtime_service_lock_invalid",
|
||||
);
|
||||
// rename() may replace an empty directory; the publish step must not.
|
||||
expect(await readdir(lock)).toEqual([]);
|
||||
expect(await lockEntries(dataDir)).toEqual(["runtime-service.lock"]);
|
||||
},
|
||||
);
|
||||
|
||||
it("removes staging and cleared-lease debris after acquiring", async () => {
|
||||
const dataDir = await directory("runtime-service-debris-");
|
||||
const staging = path.join(dataDir, "runtime-service.lock.tmp-crashed");
|
||||
await mkdir(staging, { mode: 0o700 });
|
||||
await writeFile(
|
||||
path.join(staging, "owner.json"),
|
||||
JSON.stringify({ pid: 9 }),
|
||||
);
|
||||
await mkdir(path.join(dataDir, "runtime-service.lock.stale.crashed"), {
|
||||
mode: 0o700,
|
||||
});
|
||||
const service = new RuntimeService({
|
||||
dataDir,
|
||||
createSupervisor: supervisor,
|
||||
pid: 4321,
|
||||
bootId: () => CURRENT_BOOT,
|
||||
});
|
||||
|
||||
await service.start();
|
||||
expect(await lockEntries(dataDir)).toEqual(["runtime-service.lock"]);
|
||||
expect(await ownerOf(dataDir)).toMatchObject({ pid: 4321 });
|
||||
await service.stop();
|
||||
expect(await lockEntries(dataDir)).toEqual([]);
|
||||
});
|
||||
|
||||
it("reclaims a stale validated lease", async () => {
|
||||
const dataDir = await directory("runtime-service-stale-");
|
||||
await seedLock(dataDir, { pid: 777 });
|
||||
const service = new RuntimeService({
|
||||
dataDir,
|
||||
createSupervisor: supervisor,
|
||||
pid: 4321,
|
||||
processIsAlive: () => false,
|
||||
});
|
||||
|
||||
await service.start();
|
||||
expect(await ownerOf(dataDir)).toMatchObject({ pid: 4321 });
|
||||
await service.stop();
|
||||
});
|
||||
|
||||
@@ -386,7 +610,7 @@ describe("RuntimeService", () => {
|
||||
const real = path.join(parent, "real");
|
||||
const link = path.join(parent, "data");
|
||||
await mkdir(real, { mode: 0o755 });
|
||||
await import("node:fs/promises").then(({ symlink }) => symlink(real, link));
|
||||
await symlink(real, link);
|
||||
const service = new RuntimeService({
|
||||
dataDir: link,
|
||||
createSupervisor: () => supervisor(),
|
||||
|
||||
@@ -1,12 +1,14 @@
|
||||
import { execFileSync } from "node:child_process";
|
||||
import { randomUUID } from "node:crypto";
|
||||
import { readFileSync, type Stats } from "node:fs";
|
||||
import {
|
||||
chmod,
|
||||
lstat,
|
||||
mkdir,
|
||||
readdir,
|
||||
readFile,
|
||||
rename,
|
||||
rm,
|
||||
rmdir,
|
||||
writeFile,
|
||||
} from "node:fs/promises";
|
||||
import path from "node:path";
|
||||
@@ -35,6 +37,14 @@ export interface RuntimeServiceOptions {
|
||||
): ManagedRuntime | Promise<ManagedRuntime>;
|
||||
pid?: number;
|
||||
processIsAlive?: (pid: number) => boolean;
|
||||
/** Identity of the current boot (`kern.bootsessionuuid` on macOS,
|
||||
* `/proc/sys/kernel/random/boot_id` on Linux, `null` when unknown).
|
||||
* Injectable so tests can simulate a reboot without one. */
|
||||
bootId?: () => string | null;
|
||||
/** Wall-clock epoch (ms) the kernel reports for the current boot, or `null`
|
||||
* when unknown. Consulted only for a legacy lock directory without
|
||||
* owner.json; see removeUnownedLease. Injectable for tests. */
|
||||
legacyBootTime?: () => number | null;
|
||||
diagnostics?: DiagnosticSink;
|
||||
}
|
||||
|
||||
@@ -186,29 +196,81 @@ export class RuntimeService {
|
||||
const lock = this.lockPath();
|
||||
for (let attempt = 0; attempt < 2; attempt += 1) {
|
||||
try {
|
||||
await mkdir(lock, { mode: 0o700 });
|
||||
const identity = await lstat(lock);
|
||||
this.lease = { dev: identity.dev, ino: identity.ino };
|
||||
try {
|
||||
await privateWrite(
|
||||
path.join(lock, "owner.json"),
|
||||
JSON.stringify({
|
||||
pid: this.pid(),
|
||||
created_at: new Date().toISOString(),
|
||||
}),
|
||||
);
|
||||
} catch (error) {
|
||||
await this.releaseLease();
|
||||
throw error;
|
||||
}
|
||||
return;
|
||||
await this.publishLease(lock);
|
||||
break;
|
||||
} catch (error) {
|
||||
if ((error as NodeJS.ErrnoException).code !== "EEXIST" || attempt > 0) {
|
||||
if (!leaseTaken(error) || attempt > 0) {
|
||||
throw new Error("runtime_service_lock_unavailable", { cause: error });
|
||||
}
|
||||
await this.removeStaleLease(lock);
|
||||
}
|
||||
}
|
||||
await this.sweepLeaseDebris();
|
||||
}
|
||||
|
||||
/** Publish the lease in one step: the lock directory becomes visible with
|
||||
* owner.json (pid, created_at, boot_id) already inside it, so a power cut
|
||||
* can leave a staging directory behind but never a lock without an owner.
|
||||
* The old mkdir -> write sequence had exactly that window (issue #184). */
|
||||
private async publishLease(lock: string): Promise<void> {
|
||||
const staging = `${lock}${STAGING_INFIX}${randomUUID()}`;
|
||||
await mkdir(staging, { mode: 0o700 });
|
||||
try {
|
||||
await privateWrite(
|
||||
path.join(staging, "owner.json"),
|
||||
JSON.stringify({
|
||||
pid: this.pid(),
|
||||
created_at: new Date().toISOString(),
|
||||
boot_id: this.currentBootId(),
|
||||
}),
|
||||
);
|
||||
// POSIX rename() silently REPLACES an existing *empty* directory, so on
|
||||
// its own it is not an exclusive create. A non-empty lock (every lease
|
||||
// this version publishes) makes it fail with EEXIST/ENOTEMPTY. An empty
|
||||
// lock can only come from an older version's two-step mkdir -> write:
|
||||
// either power-cut debris, or a live acquire in flight. The latter must
|
||||
// not be overwritten, and the former has to go through
|
||||
// removeStaleLease's legacy rule rather than be replaced blindly, so an
|
||||
// existing lock of any kind is reported as taken here. The residual race
|
||||
// (an old binary mkdir-ing between this check and the rename) needs an
|
||||
// old and a new Runtime starting on the same home within microseconds;
|
||||
// this version never creates an empty lock, since releaseLease also
|
||||
// removes it with a single rename.
|
||||
if (await pathExists(lock)) throw leaseTakenError();
|
||||
await rename(staging, lock);
|
||||
} catch (error) {
|
||||
await rm(staging, { recursive: true, force: true }).catch(
|
||||
() => undefined,
|
||||
);
|
||||
throw error;
|
||||
}
|
||||
// rename() keeps the inode, so this is the published directory's identity;
|
||||
// releaseLease compares it before touching the lock.
|
||||
const identity = await lstat(lock);
|
||||
this.lease = { dev: identity.dev, ino: identity.ino };
|
||||
}
|
||||
|
||||
/** Remove staging and cleared-lease directories left behind by a crash.
|
||||
* Runs only while this process holds the lease, so a staging directory is
|
||||
* either debris or belongs to a competitor that has already lost (its
|
||||
* rename cannot succeed over our non-empty lock). Best-effort. */
|
||||
private async sweepLeaseDebris(): Promise<void> {
|
||||
const base = path.basename(this.lockPath());
|
||||
const names = await readdir(this.options.dataDir).catch((): string[] => []);
|
||||
await Promise.all(
|
||||
names
|
||||
.filter(
|
||||
(name) =>
|
||||
name.startsWith(`${base}${STAGING_INFIX}`) ||
|
||||
name.startsWith(`${base}${STALE_INFIX}`),
|
||||
)
|
||||
.map((name) =>
|
||||
rm(path.join(this.options.dataDir, name), {
|
||||
recursive: true,
|
||||
force: true,
|
||||
}).catch(() => undefined),
|
||||
),
|
||||
);
|
||||
}
|
||||
|
||||
private async removeStaleLease(lock: string): Promise<void> {
|
||||
@@ -223,25 +285,81 @@ export class RuntimeService {
|
||||
) {
|
||||
throw new Error("runtime_service_lock_unsafe");
|
||||
}
|
||||
let raw: string;
|
||||
try {
|
||||
raw = await readFile(path.join(lock, "owner.json"), "utf8");
|
||||
} catch (error) {
|
||||
if ((error as NodeJS.ErrnoException).code === "ENOENT") {
|
||||
await this.removeUnownedLease(lock, info);
|
||||
return;
|
||||
}
|
||||
throw new Error("runtime_service_lock_invalid", { cause: error });
|
||||
}
|
||||
let owner: unknown;
|
||||
try {
|
||||
owner = JSON.parse(await readFile(path.join(lock, "owner.json"), "utf8"));
|
||||
owner = JSON.parse(raw);
|
||||
} catch {
|
||||
throw new Error("runtime_service_lock_invalid");
|
||||
}
|
||||
const pid =
|
||||
owner && typeof owner === "object"
|
||||
? (owner as { pid?: unknown }).pid
|
||||
: undefined;
|
||||
if (!owner || typeof owner !== "object" || Array.isArray(owner)) {
|
||||
throw new Error("runtime_service_lock_invalid");
|
||||
}
|
||||
// A lease written during another boot cannot have a living owner: every
|
||||
// process of that boot is gone, and a PID that answers the liveness probe
|
||||
// now merely reused the number. Without this, a forced power-off left the
|
||||
// lock behind, the reboot recycled the owner's PID, and launchd's
|
||||
// KeepAlive restarted the wrapper into runtime_service_already_running
|
||||
// without end (issue #184). The kernel's boot identity is compared, not
|
||||
// the wall clock: the clock can jump forward while a lease is held (no
|
||||
// RTC, VM resume, NTP or manual correction) and make a live lease look
|
||||
// older than the boot. When either side is unknown (a lease from an older
|
||||
// version, an unsupported platform) the PID probe below decides, as before.
|
||||
const ownerBoot = (owner as { boot_id?: unknown }).boot_id;
|
||||
const currentBoot = this.currentBootId();
|
||||
if (
|
||||
typeof ownerBoot === "string" &&
|
||||
ownerBoot !== "" &&
|
||||
currentBoot !== null &&
|
||||
ownerBoot !== currentBoot
|
||||
) {
|
||||
await this.clearLeaseDir(lock);
|
||||
return;
|
||||
}
|
||||
const pid = (owner as { pid?: unknown }).pid;
|
||||
if (typeof pid !== "number" || !Number.isInteger(pid) || pid <= 1) {
|
||||
throw new Error("runtime_service_lock_invalid");
|
||||
}
|
||||
if ((this.options.processIsAlive ?? processIsAlive)(pid)) {
|
||||
throw new Error("runtime_service_already_running");
|
||||
}
|
||||
const stale = `${lock}.stale.${randomUUID()}`;
|
||||
await this.clearLeaseDir(lock);
|
||||
}
|
||||
|
||||
/** A lock directory without owner.json. This version publishes owner.json
|
||||
* together with the lock and releases it with one rename, so such a
|
||||
* directory was written by an older version (mkdir -> write, where a power
|
||||
* cut in between leaves it empty) or by hand. It names no PID and no boot,
|
||||
* so the only usable evidence is "created before the current boot": clear
|
||||
* it then, and otherwise keep the previous refusal
|
||||
* (runtime_service_lock_invalid), including when the boot time is unknown.
|
||||
*
|
||||
* The kernel boot time and the directory mtime are both wall-clock values,
|
||||
* which is why lease ownership itself no longer depends on them. Here a
|
||||
* forward clock jump can only hurt if an older binary is live inside its
|
||||
* mkdir -> write window at that very moment; a backward jump only keeps the
|
||||
* old refusal. */
|
||||
private async removeUnownedLease(lock: string, info: Stats): Promise<void> {
|
||||
const bootTime = (this.options.legacyBootTime ?? currentBootTimeMs)();
|
||||
if (bootTime === null || !(info.mtimeMs < bootTime)) {
|
||||
throw new Error("runtime_service_lock_invalid");
|
||||
}
|
||||
await this.clearLeaseDir(lock);
|
||||
}
|
||||
|
||||
private async clearLeaseDir(lock: string): Promise<void> {
|
||||
const stale = `${lock}${STALE_INFIX}${randomUUID()}`;
|
||||
await rename(lock, stale);
|
||||
await rm(stale, { recursive: true });
|
||||
await rm(stale, { recursive: true, force: true });
|
||||
}
|
||||
|
||||
private async releaseLease(): Promise<void> {
|
||||
@@ -252,8 +370,19 @@ export class RuntimeService {
|
||||
const current = await lstat(lock).catch(() => null);
|
||||
if (!current || current.dev !== lease.dev || current.ino !== lease.ino)
|
||||
return;
|
||||
await rm(path.join(lock, "owner.json"), { force: true });
|
||||
await rmdir(lock).catch(() => undefined);
|
||||
// Take the lock away with one rename so no observer ever sees it without
|
||||
// owner.json: an owner-less lock is left to mean legacy debris.
|
||||
const released = `${lock}${STALE_INFIX}${randomUUID()}`;
|
||||
try {
|
||||
await rename(lock, released);
|
||||
} catch {
|
||||
return;
|
||||
}
|
||||
await rm(released, { recursive: true, force: true }).catch(() => undefined);
|
||||
}
|
||||
|
||||
private currentBootId(): string | null {
|
||||
return (this.options.bootId ?? currentBootId)();
|
||||
}
|
||||
|
||||
private async writeReceipt(
|
||||
@@ -302,6 +431,91 @@ export class RuntimeService {
|
||||
}
|
||||
}
|
||||
|
||||
const STAGING_INFIX = ".tmp-";
|
||||
const STALE_INFIX = ".stale.";
|
||||
|
||||
function leaseTaken(error: unknown): boolean {
|
||||
const code = (error as NodeJS.ErrnoException | undefined)?.code;
|
||||
return code === "EEXIST" || code === "ENOTEMPTY";
|
||||
}
|
||||
|
||||
function leaseTakenError(): NodeJS.ErrnoException {
|
||||
return Object.assign(new Error("runtime_service_lock_exists"), {
|
||||
code: "EEXIST",
|
||||
});
|
||||
}
|
||||
|
||||
async function pathExists(target: string): Promise<boolean> {
|
||||
try {
|
||||
await lstat(target);
|
||||
return true;
|
||||
} catch (error) {
|
||||
if ((error as NodeJS.ErrnoException).code === "ENOENT") return false;
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
function sysctl(name: string): string {
|
||||
return execFileSync("/usr/sbin/sysctl", ["-n", name], {
|
||||
encoding: "utf8",
|
||||
timeout: 1000,
|
||||
maxBuffer: 4096,
|
||||
killSignal: "SIGKILL",
|
||||
shell: false,
|
||||
stdio: ["ignore", "pipe", "ignore"],
|
||||
env: { PATH: "/usr/bin:/bin:/usr/sbin:/sbin", LANG: "C", LC_ALL: "C" },
|
||||
});
|
||||
}
|
||||
|
||||
let cachedBootId: string | null | undefined;
|
||||
|
||||
/** The kernel's identity for the current boot, cached per process (it cannot
|
||||
* change while the process lives). `null` on unsupported platforms or when
|
||||
* the source cannot be read; the lease logic then falls back to the PID
|
||||
* probe. */
|
||||
function currentBootId(): string | null {
|
||||
if (cachedBootId === undefined) cachedBootId = readBootId();
|
||||
return cachedBootId;
|
||||
}
|
||||
|
||||
function readBootId(): string | null {
|
||||
try {
|
||||
let raw: string;
|
||||
if (process.platform === "darwin") raw = sysctl("kern.bootsessionuuid");
|
||||
else if (process.platform === "linux")
|
||||
raw = readFileSync("/proc/sys/kernel/random/boot_id", "utf8");
|
||||
else return null;
|
||||
const id = raw.trim();
|
||||
return /^[0-9A-Za-z-]{8,64}$/.test(id) ? id : null;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
/** Wall-clock epoch (ms) the kernel reports for the current boot, or `null`.
|
||||
* Used only for owner-less legacy lock directories. */
|
||||
function currentBootTimeMs(): number | null {
|
||||
try {
|
||||
if (process.platform === "darwin") {
|
||||
const match = /\bsec\s*=\s*(\d+)\s*,\s*usec\s*=\s*(\d+)/.exec(
|
||||
sysctl("kern.boottime"),
|
||||
);
|
||||
return match
|
||||
? Number(match[1]) * 1000 + Math.floor(Number(match[2]) / 1000)
|
||||
: null;
|
||||
}
|
||||
if (process.platform === "linux") {
|
||||
const match = /^btime\s+(\d+)\s*$/m.exec(
|
||||
readFileSync("/proc/stat", "utf8"),
|
||||
);
|
||||
return match ? Number(match[1]) * 1000 : null;
|
||||
}
|
||||
} catch {
|
||||
// Unknown boot time: the caller keeps refusing.
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
async function privateWrite(target: string, value: string): Promise<void> {
|
||||
const temporary = `${target}.${randomUUID()}.tmp`;
|
||||
try {
|
||||
|
||||
@@ -0,0 +1,101 @@
|
||||
import { mkdtemp, readFile, rm, writeFile } from "node:fs/promises";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import {
|
||||
sdkMigrationLogPath,
|
||||
writeSdkMigrationLog,
|
||||
} from "./sdk-migration-log.js";
|
||||
|
||||
async function events(dataDir: string): Promise<string[]> {
|
||||
return (await readFile(sdkMigrationLogPath(dataDir), "utf8"))
|
||||
.trim()
|
||||
.split("\n")
|
||||
.map((line) => JSON.parse(line).event);
|
||||
}
|
||||
|
||||
describe("SDK migration log ordering", () => {
|
||||
it("preserves invocation order for concurrent progress and completion writes", async () => {
|
||||
const dataDir = await mkdtemp(
|
||||
path.join(os.tmpdir(), "migration-log-order-"),
|
||||
);
|
||||
try {
|
||||
const expected = Array.from(
|
||||
{ length: 100 },
|
||||
(_, i) => `migration.progress.${i}`,
|
||||
);
|
||||
expected.push("migration.finished");
|
||||
await Promise.all(
|
||||
expected.map((event) =>
|
||||
writeSdkMigrationLog(dataDir, {
|
||||
writer: "desktop",
|
||||
event,
|
||||
outcome: "started",
|
||||
}),
|
||||
),
|
||||
);
|
||||
expect(await events(dataDir)).toEqual(expected);
|
||||
} finally {
|
||||
await rm(dataDir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
it("awaiting the terminal event also drains preceding progress writes", async () => {
|
||||
const dataDir = await mkdtemp(
|
||||
path.join(os.tmpdir(), "migration-log-drain-"),
|
||||
);
|
||||
const pending: Promise<void>[] = [];
|
||||
try {
|
||||
const expected = Array.from(
|
||||
{ length: 50 },
|
||||
(_, i) => `migration.progress.${i}`,
|
||||
);
|
||||
for (const event of expected) {
|
||||
pending.push(
|
||||
writeSdkMigrationLog(dataDir, {
|
||||
writer: "desktop",
|
||||
event,
|
||||
outcome: "started",
|
||||
}),
|
||||
);
|
||||
}
|
||||
await writeSdkMigrationLog(dataDir, {
|
||||
writer: "desktop",
|
||||
event: "migration.finished",
|
||||
outcome: "succeeded",
|
||||
});
|
||||
expect(await events(dataDir)).toEqual([
|
||||
...expected,
|
||||
"migration.finished",
|
||||
]);
|
||||
} finally {
|
||||
await Promise.all(pending);
|
||||
await rm(dataDir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
it("does not let a failed append block subsequent writes to the same home", async () => {
|
||||
const dataDir = await mkdtemp(
|
||||
path.join(os.tmpdir(), "migration-log-recovery-"),
|
||||
);
|
||||
try {
|
||||
await writeFile(path.join(dataDir, "logs"), "not a directory");
|
||||
await expect(
|
||||
writeSdkMigrationLog(dataDir, {
|
||||
writer: "desktop",
|
||||
event: "migration.progress",
|
||||
outcome: "started",
|
||||
}),
|
||||
).resolves.toBeUndefined();
|
||||
await rm(path.join(dataDir, "logs"));
|
||||
await writeSdkMigrationLog(dataDir, {
|
||||
writer: "desktop",
|
||||
event: "migration.finished",
|
||||
outcome: "succeeded",
|
||||
});
|
||||
expect(await events(dataDir)).toEqual(["migration.finished"]);
|
||||
} finally {
|
||||
await rm(dataDir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -33,6 +33,9 @@ export const SDK_MIGRATION_LOG_NAME = "sdk-migration.log";
|
||||
const MAX_BYTES = 4 * 1024 * 1024;
|
||||
const SAFE_TEXT = /^[A-Za-z0-9_.:/@+-]{0,160}$/;
|
||||
const MAX_ITEMS = 32;
|
||||
// Preserve invocation order within this process, including fire-and-forget
|
||||
// progress followed by an awaited terminal event. Homes remain independent.
|
||||
const pendingWrites = new Map<string, Promise<void>>();
|
||||
|
||||
export function sdkMigrationLogPath(dataDir: string): string {
|
||||
return path.join(dataDir, "logs", SDK_MIGRATION_LOG_NAME);
|
||||
@@ -103,17 +106,28 @@ export async function writeSdkMigrationLog(
|
||||
platform: process.platform,
|
||||
...(entry.details === undefined ? {} : { details: entry.details }),
|
||||
});
|
||||
const target = sdkMigrationLogPath(dataDir);
|
||||
await mkdir(path.dirname(target), { recursive: true, mode: 0o700 });
|
||||
const size = await stat(target).then(
|
||||
(info) => info.size,
|
||||
() => 0,
|
||||
);
|
||||
if (size > MAX_BYTES) await rename(target, `${target}.1`);
|
||||
await appendFile(target, `${JSON.stringify(line)}\n`, {
|
||||
encoding: "utf8",
|
||||
mode: 0o600,
|
||||
});
|
||||
const target = path.resolve(sdkMigrationLogPath(dataDir));
|
||||
const previous = pendingWrites.get(target) ?? Promise.resolve();
|
||||
const pending = previous
|
||||
.catch(() => undefined)
|
||||
.then(async () => {
|
||||
await mkdir(path.dirname(target), { recursive: true, mode: 0o700 });
|
||||
const size = await stat(target).then(
|
||||
(info) => info.size,
|
||||
() => 0,
|
||||
);
|
||||
if (size > MAX_BYTES) await rename(target, `${target}.1`);
|
||||
await appendFile(target, `${JSON.stringify(line)}\n`, {
|
||||
encoding: "utf8",
|
||||
mode: 0o600,
|
||||
});
|
||||
});
|
||||
pendingWrites.set(target, pending);
|
||||
try {
|
||||
await pending;
|
||||
} finally {
|
||||
if (pendingWrites.get(target) === pending) pendingWrites.delete(target);
|
||||
}
|
||||
} catch {
|
||||
// Never let the log decide whether a migration or a startup happens.
|
||||
}
|
||||
|
||||
@@ -36,7 +36,7 @@ MEMMY_LICENSE = (
|
||||
MEMMY_PROVENANCE = MEMMY_LICENSE.with_name("PROVENANCE.md")
|
||||
MEMMY_PIN = "25acd5f29fc7df73b943713abe61f1b2f9839887"
|
||||
HY_MEMORY_NAME = "hy-memory"
|
||||
HY_MEMORY_VERSION = "2.0.6"
|
||||
HY_MEMORY_VERSION = "2.0.8"
|
||||
|
||||
|
||||
class SbomError(RuntimeError):
|
||||
|
||||
@@ -12,7 +12,7 @@ test("installation docs disclose publication and lifecycle boundaries", async ()
|
||||
"utf8",
|
||||
);
|
||||
|
||||
assert.match(readme, /`hy-memory==2\.0\.6`/);
|
||||
assert.match(readme, /`hy-memory==2\.0\.8`/);
|
||||
assert.match(readme, /`v1\.0\.5` 的发行目标是 Linux x64\/arm64 Runtime/);
|
||||
assert.match(readme, /Linux 制品由 GitHub Actions/);
|
||||
assert.match(readme, /macOS.*Developer ID 签名、公证/s);
|
||||
|
||||
@@ -13,7 +13,7 @@ spec.loader.exec_module(module)
|
||||
|
||||
|
||||
def test_hy_memory_notice_comes_from_the_installed_dist_info(tmp_path: Path) -> None:
|
||||
dist_info = tmp_path / "hy_memory-2.0.6.dist-info"
|
||||
dist_info = tmp_path / "hy_memory-2.0.8.dist-info"
|
||||
licenses = dist_info / "licenses"
|
||||
licenses.mkdir(parents=True)
|
||||
(dist_info / "METADATA").write_text(
|
||||
@@ -21,7 +21,7 @@ def test_hy_memory_notice_comes_from_the_installed_dist_info(tmp_path: Path) ->
|
||||
(
|
||||
"Metadata-Version: 2.4",
|
||||
"Name: hy-memory",
|
||||
"Version: 2.0.6",
|
||||
"Version: 2.0.8",
|
||||
"License-Expression: Apache-2.0",
|
||||
"",
|
||||
)
|
||||
@@ -38,7 +38,7 @@ def test_hy_memory_notice_comes_from_the_installed_dist_info(tmp_path: Path) ->
|
||||
notice = module.hy_memory_notice(dist_info)
|
||||
|
||||
assert notice["name"] == "hy-memory"
|
||||
assert notice["version"] == "2.0.6"
|
||||
assert notice["version"] == "2.0.8"
|
||||
assert notice["license"] == "Apache-2.0"
|
||||
assert notice["copyright"] == "Copyright (C) 2026 Tencent. All rights reserved."
|
||||
assert "Licensed under the Apache License" in notice["text"]
|
||||
|
||||
+1
-1
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
|
||||
|
||||
[project]
|
||||
name = "hy-memory"
|
||||
version = "2.0.6"
|
||||
version = "2.0.8"
|
||||
description = "HY Memory - Industrial-grade dual-system cognitive memory framework"
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.9"
|
||||
|
||||
@@ -0,0 +1,140 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""对话、记忆正文和模型输出只进 DEBUG,不进 INFO / WARNING。
|
||||
|
||||
MemHub 等调用方默认以 INFO 运行;这些文本一旦进 INFO,就会写进常驻的
|
||||
hy_memory.log。这里逐条锁住曾经把正文打在 INFO / WARNING 的日志。
|
||||
"""
|
||||
|
||||
import logging
|
||||
|
||||
import pytest
|
||||
|
||||
from hy_memory.agent.reconciler import ReconcileOp as AgentReconcileOp
|
||||
from hy_memory.agent.reconciler import _log_ops_detail
|
||||
from hy_memory.coding.extractor import CodingMemoryExtractor
|
||||
from hy_memory.coding.fact_reconciler import WorkingFactReconciler
|
||||
from hy_memory.coding.judge import classify_and_rewrite_queries
|
||||
from hy_memory.coding.reconciler import CodingMemoryReconciler
|
||||
from hy_memory.coding.types import CodingMemory, WorkingFact
|
||||
|
||||
from tests.coding._helpers import MockLLMProvider
|
||||
|
||||
SECRET = "SECRET-CONVERSATION-TEXT-42"
|
||||
|
||||
|
||||
def _above_debug(caplog):
|
||||
return [
|
||||
r.getMessage() for r in caplog.records
|
||||
if r.name.startswith("hy_memory") and r.levelno > logging.DEBUG
|
||||
]
|
||||
|
||||
|
||||
def _debug(caplog):
|
||||
return [
|
||||
r.getMessage() for r in caplog.records
|
||||
if r.name.startswith("hy_memory") and r.levelno == logging.DEBUG
|
||||
]
|
||||
|
||||
|
||||
def _assert_secret_only_at_debug(caplog):
|
||||
assert not [m for m in _above_debug(caplog) if SECRET in m]
|
||||
assert [m for m in _debug(caplog) if SECRET in m]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
class TestStoreInsertLogs:
|
||||
async def test_experience_task_not_at_info(self, coding_store, caplog):
|
||||
caplog.set_level(logging.DEBUG, logger="hy_memory")
|
||||
await coding_store.insert(CodingMemory(
|
||||
experience_id="e1", user_id="u1", task=SECRET, solution="S",
|
||||
))
|
||||
info = [m for m in _above_debug(caplog) if "inserted experience_id=e1" in m]
|
||||
assert info
|
||||
_assert_secret_only_at_debug(caplog)
|
||||
|
||||
async def test_fact_statement_not_at_info(self, coding_store, caplog):
|
||||
caplog.set_level(logging.DEBUG, logger="hy_memory")
|
||||
await coding_store.insert_fact(WorkingFact(
|
||||
fact_id="f1", user_id="u1", agent_id="a1", session_id="s1",
|
||||
workspace_id="ws/a", statement=SECRET,
|
||||
))
|
||||
info = [m for m in _above_debug(caplog) if "inserted fact_id=f1" in m]
|
||||
assert info
|
||||
_assert_secret_only_at_debug(caplog)
|
||||
|
||||
|
||||
class TestParseFailureWarnings:
|
||||
"""解析失败仍发 WARNING,但只带长度,模型输出进 DEBUG。"""
|
||||
|
||||
@pytest.mark.parametrize("parse", [
|
||||
CodingMemoryExtractor._parse_output,
|
||||
CodingMemoryReconciler._parse_ops,
|
||||
WorkingFactReconciler._parse_ops,
|
||||
])
|
||||
def test_warning_keeps_length_drops_text(self, parse, caplog):
|
||||
caplog.set_level(logging.DEBUG, logger="hy_memory")
|
||||
bad = f"not json {SECRET}"
|
||||
parse(bad)
|
||||
warnings = [
|
||||
r.getMessage() for r in caplog.records
|
||||
if r.levelno == logging.WARNING and "failed to parse" in r.getMessage()
|
||||
]
|
||||
assert warnings and f"({len(bad)} chars)" in warnings[0]
|
||||
_assert_secret_only_at_debug(caplog)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
class TestJudgeLogs:
|
||||
async def test_rewrite_not_at_info(self, caplog):
|
||||
caplog.set_level(logging.DEBUG, logger="hy_memory")
|
||||
llm = MockLLMProvider(f"1\n{SECRET}")
|
||||
result = await classify_and_rewrite_queries(["q"], llm)
|
||||
assert result["rewrite_query"] == SECRET
|
||||
_assert_secret_only_at_debug(caplog)
|
||||
|
||||
async def test_unrecognized_head_not_at_warning(self, caplog):
|
||||
caplog.set_level(logging.DEBUG, logger="hy_memory")
|
||||
llm = MockLLMProvider(f"{SECRET}\nrewrite")
|
||||
result = await classify_and_rewrite_queries(["q"], llm)
|
||||
assert result["ok"] is False
|
||||
_assert_secret_only_at_debug(caplog)
|
||||
|
||||
|
||||
def test_reconciler_ops_detail_is_debug(caplog):
|
||||
caplog.set_level(logging.DEBUG, logger="hy_memory")
|
||||
_log_ops_detail([AgentReconcileOp(op="ADD", content=SECRET)])
|
||||
_assert_secret_only_at_debug(caplog)
|
||||
|
||||
|
||||
class _StopAfterLogging(Exception):
|
||||
pass
|
||||
|
||||
|
||||
def _add_client():
|
||||
from types import SimpleNamespace
|
||||
from unittest.mock import AsyncMock
|
||||
|
||||
from hy_memory.client import HyMemoryClient
|
||||
|
||||
client = HyMemoryClient.__new__(HyMemoryClient)
|
||||
client._mode = "lite"
|
||||
client._config = SimpleNamespace(coding=None)
|
||||
writer = SimpleNamespace(
|
||||
VERSION="test", write=AsyncMock(side_effect=_StopAfterLogging),
|
||||
)
|
||||
client._registry = SimpleNamespace(get_writer=AsyncMock(return_value=writer))
|
||||
return client
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.parametrize("data", [
|
||||
SECRET,
|
||||
[{"role": "user", "content": SECRET}],
|
||||
])
|
||||
async def test_add_does_not_log_input_at_info(data, caplog):
|
||||
caplog.set_level(logging.DEBUG, logger="hy_memory")
|
||||
client = _add_client()
|
||||
with pytest.raises(_StopAfterLogging):
|
||||
await client._async_add_impl(data, user_id="u1")
|
||||
assert [m for m in _above_debug(caplog) if m.startswith("[add] writing")]
|
||||
_assert_secret_only_at_debug(caplog)
|
||||
@@ -0,0 +1,229 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""经验与事实记下对话时间 memory_at,并以秒级 epoch 对外返回(与 chat 一致)。
|
||||
|
||||
之前生产力链路丢了 add() 传进来的 memory_at,coding_memory_meta / working_fact
|
||||
也没有这一列,MemHub 里所有经验都显示"时间未知"。
|
||||
"""
|
||||
|
||||
import sqlite3
|
||||
import time
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from types import SimpleNamespace
|
||||
|
||||
import pytest
|
||||
|
||||
from hy_memory.client import HyMemoryClient
|
||||
from hy_memory.coding.extractor import CodingMemoryExtractor
|
||||
from hy_memory.coding.reconciler import CodingMemoryReconciler
|
||||
from hy_memory.coding.store import CodingMemoryStore, _CREATE_TABLE_SQL, _CREATE_FACT_TABLE_SQL
|
||||
from hy_memory.coding.types import CodingMemory, WorkingFact
|
||||
from hy_memory.coding.writer import CodingWriter
|
||||
|
||||
from tests.coding._helpers import MockLLMProvider
|
||||
from tests.coding.test_list_and_stats_by_scene import _client, _unwrapped
|
||||
from tests.coding.test_write_scene_routing import _messages
|
||||
from tests.coding.test_writer_e2e import _coding_messages, _extract_resp, _reconcile_resp
|
||||
|
||||
T1 = datetime(2026, 10, 1, 9, 30, tzinfo=timezone.utc)
|
||||
T2 = T1 + timedelta(days=3)
|
||||
|
||||
|
||||
def _epoch(value: datetime) -> int:
|
||||
return int(value.timestamp())
|
||||
|
||||
|
||||
def _writer(store, embed, llm):
|
||||
return CodingWriter(
|
||||
store=store,
|
||||
extractor=CodingMemoryExtractor(llm),
|
||||
reconciler=CodingMemoryReconciler(store, embed, llm),
|
||||
llm_provider=llm, embed_service=embed,
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
class TestStore:
|
||||
async def test_round_trip_as_epoch(self, coding_store):
|
||||
await coding_store.insert(CodingMemory(
|
||||
experience_id="e1", user_id="u", task="T", solution="S", memory_at=T1,
|
||||
))
|
||||
await coding_store.insert_fact(WorkingFact(
|
||||
fact_id="f1", user_id="u", statement="x", memory_at=T1,
|
||||
))
|
||||
exp = await coding_store.get_by_id("e1", user_id="u")
|
||||
fact = await coding_store.get_fact("f1", user_id="u")
|
||||
assert _epoch(exp.memory_at) == _epoch(T1)
|
||||
assert exp.to_dict()["memory_at"] == _epoch(T1)
|
||||
assert fact.to_dict()["memory_at"] == _epoch(T1)
|
||||
|
||||
async def test_missing_time_stays_unknown(self, coding_store):
|
||||
await coding_store.insert(CodingMemory(
|
||||
experience_id="e1", user_id="u", task="T", solution="S",
|
||||
))
|
||||
exp = await coding_store.get_by_id("e1", user_id="u")
|
||||
assert exp.memory_at is None and exp.to_dict()["memory_at"] is None
|
||||
|
||||
@pytest.mark.parametrize("old, new", [(T1, T2), (T2, T1), (T2, None)])
|
||||
async def test_update_keeps_the_later_time(self, coding_store, old, new):
|
||||
await coding_store.insert(CodingMemory(
|
||||
experience_id="e1", user_id="u", task="T", solution="S",
|
||||
session_id="s", memory_at=old,
|
||||
))
|
||||
await coding_store.update(CodingMemory(
|
||||
experience_id="e1", user_id="u", task="T", solution="S2",
|
||||
session_id="s", memory_at=new,
|
||||
))
|
||||
exp = await coding_store.get_by_id("e1", user_id="u")
|
||||
assert exp.solution == "S2"
|
||||
assert _epoch(exp.memory_at) == _epoch(T2)
|
||||
|
||||
async def test_edit_keeps_the_time(self, coding_store):
|
||||
await coding_store.insert(CodingMemory(
|
||||
experience_id="e1", user_id="u", task="T", solution="S", memory_at=T1,
|
||||
))
|
||||
await coding_store.edit_experience("e1", user_id="u", solution="S2")
|
||||
exp = await coding_store.get_by_id("e1", user_id="u")
|
||||
assert _epoch(exp.memory_at) == _epoch(T1)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.parametrize("variant", ["working", "coding"])
|
||||
async def test_existing_database_gains_the_column_on_open(coding_config, mock_embed, variant):
|
||||
store = CodingMemoryStore(coding_config, embed_service=mock_embed, variant=variant)
|
||||
# 先按老 schema(没有 memory_at)建库并放一行
|
||||
conn = sqlite3.connect(store._db_path)
|
||||
conn.executescript(_CREATE_TABLE_SQL.replace(" memory_at INTEGER,\n", ""))
|
||||
conn.executescript(_CREATE_FACT_TABLE_SQL.replace(" memory_at INTEGER,\n", ""))
|
||||
conn.execute(
|
||||
"INSERT INTO coding_memory_meta (experience_id, user_id, task, solution, "
|
||||
"created_at, updated_at) VALUES ('old', 'u', 'T', 'S', '2026-01-01T00:00:00', "
|
||||
"'2026-01-01T00:00:00')"
|
||||
)
|
||||
conn.commit()
|
||||
conn.close()
|
||||
|
||||
await store.initialize()
|
||||
try:
|
||||
old = await store.get_by_id("old", user_id="u")
|
||||
assert old is not None and old.memory_at is None
|
||||
await store.insert_fact(WorkingFact(
|
||||
fact_id="f1", user_id="u", statement="x", memory_at=T1,
|
||||
))
|
||||
assert _epoch((await store.get_fact("f1")).memory_at) == _epoch(T1)
|
||||
finally:
|
||||
await store.close()
|
||||
# 再开一次不出错(ADD COLUMN 是幂等的)
|
||||
again = CodingMemoryStore(coding_config, embed_service=mock_embed, variant=variant)
|
||||
await again.initialize()
|
||||
await again.close()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
class TestWriter:
|
||||
def _llm(self):
|
||||
return MockLLMProvider([
|
||||
_extract_resp(
|
||||
[{"task": "打包发布", "solution": "S", "confidence": 0.9}],
|
||||
[{"statement": "发布脚本在 scripts/publish_package.sh"}],
|
||||
),
|
||||
_reconcile_resp([{"draft_idx": 0, "action": "ADD"}]),
|
||||
])
|
||||
|
||||
async def test_conversation_time_reaches_both_layers(self, coding_store, mock_embed):
|
||||
resp = await _writer(coding_store, mock_embed, self._llm()).write(
|
||||
_coding_messages(), user_id="u", session_id="s", memory_at=T1,
|
||||
)
|
||||
exp = await coding_store.get_by_id(resp["experience_ids"][0], user_id="u")
|
||||
fact = await coding_store.get_fact(resp["fact_ids"][0], user_id="u")
|
||||
assert _epoch(exp.memory_at) == _epoch(T1)
|
||||
assert _epoch(fact.memory_at) == _epoch(T1)
|
||||
|
||||
async def test_defaults_to_request_time_like_chat(self, coding_store, mock_embed):
|
||||
before = int(time.time())
|
||||
resp = await _writer(coding_store, mock_embed, self._llm()).write(
|
||||
_coding_messages(), user_id="u", session_id="s",
|
||||
)
|
||||
exp = await coding_store.get_by_id(resp["experience_ids"][0], user_id="u")
|
||||
fact = await coding_store.get_fact(resp["fact_ids"][0], user_id="u")
|
||||
for stamp in (_epoch(exp.memory_at), _epoch(fact.memory_at)):
|
||||
assert before <= stamp <= int(time.time()) + 1
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_add_passes_the_time_to_the_productivity_writer(monkeypatch):
|
||||
seen = {}
|
||||
|
||||
async def _judge(*args, **kwargs):
|
||||
return "working"
|
||||
|
||||
class _Writer:
|
||||
async def write(self, *args, **kwargs):
|
||||
seen.update(kwargs)
|
||||
return {"success": True, "experience_ids": [], "fact_ids": []}
|
||||
|
||||
async def _get_writer(*, variant):
|
||||
return _Writer()
|
||||
|
||||
monkeypatch.setattr("hy_memory.client.classify_messages_scene", _judge)
|
||||
ns = SimpleNamespace(
|
||||
_config=SimpleNamespace(coding=SimpleNamespace(enable=True)),
|
||||
_coding_llm_provider=object(),
|
||||
_cache=None,
|
||||
_get_coding_writer=_get_writer,
|
||||
_normalize_scene=HyMemoryClient._normalize_scene,
|
||||
_scene_variant=HyMemoryClient._scene_variant,
|
||||
_parse_input=HyMemoryClient._parse_input,
|
||||
)
|
||||
await HyMemoryClient._async_add_impl(
|
||||
ns, _messages(), user_id="u1", agent_id="a1", memory_at=T1,
|
||||
)
|
||||
assert seen["memory_at"] == T1
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_list_get_and_search_return_epoch_seconds(coding_config, mock_embed):
|
||||
coding = CodingMemoryStore(coding_config, embed_service=mock_embed, variant="coding")
|
||||
await coding.initialize()
|
||||
try:
|
||||
await coding.insert(CodingMemory(
|
||||
experience_id="e1", user_id="u", task="打包发布", solution="S", memory_at=T1,
|
||||
))
|
||||
await coding.insert_fact(WorkingFact(
|
||||
fact_id="f1", user_id="u", statement="打包发布", memory_at=T2,
|
||||
derived_experience_ids=["e1"],
|
||||
))
|
||||
client = _client([], {"coding": coding})
|
||||
list_memories = _unwrapped("async_list_memories")
|
||||
|
||||
page = await list_memories(client, user_id="u", scene="coding")
|
||||
assert page["experiences"][0]["memory_at"] == _epoch(T1)
|
||||
|
||||
merged = await list_memories(client, user_id="u", scene="all")
|
||||
assert merged["memories"][0]["memory_at"] == _epoch(T1)
|
||||
|
||||
exp = await coding.get_by_id("e1", user_id="u")
|
||||
got = HyMemoryClient._format_productivity_get(
|
||||
exp, kind="experience", variant="coding", request_id="r", elapsed_ms=0,
|
||||
)
|
||||
assert got["memory_at"] == _epoch(T1)
|
||||
|
||||
client._embed_service = mock_embed
|
||||
client._coding_store = coding
|
||||
|
||||
async def _ready(variant="working"):
|
||||
return None
|
||||
|
||||
client._get_coding_writer = _ready
|
||||
client._cache = None
|
||||
resp = await HyMemoryClient._search_coding(
|
||||
client, target_query="打包发布", user_ids=["u"], workspace_id=None,
|
||||
branch=None, limit=5, working_fact_min_score=0.0,
|
||||
working_experience_min_score=0.0, request_id="r",
|
||||
t0=time.perf_counter(), variant="coding",
|
||||
)
|
||||
assert resp["facts"][0]["memory_at"] == _epoch(T2)
|
||||
assert resp["experiences"][0]["memory_at"] == _epoch(T1)
|
||||
unified = HyMemoryClient._search_lane_items("coding", resp)
|
||||
assert {item["memory_at"] for item in unified} == {_epoch(T1), _epoch(T2)}
|
||||
finally:
|
||||
await coding.close()
|
||||
@@ -0,0 +1,51 @@
|
||||
"""The guarded to_thread adapter preserves asyncio.to_thread context semantics."""
|
||||
import asyncio
|
||||
import contextvars
|
||||
import importlib.util
|
||||
from pathlib import Path
|
||||
import unittest
|
||||
|
||||
|
||||
# This helper is dependency-free; exercise it without initializing the SDK.
|
||||
_spec = importlib.util.spec_from_file_location(
|
||||
"async_executor_under_test",
|
||||
Path(__file__).resolve().parents[1] / "hy_memory/utils/async_executor.py",
|
||||
)
|
||||
_executor = importlib.util.module_from_spec(_spec)
|
||||
_spec.loader.exec_module(_executor)
|
||||
|
||||
|
||||
class ThreadContextTests(unittest.IsolatedAsyncioTestCase):
|
||||
async def test_copies_context_and_does_not_propagate_worker_changes(self):
|
||||
request_id = contextvars.ContextVar("request_id", default="missing")
|
||||
request_id.set("caller")
|
||||
|
||||
def work(prefix, *, suffix):
|
||||
value = request_id.get()
|
||||
request_id.set("worker")
|
||||
return prefix + value + suffix
|
||||
|
||||
result = await _executor.to_thread_guarded(work, "[", suffix="]")
|
||||
self.assertEqual(result, "[caller]")
|
||||
self.assertEqual(request_id.get(), "caller")
|
||||
|
||||
async def test_concurrent_callers_keep_separate_contexts(self):
|
||||
request_id = contextvars.ContextVar("request_id", default="missing")
|
||||
|
||||
async def call(value):
|
||||
request_id.set(value)
|
||||
await asyncio.sleep(0)
|
||||
return await _executor.to_thread_guarded(request_id.get)
|
||||
|
||||
self.assertEqual(await asyncio.gather(call("a"), call("b")), ["a", "b"])
|
||||
|
||||
async def test_context_is_copied_when_coroutine_runs(self):
|
||||
request_id = contextvars.ContextVar("request_id", default="missing")
|
||||
request_id.set("created")
|
||||
pending = _executor.to_thread_guarded(request_id.get)
|
||||
request_id.set("awaited")
|
||||
self.assertEqual(await pending, "awaited")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -90,7 +90,10 @@ class TestEmbedInfoLog:
|
||||
assert payload["provider"] == "openai"
|
||||
assert payload["model"] == "text-embedding-test"
|
||||
assert payload["api"] == "https://example.com/v1/embeddings"
|
||||
assert payload["request"]["input"] == ["hello world"]
|
||||
# 待向量化的原文不进 INFO,只记每条字符数
|
||||
assert "input" not in payload["request"]
|
||||
assert payload["request"]["input_chars"] == [11]
|
||||
assert "hello world" not in info_msgs[0]
|
||||
assert payload["request"]["dimensions"] == 8
|
||||
assert payload["response"]["vector_dims"] == [8]
|
||||
assert payload["response"]["vector_count"] == 1
|
||||
|
||||
@@ -0,0 +1,29 @@
|
||||
"""User text may quote tokenizer control spellings without using control tokens."""
|
||||
|
||||
import pytest
|
||||
import tiktoken
|
||||
|
||||
from hy_memory.core.embed_service import _count_tokens, _truncate_for_embedding
|
||||
from hy_memory.utils.token_count import count_tokens
|
||||
|
||||
|
||||
@pytest.mark.parametrize('counter', [count_tokens, _count_tokens])
|
||||
@pytest.mark.parametrize('marker', ['<|endoftext|>', '<|fim_prefix|>'])
|
||||
def test_literal_special_tokens_are_counted_as_ordinary_text(marker, counter):
|
||||
text = marker + '你好世界' * 20
|
||||
expected = len(tiktoken.get_encoding('cl100k_base').encode_ordinary(text))
|
||||
assert counter(text) == expected
|
||||
|
||||
|
||||
def test_literal_special_token_does_not_bypass_embedding_limit():
|
||||
text = '<|endoftext|>' + '你好世界' * 3000
|
||||
encoder = tiktoken.get_encoding('cl100k_base')
|
||||
truncated = _truncate_for_embedding(text)
|
||||
assert truncated != text
|
||||
assert len(encoder.encode_ordinary(truncated)) <= 8192 - 64
|
||||
assert truncated.startswith('<|endoftext|>')
|
||||
|
||||
|
||||
def test_short_literal_special_token_text_is_preserved():
|
||||
text = 'Explain <|endoftext|> as a literal string.'
|
||||
assert _truncate_for_embedding(text) == text
|
||||
@@ -1,4 +1,4 @@
|
||||
"""Pipeline log(始终)与 trace(可关)分离。"""
|
||||
"""Pipeline log(默认开,可关)与 trace(可关)分离。"""
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
@@ -10,6 +10,7 @@ from hy_memory.config import MemoryConfig
|
||||
from hy_memory.data.cache_sqlite import SqliteCache
|
||||
from hy_memory.utils.pipeline_observability import (
|
||||
is_llm_content_logging_enabled,
|
||||
is_pipeline_log_enabled,
|
||||
is_pipeline_trace_enabled,
|
||||
resolve_pipeline_log_dir,
|
||||
)
|
||||
@@ -39,6 +40,14 @@ class TestPipelineObservabilityConfig:
|
||||
monkeypatch.setenv("MEMORY_DATA_DIR", str(tmp_path / "data"))
|
||||
assert resolve_pipeline_log_dir() == str(tmp_path / "data" / "logs" / "pipeline")
|
||||
|
||||
def test_log_enabled_default(self, monkeypatch):
|
||||
monkeypatch.delenv("MEMORY_PIPELINE_LOG_ENABLED", raising=False)
|
||||
assert is_pipeline_log_enabled() is True
|
||||
|
||||
def test_log_disabled(self, monkeypatch):
|
||||
monkeypatch.setenv("MEMORY_PIPELINE_LOG_ENABLED", "false")
|
||||
assert is_pipeline_log_enabled() is False
|
||||
|
||||
def test_trace_enabled_default(self, monkeypatch):
|
||||
monkeypatch.delenv("MEMORY_PIPELINE_TRACE_ENABLED", raising=False)
|
||||
assert is_pipeline_trace_enabled() is True
|
||||
@@ -91,6 +100,98 @@ class TestObservabilityHookBehavior:
|
||||
|
||||
run(_test())
|
||||
|
||||
def test_log_flag_on_writes_jsonl(self, env_setup, monkeypatch):
|
||||
"""显式开启与默认一致:照常写 JSONL。"""
|
||||
monkeypatch.setenv("MEMORY_PIPELINE_LOG_ENABLED", "true")
|
||||
monkeypatch.setenv("MEMORY_PIPELINE_TRACE_ENABLED", "false")
|
||||
|
||||
async def _test():
|
||||
from datetime import date
|
||||
from hy_memory.utils.pipeline_observability import (
|
||||
drain_pipeline_log_tasks,
|
||||
install_pipeline_observability,
|
||||
)
|
||||
|
||||
config = MemoryConfig.from_dict({"cache": {"backend": "sqlite"}})
|
||||
cache = SqliteCache(config)
|
||||
await cache.initialize()
|
||||
install_pipeline_observability(cache, log_label="test")
|
||||
await cache.store_pipeline_log(
|
||||
request_id="req-on", user_id="u1", agent_id="a1", step="S1_EMBED",
|
||||
)
|
||||
await drain_pipeline_log_tasks()
|
||||
|
||||
log_file = Path(env_setup) / "default" / f"{date.today().isoformat()}.log"
|
||||
assert json.loads(log_file.read_text(encoding="utf-8").strip())["step"] == "S1_EMBED"
|
||||
await cache.close()
|
||||
|
||||
run(_test())
|
||||
|
||||
def test_log_off_trace_off_is_noop(self, env_setup, monkeypatch):
|
||||
"""两者都关:不包装、不写文件、不写库,store_pipeline_log 直接返回 True。"""
|
||||
monkeypatch.setenv("MEMORY_PIPELINE_LOG_ENABLED", "false")
|
||||
monkeypatch.setenv("MEMORY_PIPELINE_TRACE_ENABLED", "false")
|
||||
|
||||
async def _test():
|
||||
from hy_memory.utils.pipeline_observability import (
|
||||
_noop_store_pipeline_log,
|
||||
drain_pipeline_log_tasks,
|
||||
install_pipeline_observability,
|
||||
)
|
||||
|
||||
config = MemoryConfig.from_dict({"cache": {"backend": "sqlite"}})
|
||||
cache = SqliteCache(config)
|
||||
await cache.initialize()
|
||||
original_store = cache.store_pipeline_log
|
||||
install_pipeline_observability(cache, log_label="test")
|
||||
assert cache.store_pipeline_log is _noop_store_pipeline_log
|
||||
|
||||
ok = await cache.store_pipeline_log(
|
||||
request_id="req-off", user_id="ns__u1", agent_id="a1",
|
||||
step="S1_EMBED", prompt="p", response="r",
|
||||
)
|
||||
await drain_pipeline_log_tasks()
|
||||
|
||||
assert ok is True
|
||||
assert not Path(env_setup).exists()
|
||||
# 原始 store 仍能写库,说明表本身可用;空操作确实没写
|
||||
assert await cache.get_pipeline_logs(request_id="req-off") == []
|
||||
await original_store(
|
||||
request_id="req-probe", user_id="u1", agent_id="a1",
|
||||
step="PROBE", prompt="", response="",
|
||||
)
|
||||
assert len(await cache.get_pipeline_logs(request_id="req-probe")) == 1
|
||||
await cache.close()
|
||||
|
||||
run(_test())
|
||||
|
||||
def test_log_off_trace_on_writes_db_only(self, env_setup, monkeypatch):
|
||||
"""只关 Log:不写 JSONL,Trace 照常落库。"""
|
||||
monkeypatch.setenv("MEMORY_PIPELINE_LOG_ENABLED", "false")
|
||||
monkeypatch.setenv("MEMORY_PIPELINE_TRACE_ENABLED", "true")
|
||||
|
||||
async def _test():
|
||||
from hy_memory.utils.pipeline_observability import (
|
||||
drain_pipeline_log_tasks,
|
||||
install_pipeline_observability,
|
||||
)
|
||||
|
||||
config = MemoryConfig.from_dict({"cache": {"backend": "sqlite"}})
|
||||
cache = SqliteCache(config)
|
||||
await cache.initialize()
|
||||
install_pipeline_observability(cache, log_label="test")
|
||||
await cache.store_pipeline_log(
|
||||
request_id="req-db", user_id="u1", agent_id="a1", step="S1_EMBED",
|
||||
)
|
||||
await drain_pipeline_log_tasks()
|
||||
|
||||
assert not Path(env_setup).exists()
|
||||
logs = await cache.get_pipeline_logs(request_id="req-db")
|
||||
assert [row["step"] for row in logs] == ["S1_EMBED"]
|
||||
await cache.close()
|
||||
|
||||
run(_test())
|
||||
|
||||
def test_store_returns_before_slow_persist(self, env_setup, monkeypatch):
|
||||
"""慢 DB / 慢磁盘不得堵住 await store_pipeline_log。"""
|
||||
monkeypatch.setenv("MEMORY_PIPELINE_TRACE_ENABLED", "true")
|
||||
|
||||
Reference in new issue
Block a user