Compare commits

...
Author SHA1 Message Date
alvinfeiandClaude Opus 5.5 b1b3e0a291 fix(sdk): drop the user-turn filter for basic_info
Owner decision on #248: the writer no longer filters basic_info against
the user turns. The placeholder guard (with healing on read) and the
prompt hard rules stay; the guard alone keeps an assistant quoting
"name is 用户" out of the profile.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-11-01 08:18:32 +08:00
alvinfeiandClaude Opus 5.5 8895fea775 release: hy-memory 2.0.9
Placeholder values no longer overwrite the user's basic profile (L0). Not
yet published to PyPI.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-11-01 04:56:01 +08:00
alvinfeiandClaude Opus 5.5 78882f3300 fix(sdk): placeholder values never overwrite the basic profile
A weak extraction model filled basic_info with "用户", "The user" or "N/A"
when the user stated nothing, and each placeholder superseded the real L0
value. upsert_basic_profile now drops placeholder values and writes no new
version when nothing is left; profile assembly skips stored placeholders.
The writer keeps a basic_info field only when its value appears in a user
message of the request. The zh and en extraction prompts add a hard rule
and a small-talk counter-example.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-11-01 04:56:01 +08:00
feitianxiang 988667cd87 Merge pull request #224 from Tencent-Hunyuan/chore/build-36-docs
chore(release): bump MemHub build number to 36 for v1.0.5
2026-10-28 22:07:22 +08:00
6 changed files with 338 additions and 1 deletions

No files matched your search

+21
View File
@@ -94,6 +94,27 @@ authority on first launch.
`hy_node_permissions` proof table. Evidence, governed memories, model
profiles, SDK stores, and source/host grants are preserved.
## Hy Memory 2.0.9 — 2026-11-01
- Placeholder values no longer overwrite the user's basic profile (L0). A
small extraction model filled `basic_info` with `{"name": "用户"}`,
`The user` or `N/A` when the user had said nothing about themselves, and
each placeholder superseded the real value, so the L0 chain read
费天翔 → Fei → 用户 → The user. Two defences:
- `upsert_basic_profile` drops placeholder values: empty, `用户`,
`the user`, `user`, `n/a`, `na`, `none`, `null`, `unknown`, `未提供`,
`未知`, `不详`, `-`, `?` and the field's own name, compared
case-insensitively with whitespace and punctuation stripped. When nothing
is left it writes no new version and no new L0 head. Placeholders already
stored in the chain no longer override earlier real values when the
profile is assembled, so `get_user_profile` of an affected user shows the
last real value again without any migration. This also stops an
assistant quoting an old memory ("The user's name is 用户.") from being
written back.
- The extraction prompts (zh and en) state the rule: only what the user
said in a `user` turn, never the field name or a placeholder, `{}`
otherwise, with a small-talk counter-example.
## Hy Memory 2.0.8 — 2026-10-28
- Published
+33
View File
@@ -682,6 +682,23 @@ Output:
```
John's occupation and location are HIS attributes, so they go into `memory` with "John" as the subject. They must NEVER land in `basic_info.occupation` / `basic_info.location` — `basic_info` is exclusively the user's own profile. Only `name` (Tim, from the user message's `name` field) belongs there.
## Example 11 — Counter-example: small talk has no basic_info
New Messages:
[{"role": "user", "content": "Hi"},
{"role": "assistant", "content": "Hello! The user's name is The user. How can I help?"}]
Memory Date: 2025-09-03
Output:
```json
{
"memory": [],
"intentions": [],
"basic_info": {}
}
```
The user stated nothing about themselves. The assistant's line is not a user turn, and "The user" is a placeholder, not a name. Output `{}` — never `{"name": "The user"}` or `{"name": "user"}`.
---
"""
@@ -749,6 +766,14 @@ Rules:
fields (e.g. name, age, location, occupation, employer) **explicitly stated**
in the NEW user messages. Never infer, never invent. If no field is explicitly
stated, output `{{}}`.
* HARD RULE: fill a field ONLY if the user stated it in a `user` turn.
Anything said only by the assistant (including the assistant quoting an
earlier memory such as "The user's name is X") does NOT count. If nothing
qualifies, output `"basic_info": {{}}`.
* NEVER write the field name itself or a placeholder as a value ("user",
"The user", "用户", "name", "N/A", "unknown", "none", "-"). Omit the field.
* Negative example: small talk such as `[user]: Hi` / `[assistant]: How can
I help?`, or `[user]: Reply with exactly: PONG` → `"basic_info": {{}}`.
* Exception for `name`: the `name` field on the user's own message, or a leading
line "The user's name is X" / "用户名叫X", is an authoritative source for
`name` — even though the user never said it out loud.
@@ -817,6 +842,14 @@ Rules:
fields (e.g. name, age, location, occupation, employer) **explicitly stated**
in the NEW user messages. Never infer, never invent. If no field is explicitly
stated, output `{{}}`.
* HARD RULE: fill a field ONLY if the user stated it in a `user` turn.
Anything said only by the assistant (including the assistant quoting an
earlier memory such as "The user's name is X") does NOT count. If nothing
qualifies, output `"basic_info": {{}}`.
* NEVER write the field name itself or a placeholder as a value ("user",
"The user", "用户", "name", "N/A", "unknown", "none", "-"). Omit the field.
* Negative example: small talk such as `[user]: Hi` / `[assistant]: How can
I help?`, or `[user]: Reply with exactly: PONG` → `"basic_info": {{}}`.
* Exception for `name`: the `name` field on the user's own message, or a leading
line "The user's name is X" / "用户名叫X", is an authoritative source for
`name` — even though the user never said it out loud.
+31
View File
@@ -365,6 +365,13 @@ EXTRACT_PROMPT_ZH = """## 最近 k 条消息
* basic_info:**只装用户本人的画像**。
仅当 NEW user messages 中**明确**陈述稳定画像字段(如 姓名、年龄、所在地、职业、雇主)时才填写。
绝不推断、绝不编造。没有任何字段被明确陈述时输出 `{{}}`。
* **硬性规则**:只有用户在 `user` 回合里**明确说出**的字段才能填写。只在
assistant 回合出现的内容(包括 assistant 引用旧记忆,如"The user's name is X"
/ "用户名是X")一律不算。没有符合条件的字段时,必须输出 `"basic_info": {{}}`。
* **绝不**把字段名本身或占位符当值写入("用户"、"The user"、"user"、"name"、
"姓名"、"N/A"、"unknown"、"未知"、"未提供"、"-"),宁可省略该字段。
* 负例:寒暄如 `[user]: 你好` / `[assistant]: 我可以帮你。`,或
`[user]: Reply with exactly: PONG` → `"basic_info": {{}}`。
* `name` 的例外:用户自己那条消息上的 `name` 字段,或文首形如
"用户名叫X" / "The user's name is X" 的那一行,都是 `name` 的权威来源
——即使用户本人从未开口自报姓名。
@@ -440,6 +447,13 @@ EXTRACT_PROMPT_ZH_NO_INTENTION = """## 最近 k 条消息
* basic_info:**只装用户本人的画像**。
仅当 NEW user messages 中**明确**陈述稳定画像字段(如 姓名、年龄、所在地、职业、雇主)时才填写。
绝不推断、绝不编造。没有任何字段被明确陈述时输出 `{{}}`。
* **硬性规则**:只有用户在 `user` 回合里**明确说出**的字段才能填写。只在
assistant 回合出现的内容(包括 assistant 引用旧记忆,如"The user's name is X"
/ "用户名是X")一律不算。没有符合条件的字段时,必须输出 `"basic_info": {{}}`。
* **绝不**把字段名本身或占位符当值写入("用户"、"The user"、"user"、"name"、
"姓名"、"N/A"、"unknown"、"未知"、"未提供"、"-"),宁可省略该字段。
* 负例:寒暄如 `[user]: 你好` / `[assistant]: 我可以帮你。`,或
`[user]: Reply with exactly: PONG` → `"basic_info": {{}}`。
* `name` 的例外:用户自己那条消息上的 `name` 字段,或文首形如
"用户名叫X" / "The user's name is X" 的那一行,都是 `name` 的权威来源
——即使用户本人从未开口自报姓名。
@@ -669,6 +683,23 @@ Memory Date:2025-09-02
```
职业和所在地是 **John 的**属性,所以进 `memory` 并以"John"作主体,**绝不能**落到 `basic_info` 的 `occupation` / `location` —— `basic_info` 专属于用户本人。只有 `name`(Tim,来自用户消息的 `name` 字段)才该放进去。注意输出语言随输入语言(此例为英文)。
## 示例 11 —— 反例:寒暄没有 basic_info
新消息:
[{"role": "user", "content": "你好"},
{"role": "assistant", "content": "你好!我可以帮你。之前记得 The user's name is 用户."}]
Memory Date:2025-09-03
输出:
```json
{
"memory": [],
"intentions": [],
"basic_info": {}
}
```
用户没有说任何关于自己的信息。assistant 那句不是 user 回合,"用户" 也是占位符而不是名字。必须输出 `{}`——**绝不能**输出 `{"name": "用户"}`。
---
"""
+77
View File
@@ -26,6 +26,8 @@ from __future__ import annotations
import asyncio
import logging
import re
import unicodedata
import uuid
import weakref
from dataclasses import dataclass
@@ -208,6 +210,76 @@ def _sanitize_kv(
return result
# ================================================================
# 占位符守卫(P0-1)
# ================================================================
# 弱模型(如 qwen3.5-2b)在用户什么都没说时,会把 schema 字段说明里的词当值
# 填进 basic_info:{"name": "用户"} / "The user" / "N/A"。每个占位符都会
# supersede 掉真实值,演化链变成 费天翔 → Fei → 用户 → The user。
# 规则:占位符永远不入库;读侧组装 KV 时占位符也不覆盖已有真实值。
_PLACEHOLDER_VALUES = frozenset({
"用户", "该用户", "本用户", "用户本人",
"the user", "the user's", "user", "a user", "this user",
"n/a", "na", "none", "null", "nil", "unknown", "unspecified",
"not provided", "not specified", "not mentioned",
"未提供", "未知", "不详", "未说明", "未提及", "无",
"-", "?",
})
# 内置字段的中文标签:弱模型也会把字段说明的中文词当值。
_FIELD_LABELS: Dict[str, tuple] = {
"name": ("姓名", "名字", "用户名", "用户姓名", "用户名字"),
"age": ("年龄", "用户年龄"),
"location": ("所在地", "地点", "城市", "用户所在地"),
"occupation": ("职业", "工作", "用户职业"),
"employer": ("雇主", "公司", "用户雇主"),
}
# 去掉标点/符号后再比较,让 "The user." / "用户。" / "N/A" 都命中。
# "-" / "?" 本身就是纯标点,剥完为空串,同样判为占位符。
_PUNCT_RE = re.compile(r"[\s\.,;:!?'\"`~()\[\]{}<>/\\|_\-\u3000-\u303f\uff00-\uff0f\uff1a-\uff20]+")
def _placeholder_key(value: str) -> str:
s = unicodedata.normalize("NFKC", str(value or "")).strip().lower()
s = s.replace("\u2019", "'")
s = _PUNCT_RE.sub(" ", s)
return " ".join(s.split())
_PLACEHOLDER_KEYS = frozenset(_placeholder_key(v) for v in _PLACEHOLDER_VALUES)
def is_placeholder_value(field_name: str, value: Any) -> bool:
"""value 是否为占位符(空 / 用户 / The user / N/A / 字段名本身 等)。
大小写不敏感、首尾空白与标点不敏感。数字(如 age=30)永远不是占位符。
"""
if value is None:
return True
if isinstance(value, bool):
return False
if isinstance(value, (int, float)):
return False
key = _placeholder_key(value)
if not key:
return True
if key in _PLACEHOLDER_KEYS:
return True
field_key = _placeholder_key(field_name)
if field_key and key in (field_key, field_key.replace("_", " ")):
return True
for label in _FIELD_LABELS.get(field_name, ()):
if key == _placeholder_key(label):
return True
return False
def _drop_placeholders(kv: Dict[str, Any]) -> Dict[str, Any]:
return {k: v for k, v in kv.items() if not is_placeholder_value(k, v)}
# ================================================================
# 公共入口:upsert_basic_profile
# ================================================================
@@ -287,6 +359,8 @@ async def _upsert_basic_profile_unlocked(
# L0 Basic Info 从输入、唯一性到落盘都只属于 user_id。
new_kv = _sanitize_kv(kv or {}, allowed_fields=allowed_fields)
# 单调性守卫:占位符永远不覆盖已有值,过滤后为空就不写新版本、不产生新链头。
new_kv = _drop_placeholders(new_kv)
if not new_kv:
return BasicProfileUpsertResult(
success=True, changed=False, reason="no valid fields after sanitize"
@@ -513,5 +587,8 @@ def _assemble_full_kv_from_nodes(
for k, v in kv.items():
if allowed_set is not None and k not in allowed_set:
continue
# 历史上已落盘的占位符版本(如 name=用户)不覆盖更早的真实值。
if is_placeholder_value(k, v):
continue
full_kv[k] = v
return full_kv
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "hy-memory"
version = "2.0.8"
version = "2.0.9"
description = "HY Memory - Industrial-grade dual-system cognitive memory framework"
readme = "README.md"
requires-python = ">=3.9"
+175
View File
@@ -0,0 +1,175 @@
"""P0-1:占位符不得污染 L0 基础画像。
弱模型(qwen3.5-2b)在用户什么都没说时把 basic_info 填成 {"name": "用户"} /
"The user" / "N/A",每个占位符都 supersede 掉真实值,演化链变成
费天翔 → Fei → 用户 → The user。防御:(a) upsert 占位符守卫 + 读侧愈合;
(c) prompt 硬约束。验收 2(assistant 引用 "name is 用户")由占位符守卫单独兜住。
本文件覆盖改进建议第 10 节验收 1–4。
"""
import asyncio
from datetime import datetime, timedelta
import pytest
from hy_memory.agent.extractor import EXTRACT_FEW_SHOT_EN
from hy_memory.agent.prompts_zh import EXTRACT_FEW_SHOT_ZH
from hy_memory.agent.tools.basic_profile import (
_assemble_full_kv_from_nodes,
is_placeholder_value,
upsert_basic_profile,
)
from hy_memory.models.memory import MemoryLayer, MemoryNode, MemoryStatus
FIELDS = ["name", "age", "location", "occupation", "employer"]
class FakeStore:
def __init__(self, nodes=None):
self.nodes = list(nodes or [])
async def list_by_user(self, **kwargs):
return [n for n in self.nodes if n.user_id == kwargs.get("user_id")]
async def get_by_id(self, node_id):
return next((n for n in self.nodes if n.node_id == node_id), None)
async def upsert(self, node):
self.nodes.append(node)
async def update_payload(self, node_id, updates):
node = await self.get_by_id(node_id)
for key, value in updates.items():
if key == "status" and isinstance(value, str):
value = MemoryStatus(value)
setattr(node, key, value)
class FakeEmbed:
async def embed_queued(self, text):
return [0.1] * 8
def l0_versions(store):
return [n for n in store.nodes if n.layer == MemoryLayer.L0_BASIC_INFO]
def write_basic_info(store, llm_basic_info):
"""模拟 writer 的 basic_info 段:LLM 给出的非空 basic_info 直接 upsert。"""
if not llm_basic_info:
return None
return asyncio.run(upsert_basic_profile(
user_id="u1", kv=llm_basic_info, vector_store=store,
embed_service=FakeEmbed(), allowed_fields=FIELDS,
))
# ── 验收 1:寒暄不产生新的 L0 版本 ──
def test_small_talk_creates_no_l0_version():
# [user]: 你好 / [assistant]: 我可以帮你。弱模型输出 {"name": "用户"}
store = FakeStore()
write_basic_info(store, {"name": "用户"})
write_basic_info(store, {})
assert l0_versions(store) == []
def test_placeholder_only_upsert_writes_nothing():
store = FakeStore()
for kv in ({"name": "用户"}, {"name": "The user"}, {"name": "N/A"},
{"location": "unknown", "occupation": "未提供"}, {"name": "name"}):
res = write_basic_info(store, kv)
assert res.success and not res.changed
assert l0_versions(store) == []
# ── 验收 2:assistant 引用 "name is 用户" 不写入(仅靠占位符守卫)──
def test_assistant_quoting_placeholder_does_not_write():
# [assistant]: 那条 The user's name is 用户. 更说明问题
store = FakeStore()
for kv in ({"name": "用户"}, {"name": "The user"}, {"name": "The user."}):
res = write_basic_info(store, kv)
assert res.success and not res.changed
assert l0_versions(store) == []
# ── 验收 3:用户自报姓名 ──
def test_user_states_name():
# [user]: 我叫费天翔,常住杭州,做 AI 基础设施。
store = FakeStore()
res = write_basic_info(store, {
"name": "费天翔", "location": "杭州", "occupation": "AI 基础设施从业者",
})
assert res.success and res.created
assert l0_versions(store)[0].custom["basic_info_kv"] == {
"name": "费天翔", "location": "杭州", "occupation": "AI 基础设施从业者",
}
# ── 验收 4:已有真实值不被后续占位符覆盖 ──
def test_existing_name_not_overwritten_by_placeholder():
store = FakeStore()
write_basic_info(store, {"name": "费天翔"})
assert len(l0_versions(store)) == 1
for kv in ({"name": "用户"}, {"name": "The user."}, {"name": "N/A"}):
res = write_basic_info(store, kv)
assert not res.changed
assert len(l0_versions(store)) == 1
assert _assemble_full_kv_from_nodes(list(store.nodes)) == {"name": "费天翔"}
def test_mixed_kv_keeps_real_fields_and_drops_placeholders():
store = FakeStore()
res = write_basic_info(store, {"name": "用户", "location": "杭州"})
assert res.diff_kv == {"location": "杭州"}
def test_already_poisoned_chain_heals_on_read():
now = datetime.utcnow()
def node(nid, kv, days_ago):
return MemoryNode(
node_id=nid, user_id="u1", agent_id="",
layer=MemoryLayer.L0_BASIC_INFO, content="profile",
custom={"basic_info_kv": kv},
gmt_created=now - timedelta(days=days_ago),
)
chain = [
node("a", {"name": "费天翔"}, 4), node("b", {"name": "Fei"}, 3),
node("c", {"name": "用户"}, 2), node("d", {"name": "The user"}, 1),
]
assert _assemble_full_kv_from_nodes(chain) == {"name": "Fei"}
@pytest.mark.parametrize("value", [
"", " ", "用户", "用户。", "The user", "the user.", "THE USER", "user",
"The user's", "n/a", "N/A", "na", "None", "null", "unknown", "未提供",
"未知", "不详", "-", "?", "name", " Name ", "姓名",
])
def test_placeholder_values(value):
assert is_placeholder_value("name", value)
@pytest.mark.parametrize("field,value", [
("name", "费天翔"), ("name", "Fei"), ("name", "Username42"),
("age", 30), ("age", 0), ("location", "杭州"), ("employer", "Tencent"),
])
def test_real_values_are_not_placeholders(field, value):
assert not is_placeholder_value(field, value)
# ── (c) prompt 硬约束 ──
def test_prompts_carry_hard_rule_and_small_talk_counter_example():
from hy_memory.agent import extractor, prompts_zh
en_rules = [v for v in vars(extractor).values()
if isinstance(v, str) and "HARD RULE: fill a field ONLY" in v]
zh_rules = [v for v in vars(prompts_zh).values()
if isinstance(v, str) and "**硬性规则**" in v]
assert len(en_rules) >= 2 and len(zh_rules) >= 2
assert "small talk has no basic_info" in EXTRACT_FEW_SHOT_EN
assert "寒暄没有 basic_info" in EXTRACT_FEW_SHOT_ZH