fix(sdk): placeholder values never overwrite the basic profile
A weak extraction model filled basic_info with "用户", "The user" or "N/A" when the user stated nothing, and each placeholder superseded the real L0 value. upsert_basic_profile now drops placeholder values and writes no new version when nothing is left; profile assembly skips stored placeholders. The writer keeps a basic_info field only when its value appears in a user message of the request. The zh and en extraction prompts add a hard rule and a small-talk counter-example. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
988667cd87
commit
78882f3300
5 files changed
+489
-1
No files matched your search
@@ -682,6 +682,23 @@ Output:
|
||||
```
|
||||
John's occupation and location are HIS attributes, so they go into `memory` with "John" as the subject. They must NEVER land in `basic_info.occupation` / `basic_info.location` — `basic_info` is exclusively the user's own profile. Only `name` (Tim, from the user message's `name` field) belongs there.
|
||||
|
||||
## Example 11 — Counter-example: small talk has no basic_info
|
||||
|
||||
New Messages:
|
||||
[{"role": "user", "content": "Hi"},
|
||||
{"role": "assistant", "content": "Hello! The user's name is The user. How can I help?"}]
|
||||
Memory Date: 2025-09-03
|
||||
|
||||
Output:
|
||||
```json
|
||||
{
|
||||
"memory": [],
|
||||
"intentions": [],
|
||||
"basic_info": {}
|
||||
}
|
||||
```
|
||||
The user stated nothing about themselves. The assistant's line is not a user turn, and "The user" is a placeholder, not a name. Output `{}` — never `{"name": "The user"}` or `{"name": "user"}`.
|
||||
|
||||
---
|
||||
"""
|
||||
|
||||
@@ -749,6 +766,14 @@ Rules:
|
||||
fields (e.g. name, age, location, occupation, employer) **explicitly stated**
|
||||
in the NEW user messages. Never infer, never invent. If no field is explicitly
|
||||
stated, output `{{}}`.
|
||||
* HARD RULE: fill a field ONLY if the user stated it in a `user` turn.
|
||||
Anything said only by the assistant (including the assistant quoting an
|
||||
earlier memory such as "The user's name is X") does NOT count. If nothing
|
||||
qualifies, output `"basic_info": {{}}`.
|
||||
* NEVER write the field name itself or a placeholder as a value ("user",
|
||||
"The user", "用户", "name", "N/A", "unknown", "none", "-"). Omit the field.
|
||||
* Negative example: small talk such as `[user]: Hi` / `[assistant]: How can
|
||||
I help?`, or `[user]: Reply with exactly: PONG` → `"basic_info": {{}}`.
|
||||
* Exception for `name`: the `name` field on the user's own message, or a leading
|
||||
line "The user's name is X" / "用户名叫X", is an authoritative source for
|
||||
`name` — even though the user never said it out loud.
|
||||
@@ -817,6 +842,14 @@ Rules:
|
||||
fields (e.g. name, age, location, occupation, employer) **explicitly stated**
|
||||
in the NEW user messages. Never infer, never invent. If no field is explicitly
|
||||
stated, output `{{}}`.
|
||||
* HARD RULE: fill a field ONLY if the user stated it in a `user` turn.
|
||||
Anything said only by the assistant (including the assistant quoting an
|
||||
earlier memory such as "The user's name is X") does NOT count. If nothing
|
||||
qualifies, output `"basic_info": {{}}`.
|
||||
* NEVER write the field name itself or a placeholder as a value ("user",
|
||||
"The user", "用户", "name", "N/A", "unknown", "none", "-"). Omit the field.
|
||||
* Negative example: small talk such as `[user]: Hi` / `[assistant]: How can
|
||||
I help?`, or `[user]: Reply with exactly: PONG` → `"basic_info": {{}}`.
|
||||
* Exception for `name`: the `name` field on the user's own message, or a leading
|
||||
line "The user's name is X" / "用户名叫X", is an authoritative source for
|
||||
`name` — even though the user never said it out loud.
|
||||
|
||||
@@ -365,6 +365,13 @@ EXTRACT_PROMPT_ZH = """## 最近 k 条消息
|
||||
* basic_info:**只装用户本人的画像**。
|
||||
仅当 NEW user messages 中**明确**陈述稳定画像字段(如 姓名、年龄、所在地、职业、雇主)时才填写。
|
||||
绝不推断、绝不编造。没有任何字段被明确陈述时输出 `{{}}`。
|
||||
* **硬性规则**:只有用户在 `user` 回合里**明确说出**的字段才能填写。只在
|
||||
assistant 回合出现的内容(包括 assistant 引用旧记忆,如"The user's name is X"
|
||||
/ "用户名是X")一律不算。没有符合条件的字段时,必须输出 `"basic_info": {{}}`。
|
||||
* **绝不**把字段名本身或占位符当值写入("用户"、"The user"、"user"、"name"、
|
||||
"姓名"、"N/A"、"unknown"、"未知"、"未提供"、"-"),宁可省略该字段。
|
||||
* 负例:寒暄如 `[user]: 你好` / `[assistant]: 我可以帮你。`,或
|
||||
`[user]: Reply with exactly: PONG` → `"basic_info": {{}}`。
|
||||
* `name` 的例外:用户自己那条消息上的 `name` 字段,或文首形如
|
||||
"用户名叫X" / "The user's name is X" 的那一行,都是 `name` 的权威来源
|
||||
——即使用户本人从未开口自报姓名。
|
||||
@@ -440,6 +447,13 @@ EXTRACT_PROMPT_ZH_NO_INTENTION = """## 最近 k 条消息
|
||||
* basic_info:**只装用户本人的画像**。
|
||||
仅当 NEW user messages 中**明确**陈述稳定画像字段(如 姓名、年龄、所在地、职业、雇主)时才填写。
|
||||
绝不推断、绝不编造。没有任何字段被明确陈述时输出 `{{}}`。
|
||||
* **硬性规则**:只有用户在 `user` 回合里**明确说出**的字段才能填写。只在
|
||||
assistant 回合出现的内容(包括 assistant 引用旧记忆,如"The user's name is X"
|
||||
/ "用户名是X")一律不算。没有符合条件的字段时,必须输出 `"basic_info": {{}}`。
|
||||
* **绝不**把字段名本身或占位符当值写入("用户"、"The user"、"user"、"name"、
|
||||
"姓名"、"N/A"、"unknown"、"未知"、"未提供"、"-"),宁可省略该字段。
|
||||
* 负例:寒暄如 `[user]: 你好` / `[assistant]: 我可以帮你。`,或
|
||||
`[user]: Reply with exactly: PONG` → `"basic_info": {{}}`。
|
||||
* `name` 的例外:用户自己那条消息上的 `name` 字段,或文首形如
|
||||
"用户名叫X" / "The user's name is X" 的那一行,都是 `name` 的权威来源
|
||||
——即使用户本人从未开口自报姓名。
|
||||
@@ -669,6 +683,23 @@ Memory Date:2025-09-02
|
||||
```
|
||||
职业和所在地是 **John 的**属性,所以进 `memory` 并以"John"作主体,**绝不能**落到 `basic_info` 的 `occupation` / `location` —— `basic_info` 专属于用户本人。只有 `name`(Tim,来自用户消息的 `name` 字段)才该放进去。注意输出语言随输入语言(此例为英文)。
|
||||
|
||||
## 示例 11 —— 反例:寒暄没有 basic_info
|
||||
|
||||
新消息:
|
||||
[{"role": "user", "content": "你好"},
|
||||
{"role": "assistant", "content": "你好!我可以帮你。之前记得 The user's name is 用户."}]
|
||||
Memory Date:2025-09-03
|
||||
|
||||
输出:
|
||||
```json
|
||||
{
|
||||
"memory": [],
|
||||
"intentions": [],
|
||||
"basic_info": {}
|
||||
}
|
||||
```
|
||||
用户没有说任何关于自己的信息。assistant 那句不是 user 回合,"用户" 也是占位符而不是名字。必须输出 `{}`——**绝不能**输出 `{"name": "用户"}`。
|
||||
|
||||
---
|
||||
"""
|
||||
|
||||
|
||||
@@ -26,6 +26,8 @@ from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
import re
|
||||
import unicodedata
|
||||
import uuid
|
||||
import weakref
|
||||
from dataclasses import dataclass
|
||||
@@ -208,6 +210,147 @@ def _sanitize_kv(
|
||||
return result
|
||||
|
||||
|
||||
# ================================================================
|
||||
# 占位符守卫(P0-1)
|
||||
# ================================================================
|
||||
# 弱模型(如 qwen3.5-2b)在用户什么都没说时,会把 schema 字段说明里的词当值
|
||||
# 填进 basic_info:{"name": "用户"} / "The user" / "N/A"。每个占位符都会
|
||||
# supersede 掉真实值,演化链变成 费天翔 → Fei → 用户 → The user。
|
||||
# 规则:占位符永远不入库;读侧组装 KV 时占位符也不覆盖已有真实值。
|
||||
|
||||
_PLACEHOLDER_VALUES = frozenset({
|
||||
"用户", "该用户", "本用户", "用户本人",
|
||||
"the user", "the user's", "user", "a user", "this user",
|
||||
"n/a", "na", "none", "null", "nil", "unknown", "unspecified",
|
||||
"not provided", "not specified", "not mentioned",
|
||||
"未提供", "未知", "不详", "未说明", "未提及", "无",
|
||||
"-", "?",
|
||||
})
|
||||
|
||||
# 内置字段的中文标签:弱模型也会把字段说明的中文词当值。
|
||||
_FIELD_LABELS: Dict[str, tuple] = {
|
||||
"name": ("姓名", "名字", "用户名", "用户姓名", "用户名字"),
|
||||
"age": ("年龄", "用户年龄"),
|
||||
"location": ("所在地", "地点", "城市", "用户所在地"),
|
||||
"occupation": ("职业", "工作", "用户职业"),
|
||||
"employer": ("雇主", "公司", "用户雇主"),
|
||||
}
|
||||
|
||||
# 去掉标点/符号后再比较,让 "The user." / "用户。" / "N/A" 都命中。
|
||||
# "-" / "?" 本身就是纯标点,剥完为空串,同样判为占位符。
|
||||
_PUNCT_RE = re.compile(r"[\s\.,;:!?'\"`~()\[\]{}<>/\\|_\-\u3000-\u303f\uff00-\uff0f\uff1a-\uff20]+")
|
||||
|
||||
|
||||
def _placeholder_key(value: str) -> str:
|
||||
s = unicodedata.normalize("NFKC", str(value or "")).strip().lower()
|
||||
s = s.replace("\u2019", "'")
|
||||
s = _PUNCT_RE.sub(" ", s)
|
||||
return " ".join(s.split())
|
||||
|
||||
|
||||
_PLACEHOLDER_KEYS = frozenset(_placeholder_key(v) for v in _PLACEHOLDER_VALUES)
|
||||
|
||||
|
||||
def is_placeholder_value(field_name: str, value: Any) -> bool:
|
||||
"""value 是否为占位符(空 / 用户 / The user / N/A / 字段名本身 等)。
|
||||
|
||||
大小写不敏感、首尾空白与标点不敏感。数字(如 age=30)永远不是占位符。
|
||||
"""
|
||||
if value is None:
|
||||
return True
|
||||
if isinstance(value, bool):
|
||||
return False
|
||||
if isinstance(value, (int, float)):
|
||||
return False
|
||||
key = _placeholder_key(value)
|
||||
if not key:
|
||||
return True
|
||||
if key in _PLACEHOLDER_KEYS:
|
||||
return True
|
||||
field_key = _placeholder_key(field_name)
|
||||
if field_key and key in (field_key, field_key.replace("_", " ")):
|
||||
return True
|
||||
for label in _FIELD_LABELS.get(field_name, ()):
|
||||
if key == _placeholder_key(label):
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def _drop_placeholders(kv: Dict[str, Any]) -> Dict[str, Any]:
|
||||
return {k: v for k, v in kv.items() if not is_placeholder_value(k, v)}
|
||||
|
||||
|
||||
# ================================================================
|
||||
# 来源守卫(P0-1):basic_info 只认 user 回合
|
||||
# ================================================================
|
||||
# 抽取器偶尔把 assistant 回合里的内容(包括 assistant 引用的旧占位记忆
|
||||
# "The user's name is 用户.")当成用户画像再写一次。writer 把 user 回合原文
|
||||
# 交给 filter_basic_info_by_user_text(),值不在 user 原文里的字段一律丢弃。
|
||||
#
|
||||
# 匹配规则(大小写、空白不敏感,NFKC 归一):
|
||||
# - 严格字段(name / age / location / employer):值必须作为子串出现在 user
|
||||
# 原文里。"我叫费天翔" → name=费天翔 通过;"Fei Tianxiang" 与 "fei tianxiang"
|
||||
# 通过;"杭州" 写成 "Hangzhou" 则丢弃(宁缺毋滥:身份字段错一次就会
|
||||
# supersede 掉真实值)。
|
||||
# - 其余字段(occupation 及业务自定义字段):允许模型轻度改写。只要值与 user
|
||||
# 原文的最长公共子串 ≥ 2 个字符且覆盖值长度的一半以上即通过,例如
|
||||
# "AI 基础设施从业者" vs "做 AI 基础设施"(公共子串 "ai基础设施" 覆盖 6/9)。
|
||||
# 完全凭空的值(与 user 原文无公共片段)仍被丢弃。
|
||||
|
||||
STRICT_SOURCE_FIELDS = frozenset({"name", "age", "location", "employer"})
|
||||
|
||||
|
||||
def _match_key(text: Any) -> str:
|
||||
s = unicodedata.normalize("NFKC", str(text if text is not None else "")).lower()
|
||||
s = s.replace("\u2019", "'")
|
||||
return "".join(s.split())
|
||||
|
||||
|
||||
def _longest_common_substring(a: str, b: str) -> int:
|
||||
if not a or not b:
|
||||
return 0
|
||||
best = 0
|
||||
prev = [0] * (len(b) + 1)
|
||||
for ca in a:
|
||||
cur = [0] * (len(b) + 1)
|
||||
for j, cb in enumerate(b, 1):
|
||||
if ca == cb:
|
||||
cur[j] = prev[j - 1] + 1
|
||||
if cur[j] > best:
|
||||
best = cur[j]
|
||||
prev = cur
|
||||
return best
|
||||
|
||||
|
||||
def value_in_user_text(field_name: str, value: Any, user_text: str) -> bool:
|
||||
"""value 是否出自 user 原文(规则见上方注释)。"""
|
||||
v = _match_key(value)
|
||||
t = _match_key(user_text)
|
||||
if not v or not t:
|
||||
return False
|
||||
if v in t:
|
||||
return True
|
||||
if field_name in STRICT_SOURCE_FIELDS:
|
||||
return False
|
||||
common = _longest_common_substring(v, t)
|
||||
return common >= 2 and common * 2 >= len(v)
|
||||
|
||||
|
||||
def filter_basic_info_by_user_text(
|
||||
kv: Dict[str, Any],
|
||||
user_text: Optional[str],
|
||||
) -> Dict[str, Any]:
|
||||
"""只保留值出现在 user 原文里的字段;user_text 为空时整份丢弃。"""
|
||||
if not isinstance(kv, dict) or not kv:
|
||||
return {}
|
||||
if not (user_text or "").strip():
|
||||
return {}
|
||||
return {
|
||||
k: v for k, v in kv.items()
|
||||
if value_in_user_text(k, v, user_text or "")
|
||||
}
|
||||
|
||||
|
||||
# ================================================================
|
||||
# 公共入口:upsert_basic_profile
|
||||
# ================================================================
|
||||
@@ -287,6 +430,8 @@ async def _upsert_basic_profile_unlocked(
|
||||
# L0 Basic Info 从输入、唯一性到落盘都只属于 user_id。
|
||||
|
||||
new_kv = _sanitize_kv(kv or {}, allowed_fields=allowed_fields)
|
||||
# 单调性守卫:占位符永远不覆盖已有值,过滤后为空就不写新版本、不产生新链头。
|
||||
new_kv = _drop_placeholders(new_kv)
|
||||
if not new_kv:
|
||||
return BasicProfileUpsertResult(
|
||||
success=True, changed=False, reason="no valid fields after sanitize"
|
||||
@@ -513,5 +658,8 @@ def _assemble_full_kv_from_nodes(
|
||||
for k, v in kv.items():
|
||||
if allowed_set is not None and k not in allowed_set:
|
||||
continue
|
||||
# 历史上已落盘的占位符版本(如 name=用户)不覆盖更早的真实值。
|
||||
if is_placeholder_value(k, v):
|
||||
continue
|
||||
full_kv[k] = v
|
||||
return full_kv
|
||||
@@ -41,7 +41,10 @@ from ..core.workload import (
|
||||
)
|
||||
from ..agent.mem_agent import MemAgent, ProcessMode
|
||||
from ..agent.reconciler import MemoryReconciler
|
||||
from ..agent.tools.basic_profile import upsert_basic_profile
|
||||
from ..agent.tools.basic_profile import (
|
||||
filter_basic_info_by_user_text,
|
||||
upsert_basic_profile,
|
||||
)
|
||||
from ..models.memory import MemoryNode, MemoryLayer, MemoryStatus, SourceType
|
||||
from ..data.vector_store import create_vector_store
|
||||
from ..data.vector_store_base import VectorStoreBase
|
||||
@@ -3305,6 +3308,20 @@ class MemoryWriter(WritePipeline):
|
||||
tool_calls_raw = None # v0.1.5.13+ 不再有真实 LLM tool_calls;保留字段做兼容
|
||||
tool_results_summary: List[Dict[str, Any]] = []
|
||||
|
||||
# 来源守卫:basic_info 只认 user 回合。值不在任何 user 消息里的字段
|
||||
# (例如 assistant 引用旧占位记忆 "The user's name is 用户.")一律丢弃;
|
||||
# 有 messages 但没有 user 消息时整份丢弃。纯文本 add(无 messages)的
|
||||
# content 即调用方给出的用户原文,按 user 回合处理。
|
||||
if isinstance(basic_info_raw, dict) and basic_info_raw:
|
||||
_user_text = self._basic_info_user_text(request)
|
||||
_kept = filter_basic_info_by_user_text(basic_info_raw, _user_text)
|
||||
if len(_kept) != len(basic_info_raw):
|
||||
logger.info(
|
||||
f"[write] basic_info dropped fields not stated in user turns: "
|
||||
f"{sorted(set(basic_info_raw) - set(_kept))}"
|
||||
)
|
||||
basic_info_raw = _kept
|
||||
|
||||
if isinstance(basic_info_raw, dict) and basic_info_raw:
|
||||
try:
|
||||
_bp_result = await upsert_basic_profile(
|
||||
@@ -3580,6 +3597,27 @@ class MemoryWriter(WritePipeline):
|
||||
"llm_total_tokens": getattr(_er, "tokens_used", 0) or 0,
|
||||
}
|
||||
|
||||
@staticmethod
|
||||
def _basic_info_user_text(request) -> str:
|
||||
"""basic_info 来源校验用的 user 原文。
|
||||
|
||||
- 有 messages:只取 role=user 的 content 与 user 自带的 ``name``(name 是
|
||||
basic_info.name 的权威来源);没有 user 消息时返回空串(整份丢弃)。
|
||||
- 无 messages(纯文本 add):content 本身就是用户原文。
|
||||
"""
|
||||
messages = getattr(request, "messages", None) or []
|
||||
if not messages:
|
||||
return str(getattr(request, "content", "") or "")
|
||||
parts: List[str] = []
|
||||
for m in messages:
|
||||
if getattr(m, "role", None) != "user":
|
||||
continue
|
||||
name = getattr(m, "name", None)
|
||||
if name:
|
||||
parts.append(str(name))
|
||||
parts.append(str(getattr(m, "content", "") or ""))
|
||||
return "\n".join(parts)
|
||||
|
||||
@staticmethod
|
||||
def _mark_agent_failure(
|
||||
response: WriteResponse,
|
||||
|
||||
@@ -0,0 +1,238 @@
|
||||
"""P0-1:占位符不得污染 L0 基础画像。
|
||||
|
||||
弱模型(qwen3.5-2b)在用户什么都没说时把 basic_info 填成 {"name": "用户"} /
|
||||
"The user" / "N/A",每个占位符都 supersede 掉真实值,演化链变成
|
||||
费天翔 → Fei → 用户 → The user。三层防御:
|
||||
(a) upsert 单调性守卫;(b) writer 只认 user 回合;(c) prompt 硬约束。
|
||||
本文件锁住 (a)(b) 与 (c) 的 prompt 文案,覆盖改进建议第 10 节验收 1–4。
|
||||
"""
|
||||
import asyncio
|
||||
from datetime import datetime, timedelta
|
||||
|
||||
import pytest
|
||||
|
||||
from hy_memory.agent.extractor import EXTRACT_FEW_SHOT_EN
|
||||
from hy_memory.agent.prompts_zh import EXTRACT_FEW_SHOT_ZH
|
||||
from hy_memory.agent.tools.basic_profile import (
|
||||
_assemble_full_kv_from_nodes,
|
||||
filter_basic_info_by_user_text,
|
||||
is_placeholder_value,
|
||||
upsert_basic_profile,
|
||||
value_in_user_text,
|
||||
)
|
||||
from hy_memory.models.memory import MemoryLayer, MemoryNode, MemoryStatus
|
||||
from hy_memory.pipelines.base import ChatMessage, WriteRequest
|
||||
from hy_memory.pipelines.writer import MemoryWriter
|
||||
|
||||
FIELDS = ["name", "age", "location", "occupation", "employer"]
|
||||
|
||||
|
||||
class FakeStore:
|
||||
def __init__(self, nodes=None):
|
||||
self.nodes = list(nodes or [])
|
||||
|
||||
async def list_by_user(self, **kwargs):
|
||||
return [n for n in self.nodes if n.user_id == kwargs.get("user_id")]
|
||||
|
||||
async def get_by_id(self, node_id):
|
||||
return next((n for n in self.nodes if n.node_id == node_id), None)
|
||||
|
||||
async def upsert(self, node):
|
||||
self.nodes.append(node)
|
||||
|
||||
async def update_payload(self, node_id, updates):
|
||||
node = await self.get_by_id(node_id)
|
||||
for key, value in updates.items():
|
||||
if key == "status" and isinstance(value, str):
|
||||
value = MemoryStatus(value)
|
||||
setattr(node, key, value)
|
||||
|
||||
|
||||
class FakeEmbed:
|
||||
async def embed_queued(self, text):
|
||||
return [0.1] * 8
|
||||
|
||||
|
||||
def l0_versions(store):
|
||||
return [n for n in store.nodes if n.layer == MemoryLayer.L0_BASIC_INFO]
|
||||
|
||||
|
||||
def write_basic_info(store, request, llm_basic_info):
|
||||
"""模拟 writer 的 basic_info 段:来源守卫 → upsert。"""
|
||||
user_text = MemoryWriter._basic_info_user_text(request)
|
||||
kv = filter_basic_info_by_user_text(llm_basic_info, user_text)
|
||||
if not kv:
|
||||
return None
|
||||
return asyncio.run(upsert_basic_profile(
|
||||
user_id=request.user_id, kv=kv, vector_store=store,
|
||||
embed_service=FakeEmbed(), allowed_fields=FIELDS,
|
||||
))
|
||||
|
||||
|
||||
def req(*msgs, content=""):
|
||||
return WriteRequest(
|
||||
content=content,
|
||||
messages=[ChatMessage(role=r, content=c) for r, c in msgs],
|
||||
user_id="u1",
|
||||
)
|
||||
|
||||
|
||||
# ── 验收 1:寒暄不产生新的 L0 版本 ──
|
||||
|
||||
def test_small_talk_creates_no_l0_version():
|
||||
store = FakeStore()
|
||||
r = req(("user", "你好"), ("assistant", "我可以帮你。"))
|
||||
write_basic_info(store, r, {"name": "用户"})
|
||||
write_basic_info(store, r, {})
|
||||
assert l0_versions(store) == []
|
||||
|
||||
|
||||
def test_placeholder_only_upsert_writes_nothing():
|
||||
store = FakeStore()
|
||||
for kv in ({"name": "用户"}, {"name": "The user"}, {"name": "N/A"},
|
||||
{"location": "unknown", "occupation": "未提供"}, {"name": "name"}):
|
||||
res = asyncio.run(upsert_basic_profile(
|
||||
user_id="u1", kv=kv, vector_store=store,
|
||||
embed_service=FakeEmbed(), allowed_fields=FIELDS,
|
||||
))
|
||||
assert res.success and not res.changed
|
||||
assert l0_versions(store) == []
|
||||
|
||||
|
||||
# ── 验收 2:assistant 引用 "name is 用户" 不写入 ──
|
||||
|
||||
def test_assistant_quoting_placeholder_does_not_write():
|
||||
store = FakeStore()
|
||||
r = req(
|
||||
("user", "你看那条记忆"),
|
||||
("assistant", "那条 The user's name is 用户. 更说明问题"),
|
||||
)
|
||||
write_basic_info(store, r, {"name": "用户"})
|
||||
assert l0_versions(store) == []
|
||||
|
||||
|
||||
def test_value_only_in_assistant_turn_is_dropped():
|
||||
r = req(("user", "帮我查下天气"), ("assistant", "好的,费天翔,杭州今天晴。"))
|
||||
text = MemoryWriter._basic_info_user_text(r)
|
||||
assert filter_basic_info_by_user_text(
|
||||
{"name": "费天翔", "location": "杭州"}, text
|
||||
) == {}
|
||||
|
||||
|
||||
def test_no_user_messages_drops_basic_info():
|
||||
r = req(("assistant", "我叫费天翔,常住杭州"))
|
||||
assert MemoryWriter._basic_info_user_text(r) == ""
|
||||
assert filter_basic_info_by_user_text({"name": "费天翔"}, "") == {}
|
||||
|
||||
|
||||
# ── 验收 3:用户自报姓名 ──
|
||||
|
||||
def test_user_states_name():
|
||||
store = FakeStore()
|
||||
r = req(("user", "我叫费天翔,常住杭州,做 AI 基础设施。"))
|
||||
res = write_basic_info(store, r, {
|
||||
"name": "费天翔", "location": "杭州", "occupation": "AI 基础设施从业者",
|
||||
})
|
||||
assert res.success and res.created
|
||||
assert l0_versions(store)[0].custom["basic_info_kv"] == {
|
||||
"name": "费天翔", "location": "杭州", "occupation": "AI 基础设施从业者",
|
||||
}
|
||||
|
||||
|
||||
def test_user_message_name_attribute_counts_as_user_text():
|
||||
r = WriteRequest(
|
||||
messages=[ChatMessage(role="user", name="Tim", content="What's up?")],
|
||||
user_id="u1",
|
||||
)
|
||||
text = MemoryWriter._basic_info_user_text(r)
|
||||
assert filter_basic_info_by_user_text({"name": "Tim"}, text) == {"name": "Tim"}
|
||||
|
||||
|
||||
def test_plain_text_add_is_user_text():
|
||||
r = WriteRequest(content="I'm Fei Tianxiang, 35, living in Hangzhou.", user_id="u1")
|
||||
text = MemoryWriter._basic_info_user_text(r)
|
||||
assert filter_basic_info_by_user_text(
|
||||
{"name": "fei tianxiang", "age": 35, "location": "Hangzhou"}, text
|
||||
) == {"name": "fei tianxiang", "age": 35, "location": "Hangzhou"}
|
||||
|
||||
|
||||
def test_strict_fields_require_containment_loose_fields_allow_rewording():
|
||||
text = "我叫费天翔,常住杭州,做 AI 基础设施。"
|
||||
assert value_in_user_text("name", "费天翔", text)
|
||||
assert not value_in_user_text("location", "Hangzhou", text)
|
||||
assert value_in_user_text("occupation", "AI 基础设施从业者", text)
|
||||
assert not value_in_user_text("occupation", "软件工程师", text)
|
||||
|
||||
|
||||
# ── 验收 4:已有真实值不被后续占位符覆盖 ──
|
||||
|
||||
def test_existing_name_not_overwritten_by_placeholder():
|
||||
store = FakeStore()
|
||||
write_basic_info(store, req(("user", "我叫费天翔")), {"name": "费天翔"})
|
||||
assert len(l0_versions(store)) == 1
|
||||
for kv in ({"name": "用户"}, {"name": "The user."}, {"name": "N/A"}):
|
||||
res = asyncio.run(upsert_basic_profile(
|
||||
user_id="u1", kv=kv, vector_store=store,
|
||||
embed_service=FakeEmbed(), allowed_fields=FIELDS,
|
||||
))
|
||||
assert not res.changed
|
||||
assert len(l0_versions(store)) == 1
|
||||
assert _assemble_full_kv_from_nodes(list(store.nodes)) == {"name": "费天翔"}
|
||||
|
||||
|
||||
def test_mixed_kv_keeps_real_fields_and_drops_placeholders():
|
||||
store = FakeStore()
|
||||
res = asyncio.run(upsert_basic_profile(
|
||||
user_id="u1", kv={"name": "用户", "location": "杭州"}, vector_store=store,
|
||||
embed_service=FakeEmbed(), allowed_fields=FIELDS,
|
||||
))
|
||||
assert res.diff_kv == {"location": "杭州"}
|
||||
|
||||
|
||||
def test_already_poisoned_chain_heals_on_read():
|
||||
now = datetime.utcnow()
|
||||
|
||||
def node(nid, kv, days_ago):
|
||||
return MemoryNode(
|
||||
node_id=nid, user_id="u1", agent_id="",
|
||||
layer=MemoryLayer.L0_BASIC_INFO, content="profile",
|
||||
custom={"basic_info_kv": kv},
|
||||
gmt_created=now - timedelta(days=days_ago),
|
||||
)
|
||||
|
||||
chain = [
|
||||
node("a", {"name": "费天翔"}, 4), node("b", {"name": "Fei"}, 3),
|
||||
node("c", {"name": "用户"}, 2), node("d", {"name": "The user"}, 1),
|
||||
]
|
||||
assert _assemble_full_kv_from_nodes(chain) == {"name": "Fei"}
|
||||
|
||||
|
||||
@pytest.mark.parametrize("value", [
|
||||
"", " ", "用户", "用户。", "The user", "the user.", "THE USER", "user",
|
||||
"The user's", "n/a", "N/A", "na", "None", "null", "unknown", "未提供",
|
||||
"未知", "不详", "-", "?", "name", " Name ", "姓名",
|
||||
])
|
||||
def test_placeholder_values(value):
|
||||
assert is_placeholder_value("name", value)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("field,value", [
|
||||
("name", "费天翔"), ("name", "Fei"), ("name", "Username42"),
|
||||
("age", 30), ("age", 0), ("location", "杭州"), ("employer", "Tencent"),
|
||||
])
|
||||
def test_real_values_are_not_placeholders(field, value):
|
||||
assert not is_placeholder_value(field, value)
|
||||
|
||||
|
||||
# ── (c) prompt 硬约束 ──
|
||||
|
||||
def test_prompts_carry_hard_rule_and_small_talk_counter_example():
|
||||
from hy_memory.agent import extractor, prompts_zh
|
||||
|
||||
en_rules = [v for v in vars(extractor).values()
|
||||
if isinstance(v, str) and "HARD RULE: fill a field ONLY" in v]
|
||||
zh_rules = [v for v in vars(prompts_zh).values()
|
||||
if isinstance(v, str) and "**硬性规则**" in v]
|
||||
assert len(en_rules) >= 2 and len(zh_rules) >= 2
|
||||
assert "small talk has no basic_info" in EXTRACT_FEW_SHOT_EN
|
||||
assert "寒暄没有 basic_info" in EXTRACT_FEW_SHOT_ZH
|
||||
Reference in new issue
Block a user