Commit Graph
2062 Commits
Author SHA1 Message Date
xu jiaming c187dfdc73 fix(mcp): read total_datasets so the fetch-all early-exit works (#19020)
Fix #19019 — MCP _fetch_all_datasets() used wrong total key (total vs total_datasets), causing extra empty-page requests. Now reads total_datasets with fallback.
2026-09-02 19:05:50 +08:00
buua436 cf441d4c72 fix: align wiki alteration provenance (#19140) 2026-09-02 18:03:15 +08:00
Wang Qi b4a3ff3c64 Fix: link to existing dataset clean the document chunks (#19143) 2026-09-02 16:00:37 +08:00
Wang Qi 61492c0362 Refactor: add total/returned entities/relations 2 (#19114)
Follow on #19095
2026-09-02 14:28:58 +08:00
Jin Hai 988dfda4c9 Fix: Use default pipeline configuration when parsing documents via HTTP API- #13297 (#19116)
Refer to #13297

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-09-02 06:12:17 +08:00
880876f60f fix(agentic_rag): table-aware retrieval truncation fixes + action-session loop guards (#19112)
Fixes a family of table-retrieval truncation bugs and an action-session
infinite loop in the agentic RAG medium path, all root-caused on a
tabular-reasoning question

1. action_session turn cap enforced (action_session.py)
_route checked _pending_calls before attempts, so a model that kept
emitting tool calls never hit the 4-turn finalize — a single session
could burn the whole pass budget (219 retrievals / 600s observed). Added
_route_after_tool: after a tool batch, the turn budget is checked and
the session converges to finalize instead of looping back to run_action.
2. Table chunks kept whole through retrieval (search.py,
text_processing.py, action_session.py)
Three truncation points hid mid-table answer rows from the session
model:
- grep_search narrowed table chunks to 700 chars
(_GREP_OUT_CHARS_PER_CHUNK)
- _narrow_content only passed HTML tables (<table>/<tr/<td) whole —
markdown pipe tables were sentence-window narrowed
- _admit_evidence truncated the model-facing output to 1200 chars even
though the pool held the full chunk,non-table chunks keep the existing
truncation paths unchanged.

3. SCA sees more evidence (agentic_rag_graph.py, sufficient_context.py)
_SCA_VIEW_CAP 24 → 60; _SCA_CLAIMS_CONTEXT_MAX 9000 → 48000;
table-bearing evidence anchors rendered in full (bounded by the context
budget).

4. Supporting: runtime instrumentation for rounds/slots/retrievals, LLM
token accounting, slot-evidence preservation through SCA, dedup
checkpoints. Timeout/budget constants restored to upstream defaults.

---------

Co-authored-by: Loong <213221738@seu.edu.cn>
Co-authored-by: Hermes Agent <hermes-agent@localhost>
2026-09-01 21:58:29 +08:00
MarMar Labs cd27ae4c6e fix(memory): put the keyword weight in the text slot of the fusion expression (#18989)
Fix memory search hybrid weights inverted (slot 1 is vector weight, not keyword). Now matches build_fusion_expr. At 0.7, memory search becomes keyword-dominant (was vector-dominant).
2026-09-01 16:14:36 +08:00
Wang Qi fe63e09781 Refactor: add total/returned entities/relations (#19095) 2026-09-01 16:01:59 +08:00
Martin Civáň 867e4187eb Fix don't send an empty tools array (#19015)
Fix non-streaming chat sending empty tools=[] (rejected by strict providers). Restored mdl.is_tools check and added helper to omit tools field when unbound.
2026-09-01 13:55:37 +08:00
euvre 2af732d607 fix(chat): images uploaded in chat UI were ignored by vision models (agentic loop / Go attachment drop) (#18884) 2026-08-31 19:35:02 +08:00
Martin Civáň 1757425903 fix: retrieve with the assistant's configured settings (#19016)
Fix agentic search ignoring chat retrieval settings — now passes knn_top_k, weights, threshold, and top_n. knn_top_k critical for HNSW recall on large indices.
2026-08-31 19:03:28 +08:00
MarMar Labs 5e1898e293 fix(dify): search the dataset owner's index, not the caller's (#18987)
Fix team-member Dify retrieval searching wrong tenant index. Changed tenant_id to kb.tenant_id in both retrieval_by_children and kg_retriever.
2026-08-31 17:11:40 +08:00
Josh Baptiste 5af3bf2f4b Fix: KnowledgebaseService.dfs_update crashes on stored None parser_config values (#15964)
Fix #15962 — replace bare assert isinstance in KnowledgebaseService.dfs_update with safe conditions. Stored None values no longer crash table-parser parses. Matches DocumentService fix from #12815.
2026-08-31 16:19:54 +08:00
georgelichenandChen Li f300b154b0 fix(chat): keep reasoning chats grounded when LLM lacks tool calling (#18983)
Fix reasoning + KB requests using models without tool-calling — now fall back to async_chat RAG path instead of answering from general knowledge. 

Co-authored-by: Chen Li <george_lichne@163.com>
2026-08-31 14:28:12 +08:00
b149a392f4 fix(api): return parsing-status counts at the top level of each dataset (#17595) (#17611)
Fix #17595 — parsing status counts now return as top-level fields per documentation, not nested under parsing_status.

Signed-off-by: Anai-Guo <antai12232931@anaiguo.com>
Signed-off-by: Anai-Guo <antai12232931@outlook.com>
Co-authored-by: Anai-Guo <antai12232931@anaiguo.com>
Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-08-31 13:52:08 +08:00
Yingfeng 862ee95df2 Refine agentic search (#18798) 2026-08-28 23:56:14 +08:00
Wang Qi 66113709cc Feature: add highlight for retrieval testing and search (#18981) 2026-08-28 19:26:41 +08:00
706e87af9f fix(admin): stop silent superuser creation on auth probe and harden bootstrap (#18702)
Fix CWE-798/306 admin security: check_admin() silently created default admin accounts on any anonymous request. Now no longer creates accounts. Bootstrap uses env/random password. Login throttling added. Admin port bound to localhost. 

---------

Co-authored-by: jindou <jindou@local>
Co-authored-by: maoyifeng <myfbee5@163.com>
2026-08-28 17:44:52 +08:00
abdabf883b fix(rag): key Infinity field_map by original column names, not pinyin (#18318)
Fixes #18287

Fix SQL retrieval field mapping: field_map used pinyin keys but chunk_data stored original names. Now keyed by original name on Infinity/OceanBase/SereneDB. Non-ASCII columns now work.

Co-authored-by: Taranum01 <Taranum01@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Taranum01 <50813317+Taranum01@users.noreply.github.com>
Co-authored-by: Taranum01 <taranumwasu@users.noreply.github.com>
2026-08-28 15:15:07 +08:00
天海蒼灆andCursor 9ca62e8da3 fix: include filename in file download Content-Disposition header (#17105)
### Summary

GET /api/v1/files/{id} now sets attachment filename for both Python and
Go handlers so browsers can save downloads with the correct name.

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-28 14:42:04 +08:00
Jin Hai 191d62a698 Py: add update_time in select list when list data sync logs (#18867)
Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-08-28 14:22:37 +08:00
MarMar Labs d132352edb fix(file): stop is_parent_folder_exist raising on a missing parent (#18930)
Fix POST /api/v1/files 500 error when parent_id doesn't exist. Removed dead delete_folder_by_pf_id call (mismatched args, no tenant scope). Now returns False and documented error response.
2026-08-28 13:52:35 +08:00
Jin Hai d37486e295 Python && Go: fix parse status (#18841)
Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-08-28 12:47:41 +08:00
MarMar Labs 439c3be9cb fix(dataset): rebuild the navigation tree in the dataset owner index (#18932)
Fix generate_nav tenant mismatch: wipe used dataset owner's tenant_id but rebuild used caller's. Team members could delete owner's navigation tree and rebuild into their own. Now both use kb.tenant_id.
2026-08-28 11:58:31 +08:00
ChendongPang 03d0ad2ace fix(api): derive SQL retrieval tenant from dataset owner, not chat tenant (#18518) 2026-08-28 09:57:12 +08:00
Wang Qi f74e04581a Refactor: add page_size support (#18876) 2026-08-28 09:37:44 +08:00
buua436 0411c52904 fix: honor generation parameter enable flags (#18880) 2026-08-27 20:22:55 +08:00
Lynn b224f338fa Fix: rm default model in create and display builtin model (#18875) 2026-08-27 19:00:05 +08:00
Wang Qi fbe1083d65 Fix rerank_candidates_count validation error (#18863) 2026-08-27 17:05:47 +08:00
Wang Qi fb5ee011d3 Fix: hide skills folder in REST API (#18859) 2026-08-27 16:21:10 +08:00
RocandJin Hai 8b75e86b5f fix: support .wmf file extension case-insensitively and remove duplicate avi (#15498)
### Summary

Fix `.wmf` file type detection by matching lowercased filename
normalization in both Python and Go extension checks.

This PR also removes a duplicate `avi` entry from the Python visual file
type regex.

---------

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-08-27 15:58:55 +08:00
SYED ALI ABBAS RAHIL 82e4147b83 fix(api): honor dialog system prompt in reasoning-enabled chat (#18842)
### Problem

When a chat assistant has a configured system prompt and the reasoning
level is set to anything other than `NONE`, the request goes through
`rag_agent()` / the agentic RAG harness. That path generated the final
answer with a fixed `FINAL_ANSWER_SYSTEM` prompt and never injected the
user-configured system prompt, so the assistant ignored
role/tone/language instructions (reported in #18833).
2026-08-27 14:53:06 +08:00
Wang Qi 68b893cd1a Refactor: Add rerank_candidates_count, knn top_k and num_candidates to /api/v1/retrieval (#18768) 2026-08-26 20:38:19 +08:00
Aeovy 5b3a66fd8c fix(mineru): Fix MinerU Rendering Stability, Error Visibility, and Code Block Parsing Issues (#18617) 2026-08-26 20:12:41 +08:00
Burak Bayırandkriptoburak 0201c1a557 feat(data-source): add Xquik X search connector (#18714)
### Summary

Add Xquik as a new data source for X (Twitter) post ingestion. Supports Latest/Top queries, cursor pagination, incremental sync, and maps engagement/media to RAGFlow documents. Stops duplicate requests on repeated cursors. Tests and docs included.

---------

Co-authored-by: kriptoburak <kriptoburak@users.noreply.github.com>
2026-08-26 17:03:38 +08:00
MarMar Labs d3542463c9 docs(api): spell the recommendation endpoint the way the server registers it (#18764)
### Summary

Update comments
2026-08-26 12:10:05 +08:00
Wang Qi fd7cd3289a Fix total as the total, follow Go (#18750) 2026-08-25 19:25:43 +08:00
Wang Qi 92c69db79e Refactor: expose knn top_k and knn num_candidaties (#18737)
Follow on #18300
2026-08-25 17:16:36 +08:00
Mpider-San aa57022bfd Fix: cross-KB document id collision in data source sync (#18116) (#18244)
### What problem does this PR solve?

Closes #18116.

Fix cross-KB document ID collision (#18116). Older ID derivation excluded kb_id, causing two KBs to fight over one row. Unified derivation to be KB-scoped, with fallback for existing rows. Stranded rows are reclaimed, dataset deletion now cleans up state, and upgraded instances update in place. Tests added.
2026-08-25 15:15:46 +08:00
buua436 3a836911d1 fix: support hierarchical Wiki topic navigation (#18721) 2026-08-25 13:59:27 +08:00
Paul Yao 647cddc3cc fix(admin): display AWS S3 status correctly (#17300)
Related to #17294
2026-08-25 12:24:24 +08:00
Ali Farhan 03d2835731 fix(api): make the API-key subject carve-out on POST /messages reachable (#18026)
### Summary

Fix add_message so API-key callers can pass user_id (previously ignored because Python checked a Go-only flag). Now uses g.auth_type == AUTH_API. JWT/session callers stay bound to the authenticated principal. Tests added; backward compatible.
2026-08-25 12:06:51 +08:00
Wang Qi 4c6f745c1e Refactor: rename prefetch_size to rerank_candidates (#18718) 2026-08-25 11:35:53 +08:00
Yash Raj PandeyandJin Hai 71413c9fc6 fix: classify .wmf files as VISUAL instead of OTHER (#15822)
Signed-off-by: Yash Raj Pandey <yashpn62@gmail.com>
Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-08-25 11:12:45 +08:00
He Wang 2ef9d373ee feat(bedrock): support API key authentication (#18301)
### Summary

Closes #17848.

This replaces #17958 with a smaller implementation focused on the
original requirement. The previous PR gradually expanded beyond the
necessary scope; the optional custom endpoint extension is intentionally
excluded from this version.

- Add request-scoped Bedrock API key authentication to the Python
Bedrock adapters and Go chat runtime.
- Discover and persist models available to API key instances, so users
do not need to manually enter the model type, model name, or maximum
token count.
- Add API Key mode to the existing Bedrock settings UI and refresh the
model and default-model lists after instance changes.
- Preserve selected models during credential-only updates while keeping
the existing SigV4 authentication modes unchanged.
- Return a clear error for unsupported API-key reranking.
- Document instance-scoped authentication and short-term API key
guidance.
2026-08-25 11:08:24 +08:00
Öndery 65d8301ea3 fix(models): pass selected models into verify and catch constructor errors (#17132)
Two fixes not yet on main:

Pass picker-selected models into Verify – useVerifyProvider now includes modelInfoRef in the verify payload so the backend can verify the user's actual selection.

Guard model construction – Wrap EmbeddingModel/ChatModel init in try/except so failures are recorded as per-model FAIL instead of aborting the whole verify call.

Also: later failures now overwrite previous successes per review feedback.
2026-08-25 11:07:08 +08:00
Bharadwaj PendyalaandJin Hai 89fc1abac9 fix(chat): stop rag_agent forwarding message bookkeeping keys to the provider (#18663)
### Summary

Fixes #18653. Groq rejects every chat request on v0.27.0 with `property
'conversationId' is unsupported`, while the same chat works on Ollama.

Closes #18653

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-08-25 10:36:27 +08:00
Sbaaoui Idriss bfe07974dc fix: normalize model types in _factory_model_types and update calculate_model_type (#18685) 2026-08-25 09:46:46 +08:00
Wang Qi 5659476a30 Fix: correct rerank #17296 - add prefetch_size (#18300) 2026-08-24 16:09:07 +08:00
陈志谦andjindou af4651cce3 security: remove leaked OAuth credentials and stop logging user secrets (#18658)
remove leaked OAuth credentials and stop logging user secrets

Co-authored-by: jindou <jindou@local>
2026-08-24 15:53:19 +08:00