Commit Graph
318 Commits
Author SHA1 Message Date
Wang Qi b7e577ce48 Refactor: combine 3 retrieval/search API as one (#19303) 2026-09-08 11:41:46 +08:00
Manan Bansal 03d861a926 fix(api): allow setting language on dataset creation (#15703) (#16242)
Fix #15703 — dataset create endpoint now accepts language.
2026-09-08 10:53:40 +08:00
Wang Qi f8b0a1de19 Refactor: update dataset API should not have ext (#19272) 2026-09-07 14:25:15 +08:00
Wang Qi 6ec9c6a737 Fix: get rid of graphrag raptor in parser_config in REST API (#19161) 2026-09-03 09:40:52 +08:00
xu jiaming c187dfdc73 fix(mcp): read total_datasets so the fetch-all early-exit works (#19020)
Fix #19019 — MCP _fetch_all_datasets() used wrong total key (total vs total_datasets), causing extra empty-page requests. Now reads total_datasets with fallback.
2026-09-02 19:05:50 +08:00
Wang Qi 66113709cc Feature: add highlight for retrieval testing and search (#18981) 2026-08-28 19:26:41 +08:00
706e87af9f fix(admin): stop silent superuser creation on auth probe and harden bootstrap (#18702)
Fix CWE-798/306 admin security: check_admin() silently created default admin accounts on any anonymous request. Now no longer creates accounts. Bootstrap uses env/random password. Login throttling added. Admin port bound to localhost. 

---------

Co-authored-by: jindou <jindou@local>
Co-authored-by: maoyifeng <myfbee5@163.com>
2026-08-28 17:44:52 +08:00
天海蒼灆andCursor 9ca62e8da3 fix: include filename in file download Content-Disposition header (#17105)
### Summary

GET /api/v1/files/{id} now sets attachment filename for both Python and
Go handlers so browsers can save downloads with the correct name.

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-28 14:42:04 +08:00
Wang Qi f74e04581a Refactor: add page_size support (#18876) 2026-08-28 09:37:44 +08:00
Wang Qi fbe1083d65 Fix rerank_candidates_count validation error (#18863) 2026-08-27 17:05:47 +08:00
RocandJin Hai 8b75e86b5f fix: support .wmf file extension case-insensitively and remove duplicate avi (#15498)
### Summary

Fix `.wmf` file type detection by matching lowercased filename
normalization in both Python and Go extension checks.

This PR also removes a duplicate `avi` entry from the Python visual file
type regex.

---------

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-08-27 15:58:55 +08:00
Burak Bayırandkriptoburak 0201c1a557 feat(data-source): add Xquik X search connector (#18714)
### Summary

Add Xquik as a new data source for X (Twitter) post ingestion. Supports Latest/Top queries, cursor pagination, incremental sync, and maps engagement/media to RAGFlow documents. Stops duplicate requests on repeated cursors. Tests and docs included.

---------

Co-authored-by: kriptoburak <kriptoburak@users.noreply.github.com>
2026-08-26 17:03:38 +08:00
Wang Qi 92c69db79e Refactor: expose knn top_k and knn num_candidaties (#18737)
Follow on #18300
2026-08-25 17:16:36 +08:00
Paul Yao 647cddc3cc fix(admin): display AWS S3 status correctly (#17300)
Related to #17294
2026-08-25 12:24:24 +08:00
Yash Raj PandeyandJin Hai 71413c9fc6 fix: classify .wmf files as VISUAL instead of OTHER (#15822)
Signed-off-by: Yash Raj Pandey <yashpn62@gmail.com>
Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-08-25 11:12:45 +08:00
Sbaaoui Idriss bfe07974dc fix: normalize model types in _factory_model_types and update calculate_model_type (#18685) 2026-08-25 09:46:46 +08:00
Wang Qi 76bff4c915 Fix: switch image/ppt/email/audio from General to pipeline, or switch back, it report error (#18555) 2026-08-20 17:34:51 +08:00
d82c48eaf1 fix: allow .bmp uploads in filename type whitelist (#17830)
## Summary
- Frontend `Images` already includes `bmp`
(`web/src/constants/common.ts`), and picture parsing already accepts
`.bmp` (`internal/parser/parser/picture_parser.go`, `rag/app/picture.py`
via Pillow).
- This PR adds `bmp` to both whitelist sites and a small Go unit test.


Co-authored-by: zhangjiangshan1 <zhangjiangshan1@kingsoft.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-19 15:04:33 +08:00
Wang Qi 86750bd24e Fix: some pdf with chinese name cannot preview (#18422) 2026-08-18 14:54:10 +08:00
zws202606 d02723ca76 feat: Gaussdb adaptation (#17703) 2026-08-18 09:53:11 +08:00
Wang Qi 57b907cd1e Refactor: do not include compliation chunks in retrieval testing (#18364)
Retrieval Testing: do not include compilation result.
Search: include compilation result.
2026-08-17 16:19:05 +08:00
Wang Qi bd876c796e Fix: restrict xxx_ids to limit 100 (#18074) 2026-08-11 13:28:52 +08:00
Charles 42373a8229 fix(api): map internal RetCode values to valid HTTP statuses in build_error_result (#18009)
### Summary

Fixes #17980.
2026-08-10 11:30:11 +08:00
deadtrickster 8c8c1e7df7 fix(api): default delimiter must be a real newline, not the r"\n" escape (#17591) 2026-08-04 16:16:34 +08:00
buua436 3e11914144 fix: normalize legacy parser configuration (#17761) 2026-08-04 13:49:44 +08:00
Jin Hai 86021932ae Go: fix warnings (#17738)
Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-08-03 21:30:01 +08:00
buua436 3e7cfbe052 refa: simplify RAPTOR tree clustering and configuration (#17614) 2026-08-03 17:46:50 +08:00
Wang Qi 4b619f699b Refactor: consolidate page and page_size (#17597) 2026-07-31 14:08:44 +08:00
Wang Qi 0969c04eca Feature: GET /datasets to support ids array (#17606) 2026-07-31 13:23:02 +08:00
Wang Qi 6e0e495929 Fix: upload document might be hung the whole ragflow (#17537) 2026-07-30 13:54:07 +08:00
Harsh Kashyap 404a5cc33b fix(admin,api): honor STORAGE_IMPL when surfacing file_store backend (closes #17294) (#17441)
## Summary

When `STORAGE_IMPL=AWS_S3`, the Admin Service status page keeps showing
MinIO. Three things conspire:

1. `admin/server/config.py::load_configurations` only knows the
   `minio` and `minio_0` config keys. An `s3` block lands on the
   `case _:` branch and logs `Unknown configuration key: s3`
   (issue #17294).
2. `admin/server/services.py::ServiceMgr.get_all_services` filters
   retrieval services by `DOC_ENGINE` but has no equivalent filter
   for `file_store` services by `STORAGE_IMPL`. A stale MinIO block
   is returned regardless of the active backend.
3. The wired health check is hardcoded to `check_minio_alive`, which
   calls `settings.MINIO['host']`. With `STORAGE_IMPL=AWS_S3` that
   block is uninitialized, so the check always times out.


Fixes #17294
2026-07-29 11:13:20 +08:00
Jin Hai 7f21a7ba18 Go: add context, part14 (#17446)
Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-07-28 19:05:59 +08:00
Wang Qi 4d772a9365 Fix update 'Child chunk are used for retrieval' does not take effect (#900) (#17251) 2026-07-22 19:05:32 +08:00
bd7a3bb878 Fix: align go provider apis with python apis (#16867)
### Summary

As title.

---------

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
Co-authored-by: Wang Qi <wangq8@outlook.com>
2026-07-14 21:06:11 +08:00
Haruko386 7a20920b12 fix: unable to send email for py backend (#16888)
### Summary

As title

Can send email now
<img width="2313" height="1941" alt="image"
src="https://github.com/user-attachments/assets/c129b3c7-7e40-4bdd-8430-f3b402b9cab1"
/>
<img width="1256" height="2760" alt="f6ba88c56fe3945a6d28564fc4d4c54f"
src="https://github.com/user-attachments/assets/ae2dbe0c-e2d4-494f-8b86-03e959ef4808"
/>
2026-07-14 14:59:06 +08:00
euvre 3bfad1f00e fix: correct model type mappings and improve system setting persistence (#16501) 2026-07-13 16:42:55 +08:00
Harsh KashyapandHarsh Kashyap 289ca28ce2 Fix OpenAI agent stream chunk shape (#16402)
### What problem does this PR solve?

Closes #8175.

The Agent OpenAI-compatible streaming path uses `get_data_openai(...,
stream=True)`, but that helper currently returns a minimal chunk shape.
The main OpenAI-compatible chat endpoint already includes chunk metadata
such as `created`, `system_fingerprint`, `usage`, `logprobs`, and
assistant role/tool placeholders.

This PR aligns the Agent stream helper with that existing
OpenAI-compatible chunk shape while keeping the current `delta.content`
behavior and existing reference injection path intact.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):

### Verification

- `./.venv/bin/python -m pytest
test/unit_test/api/utils/test_api_utils.py -q`
- `python3 -m py_compile api/utils/api_utils.py
test/unit_test/api/utils/test_api_utils.py`
- `uvx ruff check api/utils/api_utils.py
test/unit_test/api/utils/test_api_utils.py`

---------

Co-authored-by: Harsh Kashyap <harshkashyap@Harshs-MacBook-Pro.local>
2026-07-10 10:59:32 +08:00
buua436 6a77523bf0 refa: resolve tenant model refs consistently (#16744) 2026-07-09 14:02:08 +08:00
Lynn 0ae5961e1c Feat: v0.27.0 model provider (#16604) 2026-07-08 09:47:29 +08:00
Harsh KashyapandHarsh Kashyap 98189cd20a Fix OpenAI response created timestamp (#16401)
## What this fixes

Closes #16400.

`get_data_openai()` currently returns `created: null` when callers do
not pass a timestamp, and it replaces explicit timestamp values with the
current time. This makes non-streaming OpenAI-compatible responses
inconsistent with the expected integer `created` timestamp field.

## Change

- Preserve explicit `created` values when provided.
- Default non-streaming responses to `int(time.time())` when `created`
is not provided.
- Add focused unit coverage for default timestamps, explicit timestamps,
and unchanged streaming chunk shape.

## Verification

- `./.venv/bin/python -m pytest
test/unit_test/api/utils/test_api_utils.py -q`
- `python3 -m py_compile api/utils/api_utils.py
test/unit_test/api/utils/test_api_utils.py`
- `uvx ruff check api/utils/api_utils.py
test/unit_test/api/utils/test_api_utils.py`

---------

Co-authored-by: Harsh Kashyap <harshkashyap@Harshs-MacBook-Pro.local>
2026-07-06 14:16:16 +08:00
7da4f200e5 fix(agent): enable MCP file preview via doc_id (#15399)
## Summary
MCP-wrapped agents could only force-download files looked up by
`doc_id`. This adds an explicit preview path and inline response headers
for previewable file types.

- **New** `GET /api/v1/agents/attachments/{attachment_id}/preview` —
inline preview for PDFs, images, and other safe types (pass `ext` and/or
`mime_type`)
- **Improved** `GET /api/v1/documents/{doc_id}/preview` — sets inline
disposition using the document filename
- **Improved** attachment download routing — resolves `mime_type` /
`ext` query params (no default `markdown`), supports
`disposition=inline`
- **DocGenerator output** — includes URL-encoded `preview_url` for MCP
clients
- **Legacy `/document/download/...` aliases** — still use download
semantics; MCP clients should call `/preview` explicitly

Fixes #15398

## Test plan
- [x] `pytest test/unit_test/api/utils/test_file_response_headers.py`
(6/6)

---------

Co-authored-by: MkDev11 <mkdev11@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Ling Qin <qinling0210@163.com>
2026-07-03 19:56:01 +08:00
Wang Qi 6a4b9be426 Refactor: reformat all code for lefthook using ruff and gofmt (#16585) 2026-07-03 12:53:39 +08:00
Kevin Hu 62f94cd59b Feat: Add knowledge compilation workflows (#16515)
## Summary
- Add knowledge compilation template APIs, services, and builtin
template seed data
- Add advanced knowledge compile structure/artifact/RAPTOR workflow
support
- Update parsing, dataset/document APIs, and supporting services for
compilation workflows
2026-07-02 23:22:07 +08:00
27c9a093bd Fix: close MCP sessions after canvas execution to prevent connection leaks (#13295)
### What problem does this PR solve?

Closes #12962

MCPToolCallSessions created during agent execution (in `Agent.__init__`)
are never explicitly closed. Each session starts its own event loop
thread and opens an SSE/HTTP connection to the MCP server. When the
canvas goes out of scope, these threads and connections remain alive
indefinitely, accumulating over time and causing resource exhaustion
after prolonged use.

### Solution

1. Add a `Graph.close()` method that iterates all components, finds
MCPToolCallSessions held by Agent tools, and calls `close_sync()` on
each to properly shut down the event loop, thread, and connection.
2. Call `canvas.close()` in `finally` blocks after `canvas.run()`
completes in `canvas_service.py` and `canvas_app.py`.
3. Move MCP session cleanup to `finally` blocks in `test_tool` endpoint
(`mcp_server_app.py`) and `get_mcp_tools` (`api_utils.py`) to ensure
sessions are closed even on exceptions.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: conflict-resolver <conflict-resolver@local>
Co-authored-by: Zhichang Yu <yuzhichang@gmail.com>
2026-07-02 10:57:24 +08:00
Zhichang Yu f58fae5fb7 feat(go-agent): Ported retrieval node, added Keenable web search tool (#16396)
Ported retrieval node, added Keenable web search tool
- [x] New Feature (non-breaking change which adds functionality)
2026-06-29 09:45:16 +08:00
Harsh Kashyap 8d3c3f868c fix(api): validate immutable document fields when value is zero (#16309) 2026-06-25 19:29:12 +08:00
Carl Harris a2de880b6d fix(profile): enforce profile name validation and input constraints (#15694)
### What problem does this PR solve?

The Profile **Name** field currently lacks application-level validation
and allows users to save excessively long names and unsupported special
characters.

While the database enforces a maximum length of 100 characters, neither
the frontend nor backend validates nickname format before persistence.
This can result in inconsistent user data, poor user experience, and UI
layout issues when long names wrap across multiple lines.

This PR introduces consistent frontend and backend validation for
profile names, enforces length and character constraints, provides clear
validation feedback, and prevents invalid values from being saved.

Fixes #15693

### Type of change

* [x] Bug Fix (non-breaking change which fixes an issue)
2026-06-12 11:13:18 +08:00
Wang Qi 4cbe597d7e Refactor: consolidate to use @login_required (#15652)
Refactor: consolidate to use @login_required
2026-06-05 11:35:00 +08:00
Wang Qi 1a6df01b53 Bug fix: Enhance embeding model to give better error message (#15346)
To resolve https://github.com/infiniflow/ragflow/issues/15343 enhance
the model embedding message to give extact failure message to customer.


# QWen

## Retrieval
<img width="3321" height="1033" alt="image"
src="https://github.com/user-attachments/assets/6b82921a-a3a7-4a33-a383-1cf316398ee2"
/>

## Chat
<img width="2241" height="311" alt="image"
src="https://github.com/user-attachments/assets/ec311365-62d5-407a-8915-5c8d72be9716"
/>


# SiliconFlow
## Retrieval
<img width="3321" height="1033" alt="image"
src="https://github.com/user-attachments/assets/ee2cd191-a27d-4729-b53d-2fbdb4e352cd"
/>

## Chat
<img width="1562" height="210" alt="image"
src="https://github.com/user-attachments/assets/10376a8e-a3f4-422f-bc2e-96f2a8a96448"
/>

# Baichuan
## Retrieval
<img width="3321" height="1107" alt="image"
src="https://github.com/user-attachments/assets/dcb5409d-f7fc-4804-b186-5e1ee11e09c4"
/>

## Chat
<img width="2241" height="311" alt="image"
src="https://github.com/user-attachments/assets/ec311365-62d5-407a-8915-5c8d72be9716"
/>


# Zhipu
zhipu is good.
2026-06-01 19:18:16 +08:00
Lynnandbill dc4b82523b Feat: tenant llm provider (#14595)
### What problem does this PR solve?

Python implementation of the Go-based model_provider API suite.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: bill <yibie_jingnian@163.com>
2026-05-29 17:39:41 +08:00