## Summary
Rename the repository dependency-resource directory `ragflow_deps/` to
`deps/`, and update every hardcoded reference to that directory name so
the rename does not break resource loading at build/runtime.
This is not a purely cosmetic rename: the Go runtime hardcodes the
directory name as a resource search path (e.g.
`internal/common/model_assets.go` trimming the `deps/` prefix, and the
tokenizers joining `deps/` into candidate paths), and the Dockerfiles
copy assets into `/ragflow/deps`. Those literals must move together with
the directory, otherwise assets fail to resolve.
## Two intentional exceptions (kept unchanged)
The rename deliberately keeps two names that only *coincidentally*
contained the old string:
- The published image tag `infiniflow/ragflow_deps:latest` (used by
`Dockerfile`, `Dockerfile_base`, docs, and CI `image inspect` checks).
- The CI install path `/opt/ragflow_deps` inside the CI runner image
(`build.sh` `SYSTEM_DEPS_TOKENIZER`).
## Changes
- `git mv ragflow_deps deps`.
- `build.sh`: `CL100K_TABLE_RELPATH`, native asset zip path, and related
references updated to `deps/`.
- `tools/scripts/download_dev_deps.py`: `RAGFLOW_DEPS_DIR` points at
`deps/`.
- Go runtime resource search paths (`internal/common`,
`internal/tokenizer`) updated to the `deps/` layout.
- `Dockerfile` / `Dockerfile_base`: container asset path changed to
`/ragflow/deps`; kept tag `infiniflow/ragflow_deps:latest` unchanged.
- Docs / `AGENTS.md` / CI workflow paths updated to the new directory
name.
## Test plan
- Go unit tests in `internal/common` and `internal/tokenizer` updated to
the new `deps/` layout and pass via `bash build.sh --test`.
- Python download scripts import and build URLs correctly (`py_compile`
clean).
- Added `tools/scripts/test_dep_rename_invariants.py` as a regression
guard asserting the two intentional exceptions (`/opt/ragflow_deps` and
`infiniflow/ragflow_deps`) stay intact, so a future blanket rename
cannot silently regress them.
---
## Follow-up fix (commit 2f6857095d)
The original rename used a blanket `sed s/ragflow_deps/deps/g` and only
selectively reverted `build.sh`. That left two intentionally-kept names
rewritten, which this follow-up restores:
- CI install path `/opt/ragflow_deps` — `sep-tests.yml` had been changed
to read `/opt/deps`, breaking the offline cl100k table fallback (the
cached-image path pointed at a directory that never receives the file).
Restored to `/opt/ragflow_deps`.
- Published image tag `infiniflow/ragflow_deps` — left as bare
`infiniflow/deps` in `sep-tests.yml` comments/echo, `.dockerignore`, and
`Dockerfile` / `Dockerfile_base` error messages. Restored to
`infiniflow/ragflow_deps`.
The regression guard `tools/scripts/test_dep_rename_invariants.py` was
added to prevent recurrence.
### Summary
Removed unnecessary beartype hooks from three package imports. This
prevents unwanted import-time instrumentation and dependency coupling.
based on #15258
- Update version tags in README files (including translations) from
v0.27.1 to v0.27.2
- Modify Docker image references and documentation to reflect new
version
- Update version badges and image descriptions
- Maintain consistency across all language variants of README files
Add Sitemap data source — ingests web pages from sitemap.xml with incremental sync. Supports HTML→Markdown extraction, PDF discovery, URL filtering, SSRF protection.
### What problem does this PR solve?
- Update version tags in README files (including translations) from
v0.26.4 to v0.27.0
- Modify Docker image references and documentation to reflect new
version
- Update version badges and image descriptions
- Maintain consistency across all language variants of README files
### Type of change
- [x] Documentation Update
### Summary
Fixes#18107.
`editdistance==0.8.1` (the only recent release on PyPI) has no cp313
wheels for any platform. Since this project requires exactly Python
3.13, `uv`/`pip`/`poetry` fall back to building it from source (Cython),
which fails on Windows for anyone without a working C build toolchain —
that's the PEP 517 build error in the issue.
Swapped `editdistance` for `rapidfuzz`, which ships full cp313 wheels
(win32/win_amd64/win_arm64 included) and has no build-from-source step
on any of our target platforms. The only call site was
`EntityResolution.is_similarity` in `rag/graphrag/entity_resolution.py`,
using `editdistance.eval(a, b)` to get the unweighted Levenshtein
distance between two entity names.
`rapidfuzz.distance.Levenshtein.distance(a, b)` computes the same thing
(verified identical output on several string pairs) and is used as a
direct replacement.
Ports dataset knowledge compilation (wiki/graph/tree/mindmap) to the Go
scheduler with a status contract, aligns wiki storage/retrieval with
Python, sizes prompts by content_length, and resolves embedding batch
size from provider capability.
## Summary
Remediates CVE-2026-26209 (HIGH) in `cbor2` by adding `cbor2>=5.9.0` to
`constraint-dependencies` in `pyproject.toml`.
| CVE | Severity | Package | Installed | Fixed in |
|---|---|---|---|---|
| CVE-2026-26209 | HIGH | cbor2 | 5.8.0 | 5.9.0 |
`cbor2` is a transitive dependency pulled in by `ranx` with no version
constraint, stuck at 5.8.0 because lockfile hadn't been re-resolved.
Ports the dataset knowledge compilation (wiki/graph/tree/mindmap) to the
Go scheduler with a status contract, aligns wiki storage/retrieval with
Python, and sizes prompts by content_length.
## Summary
- Pin `werkzeug>=3.1.7,<4` and refresh `uv.lock` to **3.1.8**.
- Fixes intermittent corruption of uploaded file bodies: when TCP
segments split right after multipart part headers, Werkzeug **3.1.5**
can include a leading `\r\n` in the file content
([pallets/werkzeug#3088](https://github.com/pallets/werkzeug/issues/3088);
fixed in 3.1.7).
- In RAGFlow this commonly breaks `.xlsx` parsing: ZIP/OOXML magic
(`PK\x03\x04`) no longer matches, the Excel parser falls back to CSV,
then fails with UTF-8 decode errors such as `invalid start/continuation
byte`.
Made with [Cursor](https://cursor.com)
Co-authored-by: zhangjiangshan1 <zhangjiangshan1@kingsoft.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Jin Hai <haijin.chn@gmail.com>
## Summary
Migrates `mistralai` from `==0.4.2` to `>=2.7.2,<3.0.0` to unblock the
orjson CVE fix. The old SDK pinned `orjson>=3.9.10,<3.11`, preventing
upgrade to the patched version.
| CVE | Severity | Package | Installed | Fixed in |
|---|---|---|---|---|
| CVE-2025-67221 | HIGH | orjson | 3.10.18 | 3.11.6 |
`mistralai` 2.x (the current maintained version) drops the orjson
dependency entirely. Added `orjson>=3.11.6` to `constraint-dependencies`
to pin the floor for remaining parent packages (`langgraph-sdk`,
`langsmith`, `ranx`).
## Summary
Remediates CVE-2026-59939 (HIGH) in `httplib2` by adding
`httplib2>=0.32.0` to `constraint-dependencies` in `pyproject.toml`.
| CVE | Severity | Package | Installed | Fixed in |
|---|---|---|---|---|
| CVE-2026-59939 | HIGH | httplib2 | 0.31.0 | 0.32.0 |
`httplib2` is a transitive dependency pulled in by
`google-api-python-client` and `google-auth-httplib2`. Both declare
`httplib2<1.0.0,>=0.19.0`, allowing 0.32.0 without conflict.
## Summary
Remediates CVE-2026-0994 (HIGH) in `protobuf` by adding
`protobuf>=5.29.6` to `constraint-dependencies` in `pyproject.toml`.
| CVE | Severity | Package | Installed | Fixed in |
|---|---|---|---|---|
| CVE-2026-0994 | HIGH | protobuf | 5.29.5 | 5.29.6 |
`protobuf` is a transitive dependency pulled in by `google-api-core`,
`googleapis-common-protos`, `grpcio-status`, and `opentelemetry-proto`.
It was stuck at 5.29.5 because the lockfile hadn't been re-resolved.
## Summary
Remediates three HIGH severity CVEs in `pyasn1` by bumping the existing
`constraint-dependencies` entry from `>=0.6.3` to `>=0.6.4` in
`pyproject.toml`.
| CVE | Severity | Package | Installed | Fixed in |
|---|---|---|---|---|
| CVE-2026-59884 | HIGH | pyasn1 | 0.6.3 | 0.6.4 |
| CVE-2026-59885 | HIGH | pyasn1 | 0.6.3 | 0.6.4 |
| CVE-2026-59886 | HIGH | pyasn1 | 0.6.3 | 0.6.4 |
`pyasn1` is a transitive dependency (pulled in via `google-auth` / `rsa`
/ `pyasn1-modules`). A previous constraint (`>=0.6.3`) was added for
CVE-2026-30922 but two additional vulnerabilities were disclosed that
require 0.6.4.
## Summary
Remediates two HIGH severity CVEs in `pdfminer.six` by upgrading its
parent dependency `pdfplumber` from `==0.10.4` to `==0.11.10` in
`pyproject.toml`.
| CVE | Severity | Package | Installed | Fixed in |
|---|---|---|---|---|
| CVE-2025-64512 | HIGH | pdfminer.six | 20221105 | 20251107 |
| CVE-2025-70559 | HIGH | pdfminer.six | 20221105 | 20251230 |
`pdfminer.six` is a transitive dependency exact-pinned by `pdfplumber`.
Version 0.10.4 pins `pdfminer.six==20221105` (vulnerable); version
0.11.10 pins `pdfminer.six==20260107` (patched).
## Summary
Remediates nine litellm CVEs (3 CRITICAL, 6 HIGH) by upgrading the exact
pin from `==1.82.5` to `==1.84.0` in `pyproject.toml`.
| CVE | Severity | Fixed in |
|---|---|---|
| CVE-2026-35030 | CRITICAL | 1.83.0 |
| CVE-2026-42208 | CRITICAL | 1.83.7 |
| CVE-2026-49468 | CRITICAL | 1.84.0 |
| CVE-2026-35029 | HIGH | 1.83.0 |
| CVE-2026-40217 | HIGH | 1.83.10 |
| CVE-2026-42203 | HIGH | 1.83.7 |
| CVE-2026-42271 | HIGH | 1.83.7 |
| CVE-2026-47101 | HIGH | 1.83.14 |
| CVE-2026-47102 | HIGH | 1.83.10 |
`litellm` was pinned to `==1.82.5` because versions 1.82.6 - 1.82.8 had
a broken `litellm.caching.caching` import chain, and 1.88.0 had a broken
`litellm.integrations.custom_logger` import.
Both issues are resolved in 1.84.0.
The exact pin is retained (rather than a range) due to litellm's history
of introducing regressions on minor bumps.
## Summary
Remediates five nltk CVEs by adding `nltk>=3.10.0` to
`constraint-dependencies` in `pyproject.toml`.
| CVE | Severity | Installed | Fixed in |
|---|---|---|---|
| CVE-2025-14009 | CRITICAL | 3.9.2 | 3.9.3 |
| CVE-2026-0846 | HIGH | 3.9.2 | 3.9.3 |
| CVE-2026-0847 | HIGH | 3.9.2 | — |
| CVE-2026-33231 | HIGH | 3.9.2 | 3.9.4 |
| CVE-2026-33236 | HIGH | 3.9.2 | — |
| CVE-2026-54293 | HIGH | 3.9.2 | 3.10.0 |
| CVE-2026-33230 | MEDIUM | 3.9.2 | 3.9.4 |
CVE-2026-0847 and CVE-2026-33236 have no confirmed fixed version.
`nltk` is a transitive dependency (pulled in by `crawl4ai` and
`infinity-sdk`). No package in the graph caps it below 3.10.0. It was
stuck at 3.9.2 simply because the lockfile had never been re-resolved.
## Summary
Remediates four CVEs in `lxml` and `lxml_html_clean` by upgrading
transitive dependency constraints and bumping `crawl4ai` to unblock the
resolution.
| CVE | Severity | Package | Installed | Fixed in |
|---|---|---|---|---|
| CVE-2026-41066 | HIGH | lxml | 5.4.0 | 6.1.0 |
| CVE-2026-49825 | HIGH | lxml_html_clean | 0.4.3 | 0.4.5 |
| CVE-2026-28348 | MEDIUM | lxml_html_clean | 0.4.3 | 0.4.5 |
| CVE-2026-28350 | MEDIUM | lxml_html_clean | 0.4.3 | 0.4.5 |
- Bumped `crawl4ai` from `>=0.9.0,<0.9.1` to `>=0.9.2,<0.9.3`, version
0.9.2 relaxes lxml constraint to `>=5.3,<7`. The strict `<0.9.3` upper
bound on crawl4ai is maintained per project convention
- Added `inscriptis>=2.7.3` to `constraint-dependencies`, version 2.7.3
relaxes lxml constraint to `>=5.4.0,<6.2.0`
- Added `lxml>=6.1.0` to `constraint-dependencies`
- Added `lxml_html_clean>=0.4.5` to `constraint-dependencies`
`exclude-dependencies` entry for `unclecode-litellm` (still declared by
crawl4ai 0.9.2) continues to prevent the namespace collision that
previously crashed RAGFlow.
## Verification
- `lxml` is used widely across the codebase for HTML/XML parsing. The
5.x → 6.x upgrade maintains API compatibility for standard usage
(`lxml.html`, `lxml.etree`).
- `lxml_html_clean` API is unchanged between 0.4.3 and 0.4.5.
- `crawl4ai` 0.9.2 is a patch bump from 0.9.0 with no API changes
affecting RAGFlow's usage.
- `inscriptis` 2.7.3 is a patch bump from 2.7.0.
## Testing
- Full unit test suite passes
- `uv sync` resolves cleanly
- Verified `crawl4ai` imports without litellm namespace collision
## Summary
Bumps `aiosmtplib` from `>=5.0.0` to `>=5.1.1,<6.0.0` in
`pyproject.toml`.
| CVE | Severity | Package | Installed | Fixed in |
|---|---|---|---|---|
| CVE-2026-53533 | MEDIUM | aiosmtplib | 5.0.0 | 5.1.1 |
This is a direct dependency upgrade, no override or constraint needed.
dependabot[bot]
xugangqiang
Jin Hai
Wang Qi
Sourav Rajvi
Sbaaoui Idriss
Liu An
mkaaad
VincentLambert
wenqian
qinling0210
rayhan
Charles
Yingfeng
Zhichang Yu
