ci: fix dependency compatibility failures (#2308)

* ci: fix dependency compatibility failures Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * build: constrain fastjsonschema on older Python Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * build: constrain filelock for fork compatibility Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * ci: unify source and PyPI compatibility checks * ci: build grpcio locally on older macOS ARM64 Python * ci: use macOS SDK zlib for grpcio source builds * ci: install test dependencies by workflow and platform * ci: prepare macOS OpenMP before model import checks --------- Co-authored-by: Bowen Xian (Shanghai Wicresoft Co Ltd) <v-boxian@microsoft.com> Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
2026-11-10 21:11:12 +08:00 · 2026-09-16 11:18:42 +08:00
1 parent 79633dd950
commit f431588906
@@ -0,0 +1,122 @@
+# CI compatibility baseline
+
+The three Python workflows share `constraints.txt` via a job-level, absolute
+`PIP_CONSTRAINT` path. It applies to every pip invocation, including installs
+inside `make ci-install`, later notebook/tool installs, and the released `pyqlib`
+package. A bound in this checkout's `pyproject.toml` alone does not constrain a
+package installed from PyPI. Keep the compatibility bounds in both files in
+sync; `tests/test_ci_configuration.py` checks this.
+
+This is a set of known compatibility bounds, not a fully reproducible Python
+lockfile. The workflows run `pip check` and always print installed versions for
+future diagnosis. They retain all 25 OS/Python combinations per workflow and
+disable matrix fail-fast so that one failure does not hide the others.
+
+## Why these bounds exist
+
+| Dependency | Constraint | Failure addressed |
+| --- | --- | --- |
+| MLflow | `<3.13` | Qlib uses the filesystem tracking backend, disabled by default in newer versions. |
+| filelock | `>=3.16.0,<3.30` | Fork-safety checks conflict with DataQueue's producer thread and multiprocessing workers. |
+| fastjsonschema | `<2.22` on Python 3.8/3.9 | Python 3.10 union annotations are evaluated on older interpreters. |
+| Plotly | `<7` | Qlib still imports `figure_factory.create_distplot`. |
+| lxml | `<6.1.3` | The Windows job falls back to a source distribution without libxml2 headers. |
+| OSQP | `==1.0.5` on Windows/Python 3.8 | Native imports crash with 1.1.3 and also with 0.6.7.post3. |
+| Black | `<26.1` | Use one compatible formatter for both Python and notebooks. |
+
+OSQP 1.0.5 is the wheel used by the successful full Windows/Python 3.8 job in
+[PR #2308](https://github.com/microsoft/qlib/actions/runs/30225522618/job/89854962742).
+It is selected during dependency resolution, not installed as a downgrade after
+an incompatible environment has already been built. All three workflows smoke
+test native solver imports rather than excluding Windows/Python 3.8.
+
+`MLFLOW_ALLOW_FILE_STORE=true` retains the original PR's explicit file-backend
+opt-in. The upper bound also protects source installs outside CI. Migrating
+Qlib's tracking backend is a separate change.
+
+## Install only each test job's dependencies
+
+CI calls `make ci-install` with an explicit `CI_EXTRAS` profile. The existing
+`make dev` target still installs the complete developer environment.
+
+| Source job | Extras |
+| --- | --- |
+| Regular, Windows/macOS | `dev,test,analysis,lint` |
+| Regular, Ubuntu 24.04 | `dev,test,analysis,lint,rl` |
+| Regular, Ubuntu 22.04 | `dev,test,analysis,lint,rl,docs` |
+| Slow, Windows/macOS | `dev,test,analysis` |
+| Slow, Linux | `dev,test,analysis,rl` |
+
+All source jobs additionally install `test-requirements.txt`: PyTorch is
+required for the neural-network tests even without the RL extra. NumPy/SciPy
+retain the compatibility bounds previously inherited from the RL/docs extras.
+Ubuntu jobs select CPU PyTorch wheels. torchvision/torchaudio are not used by
+the test suite and are no longer installed explicitly. No test job needs the
+package-building extra; docs dependencies are installed only where `docs-gen`
+runs. Lint checks stay in the existing regular source matrix.
+
+`tests/conftest.py` already excludes RL tests on non-Linux platforms. Installing
+RL extras there used to pull in `tianshou -> tensorboard -> grpcio`, requiring
+expensive source builds to work around incorrect macOS wheel metadata and an
+Xcode 16.4 bundled-zlib build failure. Those jobs no longer install this unused
+dependency chain, so the grpcio source-build workaround has been removed.
+Linux retains the RL extra and the same RL tests. The PyPI workflow is unchanged.
+
+Explicit imports of PyTorch, Qlib's PyTorch model registry, GeneralPTNN, and
+both dataset classes prevent missing dependencies from silently bypassing
+neural-network tests. On macOS, prepare LightGBM and its `libomp` runtime before
+these checks: importing the model package also eagerly imports LightGBM. The
+preparation step verifies the LightGBM native import before checking the model
+registry. A non-Linux footprint check rejects accidental
+reintroduction of tianshou/TensorBoard/grpcio, followed by the unchanged
+`pip check`. If a future test genuinely needs these dependencies on macOS,
+update this policy and validate native wheel metadata/imports rather than
+disabling the checks. This change does not reduce the matrix, alter test
+selection, add caches, or reduce test data sizes.
+
+## Network and workflow retries
+
+Dataset commands have at most three attempts, with a 15-minute timeout per
+attempt. `--delete_old False` makes retries noninteractive in these fresh CI
+directories, including when an earlier attempt already extracted one dataset.
+Bash command blocks use `set -euo pipefail`, so an earlier failure cannot be
+hidden by a later successful command. Exhausted retries still fail the job.
+
+The configuration-driven workflow has at most three 30-minute attempts. This
+is a bounded mitigation for the observed MLflow file-store error
+`Metric 'IC' is malformed. No data found.`, not a fix for a suspected
+read/write race. A deterministic error must still fail all attempts; do not
+introduce `continue-on-error` or silently skip assertions. Fixing file-store
+consistency should be handled separately, with a reproducer.
+
+The MLflow client performance test initializes the tracking store before
+timing repeated client construction, uses a monotonic performance clock, and
+retains the original 10ms (Linux) / 20ms (other platforms) thresholds. Cold
+plugin/store initialization is not a client-construction regression.
+
+The PyPI workflow runs outside the checkout (`RUNNER_TEMP`) and passes the
+example YAML by absolute path, so imports test the installed distribution.
+
+## Title lint and maintenance
+
+Checkout, Python setup, Node setup, and retry actions use Node 24-based releases
+(`checkout@v6`, `setup-python@v6`, `setup-node@v6`, `retry@v4`). The action
+runtime is separate from the Python and Node versions installed for tests.
+
+Node 22 satisfies commitlint's Node >=22.12 requirement. The root `package.json`
+and `package-lock.json` pin the title tooling and its transitive dependencies;
+CI uses `npm ci --ignore-scripts` and `npx --no-install`.
+
+When updating a bound, update the package metadata and constraints together,
+run the offline policy checks, and verify the full matrix, including PyPI:
+
+```sh
+python -m unittest discover -s tests -p test_ci_configuration.py
+npm ci --ignore-scripts --no-audit --no-fund
+printf '%s\n' 'ci: update compatibility baseline' | npx --no-install commitlint --config .commitlintrc.js
+```
+
+For local pip installs, set `PIP_CONSTRAINT` to the absolute path of this
+directory's `constraints.txt`. Do not infer Windows/macOS runtime correctness
+from a Linux resolver check. Remove individual bounds only after reproducing
+and verifying the upstream fix on the affected environment.
@@ -0,0 +1,19 @@
+# Shared by source and PyPI jobs through an absolute PIP_CONSTRAINT path.
+# This is a compatibility baseline, not a complete lock of every dependency.
+# Keep runtime bounds aligned with pyproject.toml; see README.md in this folder.
+
+# Qlib still uses MLflow's filesystem tracking backend.
+mlflow<3.13
+# New fork-safety checks race with the threaded DataQueue producer.
+filelock>=3.16.0,<3.30
+# 2.22 uses Python 3.10 union syntax on older interpreters.
+fastjsonschema<2.22; python_version < "3.10"
+# Qlib imports figure_factory.create_distplot, removed in Plotly 7.
+plotly<7
+# 6.1.3 falls back to a source build on Windows without libxml2 headers.
+lxml<6.1.3
+# This exact Windows/Python 3.8 wheel passed the full matrix in PR #2308.
+# Both 1.1.3 and a subsequent downgrade to 0.6.7.post3 crashed on import.
+osqp==1.0.5; sys_platform == "win32" and python_version == "3.8"
+# Use the same formatter for Python files and notebooks.
+black<26.1
@@ -0,0 +1,7 @@
+# Runtime required by both source test workflows, including non-Linux jobs
+# which do not run tests/rl. Do not rely on the RL extra to bring in PyTorch.
+torch
+# Preserve the numerical dependency bounds previously inherited from the RL
+# and docs extras while changing which optional tools are installed.
+numpy<2.0.0
+scipy<=1.15.3
@@ -18,18 +18,18 @@ jobs:
    steps:
      # This step is necessary because the lint title uses the .commitlintrc.js file in the project root directory.
      - name: Checkout Repository
-        uses: actions/checkout@v4
+        uses: actions/checkout@v6

      - name: Setup Node.js
-        uses: actions/setup-node@v4
+        uses: actions/setup-node@v6
        with:
-          node-version: '16'
+          node-version: '22'

      - name: Install commitlint
-        run: npm install --save-dev @commitlint/{config-conventional,cli}
+        run: npm ci --ignore-scripts --no-audit --no-fund

      - name: Validate PR Title with commitlint
        env:
          BODY: ${{ github.event.pull_request.title }}
        run: |
-          echo "$BODY" | npx commitlint --config .commitlintrc.js
+          printf '%s\n' "$BODY" | npx --no-install commitlint --config .commitlintrc.js
@@ -13,9 +13,13 @@ on:
 jobs:
  build:
    timeout-minutes: 120
+    env:
+      MLFLOW_ALLOW_FILE_STORE: "true"
+      PIP_CONSTRAINT: ${{ github.workspace }}/.github/ci/constraints.txt

    runs-on: ${{ matrix.os }}
    strategy:
+      fail-fast: false
      matrix:
        os: [windows-latest, ubuntu-24.04, ubuntu-22.04, macos-14, macos-15]
        # In github action, using python 3.7, pip install will not match the latest version of the package.
@@ -25,12 +29,12 @@ jobs:

    steps:
    - name: Test qlib from pip
-      uses: actions/checkout@v4
+      uses: actions/checkout@v6
      with:
        fetch-depth: 0

    - name: Set up Python ${{ matrix.python-version }}
-      uses: actions/setup-python@v4
+      uses: actions/setup-python@v6
      with:
        python-version: ${{ matrix.python-version }}

@@ -42,6 +46,13 @@ jobs:
      run: |
        python -m pip install pyqlib

+    - name: Verify dependency consistency
+      run: python -m pip check
+
+    - name: Verify native solvers on Windows Python 3.8
+      if: ${{ matrix.os == 'windows-latest' && matrix.python-version == '3.8' }}
+      run: python -c "import cvxpy, osqp; print('CVXPY', cvxpy.__version__, 'OSQP', osqp.__version__)"
+
    - name: Install Lightgbm for MacOS
      if: ${{ matrix.os == 'macos-14' || matrix.os == 'macos-15' }}
      run: |
@@ -50,11 +61,29 @@ jobs:
        python -m pip install --no-binary=:all: lightgbm

    - name: Downloads dependencies data
-      run: |
-        cd ..
-        python -m qlib.cli.data qlib_data --target_dir ~/.qlib/qlib_data/cn_data --region cn
-        cd qlib
+      uses: nick-fields/retry@v4
+      with:
+        timeout_minutes: 15
+        max_attempts: 3
+        retry_wait_seconds: 10
+        shell: bash
+        command: |
+          set -euo pipefail
+          cd "$RUNNER_TEMP"
+          python -m qlib.cli.data qlib_data --target_dir ~/.qlib/qlib_data/cn_data --region cn --delete_old False

    - name: Test workflow by config
-      run: |
-        qrun examples/benchmarks/LightGBM/workflow_config_lightgbm_Alpha158.yaml
+      uses: nick-fields/retry@v4
+      with:
+        timeout_minutes: 30
+        max_attempts: 3
+        shell: bash
+        command: |
+          set -euo pipefail
+          cd "$RUNNER_TEMP"
+          python -c "import os, pathlib, qlib; print('Testing installed Qlib:', qlib.__file__); assert pathlib.Path(os.environ['GITHUB_WORKSPACE']).resolve() not in pathlib.Path(qlib.__file__).resolve().parents"
+          qrun "$GITHUB_WORKSPACE/examples/benchmarks/LightGBM/workflow_config_lightgbm_Alpha158.yaml"
+
+    - name: Report installed dependencies
+      if: always()
+      run: python -m pip freeze
@@ -13,10 +13,14 @@ on:
 jobs:
  build:
    timeout-minutes: 180
+    env:
+      MLFLOW_ALLOW_FILE_STORE: "true"
+      PIP_CONSTRAINT: ${{ github.workspace }}/.github/ci/constraints.txt
    # we may retry for 3 times for `Unit tests with Pytest`

    runs-on: ${{ matrix.os }}
    strategy:
+      fail-fast: false
      matrix:
        os: [windows-latest, ubuntu-24.04, ubuntu-22.04, macos-14, macos-15]
        # In github action, using python 3.7, pip install will not match the latest version of the package.
@@ -26,12 +30,12 @@ jobs:

    steps:
    - name: Test qlib from source
-      uses: actions/checkout@v4
+      uses: actions/checkout@v6
      with:
        fetch-depth: 0

    - name: Set up Python ${{ matrix.python-version }}
-      uses: actions/setup-python@v4
+      uses: actions/setup-python@v6
      with:
        python-version: ${{ matrix.python-version }}

@@ -39,24 +43,40 @@ jobs:
      run: |
        python -m pip install --upgrade pip

-    - name: Installing pytorch for macos
-      if: ${{ matrix.os == 'macos-14' || matrix.os == 'macos-15' }}
-      run: |
-        python -m pip install torch torchvision torchaudio
-
    - name: Installing pytorch for ubuntu
      if: ${{ matrix.os == 'ubuntu-24.04' || matrix.os == 'ubuntu-22.04' }}
      run: |
-        python -m pip install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cpu
-
-    - name: Installing pytorch for windows
-      if: ${{ matrix.os == 'windows-latest' }}
-      run: |
-        python -m pip install torch torchvision torchaudio
+        python -m pip install torch --extra-index-url https://download.pytorch.org/whl/cpu

    - name: Set up Python tools
+      env:
+        # RL tests are Linux-only; only ubuntu-22.04 builds the docs below.
+        CI_EXTRAS: dev,test,analysis,lint${{ runner.os == 'Linux' && ',rl' || '' }}${{ matrix.os == 'ubuntu-22.04' && ',docs' || '' }}
      run: |
-        make dev
+        make ci-install
+
+    # Importing Qlib's model registry also loads LightGBM and its OpenMP runtime.
+    - name: Install Lightgbm for MacOS
+      if: ${{ matrix.os == 'macos-14' || matrix.os == 'macos-15' }}
+      run: |
+        brew update
+        brew install libomp || brew reinstall libomp
+        python -m pip install --no-binary=:all: lightgbm
+        python -c "import lightgbm; print('LightGBM', lightgbm.__version__, 'native import passed')"
+
+    - name: Verify neural network test dependencies
+      run: python -c "import torch; from qlib.contrib.model import pytorch_classes; from qlib.contrib.model.pytorch_general_nn import GeneralPTNN; from qlib.data.dataset import DatasetH, TSDatasetH; assert pytorch_classes; print('PyTorch', torch.__version__, 'model imports passed')"
+
+    - name: Verify non-Linux dependency footprint
+      if: ${{ runner.os != 'Linux' }}
+      run: python -c "from importlib.metadata import distributions; unexpected = {'tianshou', 'tensorboard', 'grpcio'} & {d.metadata['Name'].lower() for d in distributions()}; assert not unexpected, unexpected"
+
+    - name: Verify dependency consistency
+      run: python -m pip check
+
+    - name: Verify native solvers on Windows Python 3.8
+      if: ${{ matrix.os == 'windows-latest' && matrix.python-version == '3.8' }}
+      run: python -c "import cvxpy, osqp; print('CVXPY', cvxpy.__version__, 'OSQP', osqp.__version__)"

    - name: Lint with Black
      run: |
@@ -80,37 +100,41 @@ jobs:
      run: |
        make mypy
    
-    # Due to issues that cannot be automatically fixed when running `nbqa black . -l 120 --check --diff` on Jupyter notebooks,
-    # we reverted to a version of `black` earlier than 26.1.0 before performing the checks.
    - name: Check Qlib ipynb with nbqa
      run: |
-        python -m pip install "black<26.1"
        make nbqa

    - name: Test data downloads
-      run: |
-        python scripts/get_data.py qlib_data --name qlib_data_simple --target_dir ~/.qlib/qlib_data/cn_data --interval 1d --region cn
-        python scripts/get_data.py download_data --file_name rl_data.zip --target_dir tests/.data/rl
-
-    - name: Install Lightgbm for MacOS
-      if: ${{ matrix.os == 'macos-14' || matrix.os == 'macos-15' }}
-      run: |
-        brew update
-        brew install libomp || brew reinstall libomp
-        python -m pip install --no-binary=:all: lightgbm
+      uses: nick-fields/retry@v4
+      with:
+        timeout_minutes: 15
+        max_attempts: 3
+        retry_wait_seconds: 10
+        shell: bash
+        command: |
+          set -euo pipefail
+          python scripts/get_data.py qlib_data --name qlib_data_simple --target_dir ~/.qlib/qlib_data/cn_data --interval 1d --region cn --delete_old False
+          python scripts/get_data.py download_data --file_name rl_data.zip --target_dir tests/.data/rl --delete_old False

    - name: Check Qlib ipynb with nbconvert
      run: |
        make nbconvert

    - name: Test workflow by config (install from source)
-      run: |
-        python -m pip install numba
-        python qlib/cli/run.py examples/benchmarks/LightGBM/workflow_config_lightgbm_Alpha158.yaml
+      uses: nick-fields/retry@v4
+      with:
+        timeout_minutes: 30
+        max_attempts: 3
+        shell: bash
+        command: |
+          set -euo pipefail
+          python -m pip install numba
+          python -m pip check
+          python qlib/cli/run.py examples/benchmarks/LightGBM/workflow_config_lightgbm_Alpha158.yaml

    - name: Unit tests with Pytest (MacOS)
      if: ${{ matrix.os == 'macos-14' || matrix.os == 'macos-15' }}
-      uses: nick-fields/retry@v2
+      uses: nick-fields/retry@v4
      with:
        timeout_minutes: 60
        max_attempts: 3
@@ -126,10 +150,14 @@ jobs:

    - name: Unit tests with Pytest (Ubuntu and Windows)
      if: ${{ matrix.os != 'macos-13' && matrix.os != 'macos-14' && matrix.os != 'macos-15' }}
-      uses: nick-fields/retry@v2
+      uses: nick-fields/retry@v4
      with:
        timeout_minutes: 60
        max_attempts: 3
        command: |
          cd tests
          python -m pytest . -m "not slow" --durations=0
+
+    - name: Report installed dependencies
+      if: always()
+      run: python -m pip freeze
@@ -13,10 +13,14 @@ on:
 jobs:
  build:
    timeout-minutes: 720
+    env:
+      MLFLOW_ALLOW_FILE_STORE: "true"
+      PIP_CONSTRAINT: ${{ github.workspace }}/.github/ci/constraints.txt
    # we may retry for 3 times for `Unit tests with Pytest`

    runs-on: ${{ matrix.os }}
    strategy:
+      fail-fast: false
      matrix:
        os: [windows-latest, ubuntu-24.04, ubuntu-22.04, macos-14, macos-15]
        # In github action, using python 3.7, pip install will not match the latest version of the package.
@@ -26,35 +30,68 @@ jobs:

    steps:
    - name: Test qlib from source slow
-      uses: actions/checkout@v4
+      uses: actions/checkout@v6
      with:
        fetch-depth: 0

    - name: Set up Python ${{ matrix.python-version }}
-      uses: actions/setup-python@v4
+      uses: actions/setup-python@v6
      with:
        python-version: ${{ matrix.python-version }}

+    - name: Installing pytorch for ubuntu
+      if: ${{ matrix.os == 'ubuntu-24.04' || matrix.os == 'ubuntu-22.04' }}
+      run: |
+        python -m pip install torch --extra-index-url https://download.pytorch.org/whl/cpu
+
    - name: Set up Python tools
+      env:
+        # Slow tests collect the same modules, but do not run lint/docs/build.
+        CI_EXTRAS: dev,test,analysis${{ runner.os == 'Linux' && ',rl' || '' }}
      run: |
-        make dev
-
-    - name: Downloads dependencies data
-      run: |
-        python scripts/get_data.py qlib_data --name qlib_data_simple --target_dir ~/.qlib/qlib_data/cn_data --interval 1d --region cn
+        make ci-install

+    # Importing Qlib's model registry also loads LightGBM and its OpenMP runtime.
    - name: Install Lightgbm for MacOS
      if: ${{ matrix.os == 'macos-14' || matrix.os == 'macos-15' }}
      run: |
        brew update
        brew install libomp || brew reinstall libomp
        python -m pip install --no-binary=:all: lightgbm
+        python -c "import lightgbm; print('LightGBM', lightgbm.__version__, 'native import passed')"
+
+    - name: Verify neural network test dependencies
+      run: python -c "import torch; from qlib.contrib.model import pytorch_classes; from qlib.contrib.model.pytorch_general_nn import GeneralPTNN; from qlib.data.dataset import DatasetH, TSDatasetH; assert pytorch_classes; print('PyTorch', torch.__version__, 'model imports passed')"
+
+    - name: Verify non-Linux dependency footprint
+      if: ${{ runner.os != 'Linux' }}
+      run: python -c "from importlib.metadata import distributions; unexpected = {'tianshou', 'tensorboard', 'grpcio'} & {d.metadata['Name'].lower() for d in distributions()}; assert not unexpected, unexpected"
+
+    - name: Verify dependency consistency
+      run: python -m pip check
+
+    - name: Verify native solvers on Windows Python 3.8
+      if: ${{ matrix.os == 'windows-latest' && matrix.python-version == '3.8' }}
+      run: python -c "import cvxpy, osqp; print('CVXPY', cvxpy.__version__, 'OSQP', osqp.__version__)"
+
+    - name: Downloads dependencies data
+      uses: nick-fields/retry@v4
+      with:
+        timeout_minutes: 15
+        max_attempts: 3
+        retry_wait_seconds: 10
+        shell: bash
+        command: python scripts/get_data.py qlib_data --name qlib_data_simple --target_dir ~/.qlib/qlib_data/cn_data --interval 1d --region cn --delete_old False

    - name: Unit tests with Pytest
-      uses: nick-fields/retry@v2
+      uses: nick-fields/retry@v4
      with:
        timeout_minutes: 240
        max_attempts: 3
        command: |
          cd tests
          python -m pytest . -m "slow" --durations=0
+
+    - name: Report installed dependencies
+      if: always()
+      run: python -m pip freeze
@@ -45,6 +45,7 @@ tags
 .pytest_cache/
 .mypy_cache/
 .vscode/
+node_modules/

 *.swp

@@ -1,4 +1,4 @@
-.PHONY: clean deepclean prerequisite dependencies lightgbm rl develop lint docs package test analysis all install dev black pylint flake8 mypy nbqa nbconvert lint build upload docs-gen
+.PHONY: clean deepclean prerequisite dependencies lightgbm rl develop lint docs package test analysis all install dev ci-install black pylint flake8 mypy nbqa nbconvert lint build upload docs-gen
 #You can modify it according to your terminal
 SHELL := /bin/bash

@@ -12,6 +12,10 @@ PUBLIC_DIR := $(shell [ "$$READTHEDOCS" = "True" ] && echo "$$READTHEDOCS_OUTPUT
 SO_DIR := qlib/data/_libs
 SO_FILES := $(wildcard $(SO_DIR)/*.so)

+# CI selects optional dependencies by platform and workflow. Keep `make dev`
+# unchanged for contributors who want the complete development environment.
+CI_EXTRAS ?= dev,test,analysis
+
 ifeq ($(OS),Windows_NT)
    IS_WINDOWS = true
 else
@@ -110,6 +114,9 @@ install: prerequisite dependencies

 dev: prerequisite all

+ci-install: prerequisite
+	python -m pip install --no-cache-dir -r .github/ci/test-requirements.txt -e ".[$(CI_EXTRAS)]"
+
 ########################################################################################
 # Lint and pre-commit
 ########################################################################################
@@ -0,0 +1,12 @@
+{
+  "name": "qlib-ci",
+  "private": true,
+  "description": "Locked tooling for Qlib's pull request title check",
+  "engines": {
+    "node": ">=22.12.0"
+  },
+  "devDependencies": {
+    "@commitlint/cli": "21.2.2",
+    "@commitlint/config-conventional": "21.2.2"
+  }
+}
@@ -31,11 +31,10 @@ dependencies = [
  # Since version 2.1.0, pandas has deprecated the method parameter of the fillna method. 
  # qlib has updated the fillna method in PR 1987 and limited the minimum version of pandas.
  "pandas>=1.1",
-  # I encoutered an Error that the set_uri does not work when downloading artifacts in mlflow 3.1.1;
-  # But earlier versions of mlflow does not have this problem.
-  # But when I switch to 2.*.* version, another error occurs, which is even more strange...
-  "mlflow",
-  "filelock>=3.16.0",
+  # Qlib's filesystem tracking backend is disabled by default in MLflow 3.13.
+  "mlflow<3.13",
+  # filelock 3.30 rejects forks while another thread changes lock descriptor ownership.
+  "filelock>=3.16.0,<3.30",
  "redis",
  "dill",
  "fire",
@@ -47,10 +46,14 @@ dependencies = [
  "lightgbm",
  "gym",
  "cvxpy",
+  # Use the Windows/Python 3.8 wheel verified in PR #2308, including on fresh installs.
+  "osqp==1.0.5; sys_platform == 'win32' and python_version == '3.8'",
  "joblib",
  "matplotlib",
  "jupyter",
  "nbconvert",
+  # fastjsonschema 2.22 uses Python 3.10 type syntax without declaring the requirement.
+  "fastjsonschema<2.22; python_version < '3.10'",
  "pyarrow",
  "pydantic-settings",
  "setuptools-scm",
@@ -60,6 +63,7 @@ dependencies = [
 dev = [
  "pytest",
  "statsmodels",
+  "tomli; python_version < '3.11'",
 ]
 # On macos-13 system, when using python version greater than or equal to 3.10,
 # pytorch can't fully support Numpy version above 2.0, so, when you want to install torch,
@@ -71,7 +75,8 @@ rl = [
 ]

 lint = [
-  "black",
+  # Keep Python and notebook formatting on the same version family.
+  "black<26.1",
  "pylint",
  "mypy<1.5.0",
  "flake8",
@@ -97,9 +102,12 @@ package = [
 test = [
  "yahooquery",
  "baostock",
+  # Avoid source builds requiring external libxml2 headers on Windows.
+  "lxml<6.1.3",
 ]
 analysis = [
-  "plotly",
+  # Qlib still imports figure_factory.create_distplot, removed in Plotly 7.
+  "plotly<7",
  "statsmodels",
 ]
 client = [
@@ -22,10 +22,13 @@ class MLflowTest(unittest.TestCase):

        This implementation is based on the assumption creating a client is fast
        """
-        start = time.time()
+        # Measure repeated client construction, not one-time store creation,
+        # plugin discovery, or the file-store deprecation warning machinery.
+        mlflow.tracking.MlflowClient(tracking_uri=str(self.TMP_PATH))
+        start = time.perf_counter()
        for i in range(10):
            _ = mlflow.tracking.MlflowClient(tracking_uri=str(self.TMP_PATH))
-        end = time.time()
+        end = time.perf_counter()
        elapsed = end - start
        if platform.system() == "Linux":
            self.assertLess(elapsed, 1e-2)  # it can be done in less than 10ms
@@ -0,0 +1,261 @@
+# Copyright (c) Microsoft Corporation.
+# Licensed under the MIT License.
+
+"""Offline regression checks for the shared source/PyPI CI compatibility policy."""
+
+import ast
+import json
+from pathlib import Path
+import shlex
+import unittest
+
+from packaging.markers import default_environment
+from packaging.requirements import Requirement
+import yaml
+
+try:
+    import tomllib
+except ModuleNotFoundError:
+    import tomli as tomllib
+
+
+ROOT = Path(__file__).resolve().parents[1]
+WORKFLOWS = (
+    "test_qlib_from_source.yml",
+    "test_qlib_from_source_slow.yml",
+    "test_qlib_from_pip.yml",
+)
+
+
+class TestCIConfiguration(unittest.TestCase):
+    @classmethod
+    def setUpClass(cls):
+        cls.workflows = {
+            name: yaml.safe_load((ROOT / ".github/workflows" / name).read_text(encoding="utf-8")) for name in WORKFLOWS
+        }
+        cls.constraints = {
+            requirement.name: requirement
+            for line in (ROOT / ".github/ci/constraints.txt").read_text(encoding="utf-8").splitlines()
+            if line.strip() and not line.startswith("#")
+            for requirement in [Requirement(line)]
+        }
+
+    def test_constraints_match_package_metadata(self):
+        project = tomllib.loads((ROOT / "pyproject.toml").read_text(encoding="utf-8"))["project"]
+        requirements = list(project["dependencies"])
+        for extra in project["optional-dependencies"].values():
+            requirements.extend(extra)
+        requirements = {req.name: req for req in map(Requirement, requirements)}
+        for name, constraint in self.constraints.items():
+            with self.subTest(package=name):
+                self.assertEqual(constraint.specifier, requirements[name].specifier)
+                self.assertEqual(constraint.marker, requirements[name].marker)
+
+    def test_known_bad_versions_are_excluded(self):
+        for name, version in (
+            ("mlflow", "3.15.1"),
+            ("filelock", "3.30.0"),
+            ("fastjsonschema", "2.22.0"),
+            ("plotly", "7.0.0"),
+            ("lxml", "6.1.3"),
+            ("osqp", "1.1.3"),
+            ("osqp", "0.6.7.post3"),
+            ("black", "26.1.0"),
+        ):
+            with self.subTest(package=name, version=version):
+                self.assertNotIn(version, self.constraints[name].specifier)
+
+    def test_compatible_versions_are_allowed(self):
+        for name, version in (
+            ("mlflow", "2.17.2"),
+            ("mlflow", "3.12.0"),
+            ("filelock", "3.16.1"),
+            ("fastjsonschema", "2.21.2"),
+            ("plotly", "6.9.0"),
+            ("lxml", "6.1.2"),
+            ("osqp", "1.0.5"),
+            ("black", "25.12.0"),
+        ):
+            with self.subTest(package=name, version=version):
+                self.assertIn(version, self.constraints[name].specifier)
+
+    def test_platform_bounds_do_not_leak_to_other_environments(self):
+        for platform in ("win32", "linux", "darwin"):
+            for version in ("3.8", "3.9", "3.10", "3.11", "3.12"):
+                env = dict(default_environment(), sys_platform=platform, python_version=version)
+                with self.subTest(platform=platform, python=version):
+                    self.assertEqual(
+                        self.constraints["osqp"].marker.evaluate(env), platform == "win32" and version == "3.8"
+                    )
+                    self.assertEqual(self.constraints["fastjsonschema"].marker.evaluate(env), version in ("3.8", "3.9"))
+
+    def test_all_pip_subprocesses_inherit_absolute_constraints(self):
+        for name, workflow in self.workflows.items():
+            with self.subTest(workflow=name):
+                job = workflow["jobs"]["build"]
+                self.assertEqual(job["env"]["PIP_CONSTRAINT"], "${{ github.workspace }}/.github/ci/constraints.txt")
+                self.assertEqual(job["env"]["MLFLOW_ALLOW_FILE_STORE"], "true")
+                for step in job["steps"]:
+                    self.assertNotIn("PIP_CONSTRAINT", step.get("env", {}))
+
+    def test_full_matrix_is_retained_and_failures_are_not_hidden(self):
+        for name, workflow in self.workflows.items():
+            with self.subTest(workflow=name):
+                job = workflow["jobs"]["build"]
+                self.assertFalse(job["strategy"]["fail-fast"])
+                matrix = job["strategy"]["matrix"]
+                self.assertEqual(set(matrix["python-version"]), {"3.8", "3.9", "3.10", "3.11", "3.12"})
+                self.assertEqual(
+                    set(matrix["os"]), {"windows-latest", "ubuntu-24.04", "ubuntu-22.04", "macos-14", "macos-15"}
+                )
+                self.assertNotIn("exclude", matrix)
+                self.assertNotIn("continue-on-error", job)
+                for step in job["steps"]:
+                    self.assertNotIn("continue-on-error", step)
+                    self.assertNotIn("continue_on_error", step.get("with", {}))
+
+    def test_source_dependency_profiles_match_workflow_responsibilities(self):
+        for name in ("test_qlib_from_source.yml", "test_qlib_from_source_slow.yml"):
+            with self.subTest(workflow=name):
+                steps = self.workflows[name]["jobs"]["build"]["steps"]
+                install = next(step for step in steps if step["name"] == "Set up Python tools")
+                profile = "dev,test,analysis"
+                if name == "test_qlib_from_source.yml":
+                    profile += ",lint"
+                profile += "${{ runner.os == 'Linux' && ',rl' || '' }}"
+                if name == "test_qlib_from_source.yml":
+                    profile += "${{ matrix.os == 'ubuntu-22.04' && ',docs' || '' }}"
+                    docs = next(step for step in steps if step["name"] == "Make html with sphinx")
+                    self.assertEqual(docs["if"], "${{ matrix.os == 'ubuntu-22.04' }}")
+                self.assertEqual(install["env"]["CI_EXTRAS"], profile)
+                self.assertEqual(install["run"].strip(), "make ci-install")
+                self.assertNotIn("package", profile)
+                native = next(step for step in steps if step["name"] == "Verify neural network test dependencies")
+                self.assertNotIn("if", native)
+                self.assertIn("import torch", native["run"])
+                self.assertIn("import GeneralPTNN", native["run"])
+                self.assertIn("import DatasetH, TSDatasetH", native["run"])
+                self.assertIn("assert pytorch_classes", native["run"])
+                footprint = next(step for step in steps if step["name"] == "Verify non-Linux dependency footprint")
+                self.assertEqual(footprint["if"], "${{ runner.os != 'Linux' }}")
+                self.assertIn("{'tianshou', 'tensorboard', 'grpcio'}", footprint["run"])
+                self.assertIn("assert not unexpected", footprint["run"])
+                check = next(step for step in steps if step["name"] == "Verify dependency consistency")
+                self.assertLess(steps.index(install), steps.index(native))
+                self.assertLess(steps.index(native), steps.index(check))
+                self.assertLess(steps.index(footprint), steps.index(check))
+                self.assertNotIn("GRPC_PYTHON_BUILD", str(steps))
+                self.assertNotIn("PIP_NO_BINARY", str(steps))
+
+    def test_macos_lightgbm_runtime_precedes_model_imports(self):
+        for name, workflow in self.workflows.items():
+            with self.subTest(workflow=name):
+                steps = workflow["jobs"]["build"]["steps"]
+                preparation = [step for step in steps if step["name"] == "Install Lightgbm for MacOS"]
+                self.assertEqual(len(preparation), 1)
+                preparation = preparation[0]
+                self.assertEqual(preparation["if"], "${{ matrix.os == 'macos-14' || matrix.os == 'macos-15' }}")
+                self.assertIn("brew install libomp", preparation["run"])
+                consumers = []
+                for step in steps:
+                    command = step.get("run", "") + step.get("with", {}).get("command", "")
+                    if any(
+                        marker in command
+                        for marker in (
+                            "from qlib.contrib.model",
+                            "make nbconvert",
+                            "python -m pytest",
+                            "qrun ",
+                            "python qlib/cli/run.py",
+                        )
+                    ):
+                        consumers.append(step)
+                self.assertTrue(consumers)
+                for step in consumers:
+                    self.assertLess(steps.index(preparation), steps.index(step), step["name"])
+
+    def test_source_test_runtime_retains_torch_and_numerical_bounds(self):
+        requirements = {
+            req.name: req
+            for line in (ROOT / ".github/ci/test-requirements.txt").read_text(encoding="utf-8").splitlines()
+            if line.strip() and not line.startswith("#")
+            for req in [Requirement(line)]
+        }
+        self.assertEqual(set(requirements), {"torch", "numpy", "scipy"})
+        extras = tomllib.loads((ROOT / "pyproject.toml").read_text(encoding="utf-8"))["project"][
+            "optional-dependencies"
+        ]
+        previous = {req.name: req for req in map(Requirement, extras["rl"] + extras["docs"])}
+        for name, req in requirements.items():
+            self.assertEqual(req.specifier, previous[name].specifier)
+            self.assertIsNone(req.marker)
+        makefile = (ROOT / "Makefile").read_text(encoding="utf-8")
+        self.assertIn("dev: prerequisite all", makefile)
+        self.assertIn("ci-install: prerequisite", makefile)
+        self.assertIn('-r .github/ci/test-requirements.txt -e ".[$(CI_EXTRAS)]"', makefile)
+
+    def test_download_retries_are_bounded_and_noninteractive(self):
+        for name, workflow in self.workflows.items():
+            downloads = [
+                step
+                for step in workflow["jobs"]["build"]["steps"]
+                if "data download" in step["name"].lower() or "downloads dependencies" in step["name"].lower()
+            ]
+            with self.subTest(workflow=name):
+                self.assertTrue(downloads)
+                for step in downloads:
+                    options = step["with"]
+                    self.assertEqual(options["max_attempts"], 3)
+                    self.assertEqual(options["timeout_minutes"], 15)
+                    self.assertEqual(options["shell"], "bash")
+                    commands = options["command"]
+                    for line in commands.splitlines():
+                        if "qlib_data --" in line or "download_data --" in line:
+                            self.assertIn("--delete_old False", line)
+                            args = shlex.split(line)
+                            self.assertIs(ast.literal_eval(args[args.index("--delete_old") + 1]), False)
+                    if len(commands.splitlines()) > 1:
+                        self.assertIn("set -euo pipefail", commands)
+
+    def test_pypi_workflow_does_not_import_the_checkout(self):
+        steps = self.workflows["test_qlib_from_pip.yml"]["jobs"]["build"]["steps"]
+        for name in ("Downloads dependencies data", "Test workflow by config"):
+            command = next(step for step in steps if step["name"] == name)["with"]["command"]
+            self.assertIn('cd "$RUNNER_TEMP"', command)
+            self.assertLess(command.index('cd "$RUNNER_TEMP"'), command.index("python"))
+        command = next(step for step in steps if step["name"] == "Test workflow by config")["with"]["command"]
+        self.assertIn('"$GITHUB_WORKSPACE/examples/benchmarks/LightGBM/', command)
+        self.assertIn("not in pathlib.Path(qlib.__file__).resolve().parents", command)
+
+    def test_consistency_and_native_import_are_checked(self):
+        for name, workflow in self.workflows.items():
+            with self.subTest(workflow=name):
+                steps = workflow["jobs"]["build"]["steps"]
+                self.assertTrue(any(step.get("run") == "python -m pip check" for step in steps))
+                native = next(step for step in steps if step["name"] == "Verify native solvers on Windows Python 3.8")
+                self.assertIn("matrix.python-version == '3.8'", native["if"])
+                self.assertIn("matrix.os == 'windows-latest'", native["if"])
+                self.assertIn("import cvxpy, osqp", native["run"])
+                self.assertEqual(steps[-1]["name"], "Report installed dependencies")
+                self.assertEqual(steps[-1]["if"], "always()")
+
+    def test_title_lint_uses_a_lockfile_and_no_dynamic_download(self):
+        workflow = yaml.safe_load((ROOT / ".github/workflows/lint_title.yml").read_text(encoding="utf-8"))
+        steps = workflow["jobs"]["lint-title"]["steps"]
+        setup = next(step for step in steps if step["name"] == "Setup Node.js")
+        self.assertEqual(setup["with"]["node-version"], "22")
+        install = next(step for step in steps if step["name"] == "Install commitlint")
+        self.assertIn("npm ci", install["run"])
+        validate = next(step for step in steps if step["name"] == "Validate PR Title with commitlint")
+        self.assertIn("npx --no-install", validate["run"])
+        self.assertEqual(validate["env"]["BODY"], "${{ github.event.pull_request.title }}")
+        self.assertNotIn("${{", validate["run"])
+        manifest = json.loads((ROOT / "package.json").read_text(encoding="utf-8"))
+        lock = json.loads((ROOT / "package-lock.json").read_text(encoding="utf-8"))
+        self.assertEqual(manifest["devDependencies"], lock["packages"][""]["devDependencies"])
+        for name, version in manifest["devDependencies"].items():
+            self.assertEqual(version, lock["packages"]["node_modules/" + name]["version"])
+
+
+if __name__ == "__main__":
+    unittest.main()