Commit Graph
19 Commits
Author SHA1 Message Date
XianBWandCopilot 7c95268c0c fix(security)!: harden artifact serialization and loading (#2339)
* security: harden artifact serialization and loading

* ci: restore dependency and title lint compatibility

* fix: address artifact test and lint failures

* ci: handle current MLflow and Windows OSQP changes

* ci: use lxml release with cross-platform wheels

* fix: defer optional CVXPY solver imports

* docs: mark notebook model artifacts as trusted

* docs: mark position artifacts as trusted

* style: document intentional lazy import

* ci: skip broken released OSQP on Windows Python 3.8

* fix: mark online model artifacts as trusted

* test: make MLflow client timing check robust

* ci: retry nondeterministic benchmark smoke test

* fix: cap filelock before fork safety regression

* fix: use filelock release compatible with multiprocessing

* fix: preserve artifact loading compatibility after hardening

* fix(security): validate derived artifact paths and extend regressions

* fix(security): preserve extended data artifact round trips

* fix(security): retain legacy NumPy and pandas artifact classes

Allow the exact MaskedArray and SparseDtype module paths emitted by the older dependency versions in CI. Preserve restricted loading and cover both old and new paths.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(data): avoid unused trading calendar lookup in PIT normalization

PIT normalization uses report dates and calendar-day offsets, so do not request an unused Baostock calendar. Cover annual and quarterly normalization without online calendar access.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(security): require explicit trust in artifact workflows

Expose default-off artifact trust at online, delayed-training and meta-workflow entry points. Keep data loading restricted, preserve legacy default recorder calls and restored workflow defaults, and document migration and trust boundaries.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* test(workflow): cover complete DDG-DA and HIST artifact flows

Exercise real training, cache and recorder restoration, saved-task replay, signals and portfolio backtests. Fix the cache trust, grouping, relative-path, LightGBM early-stopping and HIST CPU blockers exposed by full execution.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(security)!: unify artifact trust and publish upgrade guidance

Use trusted consistently across public APIs and CLI options. Migrate pre-release saved consent only at serialization boundaries, reject invalid or conflicting settings, and preserve restricted data reads. Add a canonical upgrade guide with README, installation, changelog and runtime-error entry points.

BREAKING CHANGE: executable artifact restoration requires explicit trusted=True. HIST mappings use JSON and high-frequency cache paths must remain within artifact_root. See docs/start/artifact_migration.rst for upgrading existing workflows.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(security): remove trial-only trust compatibility

Target upgrades from pre-PR Qlib rather than artifacts generated by intermediate PR revisions. Retain restricted defaults for saved objects without consent, unify current APIs, and remove state/keyword migration branches and documentation.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* ci: avoid failed dataset probes and space download retries

Download the same existing v2 archives used by successful jobs without probing nonexistent package-version assets. Allow five download attempts 60 seconds apart; preserve the full matrix, datasets, noninteractive setup and failure propagation. Cover the exact archives and retry policy in offline CI tests.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs: add practical artifact migration examples

Show explicit consent for restored online components while preserving the original manager, schema-aware Arrow conversions, timezone-preserving ZoneInfo migration, and restricted round trips into a separate recorder. Explain shared-component scope, staging artifacts and trusted-source prerequisites.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(workflow): preserve fresh artifact workflows with scoped trust

Keep immutable cache references intact at open rolling tails. Add explicit per-load record-template consent and per-artifact collector loading options without widening other reads. Restore the original template class after parent-load failures, and support audited business-frequency and NumPy record representations through exact pickle globals.

Document the scoped APIs and cover fresh trained-model reads, cache generation, numerical round trips and refusal paths.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-10-08 17:49:08 +08:00
Chia-hung Tai 184ce34a34 [807] Move the REG_CONSTANT/EPS to constant.py. (#811)
* [807] Move the REG_CONSTANT to constant.py.

* import REG_US.

* Move EPS to constant.py.
2022-01-09 21:39:46 +08:00
you-n-g 3493f29e16 Enhance Task Dict Var (#778) 2021-12-26 00:18:44 +08:00
Christian Clauss b8d1e08010 Fix undefined names in Python code (#599)
* Update pytorch_tabnet.py

$ `flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics`
```
./qlib/qlib/contrib/model/pytorch_tabnet.py:567:38: F821 undefined name 'inp'
            self.independ.append(GLU(inp, out_dim, vbs=vbs))
                                     ^
./qlib/examples/model_rolling/task_manager_rolling.py:75:18: F821 undefined name 'task_train'
        run_task(task_train, self.task_pool, experiment_name=self.experiment_name)
                 ^
2     F821 undefined name 'task_train'
2
```

* Fix undefined names in Python code

* from qlib.model.trainer import task_train
2021-09-14 12:13:27 +08:00
Young 9303415666 refactor online serving rolling api 2021-07-29 18:13:12 +08:00
lzh222333 231bdf8608 Merge remote-tracking branch 'microsoft/main' into online_srv 2021-06-01 08:29:02 +00:00
zhupr 98eacf8f88 add test/config.py 2021-05-28 13:24:47 +08:00
lzh222333 b24af7fff6 multiprocessing support 2021-05-24 05:07:38 +00:00
lzh222333 ebd01e0de5 Online Serving V11 2021-05-14 06:44:16 +00:00
lzh222333 67c5740c83 OnlineServing V9 2021-04-29 04:30:09 +00:00
lzh222333 42f510024c update collector 2021-04-27 04:12:08 +00:00
lzh222333 319396c815 online serving V8 2021-04-25 06:26:45 +00:00
lzh222333 cec318fbfe online serving V7 2021-04-16 05:37:13 +00:00
lzh222333 c20eb5c8a6 format code 2021-04-08 03:30:24 +00:00
lzh222333 cb42e99bee bug fixed & examples fire 2021-04-07 03:33:27 +00:00
lzh222333 bd7a1c11b9 trainer & group & collect & ensemble 2021-04-02 04:27:14 +00:00
lzh222333 edcd7b1ff9 bug fixed & code format 2021-03-31 03:08:48 +00:00
lzh222333 544365f3a9 ensemble & get_exp & dataset_pickle 2021-03-31 02:39:14 +00:00
lzh222333 46cd57688e Online Serving V4 2021-03-26 04:20:25 +00:00