Commit Graph
17 Commits
Author SHA1 Message Date
XianBWandCopilot 7c95268c0c fix(security)!: harden artifact serialization and loading (#2339)
* security: harden artifact serialization and loading

* ci: restore dependency and title lint compatibility

* fix: address artifact test and lint failures

* ci: handle current MLflow and Windows OSQP changes

* ci: use lxml release with cross-platform wheels

* fix: defer optional CVXPY solver imports

* docs: mark notebook model artifacts as trusted

* docs: mark position artifacts as trusted

* style: document intentional lazy import

* ci: skip broken released OSQP on Windows Python 3.8

* fix: mark online model artifacts as trusted

* test: make MLflow client timing check robust

* ci: retry nondeterministic benchmark smoke test

* fix: cap filelock before fork safety regression

* fix: use filelock release compatible with multiprocessing

* fix: preserve artifact loading compatibility after hardening

* fix(security): validate derived artifact paths and extend regressions

* fix(security): preserve extended data artifact round trips

* fix(security): retain legacy NumPy and pandas artifact classes

Allow the exact MaskedArray and SparseDtype module paths emitted by the older dependency versions in CI. Preserve restricted loading and cover both old and new paths.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(data): avoid unused trading calendar lookup in PIT normalization

PIT normalization uses report dates and calendar-day offsets, so do not request an unused Baostock calendar. Cover annual and quarterly normalization without online calendar access.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(security): require explicit trust in artifact workflows

Expose default-off artifact trust at online, delayed-training and meta-workflow entry points. Keep data loading restricted, preserve legacy default recorder calls and restored workflow defaults, and document migration and trust boundaries.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* test(workflow): cover complete DDG-DA and HIST artifact flows

Exercise real training, cache and recorder restoration, saved-task replay, signals and portfolio backtests. Fix the cache trust, grouping, relative-path, LightGBM early-stopping and HIST CPU blockers exposed by full execution.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(security)!: unify artifact trust and publish upgrade guidance

Use trusted consistently across public APIs and CLI options. Migrate pre-release saved consent only at serialization boundaries, reject invalid or conflicting settings, and preserve restricted data reads. Add a canonical upgrade guide with README, installation, changelog and runtime-error entry points.

BREAKING CHANGE: executable artifact restoration requires explicit trusted=True. HIST mappings use JSON and high-frequency cache paths must remain within artifact_root. See docs/start/artifact_migration.rst for upgrading existing workflows.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(security): remove trial-only trust compatibility

Target upgrades from pre-PR Qlib rather than artifacts generated by intermediate PR revisions. Retain restricted defaults for saved objects without consent, unify current APIs, and remove state/keyword migration branches and documentation.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* ci: avoid failed dataset probes and space download retries

Download the same existing v2 archives used by successful jobs without probing nonexistent package-version assets. Allow five download attempts 60 seconds apart; preserve the full matrix, datasets, noninteractive setup and failure propagation. Cover the exact archives and retry policy in offline CI tests.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs: add practical artifact migration examples

Show explicit consent for restored online components while preserving the original manager, schema-aware Arrow conversions, timezone-preserving ZoneInfo migration, and restricted round trips into a separate recorder. Explain shared-component scope, staging artifacts and trusted-source prerequisites.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(workflow): preserve fresh artifact workflows with scoped trust

Keep immutable cache references intact at open rolling tails. Add explicit per-load record-template consent and per-artifact collector loading options without widening other reads. Restore the original template class after parent-load failures, and support audited business-frequency and NumPy record representations through exact pickle globals.

Document the scoped APIs and cover fresh trained-model reads, cache generation, numerical round trips and refusal paths.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-10-08 17:49:08 +08:00
Linlang 3097dcc995 fix(security): use RestrictedUnpickler in load_instance (#2153)
* fix(security): enforce RestrictedUnpickler for load_instance to prevent unsafe pickle deserialization

* fix: lint error
2026-03-10 20:45:38 +08:00
codecnotsupported bda374180a Update links to chenditc/investment_data to always point to latest release (#1877)
* Update README.md

Link to latest release.
https://docs.github.com/en/repositories/releasing-projects-on-github/linking-to-releases#linking-to-the-latest-release

* Update README.md

Link to latest release.
https://docs.github.com/en/repositories/releasing-projects-on-github/linking-to-releases#linking-to-the-latest-release

* Update README.md

Link to latest release.
https://docs.github.com/en/repositories/releasing-projects-on-github/linking-to-releases#linking-to-the-latest-release

* Update README.md

Link to latest release.
https://docs.github.com/en/repositories/releasing-projects-on-github/linking-to-releases#linking-to-the-latest-release

* Update README.md

* Update README.md
2025-01-03 13:56:49 +08:00
you-n-g 5190332c7e Add some misc features. (#1816)
* Normal mod

* Black linting

* Linting
2024-06-26 18:34:00 +08:00
you-n-g be4646b4b7 Adjust rolling api (#1594)
* Intermediate version

* Fix yaml template & Successfully run rolling

* Be compatible with benchmark

* Get same results with previous linear model

* Black formatting

* Update black

* Update the placeholder mechanism

* Update CI

* Update CI

* Upgrade Black

* Fix CI and simplify code

* Fix CI

* Move the data processing caching mechanism into utils.

* Adjusting DDG-DA

* Organize import
2023-07-14 12:16:12 +08:00
Wendi Li cd4ab998fb Update on Dynamic Benchmark (#1539)
* move config file to benchmark_dynamic & switch default sim task model to GBDT

* Update benchmark_dynamic results

* Change the default value of alpha of DDG-DA
2023-06-03 08:42:24 +08:00
Fivele-LiandYoung 370477288d fix_DDG-DA_workflow_bug (#1516)
* 1.specify group_keys=False to avoid FutureWarning;
2.fix get train_start from dict unexpected problem;

* fix black

* Add comments

* Add make file

---------

Co-authored-by: Young <afe.young@gmail.com>
2023-05-24 15:49:58 +08:00
you-n-g 32c3070b73 Refine DDG-DA (#1472)
* Run ddg-da successfully

* Support include valid; More parameters

* Support L2 reg & visualization

* Blackformat

* Enable fill_method

* Support specify handler & optim dataset

* Fix Pylint
2023-04-07 15:00:21 +08:00
Wendi Li ebb8ec34f3 [DDG-DA] Update crowd-sourced data results (#1405)
* [DDG-DA] Update crowd-sourced data experiments

* Remove internal data version

* Modify README
2023-01-03 00:15:50 +08:00
you-n-g e24ef67663 Update README.md 2022-06-14 10:53:09 +08:00
you-n-g 2d5eecb9a2 Update README.md 2022-06-14 10:52:50 +08:00
you-n-g 6edd0bf298 fix ddgda run all bug & pylint (#1031) 2022-04-03 20:43:02 +08:00
Wendi Li 0e8b94a552 Update README.md (#915)
Add the memory and disk requirement of DDG-DA.
2022-02-15 09:27:51 +08:00
you-n-g 7f274b1e4e Fix code and docs for issues (#853)
* Docs for model and strategy

* add some docs about workflow and online

* safe_load yaml

* DDG-DA paper link and comments for code
2022-01-17 13:57:44 +08:00
you-n-g 0304df0d5b Update README.md 2022-01-10 16:56:18 +08:00
Young 181ee3c070 FIX File Name 2022-01-10 16:55:20 +08:00
cf35562e84 DDG-DA paper code (#743)
* Merge data selection to main

* Update trainer for reweighter

* Typos fixed.

* update data selection interface

* successfully run exp after refactor some interface

* data selection share handler &  trainer

* fix meta model time series bug

* fix online workflow set_uri bug

* fix set_uri bug

* updawte ds docs and delay trainer bug

* docs

* resume reweighter

* add reweighting result

* fix qlib model import

* make recorder more friendly

* fix experiment workflow bug

* commit for merging master incase of conflictions

* Successful run DDG-DA with a single command

* remove unused code

* asdd more docs

* Update README.md

* Update & fix some bugs.

* Update configuration & remove debug functions

* Update README.md

* Modfify horizon from code rather than yaml

* Update performance in README.md

* fix part comments

* Remove unfinished TCTS.

* Fix some details.

* Update meta docs

* Update README.md of the benchmarks_dynamic

* Update README.md files

* Add README.md to the rolling_benchmark baseline.

* Refine the docs and link

* Rename README.md in benchmarks_dynamic.

* Remove comments.

* auto download data

Co-authored-by: wendili-cs <wendili.academic@qq.com>
Co-authored-by: demon143 <785696300@qq.com>
2022-01-10 16:52:37 +08:00