* security: harden artifact serialization and loading
* ci: restore dependency and title lint compatibility
* fix: address artifact test and lint failures
* ci: handle current MLflow and Windows OSQP changes
* ci: use lxml release with cross-platform wheels
* fix: defer optional CVXPY solver imports
* docs: mark notebook model artifacts as trusted
* docs: mark position artifacts as trusted
* style: document intentional lazy import
* ci: skip broken released OSQP on Windows Python 3.8
* fix: mark online model artifacts as trusted
* test: make MLflow client timing check robust
* ci: retry nondeterministic benchmark smoke test
* fix: cap filelock before fork safety regression
* fix: use filelock release compatible with multiprocessing
* fix: preserve artifact loading compatibility after hardening
* fix(security): validate derived artifact paths and extend regressions
* fix(security): preserve extended data artifact round trips
* fix(security): retain legacy NumPy and pandas artifact classes
Allow the exact MaskedArray and SparseDtype module paths emitted by the older dependency versions in CI. Preserve restricted loading and cover both old and new paths.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix(data): avoid unused trading calendar lookup in PIT normalization
PIT normalization uses report dates and calendar-day offsets, so do not request an unused Baostock calendar. Cover annual and quarterly normalization without online calendar access.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix(security): require explicit trust in artifact workflows
Expose default-off artifact trust at online, delayed-training and meta-workflow entry points. Keep data loading restricted, preserve legacy default recorder calls and restored workflow defaults, and document migration and trust boundaries.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* test(workflow): cover complete DDG-DA and HIST artifact flows
Exercise real training, cache and recorder restoration, saved-task replay, signals and portfolio backtests. Fix the cache trust, grouping, relative-path, LightGBM early-stopping and HIST CPU blockers exposed by full execution.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix(security)!: unify artifact trust and publish upgrade guidance
Use trusted consistently across public APIs and CLI options. Migrate pre-release saved consent only at serialization boundaries, reject invalid or conflicting settings, and preserve restricted data reads. Add a canonical upgrade guide with README, installation, changelog and runtime-error entry points.
BREAKING CHANGE: executable artifact restoration requires explicit trusted=True. HIST mappings use JSON and high-frequency cache paths must remain within artifact_root. See docs/start/artifact_migration.rst for upgrading existing workflows.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix(security): remove trial-only trust compatibility
Target upgrades from pre-PR Qlib rather than artifacts generated by intermediate PR revisions. Retain restricted defaults for saved objects without consent, unify current APIs, and remove state/keyword migration branches and documentation.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* ci: avoid failed dataset probes and space download retries
Download the same existing v2 archives used by successful jobs without probing nonexistent package-version assets. Allow five download attempts 60 seconds apart; preserve the full matrix, datasets, noninteractive setup and failure propagation. Cover the exact archives and retry policy in offline CI tests.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* docs: add practical artifact migration examples
Show explicit consent for restored online components while preserving the original manager, schema-aware Arrow conversions, timezone-preserving ZoneInfo migration, and restricted round trips into a separate recorder. Explain shared-component scope, staging artifacts and trusted-source prerequisites.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix(workflow): preserve fresh artifact workflows with scoped trust
Keep immutable cache references intact at open rolling tails. Add explicit per-load record-template consent and per-artifact collector loading options without widening other reads. Restore the original template class after parent-load failures, and support audited business-frequency and NumPy record representations through exact pickle globals.
Document the scoped APIs and cover fresh trained-model reads, cache generation, numerical round trips and refusal paths.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: use baostock to fetch trading calendar instead of Eastmoney API
* fix: lint error
* fix: lint error
* ci: enable concurrency to cancel in-progress runs for same workflow and ref
---------
Co-authored-by: Linlang Lv (iSoftStone Information) <v-llv@microsoft.com>
* fix: replace deprecated pandas fillna(method=) with ffill()/bfill()
Replace deprecated fillna(method="ffill"/"bfill") calls with modern
pandas ffill() and bfill() methods to fix FutureWarnings in pandas 2.x.
Also includes black formatting fixes for compliance.
This addresses the pandas deprecation warnings portion of issue #1981.
Other issues (date parsing, type conversion, timezone handling) will be
addressed in separate commits.
Fixes:
- Yahoo collector: 2 instances in calc_change() and adjusted_price()
- BaoStock collector: 1 instance in calc_change()
- Core utils: resam.py fillna operations
- Backtest: profit_attribution.py stock data processing
- High-freq ops: FFillNan and BFillNan operators
- Position analysis: parse_position.py weight processing
Partially addresses GitHub issue #1981
* lint with black
* lint with black
* limit minimum version of pandas
* limit minimum version of pandas
---------
Co-authored-by: Linlang <Lv.Linlang@hotmail.com>
* fixed a problem with multi index caused by the default value of groupkey
* modify group_key default value
* limit pandas verion
* format with black
* fix docs error
* fix docs error
* fixed bugs caused by pandas upgrade
* remove needless code
* reformat with black
* limit version & add docs
* fix: resolve#1892 by retriving the data page by page
* fix: resolve#1892 by retriving the data page by page
* reformat with black
---------
Co-authored-by: shengyuhong <shengyuhong@bytedance.com>
Co-authored-by: fibers <yu8582@126.com>
* #854 implement first data health checker draft
* #854 added support for qlib's data format, implemented factor check, reformatted summary
* adaptation current dataset
* format with black
* add data health check to docs
* fix sphinx error
* fix pylint error
* update code
* format with black
* format with pylint
---------
Co-authored-by: Linlang <Lv.Linlang@hotmail.com>
* Fix FutureWarning: Passing unit-less datetime64 dtype to .astype is deprecated and will raise in a future version. Pass 'datetime64[ns]' instead
* align index format while end date contains current day data
* fix black
* fix black
* optimize code
* optimize code
* optimize code
* fix ci error
* check ci error
* fix ci error
* check ci error
* check ci error
* check ci error
* check ci error
* check ci error
* check ci error
* fix ci error
* fix ci error
* fix ci error
* fix ci error
* fix ci error
---------
Co-authored-by: Cadenza-Li <362237642@qq.com>
Co-authored-by: Linlang <Lv.Linlang@hotmail.com>
* fix the bug that the HS_SYMBOLS_URL is 404
* fix bug
* format with black
* fix pylint error
* change error code
* fix ci error
* fix ci error
* optimize code
* optimize code
* add comments
---------
Co-authored-by: Linlang <Lv.Linlang@hotmail.com>
* download orderbook data
* fix CI error
* fix CI error
* test fix CI error
* test fix CI error
* test fix CI error
* test fix CI error
* test fix CI error
* test fix CI error
* test fix CI error
* test fix CI error
* test fix CI error
* test fix CI error
* test fix CI error
* test fix CI error
* test fix CI error
* test fix CI error
* test fix CI error
* test fix CI error
* test fix CI error
* test fix CI error
* test fix CI error
* optimize get_data code
* optimize get_data code
* optimize get_data code
* optimize README
---------
Co-authored-by: Linlang <v-linlanglv@microsoft.com>
* add_baostock_collector
* modify_comments
* fix_pylint_error
* solve_duplication_methods
* modified the logic of update_data_to_bin
* modified the logic of update_data_to_bin
* optimize code
* optimize pylint issue
* fix pylint error
* changes suggested by the review
* fix CI faild
* fix CI faild
* fix issue 1121
* format with black
* optimize code logic
* optimize code logic
* fix error code
* drop warning during code runs
* optimize code
* format with black
* fix bug
* format with black
* optimize code
* optimize code
* add comments
* Intermediate version
* Fix yaml template & Successfully run rolling
* Be compatible with benchmark
* Get same results with previous linear model
* Black formatting
* Update black
* Update the placeholder mechanism
* Update CI
* Update CI
* Upgrade Black
* Fix CI and simplify code
* Fix CI
* Move the data processing caching mechanism into utils.
* Adjusting DDG-DA
* Organize import
* Update YahooNormalizeUS1dExtend(#1196)
* Prevent pandas read_csv errors while running update_data_to_bin for US region
* Fix parse_index error while running update_data_to_bin for US region
* prevent pandas.read_csv error on specific symbol names
* Reordering parameters for better rendering
* removes prefix during feature_dir existence checking
* add explanation comments
* fix gramma error in doc strings
* fix typos in exchange.py
* fix typos and gramma errors
* fix typo and rename function param to avoid shading python keyword
* remove redundant parathesis; pass kwargs to parent class
* fix pyblack
* further correction
* assign -> be assigned to
* Explain data crawler structure
* Add documentation for data and feature
* Update scripts/data_collector/yahoo/README.md
Co-authored-by: you-n-g <you-n-g@users.noreply.github.com>
* Remove some confusing wording
* Add third party data source
* Fix command typo
* Update commands
Co-authored-by: you-n-g <you-n-g@users.noreply.github.com>
XianBWandCopilot
LinlangandLinlang Lv
kzhdev
shauryaMi12andLinlang
Alaa KaddourandLinlang
you-n-gandLinlang
Ben HeckmannandLinlang
codecnotsupported
Di
Lee YuntongandLeeYuntong
YangandLinlang
Hao ZhaoandLinlang
fei longandLinlang Lv
Ikko Eltociear AshimineandLinlang Lv
Andy li
zhuan
YQ Tsui
Hyeongmin Moon
lerit
Lewen Wang
YaOzI