* security: harden artifact serialization and loading * ci: restore dependency and title lint compatibility * fix: address artifact test and lint failures * ci: handle current MLflow and Windows OSQP changes * ci: use lxml release with cross-platform wheels * fix: defer optional CVXPY solver imports * docs: mark notebook model artifacts as trusted * docs: mark position artifacts as trusted * style: document intentional lazy import * ci: skip broken released OSQP on Windows Python 3.8 * fix: mark online model artifacts as trusted * test: make MLflow client timing check robust * ci: retry nondeterministic benchmark smoke test * fix: cap filelock before fork safety regression * fix: use filelock release compatible with multiprocessing * fix: preserve artifact loading compatibility after hardening * fix(security): validate derived artifact paths and extend regressions * fix(security): preserve extended data artifact round trips * fix(security): retain legacy NumPy and pandas artifact classes Allow the exact MaskedArray and SparseDtype module paths emitted by the older dependency versions in CI. Preserve restricted loading and cover both old and new paths. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(data): avoid unused trading calendar lookup in PIT normalization PIT normalization uses report dates and calendar-day offsets, so do not request an unused Baostock calendar. Cover annual and quarterly normalization without online calendar access. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(security): require explicit trust in artifact workflows Expose default-off artifact trust at online, delayed-training and meta-workflow entry points. Keep data loading restricted, preserve legacy default recorder calls and restored workflow defaults, and document migration and trust boundaries. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * test(workflow): cover complete DDG-DA and HIST artifact flows Exercise real training, cache and recorder restoration, saved-task replay, signals and portfolio backtests. Fix the cache trust, grouping, relative-path, LightGBM early-stopping and HIST CPU blockers exposed by full execution. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(security)!: unify artifact trust and publish upgrade guidance Use trusted consistently across public APIs and CLI options. Migrate pre-release saved consent only at serialization boundaries, reject invalid or conflicting settings, and preserve restricted data reads. Add a canonical upgrade guide with README, installation, changelog and runtime-error entry points. BREAKING CHANGE: executable artifact restoration requires explicit trusted=True. HIST mappings use JSON and high-frequency cache paths must remain within artifact_root. See docs/start/artifact_migration.rst for upgrading existing workflows. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(security): remove trial-only trust compatibility Target upgrades from pre-PR Qlib rather than artifacts generated by intermediate PR revisions. Retain restricted defaults for saved objects without consent, unify current APIs, and remove state/keyword migration branches and documentation. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * ci: avoid failed dataset probes and space download retries Download the same existing v2 archives used by successful jobs without probing nonexistent package-version assets. Allow five download attempts 60 seconds apart; preserve the full matrix, datasets, noninteractive setup and failure propagation. Cover the exact archives and retry policy in offline CI tests. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * docs: add practical artifact migration examples Show explicit consent for restored online components while preserving the original manager, schema-aware Arrow conversions, timezone-preserving ZoneInfo migration, and restricted round trips into a separate recorder. Explain shared-component scope, staging artifacts and trusted-source prerequisites. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(workflow): preserve fresh artifact workflows with scoped trust Keep immutable cache references intact at open rolling tails. Add explicit per-load record-template consent and per-artifact collector loading options without widening other reads. Restore the original template class after parent-load failures, and support audited business-frequency and NumPy record representations through exact pickle globals. Document the scoped APIs and cover fresh trained-model reads, cache generation, numerical round trips and refusal paths. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
HIST
- Code: https://github.com/Wentao-Xu/HIST
- Paper: HIST: A Graph-based Framework for Stock Trend Forecasting via Mining Concept-Oriented Shared InformationAdaRNN: Adaptive Learning and Forecasting for Time Series.
Stock-index mapping migration
The bundled qlib_csi300_stock_index.npy object mapping has been replaced by
qlib_csi300_stock_index.json, preserving all
735 entries. The example YAML already
uses the new file. Update the same setting in your own workflow YAML:
task:
model:
kwargs:
stock_index: "benchmarks/HIST/qlib_csi300_stock_index.json"
This path follows the example's convention of running from examples/; adjust it
to your working directory. stock2concept is a separate numeric matrix and still
uses a two-dimensional .npy loaded without pickle. Do not rename or replace that
matrix with the JSON mapping. It must include the unknown-stock row 733 (at
least 734 rows), and all mapped indices must be within its row bounds.
A previously saved HIST model retains its own path. After independently trusting
and restoring it, update model.stock_index to the matching JSON file before
prediction or further fitting. Changing YAML alone does not update a restored
model. Preserve all 735 bundled entries and their original matrix row assignments.
For a known-trusted custom mapping, re-export it from your original trusted metadata or producer into a JSON object:
- Keys must be instrument strings, matching your Qlib dataset.
- Values must be non-negative integers (not strings, floats or booleans) indexing
rows in the corresponding
stock2conceptmatrix. - Preserve each instrument's original row assignment, and verify all indices are within the matrix's row bounds.
- Point
task.model.kwargs.stock_indexat your new.jsonfile and keep the matching concept matrix.
If you only have an old object-pickled .npy, recover or regenerate the mapping
from the trusted source rather than loading an unknown file to convert it.
Changing the filename extension alone does not convert the contents. HIST
deliberately rejects the old object format because it requires executable pickle
deserialization; there is no fallback or recorder trust flag that re-enables it.
See the artifact loading migration guide for the separate policy on model/dataset artifacts. The JSON mapping change does not make pre-existing local model checkpoints or other pickle loaders safe; those inputs still require independent trust.
Full workflow regression
From the repository root, with the test and model dependencies installed:
python -m pytest tests/model/test_hist_workflow.py -m slow -q
This offline CPU regression runs real Alpha360/DatasetH preparation, one HIST training epoch, signal analysis and a six-day TopkDropout backtest in an isolated MLflow store. It checks 48 prediction/label rows, actual optimizer updates, finite reports and trading activity. Saved model/dataset objects are refused by default; explicitly trusted reloads reproduce predictions exactly, including in a fresh Python process. Metadata and concept fixtures are local JSON and numeric NumPy files, with no downloads. This is functional integration coverage, not a paper-scale model-quality benchmark.