mirror of
https://github.com/microsoft/qlib.git
synced 2026-11-13 22:41:11 +08:00
XianBWandCopilot
7c95268c0c
* security: harden artifact serialization and loading * ci: restore dependency and title lint compatibility * fix: address artifact test and lint failures * ci: handle current MLflow and Windows OSQP changes * ci: use lxml release with cross-platform wheels * fix: defer optional CVXPY solver imports * docs: mark notebook model artifacts as trusted * docs: mark position artifacts as trusted * style: document intentional lazy import * ci: skip broken released OSQP on Windows Python 3.8 * fix: mark online model artifacts as trusted * test: make MLflow client timing check robust * ci: retry nondeterministic benchmark smoke test * fix: cap filelock before fork safety regression * fix: use filelock release compatible with multiprocessing * fix: preserve artifact loading compatibility after hardening * fix(security): validate derived artifact paths and extend regressions * fix(security): preserve extended data artifact round trips * fix(security): retain legacy NumPy and pandas artifact classes Allow the exact MaskedArray and SparseDtype module paths emitted by the older dependency versions in CI. Preserve restricted loading and cover both old and new paths. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(data): avoid unused trading calendar lookup in PIT normalization PIT normalization uses report dates and calendar-day offsets, so do not request an unused Baostock calendar. Cover annual and quarterly normalization without online calendar access. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(security): require explicit trust in artifact workflows Expose default-off artifact trust at online, delayed-training and meta-workflow entry points. Keep data loading restricted, preserve legacy default recorder calls and restored workflow defaults, and document migration and trust boundaries. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * test(workflow): cover complete DDG-DA and HIST artifact flows Exercise real training, cache and recorder restoration, saved-task replay, signals and portfolio backtests. Fix the cache trust, grouping, relative-path, LightGBM early-stopping and HIST CPU blockers exposed by full execution. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(security)!: unify artifact trust and publish upgrade guidance Use trusted consistently across public APIs and CLI options. Migrate pre-release saved consent only at serialization boundaries, reject invalid or conflicting settings, and preserve restricted data reads. Add a canonical upgrade guide with README, installation, changelog and runtime-error entry points. BREAKING CHANGE: executable artifact restoration requires explicit trusted=True. HIST mappings use JSON and high-frequency cache paths must remain within artifact_root. See docs/start/artifact_migration.rst for upgrading existing workflows. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(security): remove trial-only trust compatibility Target upgrades from pre-PR Qlib rather than artifacts generated by intermediate PR revisions. Retain restricted defaults for saved objects without consent, unify current APIs, and remove state/keyword migration branches and documentation. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * ci: avoid failed dataset probes and space download retries Download the same existing v2 archives used by successful jobs without probing nonexistent package-version assets. Allow five download attempts 60 seconds apart; preserve the full matrix, datasets, noninteractive setup and failure propagation. Cover the exact archives and retry policy in offline CI tests. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * docs: add practical artifact migration examples Show explicit consent for restored online components while preserving the original manager, schema-aware Arrow conversions, timezone-preserving ZoneInfo migration, and restricted round trips into a separate recorder. Explain shared-component scope, staging artifacts and trusted-source prerequisites. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(workflow): preserve fresh artifact workflows with scoped trust Keep immutable cache references intact at open rolling tails. Add explicit per-load record-template consent and per-artifact collector loading options without widening other reads. Restore the original template class after parent-load failures, and support audited business-frequency and NumPy record representations through exact pickle globals. Document the scoped APIs and cover fresh trained-model reads, cache generation, numerical round trips and refusal paths. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
48 lines
4.1 KiB
ReStructuredText
48 lines
4.1 KiB
ReStructuredText
.. _highfreq:
|
|
|
|
========================================================================
|
|
Design of Nested Decision Execution Framework for High-Frequency Trading
|
|
========================================================================
|
|
.. currentmodule:: qlib
|
|
|
|
Introduction
|
|
============
|
|
|
|
Daily trading (e.g. portfolio management) and intraday trading (e.g. orders execution) are two hot topics in Quant investment and are usually studied separately.
|
|
|
|
To get the join trading performance of daily and intraday trading, they must interact with each other and run backtest jointly.
|
|
In order to support the joint backtest strategies at multiple levels, a corresponding framework is required. None of the publicly available high-frequency trading frameworks considers multi-level joint trading, which makes the backtesting aforementioned inaccurate.
|
|
|
|
Besides backtesting, the optimization of strategies from different levels is not standalone and can be affected by each other.
|
|
For example, the best portfolio management strategy may change with the performance of order executions(e.g. a portfolio with higher turnover may become a better choice when we improve the order execution strategies).
|
|
To achieve overall good performance, it is necessary to consider the interaction of strategies at a different levels.
|
|
|
|
Therefore, building a new framework for trading on multiple levels becomes necessary to solve the various problems mentioned above, for which we designed a nested decision execution framework that considers the interaction of strategies.
|
|
|
|
.. image:: ../_static/img/framework.svg
|
|
|
|
The design of the framework is shown in the yellow part in the middle of the figure above. Each level consists of ``Trading Agent`` and ``Execution Env``. ``Trading Agent`` has its own data processing module (``Information Extractor``), forecasting module (``Forecast Model``) and decision generator (``Decision Generator``). The trading algorithm generates the decisions by the ``Decision Generator`` based on the forecast signals output by the ``Forecast Module``, and the decisions generated by the trading algorithm are passed to the ``Execution Env``, which returns the execution results.
|
|
|
|
The frequency of the trading algorithm, decision content and execution environment can be customized by users (e.g. intraday trading, daily-frequency trading, weekly-frequency trading), and the execution environment can be nested with finer-grained trading algorithm and execution environment inside (i.e. sub-workflow in the figure, e.g. daily-frequency orders can be turned into finer-grained decisions by splitting orders within the day). The flexibility of the nested decision execution framework makes it easy for users to explore the effects of combining different levels of trading strategies and break down the optimization barriers between different levels of the trading algorithm.
|
|
|
|
The optimization for the nested decision execution framework can be implemented with the support of `QlibRL <./rl/overall.html>`_. To know more about how to use the QlibRL, go to API Reference: `RL API <../reference/api.html#rl>`_.
|
|
|
|
Example
|
|
=======
|
|
|
|
An example of a nested decision execution framework for high-frequency can be found `here <https://github.com/microsoft/qlib/blob/main/examples/nested_decision_execution/workflow.py>`_.
|
|
|
|
|
|
Besides, the above examples, here are some other related works about high-frequency trading in Qlib.
|
|
|
|
.. note::
|
|
|
|
New source builds require ``HighFreqProvider`` artifact paths, including derived
|
|
cache files, to remain inside ``artifact_root`` (the current directory by
|
|
default). Cached pickle contents still require independent trust. See
|
|
:ref:`artifact_loading_migration` before reusing existing provider configurations.
|
|
|
|
- `Prediction with high-frequency data <https://github.com/microsoft/qlib/tree/main/examples/highfreq#benchmarks-performance-predicting-the-price-trend-in-high-frequency-data>`_
|
|
- `Examples <https://github.com/microsoft/qlib/blob/main/examples/orderbook_data/>`_ to extract features from high-frequency data without fixed frequency.
|
|
- `A paper <https://github.com/microsoft/qlib/tree/high-freq-execution#high-frequency-execution>`_ for high-frequency trading.
|