qlib/examples/benchmarks_dynamic/DDG-DA

Introduction

This is the implementation of DDG-DA based on Meta Controller component provided by Qlib.

Please refer to the paper for more details: DDG-DA: Data Distribution Generation for Predictable Concept Drift Adaptation [arXiv]

Background

In many real-world scenarios, we often deal with streaming data that is sequentially collected over time. Due to the non-stationary nature of the environment, the streaming data distribution may change in unpredictable ways, which is known as concept drift. To handle concept drift, previous methods first detect when/where the concept drift happens and then adapt models to fit the distribution of the latest data. However, there are still many cases that some underlying factors of environment evolution are predictable, making it possible to model the future concept drift trend of the streaming data, while such cases are not fully explored in previous work.

Therefore, we propose a novel method DDG-DA, that can effectively forecast the evolution of data distribution and improve the performance of models. Specifically, we first train a predictor to estimate the future data distribution, then leverage it to generate training samples, and finally train models on the generated data.

Dataset

The data in the paper are private. So we conduct experiments on Qlib's public dataset. Though the dataset is different, the conclusion remains the same. By applying DDG-DA, users can see rising trends at the test phase both in the proxy models' ICs and the performances of the forecasting models.

Run the Code

After verifying that the recorder artifacts and local working files are your own and cannot be replaced by untrusted writers (see below), run from this directory:

    python workflow.py --trusted=True run

The default forecasting models are Linear. Users can choose other forecasting models by changing the forecast_model parameter when DDG-DA initializes. For example, users can try LightGBM forecasting models by running the following command:

    python workflow.py --trusted=True --conf_path=../baseline/workflow_config_lightgbm_Alpha158.yaml run

Recorder artifacts and local working files

workflow.py exposes the DDGDA workflow through the DDGDABench Fire entry point. Its trusted option defaults to False. Set it explicitly only for artifacts and caches from a verified writer in access-controlled MLflow and local storage: unrestricted pickle loading can execute code. Creating a run yourself is not enough if someone else can overwrite its files.

The option covers recorder-backed executable meta-model/task loading, including InternalData.setup, and DDG-DA's local handler/internal-data pickle cache reads; prediction, label and numerical-report artifact loads remain restricted. Lower-level callers can also pass trusted=True to MetaDatasetDS or InternalData.setup for recorder task reads, not as a global local-cache grant. A refused load is not a reason to retry automatically with trust enabled. See the artifact loading migration guide for data compatibility and migration details.

This example also saves and reuses local pickle files in working_dir, which the benchmark sets to this directory. Handler/internal-data caches default to restricted loading, which refuses executable objects such as Alpha158 or InternalData. The explicit opt-in allows these caches to be restored with ordinary pickle and emits a warning; it does not authenticate their contents. Protect working_dir, the configuration directory (also used for handler caching), and any supplied h_path from untrusted writes. Do not copy unknown cached handlers, meta-information or models into them. There is no automatic unsafe retry and no change to the global restricted loader. Other pickle APIs and workflow YAML retain their own trust requirements; only use trusted configurations and files.

Generated tasks keep lightweight handler-cache configuration references, including the path and chosen trusted setting, rather than embedding the full market data. Do not assume task["dataset"]["kwargs"]["handler"] is a file:// string. Treat saved task configurations as executable inputs; reusing an opted-in task also reuses that local-cache consent independently of a new workflow's default. An old exported task may need regeneration with its matching meta-model and configuration to obtain the current cache reference and selected policy. Loading a saved task containing a reweighter through a recorder still requires explicit recorder consent; that flag alone does not globally authorize local cache reads.

The Makefile's clean target deletes local pickle files and mlruns; preserve any results you need before using it. Deleting experiments or running a full retrain is not necessary to migrate ordinary trusted artifacts.

Full workflow regression

From the repository root, with the test and model dependencies installed:

python -m pytest tests/rolling_tests/test_ddgda.py -m slow -q

The offline regression uses deterministic local daily market data and an isolated MLflow store. Both linear and LightGBM similarity models run through feature selection, seven similarity-training windows, daily rank IC, cache restoration, 30-epoch meta-training, inferred time weights, two rolling training windows, prediction/label collection and a 40-day portfolio backtest. It checks default refusal, explicit authorization, restored predictions, delayed replay of both saved rolling tasks and non-empty numerical results without replacing workflow stages with mocks. These small integration cases verify functionality, not paper-scale performance or investment returns.

Results

The results of related methods in Qlib's public dataset can be found here

Requirements

Here are the minimal hardware requirements to run the workflow.py of DDG-DA.

• Memory: 45G
• Disk: 4G

Pytorch with CPU & RAM will be enough for this example.