Intraday realized-volatility forecasting at the one-minute grid.
The stack stitches together

• classic econometrics (Student-t EGARCH, RFSV, wavelet fractals)
• cross-asset graph neural nets (rolling correlation GCN)
• a lightweight Transformer forecaster

Everything is YAML-driven and packaged with strict typing, PyPI pinning and a Makefile for easy incremental runs.

• Date range : 2023-01-04 -> 2024-12-31
• Universe : 11 tickers
  AAPL, AMZN, GOOGL, META, MSFT, NFLX, NVDA, TSLA, SPY, QQQ, VXX
• Benchmarks : SPY, QQQ, VXX (used for the GNN residual design matrix)
• Rows : 2 137 111 minute bars
• Source : Polygon.io free API

    src/forecast_vol/          importable package
      ├─ market/               NYSE calendar
      ├─ preprocess/           gap-fill, minute features
      ├─ synergy/              HMM regimes + rolling GCN
      ├─ stochastic/           EGARCH, fractal, RFSV
      └─ attention/            Transformer predictor
    configs/                   YAML configs
    tests/                     pytests
    Makefile                   dev shortcuts
    Dockerfile                 slim Python 3.13 base
    data/                      raw, interim, processed, external
    reports/                   all plots and summaries

    git clone https://github.com/diljit22/forecast_vol.git
    cd forecast_vol

    # editable install + dev tooling
    make dev-install

    # build NYSE calendar once
    make build-sessions

    # full end-to-end pipeline (CPU)
    make pipeline

    # run tests
    make test

The default configuration is CPU-only. GPU users can point configs/*.yaml to a cuda device and install the matching wheels.

    docker build -t forecast_vol .
    docker run --rm -v "$PWD/data:/app/data" forecast_vol make pipeline

Note: for Cuda the appropriate wheels should be added.

CSV bars were downloaded from the Polygon.io minute endpoint (free tier).

Minute files must be named <TICKER>_1m.csv.
If your feed differs, tweak the loader in preprocess/restrict.py. As it crawls for files of named in this format:

    data/raw/
    └── 2023/
        ├── stock/   AAPL_1m.csv  MSFT_1m.csv  ... 8 total
        └── etf/     SPY_1m.csv   QQQ_1m.csv   VXX_1m.csv
    └── 2024/
        ├── stock/   AAPL_1m.csv  MSFT_1m.csv  ... 8 total
        └── etf/     SPY_1m.csv   QQQ_1m.csv   VXX_1m.csv

Raw Schema for the ticker data follows this format (for each minute)

Market calander data was collected from the NYSE calendar and compiled in a JSON. This is later parsed into Unix epoch ms.

market/build_sessions.py builds an NYSE calendar JSON with epoch-ms open/close pairs (09:30 to 16:00 or 09:30 to 13:00 on half-days). It is called by make build-sessions Run once:

Data is cleaned via make restrict and make resample. The former trims raw CSV to the trading grid and audits gaps while the latter fills missing minutes, synthesizes High/Low with a small epsilon wiggle and recomputes VWAP.

For each row ticker and each row the following basic features are added via the make build-dataset pipeline:

- add_active_neighbourhood : prev/next trading-day flags
- add_distance_open_close  : millis since open / until close
- add_simple_returns       : close-to-close percent return
- add_log_return           : log return of close price
- add_parkinson            : Parkinson high-low volatility
- add_realised_var         : rolling realised variance
- add_bipower              : bipower variation estimator
- add_garman               : Garman-Klass volatility
- add_iqv                  : integrated quartic variation
- add_trade_intensity      : first-difference volume proxy
- add_vov                  : volatility-of-volatility
- one hot encoding for ticker symbols

All tickers are also merged into one parquet in this step.

Labels each ticker into three volatility regimes:

• Numbers are minute bars inside each regime

• design matrix = minute returns minus benchmark residuals
• Optuna tunes (edges >= |thr|, hidden_dim, layers, lr)
• best model: thr=0.493, hidden=128, layers=3, lr=5.7e-4
• rolling daily embeddings written to data/interim/synergy_daily/

GNN correlation model Optuna sweep (top 5)

The best trial (row 1) is automatically written to data/interim/synergy_daily/ and its hyper-parameters are written to reports/synergy/best_params.yaml.

*(Full 20-trial CSV lives in reports/synergy/optuna.csv.

This stage adds three per-ticker volatility features to complement minute-level data:

Artifacts live in data/interim/stochastic.parquet
and a full CSV report is written to reports/stochastic/fit_report.csv.

Optuna-selected hyper-parameters

Chronological CV, 2023-01-04 -> 2024-12-31 (11 tickers)

95% of absolute errors fall inside +- 0.002 realized-variance units. However this is probably due to the scale of data (minute var is inherently low and despite normalization techniques I could not discern it fully)

    make test

Coverage:

• calendar logic
• feature-engineering pipe
• correlation matrix + GNN forward
• EGARCH / RFSV return finite values
• Transformer Predictor shape check

MIT license - see LICENSE.

    @software{singh2025_forecastvol,
      author = {Diljit Singh},
      title  = {forecast_vol: Minute-level volatility forecasting},
      year   = 2025,
      url    = {https://github.com/diljit22/forecast_vol}
    }