Official implementation of the experiments in "Loss Shaping Constraints for Long-Term Time Series Forecasting" (ICML 2024). Forked from Autoformer.
Environment setup:
# Create and activate conda environment
conda env create -f environment.yml
conda activate loss-shapingSet up W&B (optional):
wandb loginThe datasets can be downloaded from this link. Alternatively, run make get_dataset. The expected structure is {root_path}/{data_path}.csv.
The following examples show how to run ERM and constrained trainings using the Autoformer model on the ECL dataset.
To avoid using W&B, you can add the flag WANDB_DISABLED=true to the python command, or just use the CLI prompt when running the script.
python run.py \
--wandb_project WandbProject \
--wandb_run example-run \
--model Autoformer \
--model_id electricity \
--data custom \
--root_path ./dataset/electricity/ \
--data_path electricity.csv \
--features M \
--seq_len 96 \
--label_len 48 \
--pred_len 192 \
--e_layers 3 \
--d_layers 1 \
--factor 3 \
--enc_in 321 \
--dec_in 321 \
--c_out 321 \
--n_heads 16 \
--d_model 128 \
--d_ff 256 \
--dropout 0.2 \
--train_epochs 25 \
--patience 10 \
--batch_size 32 \
--learning_rate 0.0001 \
--is_training 1 \
--constraint_type ermpython run.py \
--wandb_project WandbProject \
--wandb_run example-run \
--model Autoformer \
--model_id electricity \
--data custom \
--root_path ./dataset/electricity/ \
--data_path electricity.csv \
--features M \
--seq_len 96 \
--label_len 48 \
--pred_len 192 \
--e_layers 3 \
--d_layers 1 \
--factor 3 \
--enc_in 321 \
--dec_in 321 \
--c_out 321 \
--n_heads 16 \
--d_model 128 \
--d_ff 256 \
--dropout 0.2 \
--train_epochs 25 \
--patience 10 \
--batch_size 32 \
--learning_rate 0.0001 \
--is_training 1 \
--constraint_type constant \
--constraint_level 0.167 \
--dual_lr 0.01 \
--dual_init 1.0Key arguments (added by us):
constraint_type: Type of constraint to use during training.ERMis unconstrained.- Supported constraint types:
erm, constant, static_linear, dynamic_linear, resilience, monotonic, static_exponential
- Supported constraint types:
constraint_level: Upper bound on loss at each timestep (for constant constraints)dual_lr: Learning rate for dual variablesdual_init: Initial value for dual variables.
Alternatively, the sweeps/ directory contains YAML files for running W&B Sweeps, which were used for the paper experiments. To run:
wandb sweep sweeps/{sweep_name}.yaml
wandb agent {sweep_id} #get the sweep_id from the previous command's output.For more details on other arguments see the argparse help, or refer to the original Autoformer repository.
Transformer-Based:
- Autoformer
- Informer
- Reformer
- Pyraformer
- iTransformer
- Nonstationary Transformer
- Vanilla Transformer
Other:
- FiLM (Feature-wise Linear Modulation)
- Electricity Consumption Load (ECL)
- ETT (Electricity Transformer Temperature)
- Weather
- Exchange Rate
- Traffic
- Influenza-Like Illness (ILI)
This implementation builds upon the Autoformer repository. We thank the original authors for their open-source contribution.