This repository accompanies Goldfeder, J., Sipple, J., Real-World Data and Calibrated Simulation Suite for Offline Training of Reinforcement Learning Agents to Optimize Energy and Emission in Office Buildings, currently under review at Neurips 2024, and builds off of Goldfeder, J., Sipple, J., (2023). A Lightweight Calibrated Simulation Enabling Efficient Offline Learning for Optimal Control of Real Buildings, BuildSys '23, November 15–16, 2023, Istanbul, Turkey
The best place to jump in is the Soft Actor Critic Demo notebook, available in notebooks/SAC_Demo.ipynb
This will walk you through:
-
Creating an RL (gym compatible) environment
-
Visualizing the env
-
Training an agent using the Tensorflow Agents Library
Before you run this notebook, make sure to go through the setup instructions below to ensure the notebook runs successfully.
Follow these steps to setup locally before you run the
notebooks/SAC_Demo.ipynb notebook. Note: this will only work on linux, as some
libraries are not supported by other operating systems.
-
Clone the repository
-
Ensure you have
protocandffmpeginstalled, as well aspython >=3.10.12 and <3.12. You can install these runningsudo apt install -y protobuf-compilerandsudo apt install -y ffmpeg -
Create a virtual environment by running
python -m venv .venv. Activate the environmentsource .venv/bin/activate. Then, install poetry withpip install poetry -
Install the dependencies by running
poetry install --with dev -
Build the
.protofiles atsmart_control/protointo python files by running:cd smart_control/proto protoc --python_out=. smart_control_building.proto \ smart_control_normalization.proto \ smart_control_reward.proto cd ../..
-
Modify the value of
VIDEO_PATH_ROOTatsmart_control/simulator/constants.py. This is the path where simulation videos will be stored -
Now in the
notebooks/SAC_Demo.ipynbnotebook, modify the values ofdata_path,metrics_path,output_data_pathandroot_dir. In particular,data_pathshould point to thesim_config.ginfile atsmart_control/configs/sim_config.gin -
Now you are ready to run the
notebooks/SAC_Demo.ipynbnotebook
In addition to our calibrated simulator, we released 6 years of data on 3 buildings, for further calibration, and to use, in conjunction with the simulator, for training and evaluating RL models. The dataset is part of Tensorflow Datasets.
Here is an Unofficial Community-run Documentation Site containing more information about the project and the codebase.
We welcome your contributions to this repository!
All open source contributors will need to sign Google's Contributor License Agreement (CLA).
Contributors are encouraged to consult the sections below for more information about code documentation, testing, and formatting.
We encourage you to document your code using docstrings. Specifically we use the Google Docstring Guidelines outlined in the Google Python Style Guide.
We encourage you to add tests to ensure your code is working as expected.
Running tests:
# run all tests:
pytest
# disable warnings:
pytest --disable-pytest-warnings
# run specific test files:
pytest --disable-pytest-warnings path/to/your/test.py
# run specific tests:
pytest --disable-pytest-warnings -k your_test_name_hereWe are using pyink to format Python code
according to
Google Python Style Guidelines.
The formatter will automatically update files inplace.
The formatter will run automatically as a pre-commit hook (see "Pre-commit Hooks" section below for more information and setup instructions).
Additionally, for contributors using the VS Code text editor, we have configured
a VS Code workspace settings file to run the formatter whenever a file is saved.
NOTE: this requires the
ms-python.black-formatter extension
for VS Code.
If you would like to run the formatter manually:
# format all the files:
pyink .
# format a specific file or directory:
pyink /path/to/file/or/dirIf you would like to perform a dry run:
# check if a file would be changed:
pyink . --check
# see what changes would be made:
pyink . --diffIf you would like to prevent certain lines of code from being formatted (for
example to leave a long line as-is), it is possible to
ignore formatting
by adding a trailing comment of # fmt: skip to the right of the line / at the
end of the expression, or by wrapping multiple lines of code between
# fmt: off and # fmt: on comments. NOTE: pyink and pylint (see section
below) may each require their own separate set of comments, however pyink
respects many pylint comments, so you are recommended to try using a pylint
comment first, and then only also add a pyink comment as necessary.
We are using isort to control the sort order
of Python import statements, specifically grouping the "smart_control" local
module imports separately in their own section below the package imports.
The import sorter will run automatically as a pre-commit hook (see "Pre-commit Hooks" section below for more information and setup instructions).
Additionally, for contributors using the VS Code text editor, we have configured
a VS Code workspace settings file to run the import sorter whenever a file is
saved. NOTE: this requires the
ms-python.isort extension
for VS Code.
If you would like to run the import sorter manually:
# sort all the files:
isort .
# sort a specific file:
isort /path/to/file.py
# sort with verbose outputs (helpful for troubleshooting):
isort -v .We are using pylint to
check for Python style formatting issues that pyink doesn't fix, to more
closely follow
Google Python style guidelines.
The style checker will NOT automatically update files inplace, but rather will
produce a report containing any errors that you will need to fix manually.
The style checker will run automatically as a pre-commit hook (see "Pre-commit Hooks" section below for more information and setup instructions).
If you would like to run the style checker manually:
# check all files:
pylint --rcfile=.pylintrc --ignore=proto smart_control
# check a specific file:
pylint --rcfile=.pylintrc --ignore=proto smart_control/path/to/file.pyTo check for a specific issue (e.g. "missing-module-docstring"), using the corresponding message code (e.g. "C0114"):
pylint smart_control --rcfile=.pylintrc --ignore=proto --disable=all --enable=C0114If you would like to prevent certain lines of code from being checked (for
example to leave a long line as-is), it is possible to
ignore formatting
for a given message (e.g. "line-too-long") by adding a trailing comment of
# pylint: disable=line-too-long to the right of the line / at the end of the
expression, or by wrapping multiple lines of code between
# pylint: disable=line-too-long and # pylint: enable=line-too-long comments.
We are using mdformat to check for
formatting errors in markdown files.
The markdown formatter will run automatically as a pre-commit hook (see "Pre-commit Hooks" section below for more information and setup instructions).
If you would like to run the markdown formatter manually:
# format specific files:
mdformat README.md
# check if a file would be changed:
mdformat README.md --checkNOTE: it would be nice to check all markdown files, however this includes all markdown files in the ".venv" folder (not desired), and the functionality for ignoring certain directories is only supported in Python 3.13+. When we upgrade we can consider updating the approach, but right now we are only targeting specific files.
The mdformat tool might not be able to format long lines containing code
fences, so some manual review may still be required.
Long lines caused by links are OK to keep as-is.
We are using pre-commit hooks to perform code formatting, import sorting, and style checking. These actions will take place on each commit.
To enable the pre-commit hooks, you must perform a one-time setup by running
pre-commit install. This will update ".git/hooks/pre-commit".
If you would like to run the pre-commit hooks without making a commit:
# run against staged files only:
pre-commit run
# run against all files:
pre-commit run --all-files
# run against a specific set of file(s):
pre-commit run --files path/to/my_file.py path/to/other_file.pyIf you encounter issues and need to clear the cache:
pre-commit cleanIf you would like to make a commit and skip the hooks (not recommended), use the
--no-verify flag with your git commit command.