In this repository, we are trying different ways to make reinforcement learning environments from Mujoco Gym and dm_control deterministic. We strive to ensure that the environments have the following important properties:

• the function reset() gives always the same initial state;
• the function step(action) such that a sequence of actions uniquely determines states and rewards;
• the function virtual_step(state, action) uniquely determines next_state and reward.

We implemented a wrapper for the environments that aims to fulfill these points, but unfortunately, it works not for all environments yet. The main reason is that Mujoco has internal variables and structures to which there is no obvious access. For example, in many environments, next_state and reward depend not only on the current states, but also on "internal physics". Nonetheless, we strive to take such things into account. Our results are described in the table below.

Here DMControlEnv(domain_name, task_name) is a simple wrapper to use usuall gym interface; DMControlEnvWithPhysics(domain_name, task_name) is a special wrapper which implicitly saves "internal physics" of environments as an additional attribute of the numpy array state.

Here GymEnv(domain_name, task_name) is a special wrapper which implicitly saves MjSimState object as an additional attribute of the numpy array state.

Support for Windows has been dropped in newer versions of mujoco-py. The latest working version is 1.50.1.68. But even here you can’t do without dancing with a tambourine.

Microsoft Visual C++ 14.0 or greater. https://visualstudio.microsoft.com/visual-cpp-build-tools/

1. Download binaries: http://www.roboti.us/download.html/mjpro150_win64.zip and activation key: http://www.roboti.us/license.html

2. Create directory %userprofile%/.mujoco/.

3. Unzip the binaries and move the key to the created directory.

4. Add full path to directory %userprofile%/.mujoco/mjpro150/bin into a variable environments PATH.

5. Download mujoco-py versions 1.50.1.68: https://files.pythonhosted.org/packages/cf/8c/64e0630b3d450244feef0688d90eab2448631e40ba6bdbd90a70b84898e7/mujoco-py-1.50.1.68.tar.gz

6. Unzip the downloaded archive to an arbitrary directory, navigate to this directory in the terminal and install mujoco-py using command: python setup.py install

1. Before each use, you must execute the commands

import os
os.add_dll_directory(os.path.join(os.path.expanduser('~'), ".mujoco", "mjpro150", "bin"))
from mujoco_py import GlfwContext
GlfwContext(True)

2. Loading environments is done via gym

import gym
env = gym.make("Ant-v3")

3. Further use of environments - habitual.

import gym
import matplotlib.pyplot as plt

env = gym.make("Ant-v3")
state = env.reset()
pixels = env.render("rgb_array")
plt.imshow(pixels)

Follow the instructions

https://github.com/openai/mujoco-py/blob/master/README.md