8000 Adds checkpoint frequencies for serial and batch Agents. by prabhatnagarajan · Pull Request #525 · chainer/chainerrl · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Adds checkpoint frequencies for serial and batch Agents. #525

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 10 commits into from
Sep 13, 2019
Merged

Adds checkpoint frequencies for serial and batch Agents. #525

merged 10 commits into from
Sep 13, 2019

Conversation

prabhatnagarajan
Copy link
Contributor

We may want to checkpoint intermediate networks during training. This PR modifies one example, and the experiments API to allow that for serial agents.

One use case of this update could be T-REX (https://arxiv.org/abs/1904.06387), which generates synthetic demonstrations by taking different checkpoints of a neural network and using them to generate demonstrations, which may be ranked according to the checkpoint order.

@prabhatnagarajan prabhatnagarajan changed the title Adds checkpoint frequencies for training serial agents Adds checkpoint frequencies for Serial and Batch Agents. Aug 20, 2019
@prabhatnagarajan
Copy link
Contributor Author

Testing:

I ran python train_ppo_ale.py --gpu -1 --checkpoint-frequency 20 --steps 100

and got the following output:

INFO:chainerrl.experiments.train_agent_batch:Saved the agent to results/20190820T214258.089467/40_checkpoint
INFO:chainerrl.experiments.train_agent_batch:Saved the agent to results/20190820T214258.089467/60_checkpoint
INFO:chainerrl.experiments.train_agent_batch:Saved the agent to results/20190820T214258.089467/80_checkpoint
INFO:chainerrl.experiments.train_agent_batch:Saved the agent to results/20190820T214258.089467/100_checkpoint
INFO:chainerrl.experiments.train_agent_batch:Saved the agent to results/20190820T214258.089467/104_finish

I also checked some of the directories, and they indeed exist.

@prabhatnagarajan
Copy link
Contributor Author

I also ran on train_dqn_ale.py. I ran the following:

python examples/atari/train_dqn_ale.py --gpu -1 --checkpoint-frequency 20 --steps 100

with the following output:

INFO:chainerrl.experiments.train_agent:Saved the agent to results/20190820T214138.012278/40_checkpoint
INFO:chainerrl.experiments.train_agent:Saved the agent to results/20190820T214138.012278/60_checkpoint
INFO:chainerrl.experiments.train_agent:Saved the agent to results/20190820T214138.012278/80_checkpoint
INFO:chainerrl.experiments.train_agent:outdir:results/20190820T214138.012278 step:100 episode:0 R:2.0```

I checked that the directories were created, and that there were files inside them.

@prabhatnagarajan prabhatnagarajan changed the title Adds checkpoint frequencies for Serial and Batch Agents. Adds checkpoint frequencies for serial and batch Agents. Aug 21, 2019
@ummavi
Copy link
Member
ummavi commented Sep 13, 2019

/test

@pfn-ci-bot
Copy link
Collaborator

Successfully created a job for commit 649b3c1:

@prabhatnagarajan prabhatnagarajan merged commit b14faec into chainer:master Sep 13, 2019
@prabhatnagarajan prabhatnagarajan deleted the chkpt_freq branch September 13, 2019 08:11
@muupan muupan added this to the v0.8 milestone Feb 6, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ummavi ummavi ummavi approved these changes

Assignees
No one assigned
Labels
enhancement
Projects
None yet
Milestone
v0.8
Development

Successfully merging this pull request may close these issues.
4 participants
@prabhatnagarajan @ummavi @pfn-ci-bot @muupan
0