Hi, I do not mean to specify the number of steps before a network update. I hope to specify the total number of environment steps in an entire training session. For example, stop training after 3.0e7 steps.

I hope to do so because when training Atari games, we usually plot the epsiodic return as a function of total number of environment steps that the agent has experienced. And I would like to specify the maximum number of environment steps, not the maximum number of episodes.

https://github.com/thu-ml/tianshou/blob/master/examples/atari/atari_ppo.py

For example, if you want to set total steps 3e7, you can set epoch=100 with --step-per-epoch=3e5: total_steps == epoch * step_per_epoch; you can change number of epoch to control the evaluation interval

Thank you! Another small question: what will happen if the game does not end when reaching the maximum number of steps in an epoch? Will the game stop immediately, or will it stop only after the current episode ends (so the actual steps will be slightly larger than the maximum steps)?

it will stop stepping env, switch to test env, do one evaluation, then continue stepping the partially stepped training env