• I have marked all applicable categories:
  • exception-raising bug
  • RL algorithm bug
  • documentation request (i.e. "X is missing from the documentation.")
  • new feature request
• I have visited the source website
• I have searched through the issue tracker for duplicates
• I have mentioned version numbers, operating system and environment, where applicable:

  import tianshou, torch, numpy, sys
  print(tianshou.__version__, torch.__version__, numpy.__version__, sys.version, sys.platform)

Hello,

It seems that episode start signals are currently not used (hidden state must be reset when such signal is encountered) when working with RNN and on-policy algorithms.
This may explain poor performance reported in #470 (comment).

More context:
SB3-Contrib Implementation: Stable-Baselines-Team/stable-baselines3-contrib#53
CleanRL (@vwxyzjn implementation): vwxyzjn/cleanrl#83