• I have marked all applicable categories:
  • exception-raising bug
  • RL algorithm bug
  • documentation request (i.e. "X is missing from the documentation.")
  • new feature request
• I have visited the source website
• I have searched through the issue tracker for duplicates
• I have mentioned version numbers, operating system and environment, where applicable:

  import tianshou, torch, sys
  print(tianshou.__version__, torch.__version__, sys.version, sys.platform)

yields

0.3.0 1.7.0 3.6.9 (default, Oct  8 2020, 12:12:24)
[GCC 8.4.0] linux

Hi,
I have a suggestion regarding loading and saving the replay buffer. My understanding is that the currently implemented solution uses pickle. However, pickle has a large memory overhead which can be problematic for large replay buffers. In my case pickling a replay buffer of about 2,5 GB took more than 17 GB of memory. In some cases this overhead is too large.

I therefore implemented saving to and loading from HDF5 files. The idea is to represent a Batch by a group in the HDF5 file whereas tensors and numpy arrays are represented by datasets. This would add a dependency on h5py.

There is still an overhead when loading the replay buffer, but it is smaller. In the example above the peak memory usage was 7,3 GB. This could possibly be improved though as I am not an expert when it comes to HDF5.

If you are interested, I can make a pull request.