@Trinkle23897 Just for you to know, I'm an engineer and for the first time, and thanks to this PR (and all the previous ones) a real case application is finally running without raising an exception 😆 Perhaps it is time to release 0.2.4 😛

Sure and much appreciate~
How about the documentation update?

Yes I'll do it now. Could you please tell me what/where to update them and what you have in mind exactly ?

https://github.com/thu-ml/tianshou/blob/master/CONTRIBUTING.md

Yes I know this, but you mean only updating this page https://tianshou.readthedocs.io/en/latest/api/tianshou.data.html ?

Exactly. I mean that you can add the new feature in https://tianshou.readthedocs.io/en/latest/api/tianshou.data.html#tianshou.data.Batch

Batch has other methods, including __getitem__(), __len__(), append(), and split():

also with buffer.

You can make the documentation show __xxx__ by modifying this line:

Added missing documentation. It turns out that I didn't modified the behaviour of Buffer by any means, it was only internal modifications, so I have not updated it.

Could it support slicing method such as batch[:, i]?
Because I find that the advance RNN hidden state (#19) needs this and otherwise I should recursively get the value.

In practice it shouldn't be very hard, but conceptually Batch is missing the notion of multidimensional samples, for instance only len is available, and it corresponds to the dimension to the first axis of the stored data. The first step would be to add a shape method, similar to numpy. After that It shouldn't be much trouble. Maybe you could open an issue.

The issue is that list cannot be supported since computing the shape of a list is very expensive and grows exponentially with depth.

Adding this property to Batch class would do the trick (note that isinstance if ... else should be used to early return [] in case of native Python scalar) :

    @property
    def shape(self) -> List[int]:
        """Return self.size."""
        if len(self.__dict__.keys()) == 0:
            return []
        else:
            data_shape = []
            for v in self.__dict__.values():
                try:
                    data_shape.append(v.shape)
                except AttributeError:
                    raise TypeError("No support for 'shape' method with "\
                                    f"type {type(v)} in class Batch.")
            return min(*data_shape) if len(data_shape) > 1 else data_shape[0]

Then, _valid_bounds(len(self), index) must be replaced by np.all(map(_valid_bounds, zip(self.shape, index)), ONLY in the case where index is a actually a list of list of indices. No need to change anything else, since the rest of the method is already compatible with multidimensional indexing (expect for list, but list is not supported by shape, so an exception will be raised before reaching this point).

NB: You can ping me on the issue/PR if you want a review.

By the way, it may be good to overload direct assignment to make sure that every list are converted to numpy. Indeed, list are a recurrent issue for slicing, stacking and concatenation, and it introduces many isinstance if ... else ... for no real advantage.