https://github.com/thu-ml/tianshou/blob/bf7841078dc5d4357ba8046efb656674af20f54b/tianshou/policy/multiagent/mapolicy.py#L63