• I have marked all applicable categories:
  • exception-raising bug
  • RL algorithm bug
  • documentation request (i.e. "X is missing from the documentation.")
  • new feature request
• I have visited the source website
• I have searched through the issue tracker for duplicates
• I have mentioned version numbers, operating system and environment, where applicable:

  import tianshou, gymnasium as gym, torch, numpy, sys
  print(tianshou.__version__, gym.__version__, torch.__version__, numpy.__version__, sys.version, sys.platform)

I have been trying Tianshou's multi-agent reinforcement learning compatibility with PettingZoo for multi-agent cooperative 2d gridworld, similar to the Prisoner-Guard example in PettingZoo: https://pettingzoo.farama.org/tutorials/environmentcreation/3-action-masking/.

I realise that for the Simultaneous Move scenario where all agents move simultaneously, the agents seem to be using the same shared action, instead of having each agent being given their own action instead. This is evident in the PettingZoo_env.py wrapper where the action seems to be shared.

Does Tianshou provide any way to allow each agent to have their own action rather than a shared action for the Simultaneous Move case?

The documentation for MARL: https://tianshou.readthedocs.io/en/master/tutorials/cheatsheet.html#multi-agent-reinforcement-learning, seems to indicate that simultaneous moves for MOBA games looks possible. It also makes sense for each agent to have their unique actions rather than shared.

Else a turn-based scenario seems to be the most probable method to fall back on, though not ideal.

Also, almost all examples between PettingZoo and Tianshou for MARL show a turn-based non-cooperative example, such as TicTacToe. But i was curious to see if there are any simultaneous move cooperative examples.