Support Basic MARL #619
Support Basic MARL #619
Conversation
This is not a good assumption. The environment's input per step can be (action, player_id) and many players share the same environment. My previous solution to this issue is to extend the API from action array to action dict, for example, https://github.com/sail-sg/envpool/blob/c2b5b6c679304c976d821e2519e17cf3e9f4f98e/envpool/python/envpool.py#L101-L109 {
"env_id": [0, 1, 2, 3],
"players.env_id": [1, 2, 0, 2, 0, 1, 1, 2]
"players.action: [x, x, x, x, x, x, x, x],
"players.id": [a, b, c, a, b, c, d, e],
}It shows 4 environments, 5 agents (a-e), and generates a batch of data with size 8. Policy a generates 0th and 3rd action for 1st and 2nd environment respectively. The full test script is https://github.com/sail-sg/envpool/blob/c2b5b6c679304c976d821e2519e17cf3e9f4f98e/envpool/dummy/dummy_envpool_test.cc#L26-L92 The above is the solution provided by envpool. And It may not be consistent with PettingZoo. I put it here just for a reference. The other 3 items look good! |
make format(required)make commit-checks(required)As I mentioned in #494, to support CTDE scheme or other schemes that need global information, the returned
infoshould contain more information.By the way, I am trying to implement some basic MARL-communication algorithms based on tianshou. Inspired by #399 (comment), I have the following design:
agent_numagents andenv_numenvs should act as a venv withagent_numxenv_numenvs.env_id=agent_numxenv_num+env_numshould be contained ininfo.env_idininfoindicates which agents are taking actions.agent_numxenv_numbuffers, the indices used for each agent should be the same.agent_numagents and the centralized module.