This is the code of our paper entilted Value-based CTDE Methods in Symmetric Two-team Markov Game: from Cooperation to Team Competition.

This code is a modification of the former pymarl to allow to train several teams at the same time in a competitive mode in the competitive SMAC environment.

We hereafter consider an installation with python virtual environments. If you consider using an other environment manager such as conda, consider to modify the following scripts.

Install the python virtual environment (python 3.6 required):

./install_venv.sh

Set up StarCraft II(on Linux). Check competitive SMAC for more details:

bash install_sc2.sh

Set up SMAC Maps:

cp -r src/envs/starcraft2/maps/SMAC_Maps/ 3rdparty/StarCraftII/Maps/

Execute the following command that play randomly at competitive SMAC to test your sc2 installation.

source env/bin/activate
python3.6 -m smac.examples.random_agents_compet.py
deactivate

You can train nine types of team with the following command: You can change the map by modifying the map_name from '3m_compet' to '3s5z_compet'.

source env/bin/activate
python3.6 src/main.py --config="config_name" --env-config=sc2_compet with env_args.map_name=3m_compet
deactivate

The parameter "config_name" defines which type of team you will train. Config files are in the folder "/src/config/algs". Here is the list of train config that are self-explanatory:

• popu_qmix_vs_heuristic
• popu_qmix_self
• popu_qmix_5
• popu_qvmix_vs_heuristic
• popu_qvmix_self
• popu_qvmix_5
• popu_maven_vs_heuristic
• popu_maven_self
• popu_maven_5

Once trained, it is possible to test your teams in different configurations.

We provide all the test scripts executed to compute Elo scores after training or win-rates along training.

For the Elo, scripts have a name prefixed "elo_". They will automatically find the 10000000th saved networks and execute the experiment as described in the paper.

For the win-rates, scripts have a name prefixed "run_test".

All those scripts sometimes require arguments that are self-explanatory.

The heuristic is implemented in the competitive SMAC and can be modified.

If you use the competitive PYMARL implementation in your own work, please cite our paper: Value-based CTDE Methods in Symmetric Two-team Markov Game: from Cooperation to Team Competition.

@inproceedings{leroy2022twoteam,
title={Value-based {CTDE} Methods in Symmetric Two-team Markov Game: from Cooperation to Team Competition},
author={Leroy, Pascal and Pisane, Jonathan and Ernst, Damien},
booktitle={Deep Reinforcement Learning Workshop NeurIPS 2022},
year={2022}
}