I couldn't make it work on ubuntu (loky fails with pickling error, multiprocessing fails with daemonic process can't have children), and it generally is of limited use to start experiments in parallel on a single machine, unless there are a lot of CPUs.

The one thing that might work and actually make sense is a ray launcher. We could also borrow a launcher from hydra, though it might be cumbersome.

An LLM with coding abilities can likely implement a ray based launcher almost out of the box