RL is flaky and one needs to report statistically significant results. We should help contributors and users with this. I think using nni for parallel execution is a good fit for that (nni is dead, and we shouldn't rely on external tools unless strictly necessary, especially those needing external config files).

Original motivation:

I run 10 runs for each config and there wasn't much difference. I guess that was an unlucky v.s. lucky seed case

Originally posted by @MischaPanch in #886 (comment)