In his talk about data.table @tdhock showed us how they use the atime R package to do continuous benchmarks using GitHub actions.
The files that define these benchmarks are here: https://github.com/Rdatatable/data.table/blob/master/.ci/atime/tests.R and an example how the results look is here: Rdatatable/data.table#6993 (comment)
It would be nice if we could do something similar for mlr3.

One issue is that our benchmarks would probably take considerably more time than the data.table ones.