I have not found a plugin for this yet, if this already exist please let me now, maybe I'm also going to look to implement it myself. For now just leaving my thoughts here.

It would be great to have permissions to a single application instead of a whole server. Maybe this can be done as a per user config. Supporting groups wouldn't be necessary and just be a convience feature instead.