In papers as well as vim, j and k scrolls. The fact that they are used to switch files instead goes against that standard.

I'd suggest using some other keybind for switching files, e.g. J and K, while having j and k perform scrolling. (Or make the keybindings configurable altogether.)