Description

There are a few issues that create friction between Python and Lmod:

• Python environments i.e. subprocess.Popen do not persist between commands
• Lmod functions are exported as functions instead of having sourceable interface
• Lmod interface is not guaranteed to be loaded e.g. in default github CI

These factors make it hard to chain python/shell commands with pushed/poped Lmod module environment

Proposal

Provide a simple Python interface that can call Lmod functions directly. A minimum interface could be:

• module list
• module load
• module unload/purge

The introspection commands could be direct calls to the Lmod interface with some basic output conversion. The main part is how the load/unload interface can look like. Here a Lmod class would carry an env dictionary with all the Lmod native environment operations like prepend_path chained accordingly. Then when requested, these can be evaluated through Lua or equivalent to have the plain python dictionary that can be fed through subprocess

Considerations

• There are a few python packages that can call lua scripts
• If two introspection commands module getenv and module getscripts (exporting the environment variable and executed scripts respectively for a given loaded environment) can be provided on the Lmod side, this would simplify writing this interface
• Supporting external scripts that source environment variables might not be easily supported
• swig supports making bindings for both languages. That could make such interfaces available to many more other languages as well