aioexec

Description

Aioexec is a simple, intuitive interface around the concurrent.futures package and asyncio's loop.run_in_executor method. Aioexec is leightweight, no dependencies and ~100 LOC.

Requirements

aioexec requires Python >= 3.7

Install

pip install aioexec

or

pipenv install aioexec

Usage

Without aioexec you usually run an executor something like this:

import aysncio
from concurrent.futures import ProcessPoolExecutor

# ...

loop = asyncio.get_event_loop()

foo = await loop.run_in_executor(
    ProcessPoolExecutor(1), lambda: my_func(foo='baz')
)

With aioexec you would do the same like this:

from aioexec import Procs

# ...

foo = await Procs(1).call(my_func, foo='baz')

You can pass both sync and async functions to an executor:

def my_sync_func(foo):
    return stuff(foo)

async def my_async_func(foo):
    return await stuff(foo)


foo = await Procs(1).call(my_sync_func, foo='baz')
foo = await Procs(1).call(my_async_func, foo='baz')

You can call a batch of functions in the same executor like this:

import asyncio
from aioexec import Procs, Call

# ...

my_values = await asyncio.gather(
    *Procs(3).batch(
        Call(my_func, foo='bar'),
        Call(my_func, foo='baz'),
        Call(my_func, foo='qux'),
    )
)

This plays nicely with comprehensions:

my_values = await asyncio.gather(
    *Procs(10).batch(
        Call(my_func, foo=i) for i in range(0, 10)
    )
)

You can also spawn a pool in a context and make multiple different calls with the same executor:

with Procs(10) as pool:

    value_a = await pool.call(my_func, foo='baz')

    value_b = await aio.gather(
        *pool.batch(
            Call(my_func, foo=i) for i in range(0, 10)
        )
    )

    # etc...

The examples from above work the same for Threads, e.g.:

from aioexec import Threads

# ...

foo = await Threads(1).call(my_func, foo='baz')

If necessary, you can pass an event loop to the executors like this:

foo = await Threads(1, my_loop).call(my_func, foo='baz')
foo = await Procs(1, my_loop).call(my_func, foo='baz')

Development / Testing

Clone the repo and install dev packages:

pipenv install --dev

Run tests:

pipenv run python make.py test