Background

Async-profiler has two options for CPU profiling: -e cpu and -e itimer.
-e cpu (default) relies on perf_events, while -e timer is based on setitimer.

Perf events typically yield more accurate sampling and make possible to collect kernel stack traces. However, perf events are not always available, e.g., because of perf_event_paranoid settings or seccomp restrictions. Perf events are often disabled in containers. Furthermore, async-profiler opens one perf_event descriptor per thread, which can be problematic for an application with many threads running under a low ulimit for the number of open file descriptors.

itimer works fine in containers, but it may suffer from inaccuracies caused by the following limitations:

• only one itimer signal can be delivered to a process at a time;
• signals are not distributed evenly between running threads;
• sampling resolution is limited by the size of jiffies.

See #272.

Proposal

Add new CPU sampling engine based on timer_create.

It combines benefits of -e cpu and -e itimer, except that it does not allow collecting kernel stacks.

• Works in containers by default.
• Does not suffer from itimer biases.
• Does not consume file descriptors.

timer_create is used in Go profiler, see https://felixge.de/2022/02/11/profiling-improvements-in-go-1.18/

Implementation

New Linux-specific event is introduced: -e ctimer. In this mode, async-profiler creates per-thread CPU timer that sends profiling signals to a thread associated with that timer. The rest of the mechanics is the same as in itimer mode.

Note: -e ctimer is not supported on macOS.