[Bug]: Flaky test failures in absl_failure_signal_handler_test on ppc64le in Fedora #1804
Description
Describe the issue
Recently, the abseil-cpp package in Fedora has been failing to build on ppc64le due to frequent flaky test failures in absl_failure_signal_handler_test.
Looking at the first failing builds in Fedora 41 and in Rawhide / Fedora 42, this could be associated with updating from kernel 6.11.3 to 6.12.4 in F41 and 6.13.0-0.rc3 to 6.13.0-0.rc4 in Rawhide, although there are other possibilities, and the flaky nature of the issue means that the the update that triggered this could have happened earlier.
Steps to reproduce the problem
There are no particularly unusual compiler flags or patches in the RPM spec file. I suspect you could reproduce this in a git checkout on any real ppc64le machine, but I can’t easily verify that because I don’t have interactive access to one.
I tried the following on emulated ppc64le hardware – anyone can do this by running a ppc64le version of Fedora 41 via qemu-user-system; I used a mock chroot with emulation via qemu-user-static.
$ git clone https://github.com/abseil/abseil-cpp.git
$ cd abseil-cpp
$ git checkout 20240722.0
$ cmake -S. -Bbuild -DABSL_BUILD_TESTING:BOOL=ON -DABSL_USE_EXTERNAL_GOOGLETEST:BOOL=ON -DABSL_FIND_GOOGLETEST:BOOL=ON
$ cmake --build build -j16 --verbose
$ ctest --test-dir build --output-on-failure --force-new-ctest-process -j16 --exclude-regex absl_symbolize_test
The --exclude-regex absl_symbolize_test is because absl_symbolize_test failed in emulation only, which I didn’t consider significant.
I was not able to reproduce this issue in emulation.
What version of Abseil are you using?
20240722.0
What operating system and version are you using?
Fedora 41 or Fedora Rawhide (future 42).
What compiler and version are you using?
In Fedora 41:
Using built-in specs.
COLLECT_GCC=/usr/bin/gcc
COLLECT_LTO_WRAPPER=/usr/libexec/gcc/ppc64le-redhat-linux/14/lto-wrapper
OFFLOAD_TARGET_NAMES=nvptx-none
OFFLOAD_TARGET_DEFAULT=1
Target: ppc64le-redhat-linux
Configured with: ../configure --enable-bootstrap --enable-languages=c,c++,fortran,objc,obj-c++,ada,go,m2,lto --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-shared --enable-threads=posix --enable-checking=release --enable-targets=powerpcle-linux --disable-multilib --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique-object --enable-linker-build-id --with-gcc-major-version-only --enable-libstdcxx-backtrace --with-libstdcxx-zoneinfo=/usr/share/zoneinfo --with-linker-hash-style=gnu --enable-plugin --enable-initfini-array --with-isl=/builddir/build/BUILD/gcc-14.2.1-build/gcc-14.2.1-20240912/obj-ppc64le-redhat-linux/isl-install --enable-offload-targets=nvptx-none --enable-offload-defaulted --without-cuda-driver --enable-gnu-indirect-function --enable-secureplt --with-long-double-128 --with-long-double-format=ieee --with-cpu-32=power8 --with-tune-32=power8 --with-cpu-64=power8 --with-tune-64=power8 --build=ppc64le-redhat-linux --with-build-config=bootstrap-lto --enable-link-serialization=1
Thread model: posix
Supported LTO compression algorithms: zlib zstd
gcc version 14.2.1 20240912 (Red Hat 14.2.1-3) (GCC)
In Fedora Rawhide:
Using built-in specs.
COLLECT_GCC=/usr/bin/gcc
COLLECT_LTO_WRAPPER=/usr/libexec/gcc/ppc64le-redhat-linux/14/lto-wrapper
OFFLOAD_TARGET_NAMES=nvptx-none
OFFLOAD_TARGET_DEFAULT=1
Target: ppc64le-redhat-linux
Configured with: ../configure --enable-bootstrap --enable-languages=c,c++,fortran,objc,obj-c++,ada,go,m2,lto --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-shared --enable-threads=posix --enable-checking=release --enable-targets=powerpcle-linux --disable-multilib --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique-object --enable-linker-build-id --with-gcc-major-version-only --enable-libstdcxx-backtrace --with-libstdcxx-zoneinfo=/usr/share/zoneinfo --with-linker-hash-style=gnu --enable-plugin --enable-initfini-array --with-isl=/builddir/build/BUILD/gcc-14.2.1-build/gcc-14.2.1-20241104/obj-ppc64le-redhat-linux/isl-install --enable-offload-targets=nvptx-none --enable-offload-defaulted --without-cuda-driver --enable-gnu-indirect-function --enable-secureplt --with-long-double-128 --with-long-double-format=ieee --with-cpu-32=power8 --with-tune-32=power8 --with-cpu-64=power8 --with-tune-64=power8 --build=ppc64le-redhat-linux --with-build-config=bootstrap-lto --enable-link-serialization=1
Thread model: posix
Supported LTO compression algorithms: zlib zstd
gcc version 14.2.1 20241104 (Red Hat 14.2.1-6) (GCC)
What build system are you using?
In Fedora 41, cmake version 3.30.5.
In Fedora Rawhide, cmake version 3.31.3.
Additional context
While I don’t have interactive access to real pp64le hardware, I’m happy to test candidate patches or do experiments to get additional information by performing non-interactive abseil-cpp package “scratch” builds on Fedora infrastructure.
Just to be sure, and since I remember #1133, I tried building abseil-cpp with LTO disabled; this did not fix the problem.