.github/workflows/cmake.yml (1)

773-776: New Benchmark Test Step Integration
This new step clearly delineates that the benchmark executable is run purely for crash testing without collecting coverage data. The use of the --benchmark_min_time=0 flag ensures the benchmark runs quickly, aligning with the PR’s objective of speeding up CI runs.

One suggestion to further improve robustness: consider adding an if: condition to execute this step only when benchmarks are enabled (e.g. by checking if matrix.cmake-args contains -DWITH_BENCHMARKS=ON). This will prevent the step from executing on configurations that do not build the benchmark executable, which could otherwise result in unexpected failures. For example:

-    - name: Test benchmarks (crashtest only, no coverage data collection)
-      run: test/benchmarks/benchmark_zlib --benchmark_min_time=0
-      working-directory: ${{ matrix.build-dir || '.' }}
+    - name: Test benchmarks (crashtest only, no coverage data collection)
+      if: contains(matrix.cmake-args, "-DWITH_BENCHMARKS=ON")
+      run: test/benchmarks/benchmark_zlib --benchmark_min_time=0
+      working-directory: ${{ matrix.build-dir || '.' }}

This conditional ensures the benchmark runs only in the appropriate matrix jobs.

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

.github/workflows/cmake.yml (3)

21-21: Benchmark Flag Consistency Check.
The flag -DWITH_BENCHMARKS=ON has been added to several job configurations (e.g. Ubuntu 20.04 GCC, Ubuntu GCC Benchmark, Ubuntu GCC ARM HF ASAN, Ubuntu GCC AARCH64 ASAN, Ubuntu GCC MIPS64, Ubuntu GCC RISC-V, and macOS Clang with a deployment target of 10.10). Please verify that this flag is intentionally enabled in all these entries and that it won’t trigger unintended benchmark executions in configurations where running benchmarks may not be desired.

Also applies to: 35-35, 175-175, 206-206, 242-242, 311-311, 592-592

---

774-778: Validate New Benchmark Crash Test Step.
A new CI step named "Test benchmarks (crashtest only, no coverage data collection)" has been introduced (lines 774–778) to run the benchmark binary using the command test/benchmarks/benchmark_zlib --benchmark_min_time=0. Confirm that this step correctly isolates benchmarks for crash detection only and does not inadvertently interfere with your coverage reports or other test outcomes.

🧰 Tools

🪛 actionlint (1.7.4)

775-775: got unexpected character '"' while lexing expression, expecting 'a'..'z', 'A'..'Z', '_', '0'..'9', ''', '}', '(', ')', '[', ']', '.', '!', '<', '>', '=', '&', '|', '*', ',', ' '. do you mean string literals? only single quotes are available for string delimiter

(expression)

---

775-775: Fix YAML String Literal in Condition.
The if condition on line 775 uses double quotes ("-DWITH_BENCHMARKS=ON"), which caused a lexing error according to actionlint. YAML in this context prefers single quotes for string literals. Please update the condition as shown below.

-      if: contains(matrix.cmake-args, "-DWITH_BENCHMARKS=ON")
+      if: contains(matrix.cmake-args, '-DWITH_BENCHMARKS=ON')

🧰 Tools

🪛 actionlint (1.7.4)

775-775: got unexpected character '"' while lexing expression, expecting 'a'..'z', 'A'..'Z', '_', '0'..'9', ''', '}', '(', ')', '[', ']', '.', '!', '<', '>', '=', '&', '|', '*', ',', ' '. do you mean string literals? only single quotes are available for string delimiter

(expression)

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

.github/workflows/cmake.yml (1)

21-21: Consistent Addition of Benchmark Flag

The changes add the -DWITH_BENCHMARKS=ON flag (and combinations thereof with sanitizer and toolchain options) across multiple matrix entries. This ensures that benchmark targets are built separately without being registered as part of the main gtest suite—aligning with the objective to eliminate misleading benchmark coverage in CI. Please verify that in all these contexts the flag does not interfere with other build options (such as sanitizer flags) and that it deliberately excludes benchmarks from standard test registration.

Also applies to: 28-28, 168-168, 200-200, 236-236, 257-257, 304-304, 317-317, 325-325, 347-347, 395-395, 585-585

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

.github/workflows/cmake.yml (1)

772-777: Review of sanitizer environment variable declarations in the benchmark step
Static analysis warnings indicate that properties such as tsan-options, lsan-options, and ubsan-options are not defined in the job’s matrix object. It is advisable to either define default values for these options in the matrix configuration or adjust their usage here to ensure safe fallback values.

🧰 Tools

🪛 actionlint (1.7.4)

774-774: property "tsan-options" is not defined in object type {asan-options: string; build-config: string; build-dir: string; build-shared: string; build-src-dir: string; cflags: string; chost: string; cmake-args: string; codecov: string; compiler: string; cxx-compiler: string; gcov-exec: string; ldflags: string; msan-options: string; name: string; os: string; packages: string; parallels-jobs: number; readonly-project-dir: bool}

(expression)

---

775-775: property "lsan-options" is not defined in object type {asan-options: string; build-config: string; build-dir: string; build-shared: string; build-src-dir: string; cflags: string; chost: string; cmake-args: string; codecov: string; compiler: string; cxx-compiler: string; gcov-exec: string; ldflags: string; msan-options: string; name: string; os: string; packages: string; parallels-jobs: number; readonly-project-dir: bool}

(expression)

---

776-776: property "ubsan-options" is not defined in object type {asan-options: string; build-config: string; build-dir: string; build-shared: string; build-src-dir: string; cflags: string; chost: string; cmake-args: string; codecov: string; compiler: string; cxx-compiler: string; gcov-exec: string; ldflags: string; msan-options: string; name: string; os: string; packages: string; parallels-jobs: number; readonly-project-dir: bool}

(expression)

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

.github/workflows/cmake.yml (1)

770-780: New CI Step for Benchmark Crash-Testing
The addition of the "Test benchmarks (crashtest only, no coverage data collection)" step is well aligned with the PR objectives. The step uses a ctest command filtered with the regex ^benchmark_zlib$, which should effectively crash-test the benchmark without affecting coverage data.

However, take note of the environment variables provided in this step:

• TSAN_OPTIONS: ${{ matrix.tsan-options || 'verbosity=0' }}:abort_on_error=1:halt_on_error=1
• LSAN_OPTIONS: ${{ matrix.lsan-options || 'verbosity=0' }}:abort_on_error=1:halt_on_error=1
• UBSAN_OPTIONS: ${{ matrix.ubsan-options || 'verbosity=0' }}:print_stacktrace=1:abort_on_error=1:halt_on_error=1

Static analysis (actionlint) has flagged that these properties (tsan-options, lsan-options, and ubsan-options) are not defined in the matrix object, which might lead to warnings or unexpected behavior. To resolve this, consider one of the following:

1. Option A: Define these properties explicitly in each matrix entry where they are applicable (even if as empty strings) so that the fallback is triggered without static analysis issues.

   For example, in each matrix entry add:

   tsan-options: ""
   lsan-options: ""
   ubsan-options: ""

2. Option B: Adjust the expressions in the environment block to remove reliance on these undefined properties if they are not needed. For instance, if crash-testing benchmarks do not require TSAN/LSAN/UBSAN options, you might remove them or provide static defaults.

🧰 Tools

🪛 actionlint (1.7.4)

777-777: property "tsan-options" is not defined in object type {asan-options: string; build-config: string; build-dir: string; build-shared: string; build-src-dir: string; cflags: string; chost: string; cmake-args: string; codecov: string; compiler: string; cxx-compiler: string; cxxflags: string; gcov-exec: string; ldflags: string; msan-options: string; name: string; os: string; packages: string; parallels-jobs: number; readonly-project-dir: bool}

(expression)

---

778-778: property "lsan-options" is not defined in object type {asan-options: string; build-config: string; build-dir: string; build-shared: string; build-src-dir: string; cflags: string; chost: string; cmake-args: string; codecov: string; compiler: string; cxx-compiler: string; cxxflags: string; gcov-exec: string; ldflags: string; msan-options: string; name: string; os: string; packages: string; parallels-jobs: number; readonly-project-dir: bool}

(expression)

---

779-779: property "ubsan-options" is not defined in object type {asan-options: string; build-config: string; build-dir: string; build-shared: string; build-src-dir: string; cflags: string; chost: string; cmake-args: string; codecov: string; compiler: string; cxx-compiler: string; cxxflags: string; gcov-exec: string; ldflags: string; msan-options: string; name: string; os: string; packages: string; parallels-jobs: number; readonly-project-dir: bool}

(expression)

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

.github/workflows/cmake.yml (2)

21-29: Benchmark Flag Inclusion in Matrix Configurations

The changes add the -DWITH_BENCHMARKS=ON flag (often combined with other flags, e.g. sanitizer options) consistently across multiple job configurations (for example, in the "Ubuntu 20.04 GCC" job at line 21, "Ubuntu GCC ASAN" at line 29, "Ubuntu GCC 32-bit" at line 91, "Ubuntu GCC MIPS64" at line 237, and "macOS Clang (Target 10.10)" at line 589). This aligns well with the PR’s objective of selectively running benchmarks.

Recommendation:
• Review that benchmarks are enabled only in configurations where they are intended, so that jobs aimed solely at library tests (or with coverage enabled) aren’t inadvertently running benchmarks.
• Optionally, consider centralizing or documenting the purpose of the added benchmark flag for easier long‐term maintenance.

Also applies to: 91-91, 169-171, 237-237, 589-589

---

772-782: New Benchmark Crash-Test Step

A new job step named "Test benchmarks (crashtest only, no coverage data collection)" has been introduced. This step conditionally runs the benchmark test (filtered via -R ^benchmark_zlib$) only when the -DWITH_BENCHMARKS=ON flag is present. This approach fulfills the PR objective by decoupling benchmark execution (as a crash test) from the main test coverage run.

Points to consider:
• The use of the condition contains(matrix.cmake-args, '-DWITH_BENCHMARKS=ON') appears appropriate to isolate benchmark runs.
• The environment settings for sanitizer options (TSAN_OPTIONS, LSAN_OPTIONS, and UBSAN_OPTIONS) fallback to a default ('verbosity=0') if not defined in the matrix; however, static analysis hints indicate these properties are not formally defined in the matrix object. To avoid potential warnings and to ensure clarity, consider explicitly defining these keys (with their optional default values) in the matrix configuration.
• Verify that filtering for ^benchmark_zlib$ accurately targets the desired benchmark without accidentally omitting or including unintended tests.

🧰 Tools

🪛 actionlint (1.7.4)

779-779: property "tsan-options" is not defined in object type {asan-options: string; build-config: string; build-dir: string; build-shared: string; build-src-dir: string; cflags: string; chost: string; cmake-args: string; codecov: string; compiler: string; cxx-compiler: string; cxxflags: string; gcov-exec: string; ldflags: string; msan-options: string; name: string; os: string; packages: string; parallels-jobs: number; readonly-project-dir: bool}

(expression)

---

780-780: property "lsan-options" is not defined in object type {asan-options: string; build-config: string; build-dir: string; build-shared: string; build-src-dir: string; cflags: string; chost: string; cmake-args: string; codecov: string; compiler: string; cxx-compiler: string; cxxflags: string; gcov-exec: string; ldflags: string; msan-options: string; name: string; os: string; packages: string; parallels-jobs: number; readonly-project-dir: bool}

(expression)

---

781-781: property "ubsan-options" is not defined in object type {asan-options: string; build-config: string; build-dir: string; build-shared: string; build-src-dir: string; cflags: string; chost: string; cmake-args: string; codecov: string; compiler: string; cxx-compiler: string; cxxflags: string; gcov-exec: string; ldflags: string; msan-options: string; name: string; os: string; packages: string; parallels-jobs: number; readonly-project-dir: bool}

(expression)

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

.github/workflows/cmake.yml (5)

28-29: Augment Ubuntu GCC ASAN Configuration

The updated lines in the Ubuntu GCC ASAN matrix entry now set:

• cxxflags: -Wno-maybe-uninitialized
• cmake-args: -DWITH_SANITIZER=Address -DWITH_BENCHMARKS=ON

This ensures that both the sanitizer and benchmark flags are enabled together. Please verify that combining these flags does not introduce any unintended interactions between sanitizer settings and benchmark build/test behavior.

---

169-170: Update Ubuntu GCC ARM HF ASAN Settings

The modification in the ARM HF ASAN block now adds -DWITH_BENCHMARKS=ON alongside the sanitizer flag:

• Line 169: cmake-args: -DCMAKE_TOOLCHAIN_FILE=cmake/toolchain-armhf.cmake -DWITH_SANITIZER=Address -DWITH_BENCHMARKS=ON
• Line 170: cxxflags: -Wno-psabi -Wno-maybe-uninitialized

These changes are consistent with the approach in other configurations. Double-check that the toolchain and compiler-specific flags continue to work correctly with benchmark builds.

---

201-201: Enable Benchmarks for Ubuntu GCC AARCH64 ASAN

The AARCH64 ASAN block now includes -DWITH_BENCHMARKS=ON along with the sanitizer flag:

• Line 201: cmake-args: -DCMAKE_TOOLCHAIN_FILE=cmake/toolchain-aarch64.cmake -DWITH_SANITIZER=Address -DWITH_BENCHMARKS=ON

This change is consistent with the other architectures. Please confirm that the additional flag does not adversely affect AARCH64 builds.

---

396-398: Configure Benchmarks for Ubuntu MinGW x86_64

The changes in the MinGW x86_64 block introduce:

• Line 396: cmake-args: -DCMAKE_TOOLCHAIN_FILE=cmake/toolchain-mingw-x86_64.cmake -DWITH_BENCHMARKS=ON
• Line 398: cxxflags: -Wno-unused-parameter

Ensure that the benchmark build is supported in the MinGW environment and that this additional flag does not conflict with any Windows-specific compiler behavior.

---

589-589: Set Benchmarks in macOS Clang (Target 10.10) Configuration

The macOS Clang configuration now appends -DWITH_BENCHMARKS=ON to the deployment target flag:

• Line 589: cmake-args: -DCMAKE_OSX_DEPLOYMENT_TARGET=10.10 -DWITH_BENCHMARKS=ON

Verify that the benchmarks are compatible with the targeted macOS deployment version and that they do not interfere with coverage collection in later CI steps.

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

.github/workflows/cmake.yml (3)

727-727: Consistency in CXXFLAGS usage.
The updated assignment on line 727 (CXXFLAGS: ${{ matrix.cxxflags }}) appears consistent with the rest of the configuration. Just ensure that every matrix entry supplies the expected value so that no configuration is inadvertently omitted.

---

772-782: New crash-test step for benchmarks without coverage collection.
The newly added CI step titled “Test benchmarks (crashtest only, no coverage data collection)” (lines 772–782) effectively isolates the execution of benchmark_zlib using a dedicated command:

-run: ctest --verbose -C Release -E benchmark_zlib --output-on-failure --max-width 120 -j ${{ matrix.parallels-jobs || '3' }}
+run: ctest --verbose -C Release -R ^benchmark_zlib$ --output-on-failure --max-width 120 -j ${{ matrix.parallels-jobs || '3' }}

This change ensures that benchmarks are crash-tested without affecting coverage metrics. It directly addresses the PR objective of decoupling benchmark execution from coverage reporting.

If needed, additional inline documentation in the CI step could help clarify its purpose for future maintainers.

🧰 Tools

🪛 actionlint (1.7.4)

779-779: property "tsan-options" is not defined in object type {asan-options: string; build-config: string; build-dir: string; build-shared: string; build-src-dir: string; cflags: string; chost: string; cmake-args: string; codecov: string; compiler: string; cxx-compiler: string; cxxflags: string; gcov-exec: string; ldflags: string; msan-options: string; name: string; os: string; packages: string; parallels-jobs: number; readonly-project-dir: bool}

(expression)

---

780-780: property "lsan-options" is not defined in object type {asan-options: string; build-config: string; build-dir: string; build-shared: string; build-src-dir: string; cflags: string; chost: string; cmake-args: string; codecov: string; compiler: string; cxx-compiler: string; cxxflags: string; gcov-exec: string; ldflags: string; msan-options: string; name: string; os: string; packages: string; parallels-jobs: number; readonly-project-dir: bool}

(expression)

---

781-781: property "ubsan-options" is not defined in object type {asan-options: string; build-config: string; build-dir: string; build-shared: string; build-src-dir: string; cflags: string; chost: string; cmake-args: string; codecov: string; compiler: string; cxx-compiler: string; cxxflags: string; gcov-exec: string; ldflags: string; msan-options: string; name: string; os: string; packages: string; parallels-jobs: number; readonly-project-dir: bool}

(expression)

---

779-781: Review environment variable defaults for sanitizers.
Static analysis tools flag that properties like tsan-options, lsan-options, and ubsan-options are not explicitly defined in the matrix (lines 779–781). Although the fallback defaults (using || 'verbosity=0') mitigate potential issues at runtime, consider defining these keys explicitly in the matrix configuration for clarity and to avoid YAML validation warnings.

🧰 Tools

🪛 actionlint (1.7.4)

779-779: property "tsan-options" is not defined in object type {asan-options: string; build-config: string; build-dir: string; build-shared: string; build-src-dir: string; cflags: string; chost: string; cmake-args: string; codecov: string; compiler: string; cxx-compiler: string; cxxflags: string; gcov-exec: string; ldflags: string; msan-options: string; name: string; os: string; packages: string; parallels-jobs: number; readonly-project-dir: bool}

(expression)

---

780-780: property "lsan-options" is not defined in object type {asan-options: string; build-config: string; build-dir: string; build-shared: string; build-src-dir: string; cflags: string; chost: string; cmake-args: string; codecov: string; compiler: string; cxx-compiler: string; cxxflags: string; gcov-exec: string; ldflags: string; msan-options: string; name: string; os: string; packages: string; parallels-jobs: number; readonly-project-dir: bool}

(expression)

---

781-781: property "ubsan-options" is not defined in object type {asan-options: string; build-config: string; build-dir: string; build-shared: string; build-src-dir: string; cflags: string; chost: string; cmake-args: string; codecov: string; compiler: string; cxx-compiler: string; cxxflags: string; gcov-exec: string; ldflags: string; msan-options: string; name: string; os: string; packages: string; parallels-jobs: number; readonly-project-dir: bool}

(expression)

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

.github/workflows/cmake.yml (1)

762-772: Introduce Dedicated Benchmark Crash-Test Step

A new CI step titled "Test benchmarks (crashtest only, no coverage data collection)" has been added. It conditionally runs only the benchmark_zlib target (using -R ^benchmark_zlib$) when benchmarks are enabled. This approach ensures that benchmarks primarily serve as crash tests, thereby eliminating the misleading coverage data.

Note: The step’s environment configuration references tsan-options, lsan-options, and ubsan-options via fallback expressions. Static analysis tools flagged these properties as undefined in the matrix type. Although the fallback defaults (e.g., 'verbosity=0') mitigate runtime issues, consider defining explicit default values for these keys in the matrix configuration to avoid potential YAML validation warnings in the future.

🧰 Tools

🪛 actionlint (1.7.4)

769-769: property "tsan-options" is not defined in object type {asan-options: string; build-config: string; build-dir: string; build-shared: string; build-src-dir: string; cflags: string; chost: string; cmake-args: string; codecov: string; compiler: string; cxx-compiler: string; cxxflags: string; gcov-exec: string; ldflags: string; msan-options: string; name: string; os: string; packages: string; parallels-jobs: number; readonly-project-dir: bool}

(expression)

---

770-770: property "lsan-options" is not defined in object type {asan-options: string; build-config: string; build-dir: string; build-shared: string; build-src-dir: string; cflags: string; chost: string; cmake-args: string; codecov: string; compiler: string; cxx-compiler: string; cxxflags: string; gcov-exec: string; ldflags: string; msan-options: string; name: string; os: string; packages: string; parallels-jobs: number; readonly-project-dir: bool}

(expression)

---

771-771: property "ubsan-options" is not defined in object type {asan-options: string; build-config: string; build-dir: string; build-shared: string; build-src-dir: string; cflags: string; chost: string; cmake-args: string; codecov: string; compiler: string; cxx-compiler: string; cxxflags: string; gcov-exec: string; ldflags: string; msan-options: string; name: string; os: string; packages: string; parallels-jobs: number; readonly-project-dir: bool}

(expression)

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

.github/workflows/cmake.yml (1)

762-772: Integrate a Dedicated Crash-Test for Benchmarks
The newly added CI step "Test benchmarks (crashtest only, no coverage data collection)" runs the benchmark_zlib target separately. This approach prevents skewing the coverage reports while still verifying the stability of the benchmarks. Additionally, since this step relies on sanitizer environment variables (e.g., tsan-options, lsan-options, ubsan-options) that are not explicitly defined in the matrix, consider adding default definitions for these keys to ensure YAML validation consistency.

🧰 Tools

🪛 actionlint (1.7.4)

769-769: property "tsan-options" is not defined in object type {asan-options: string; build-config: string; build-dir: string; build-shared: string; build-src-dir: string; cflags: string; chost: string; cmake-args: string; codecov: string; compiler: string; cxx-compiler: string; cxxflags: string; gcov-exec: string; ldflags: string; msan-options: string; name: string; os: string; packages: string; parallels-jobs: number; readonly-project-dir: bool}

(expression)

---

770-770: property "lsan-options" is not defined in object type {asan-options: string; build-config: string; build-dir: string; build-shared: string; build-src-dir: string; cflags: string; chost: string; cmake-args: string; codecov: string; compiler: string; cxx-compiler: string; cxxflags: string; gcov-exec: string; ldflags: string; msan-options: string; name: string; os: string; packages: string; parallels-jobs: number; readonly-project-dir: bool}

(expression)

---

771-771: property "ubsan-options" is not defined in object type {asan-options: string; build-config: string; build-dir: string; build-shared: string; build-src-dir: string; cflags: string; chost: string; cmake-args: string; codecov: string; compiler: string; cxx-compiler: string; cxxflags: string; gcov-exec: string; ldflags: string; msan-options: string; name: string; os: string; packages: string; parallels-jobs: number; readonly-project-dir: bool}

(expression)

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro