Summary

Signed integer underflow happening in function Ref_cmp() in dwg.c:3825

Details

The subtraction (long)pKey->absolute_ref - (long)(*ppR)->absolute_ref can underflow, triggering an undefined behavior; this will most likely result in a positive value being returned by the Ref_cmp() function. This function seems to be used twice in a bsearch(), where the sign of its result is significant for the search itself, as it determines the next iterations of the search algorithm. In case where, due to an underflow, the resulting sign is unreliable, the search will fail.

Below is the log of UBSan detecting the bug:

dwg.c:3825:35: runtime error: signed integer overflow: -9223372036854775807 - 29 cannot be represented in type 'long'

Reproduce

In the attached archive you will find the test case used to trigger this behavior. We tested your fuzzing harness compiling it with LLVM 20 and AFL++ 4.32, running on Ubuntu 20.04. To detect this issue, use the -fsanitize=signed-integer-overflow flag while compiling.

Proposed fix

Fixing this issue should be pretty simple, as you can easily detect the overflow before it happens (e.g. with some of these solutions) and return a value with a correct sign in case an overflow is going to happen.