Fix libjxl#4410 #16458

Job	Run time
Ubuntu Build scalar	2m 14s
Ubuntu Build tsan	1m 46s
Ubuntu Build debug	1m 46s
Ubuntu Build release-system-lcms2	1m 21s
Ubuntu Build asan	1m 49s
Ubuntu Build release	14m 12s
Ubuntu Build coverage	56s
Ubuntu Build msan	3m 46s
Ubuntu Build release:gcc	1m 58s
Ubuntu Build release:minimal	3m 27s
Ubuntu Build release:arm64	14m 47s
Ubuntu Build release:arm64-lowprecision	7m 30s
Ubuntu Build release-nojpeg	1m 37s
Ubuntu Build lowprecision	15m 43s
Ubuntu Build release-lcms2	1m 14s
Ubuntu Build release:gcc-old-old	1m 23s
Ubuntu Build release:clang-old-old	1m 26s
Ubuntu Build release:gcc-old	1m 49s
MacOS Build release:osx	1m 44s
Ubuntu Build release:clang-old	1m 58s
MacOS Build release:osx-15	1m 26s
	1h 23m 52s

Provide feedback