An issue was discovered in Qt 5.11. A malformed PPM image causes a division by zero and a crash in qppmhandler.cpp.
Created qt tracking bugs for this issue:
Affects: fedora-all [bug 1691638]
Created qt5 tracking bugs for this issue:
Affects: fedora-all [bug 1691637]
You can list all versions of qt3 as not affected. I verified that this code is not present in Qt 3, it was introduced in Qt 4.0.0.
qt-4.8.7-45.fc28 has been pushed to the Fedora 28 stable repository. If problems still persist, please make note of it in this bug report.
I don't see how you can come to the conclusion that rhel-*/qt=notaffected. I have seen the vulnerable code in ALL versions of Qt 4, from 4.0.0 to 4.8.7.
Backtrace from 5.11.1:
Program received signal SIGFPE, Arithmetic exception.
0x00007ffff7a741ca in scale_pbm_color (bv=12336, gv=12336, rv=12336, mx=0)
79 ../../include/QtGui/../../src/gui/painting/qrgba64.h: No such file or directory.
#0 0x00007ffff7a741ca in scale_pbm_color (bv=12336, gv=12336, rv=12336, mx=0)
#1 read_pbm_body (outImage=0x7fffffffd490, mcc=1329790976, h=3, w=<optimized out>,
type=<optimized out>, device=0x61c330) at image/qppmhandler.cpp:193
#2 QPpmHandler::read (this=0x61c8b0, image=0x7fffffffd490) at image/qppmhandler.cpp:509
#3 0x00007ffff7a46a8a in QImageReader::read (this=0x7fffffffd4e8, image=0x7fffffffd490)
#4 0x00007ffff7a470d8 in QImageReader::read (this=this@entry=0x7fffffffd4e8)
#5 0x00007ffff7a2f0da in QImage::load (this=0x7fffffffd560, fileName=..., format=<optimized out>)
#6 0x0000000000400ce2 in main (argc=2, argv=0x7fffffffd6d8) at main.cpp:14
Unable to reproduce this on Red Hat Enterprise 6 or 7 (7 running qt 4.8.7).
static inline QRgb scale_pbm_color(quint16 mx, quint16 rv, quint16 gv, quint16 bv)
return QRgba64::fromRgba64((rv * 0xffffu) / mx, (gv * 0xffffu) / mx, (bv * 0xffffu) / mx, 0xffff).toArgb32();
Looks like MX is 0 and a nice exception occurs.
Unrelated, but interesting. Division by zero is undefined behavior. gcc seems to generate a SIGFPE, whereas clang/llvm seems to generate junk and continue. Easiest way to detect this via a clang build is with -fsanitize=undefined. I'm sure there are a bunch of other knobs and switches to change the behavior.
Red Hat Enterprise Linux 7 looks like it has the responsible code even though I couldn't reproduce it.. Didn't track down where/why mx is getting set to zero, but 7 could potentially be impacted. The code doesn't appear in earlier versions to the best of my knowledge.