Description of Problem: A bug in GLUT 3.7 causes certain GLUT programs to crash when the mouse wheel is rotated. Version-Release number of selected component (if applicable): glut-3.7-8 This is tested agains the (null) public beta, but the glut package in the 8.0 release is the same version. GLUT 3.7 on the X window system improperly assumes there are no more than 3 mouse buttons, causing crashes and/or other bizarre behavior if another mouse button is pressed. For example, on XFree86 the mouse wheel is mapped to button #4, so rotating the mouse wheel while running a GLUT program can cause the GLUT program to crash. The problem occurs on line 425 of glut_event.c: menuNum = window->menu[event.xbutton.button - 1]; if event.xbutton.button >= 3, then this overflows window->menu. Simple solution: Change GLUT_MAX_MENUS in glutint.h from 3 to 5, the maximum number of buttons allowed according to the X documentation. Better solution: Put in an explicit bounds check NOTE: Only GLUT programs that use subwindows will crash. Otherwise the overflow of window->menu will read window->parent, which will be NULL, and no symptoms will be noticed. This bug has been reported to the GLUT maintainer, mjk.
Can you provide a test case? ie: An application name which I can run to test it with mousewheel and see this crash? Unless I can reproduce the problem locally with a specific app, and see it in action, then patch the problem and test to ensure the problem is fixed, I can't really change anything. Just FYI, the X protocol works with 7 buttons, not 5. Thanks
Created attachment 79525 [details] testcase. Try moving the mouse wheel while over the main window, but not over either rectangle
The man page XButtonEvent (3x) documents only mouse buttons 1 through 5. That's probably another bug, then... I've attached a test case (slightly modified version of one of the GLUT test programs).
It has been brought to my attention that the GLUT source code written by Mark Kilgard, and shipped in Red Hat Linux 7.3, 8.0 and 9, does not permit modification, and that the author of GLUT has been strict about this in the past. GLUT is also currently not maintained by the author, nor has he passed on maintenance to anyone else. As such, Red Hat can not legally fix any bugs in GLUT, and we are removing it from Red Hat Linux effective immediately. Users experiencing problems with GLUT should either contact the author of GLUT directly to request permission to fix bugs and use/redistribute, or should try "freeglut" which is an alternative implementation of GLUT written from the ground up under an open source license. I will be investigating wether or not freeglut is suitable for inclusion in Red Hat Linux based on wether or not it is maintained currently, and wether or not it is useable to people whom use GLUT for development. Red Hat Linux contains no applications or software which use GLUT, so removal of glut alone will not disrupt any software included with the distribution.