I'm using a memorex three button mouse, and its cursor jumps around while trying to move around the screen. Previous releases of Redhat didn't have this problem.
X or gpm? Anyway, it certainly doesn't have anything do with zlib. Try 6.2 and report it with XFree86 or gpm if the problem continues.