The xf86config utility warns the user not to do any probing when using the S3 Virge/VX chipset. Unfortunately, the RH 5.2 install does the probing anyways, and locks the machine up solidly (only hitting the Reset buttons works). I had to install XFree86 manually from the RPMs. If possible, use the database of "cards not to probe" in the RH installer, and/or give the user an option NOT to probe. (Currently there is only an "OK" button in the box in the install screen.) Or, perhaps, as you do with giving the user a choice between running Disk Druid or fdisk, how about an option to run either xf86config, XF86Setup, or the RedHat X setup programs? That seems like the simplest fix to me.
This was originally done because a lot of the S3V drivers were rolled over into the SVGA server instead of the S3V server like before. We have noticed that this arrangement does not work properly for all S3V cards. To circumvent this problem you choose expert mode when you first start the install. This will allow you to choose the S3V server instead of SVGA being chosen for you. We are working to make the configuration better in our next release.