I encountered big problems trying to upgrade from RH6.2 to XFree86-4.0-0.6.i386.rpm. There are a number of config directories that are moved from /usr/X11R6/lib/X11 to /etc/X11. The install fails on them. I believe the correct procedure is either rename the old directory, or mv it to /etc/X11 and put in a symlink. Each time I tried to update, it ran for a very long time, running a command like tar cBf - | tar xf - It seems to have gone insane!! It was tarring my whole hard disk, as far as I can tell. I killed it after a while (and many warnings about ignoring sockets in places like my home directory!) I tried again to do the update with the same results. I did not get the SRPM and investigate which script was going nuts.
The %triggerpostun script was messing up - this should be fixed in more current versions (starting from 0.8, I think - current is 0.16).