Running xfs in runlevel 3 makes no sense. startx probably needs to know how to start and stop it instead.
The developers slightly disagree. There are many many ways to start X -- startx is only one of them. Some sites use scripts that give their own parameters to xinit, others use xdm, kdm, or gdm. The possibilities are wide. While it is true that xfs is only useful while X is running in MOST cases, it can also be used as a networked font server, so in other cases running it while X isn't running on the local console DOES make sense. Finally, in Red Hat Linux 6.0, some sort of display manager will be the default way to log into X, in which case X will be running 99% of the time. If your machine doesn't fall into any of these categories, it is a simple matter to do a 'chkconfig --del xfs' to turn off xfs permanently from the runlevel, and then use /etc/rc.d/init.d/xfs (start|stop) to turn it on and off manually.