dev and MAKEDEV have a couple of dependencies that prevent their uninstallation in favor of devfs. This is a bug that needs to be fixed before anything based on 2.4 goes gold. openssh requires the dev rpm for some odd reason, and if that is fixed, the dev package can probably be taken out. MAKEDEV is a bigger issue, and all packages that rely on it, such as raidtools, need to be revamped to become "devfs-aware" Kudzu (see bug 22810).
There is no need to actualy remove /dev (or its contents) before using devfs - it will run regardless. When any filesystem is mounted the files under the mountpoint dissapear in favor of the mouted filesystem. In any case kernel 2.4 works perfectly well without devfs (and I understand that this is what RedHat intends to do).
Several scripts require /dev. OpenSSH *needs* the /dev/urandom device in order to start up and seed its random-number generator. Because devfs mounts over /dev, this is not likely to change.