A payload other than "cpio" makes rpm print an error much too late in the installation process. This patch adds an earlier check. It also prints a different message for delta-rpms (see SuSE's deltarpm package).
Created attachment 107058 [details] Proposed patch
I have no problem with an earlier payload check in principle, and lib/rpminstall.c is about as early as it gets for the rpm CLI. There are several other pathways that will need the check plugged however, the most important of which is rpmtsAddInstallElement() in lib/depends.c, as that performs similar "early" detection for bindings that your patch does for the rpm CLI. The other pathways are those used to install *.src.rpm. Lots of this mess needs scrap and rewrite, lest the introduction of a 4th (i.e. other than binary/src and now "drpm") flavor of *.rpm cause this bug to be dicovered again and again and again.
Now that rpm-4.4.4 supports "ustar" payloads, this is not the right check to do. The better implementation is to add a "rpmlib(PayloadIsDeltaRpm)" (or equivalent) tracking dependency when building delta rpm headers, and then failing in rpmtsCheck() if the version of rpmlib does not supprt the necessary functionality. That's perfect for binary rpm's, but source rpm installs (are there delta rpm payloads in source rpm's?) would still need additional handling because rpmtsCheck() (and the install paths) are different. I'm perfectly okay living without the check for srpm's because non-cpio payloads for srpm's is plain and simply a bad idea. WONTFIX in the sense that the included patch is the wrong check, but I'm perfectly happy with a rpmlib(...) tracking dependency for delat rpm's underneath a #ifdef in rpm sources some day.