Description of problem: On ia64 rhel5 hosts yum will install both restraint-rhts.i386 and restraint-rhts.ia64 This will cause the wrong selinux module to try to get loaded and fail. Version-Release number of selected component (if applicable): 0.22 How reproducible: Everytime Steps to Reproduce: 1. Schedule a beaker job against rhel5 ia64 and you will not get the proper selinux modules loaded. 2. Very evident in the cifs/connectathon test suite.
This is a pretty drastic change to how the harness repos are managed on Beaker and I'm not convinced that we really need to do this. Yum is already smart enough not to try and install incompatible arches, so having all arches together in one repo is okay. And there is nothing inherently wrong with installation restraint.i386 on ia64, it has all that multilib support so that you can do that if you want. The real problem is just that the /emul/ia32 multilib relocation hackery on ia64 is preventing the selinux policy from being installed properly, right? Could we move the restraint selinux policy into a noarch subpackage to avoid that problem?
I tried building noarch sub packages but the version of rpm in rhel5 is not new enough for this to be feasible. rpm-build-4.4.2.3-34.el5.x86_64.rpm https://www.redhat.com/archives/rhl-devel-list/2009-February/msg01023.html rpm version 4.6 or higher is needed. rel-eng is adamant about having separate repos for each arch. I know this may be painful but we do have $basearch to help here..
(In reply to Bill Peck from comment #3) > rel-eng is adamant about having separate repos for each arch. I think they are just cargo-culting it though. They ship i386/i686 packages in ia64 anyway, for multilib. This suggestion is really: hack up the repos to hide restraint-rhts.i386 from ia64 machines. We can do it, but it doesn't seem like the easiest or best solution. I played around some more on an ia64 machine. It seems to be a flaw in the hacky multilib support in RHEL5 ia64 that the RPM script interpreter dependencies are essentially wrong. It looks like installing restraint-rhts.i386 works properly if I install ia32el and bash.i386 first -- although you can't do that in Beaker jobs because the RHEL5 composes have Supplementary and compatlayer split out in different directories and Beaker doesn't import them. But there might be simpler workarounds we can do here... if installing restraint-rhts.i386 is really never desired, then maybe we can just set multilib_policy=best in our kickstart %post before we install the harness packages?
This *should* be a minimal reproducer, I think, except when I ran it on beaker-devel it actually succeeded (the rhts policy module was loaded, even though the %post had failed) so I'm not sure what is going on exactly... bkr workflow-simple --whiteboard 'bz1328153 repro' \ --arch ia64 --family RedHatEnterpriseLinuxServer5 \ --task /distribution/command \ --taskparam 'CMDS_TO_RUN=semodule -l | grep rhts' \ --ks-meta 'harness=restraint-rhts'
(In reply to Dan Callaghan from comment #5) It seems to only reproduce if restraint-rhts.i386 is installed *before* restraint-rhts.ia64, for some reason. The job on beaker-devel where it passed had restraint-rhts.ia64 installed first and then restraint-rhts.i386. Yum installs the packages in arbitrary order, it seems to change at random (probably based on Python dict ordering or something).
Anyway, this template hack using multilib_policy=best is enough to make yum skip installing restraint-rhts.i386: http://gerrit.beaker-project.org/4826 I would much prefer this over permanently rearranging all our harness repos.
Going to fill in some more background info here for future reference... RHEL5 ia64 can install 32-bit x86 packages (i386/i686) using the same multilib support that lets you install i386 packages on x86_64. However on ia64 there is an emulation layer which runs the i386 binaries, they can't be natively executed like on x86_64. To hack around this, when an i386 package is installed on ia64 rpm will relocate certain files into /emul/ia32-linux so that the emulation layer can kind of somehow magically work. For example: # rpm -ql bash.i386 /emul/ia32-linux/bin/bash /emul/ia32-linux/bin/sh /etc/skel/.bash_logout ... Note that this is the same bash.i386 package that would be installed on a real i386 machine, the package *actually* contains /bin/bash and /bin/sh, rpm is relocating those files at install time. As a further hack (presumably needed due to the above hack), rpm also changes the package scriptlet interpreter to prefix /emul/ia32-linux as well. So for example, installing bash.i386 runs its scriptlets with /emul/ia32-linux/bin/sh even though the scriptlet interpreter in the package itself is declared as /bin/sh. Packages should always Require their scriptlet interpreter -- that is, if the package has a %post scriptlet using /bin/sh then the package needs to Require /bin/sh so that it's available when rpm is running the scriptlet. Normally, the proper Requires is automatically added at rpm build time. So bash.i386 has a requirement on /bin/sh (and so does restraint-rhts.i386). The problem is that it *actually* needs /emul/ia32-linux/bin/sh, not /bin/sh, due to rpm's hackery mentioned above. But there is nothing which will pull that in as a dependency. And indeed there is nothing that will pull in the ia32el package, which seems to be necessary for the /emul/ia32-linux stuff to work, when you install any i386 packages. You would have to manually install it first too. That's how we end up with errors like: Non-fatal POSTIN scriptlet failure in rpm package restraint-rhts-0.1.24-1.el5_9.i386 error: %post(restraint-rhts-0.1.24-1.el5_9.i386) scriptlet failed, exit status 255 where 255 indicates that the executable, in this case /emul/ia32-linux/bin/sh, doesn't exist. These problems all seem to be unavoidable flaws in the way that RHEL5 ia64 is (ab)using rpm multilib support for its i386 emulation. I guess this is why this approach of having rpm forcibly relocate certain files was never used anywhere else since then.
(In reply to Dan Callaghan from comment #5) > bkr workflow-simple --whiteboard 'bz1328153 repro' \ > --arch ia64 --family RedHatEnterpriseLinuxServer5 \ > --task /distribution/command \ > --taskparam 'CMDS_TO_RUN=semodule -l | grep rhts' \ > --ks-meta 'harness=restraint-rhts' So this reproducer does work when it's run with released harness repos, containing restraint-0.1.24-1.el5_9. In those cases, yum installs restraint-rhts.i386 before restraint-rhts.ia64 and it breaks the policy module loading. (I'm still not actually sure *why*... is rpm skipping the second %post script after the first one fails?) When the reproducer runs with harness-testing repos containing restraint-0.1.24-1.git.14.8dfa89b.el5_9, yum installs restraint-rhts.ia64 before restraint-rhts.i386 and the policy is successfully loaded so it doesn't reproduce.
This patch was merged to the release-22 branch but the next release will be 23.0.
Beaker 23.0 has been released.