Bug 2252671

Summary: DNF5 from an x86_64 bootstrap image installs x86_64 chroot for fedora-rawhide-i386
Product: [Fedora] Fedora Reporter: Mikhail <mikhail.v.gavrilov>
Component: dnf5Assignee: Marek Blaha <mblaha>
Status: CLOSED RAWHIDE QA Contact:
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: rawhideCC: copr-team, egoode, jkolarik, mblaha, msuchy, nsella, philip.wyett, pkratoch, pmatilai, praiskup, robatino, rpm-software-management, rxguy
Target Milestone: ---Flags: egoode: needinfo-
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2023-12-11 12:38:51 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 2187792, 2187793, 2252074    
Attachments:
Description Flags
root.log
none
meson-log.txt
none
Log of " mock --root=fedora-rawhide-i386 --init "
none
Log of " mock --root=fedora-rawhide-i386 --init --yum " none

Description Mikhail 2023-12-03 19:45:13 UTC
Created attachment 2002728 [details]
root.log

Description of problem:
After upgrade mock-core-configs I find out that unable make every day mesa snapshot build.


Version-Release number of selected component (if applicable):
Last good version of mock-core-config is 39.2-1.fc40

How reproducible:
Always

Steps to Reproduce:
> mock -r fedora-rawhide-x86_64 --rebuild ~/rpmbuild/SRPMS/mesa-24.0.0-1.20231203.00.76be39c.fc40.src.rpm
INFO: mock.py version 5.2 starting (python version = 3.12.0, NVR = mock-5.2-1.fc40), args: /usr/libexec/mock/mock -r fedora-rawhide-x86_64 --rebuild /home/mikhail/rpmbuild/SRPMS/mesa-24.0.0-1.20231203.00.76be39c.fc40.src.rpm
Start(bootstrap): init plugins
INFO: selinux enabled
Finish(bootstrap): init plugins
Start: init plugins
INFO: selinux enabled
Finish: init plugins
INFO: Signal handler active
Start: run
INFO: Start(/home/mikhail/rpmbuild/SRPMS/mesa-24.0.0-1.20231203.00.76be39c.fc40.src.rpm)  Config(fedora-rawhide-x86_64)
Start: clean chroot
Finish: clean chroot
Mock Version: 5.2
INFO: Mock Version: 5.2
Start(bootstrap): chroot init
INFO: calling preinit hooks
INFO: enabled root cache
INFO: enabled package manager cache
Start(bootstrap): cleaning package manager metadata
Finish(bootstrap): cleaning package manager metadata
INFO: Using 'dnf' instead of 'dnf5' for bootstrap chroot
INFO: Package manager dnf detected and used (fallback)
Finish(bootstrap): chroot init
Start: chroot init
INFO: calling preinit hooks
INFO: enabled root cache
Start: unpacking root cache
Finish: unpacking root cache
INFO: enabled package manager cache
Start: cleaning package manager metadata
Finish: cleaning package manager metadata
INFO: enabled HW Info plugin
INFO: Package manager dnf5 detected and used (direct choice)
Start: dnf5 update
execv(/usr/bin/dnf5) failed: No such file or directory
ERROR: Exception(/home/mikhail/rpmbuild/SRPMS/mesa-24.0.0-1.20231203.00.76be39c.fc40.src.rpm) Config(fedora-rawhide-x86_64) 0 minutes 14 seconds
INFO: Results and/or logs in: /var/lib/mock/fedora-rawhide-x86_64/result
ERROR: Command failed: 
 # /usr/bin/systemd-nspawn -q -M 46e28309fb154f4fb83acdc2f0cfc0c7 -D /var/lib/mock/fedora-rawhide-x86_64-bootstrap/root -a --capability=cap_ipc_lock --bind=/tmp/mock-resolv.q6kecbwt:/etc/resolv.conf --console=pipe --setenv=TERM=vt100 --setenv=SHELL=/bin/bash --setenv=HOME=/var/lib/mock/fedora-rawhide-x86_64/root/installation-homedir --setenv=HOSTNAME=mock --setenv=PATH=/usr/bin:/bin:/usr/sbin:/sbin '--setenv=PROMPT_COMMAND=printf "\033]0;<mock-chroot>\007"' '--setenv=PS1=<mock-chroot> \s-\v\$ ' --setenv=LANG=C.UTF-8 --setenv=LC_MESSAGES=C.UTF-8 --resolv-conf=off /usr/bin/dnf5 --installroot /var/lib/mock/fedora-rawhide-x86_64/root/ -y --releasever 40 upgrade --setopt=deltarpm=False --setopt=allow_vendor_change=yes --allowerasing --setopt=tsflags=nocontexts


Actual results:
Error message about absence dnf5

Expected results:
Builded mesa

Additional info:

Comment 1 Fedora Blocker Bugs Application 2023-12-03 19:47:41 UTC
Proposed as a Blocker and Freeze Exception for 40-beta by Fedora user mikhail using the blocker tracking app because:

 Because this breaks one of important function - building Fedora packages locally.

Comment 2 Miroslav Suchý 2023-12-03 20:51:15 UTC
See
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/YGQOOIOSAWK5SQEMXAGRXPLOWEBDUPQQ/

$ mock -r fedora-rawhide-x86_64 --scrub=bootstrap  # or --scrub=all

Comment 3 Mikhail 2023-12-03 22:18:38 UTC
Created attachment 2002748 [details]
meson-log.txt

Why when I try build fedora-rawhide-i386 package manager downloads x86_64 version packages?

> mock -r fedora-rawhide-i386 --rebuild ~/rpmbuild/SRPMS/mesa-24.0.0-1.20231203.00.76be39c.fc40.src.rpm
INFO: mock.py version 5.2 starting (python version = 3.12.0, NVR = mock-5.2-1.fc40), args: /usr/libexec/mock/mock -r fedora-rawhide-i386 --rebuild /home/mikhail/rpmbuild/SRPMS/mesa-24.0.0-1.20231203.00.76be39c.fc40.src.rpm
Start(bootstrap): init plugins
INFO: selinux enabled
Finish(bootstrap): init plugins
Start: init plugins
INFO: selinux enabled
Finish: init plugins
INFO: Signal handler active
Start: run
INFO: Start(/home/mikhail/rpmbuild/SRPMS/mesa-24.0.0-1.20231203.00.76be39c.fc40.src.rpm)  Config(fedora-rawhide-i686)
Start: clean chroot
Finish: clean chroot
Mock Version: 5.2
INFO: Mock Version: 5.2
Start(bootstrap): chroot init
INFO: calling preinit hooks
INFO: enabled root cache
INFO: enabled package manager cache
Start(bootstrap): cleaning package manager metadata
Finish(bootstrap): cleaning package manager metadata
INFO: Package manager dnf5 detected and used (fallback)
Finish(bootstrap): chroot init
Start: chroot init
INFO: calling preinit hooks
INFO: enabled root cache
Start: unpacking root cache
Finish: unpacking root cache
INFO: enabled package manager cache
Start: cleaning package manager metadata
Finish: cleaning package manager metadata
INFO: enabled HW Info plugin
INFO: Package manager dnf5 detected and used (direct choice)
Start: dnf5 update
Updating and loading repositories:
 local repo                             100% |   0.0   B/s |   1.5 KiB |  00m00s
 local                                  100% |   6.2 KiB/s |   3.8 KiB |  00m01s
Repositories loaded.
Nothing to do.
Finish: dnf5 update
Finish: chroot init
INFO: Buildroot is handled by package management downloaded with a bootstrap image:
  rpm-4.19.0-3.fc40.x86_64
  rpm-sequoia-1.5.0-2.fc40.x86_64
  python3-dnf-4.18.1-2.fc40.noarch
  yum-4.18.1-2.fc40.noarch
Start: build phase for mesa-24.0.0-1.20231203.00.76be39c.fc40.src.rpm
Start: build setup for mesa-24.0.0-1.20231203.00.76be39c.fc40.src.rpm
Building target platforms: i686
Building for target i686
setting SOURCE_DATE_EPOCH=1701648000
Wrote: /builddir/build/SRPMS/mesa-24.0.0-1.20231203.00.76be39c.fc40.src.rpm
Updating and loading repositories:
 local repo                             100% |   0.0   B/s |   1.5 KiB |  00m00s
 local                                  100% |   8.5 KiB/s |   3.8 KiB |  00m00s
Repositories loaded.
Package                      Arch   Version                 Repository      Size
Installing:                                                                     
 bindgen-cli                 x86_64 0.68.1-2.fc40           local        5.9 MiB
 bison                       x86_64 3.8.2-6.fc40            local        3.5 MiB
 cargo-rpm-macros            noarch 25.2-2.fc40             local       14.0 KiB
 clang-devel                 x86_64 17.0.6-1.fc40           local       22.7 MiB
 elfutils-libelf-devel       x86_64 0.190-4.fc40            local       34.4 KiB
 expat-devel                 x86_64 2.5.0-3.fc39            local      190.6 KiB
 flex                        x86_64 2.6.4-14.fc40           local      905.3 KiB
 gcc                         x86_64 13.2.1-5.fc40           local       94.7 MiB
 gcc-c++                     x86_64 13.2.1-5.fc40           local       33.9 MiB
 gettext                     x86_64 0.22-2.fc39             local        5.0 MiB
 glslang                     x86_64 13.1.1-1.fc40           local        2.9 MiB
 kernel-headers              x86_64 6.7.0-0.rc3.git0.1.fc40 local        6.1 MiB
 libX11-devel                x86_64 1.8.7-1.fc40            local        1.0 MiB
 libXdamage-devel            x86_64 1.1.6-1.fc40            local        2.5 KiB
 libXext-devel               x86_64 1.3.5-4.fc40            local       98.8 KiB
 libXfixes-devel             x86_64 6.0.1-1.fc40            local        9.2 KiB
 libXrandr-devel             x86_64 1.5.4-1.fc40            local       21.8 KiB
 libXxf86vm-devel            x86_64 1.1.5-4.fc40            local       12.1 KiB
 libclc-devel                x86_64 17.0.6-1.fc40           local      146.2 KiB

***

And it leads build to FAIL.

The Meson build system
Version: 1.2.3
Source dir: /builddir/build/BUILD/mesa-76be39ca6dbcfb62f1ef5194661c765cc97620c7
Build dir: /builddir/build/BUILD/mesa-76be39ca6dbcfb62f1ef5194661c765cc97620c7/redhat-linux-build
Build type: native build
Project name: mesa
Project version: 24.0.0-devel

meson.build:21:0: ERROR: Compiler gcc cannot compile programs.

Comment 4 Pavel Raiskup 2023-12-04 08:15:53 UTC
Thank you for the report.  This is good point, it seems like DNF5 doesn't
respect the changed personality (DNF4 used to respect it):

https://github.com/rpm-software-management/mock/blob/92c488545cc5a7797bd99f32688218b4039acf59/mock/py/mock.py#L831-L832

Evan, can you please take a look?

Comment 5 Pavel Raiskup 2023-12-04 08:21:35 UTC
Workaround for this is:

$ mock -r fedora-rawhide-i386 --scrub=bootstrap
$ mock -r fedora-rawhide-i386 --no-bootstrap-image <srpm>

This is basically what would happen in Koji (bootstrap image disabled).

Comment 6 Dale Turner 2023-12-04 22:48:07 UTC
That workaround does not work for me.

"mock --yum ..." does.

Comment 7 Pavel Raiskup 2023-12-05 09:29:47 UTC
> That workaround does not work for me.
> "mock --yum ..." does.

Can you provide more info?  Switching to --yum on reasonable modern (EL8+)
machine is likely a no because yum is a /bin/dnf-3 (== DNF4) symlink.

> Evan, can you please take a look?

I'm setting Component=dnf5 for this issue.  Evan is on PTO right now and
it would be nice to hear opinions from DNF5 experts.

Comment 8 Pavel Raiskup 2023-12-05 09:55:03 UTC
Another work-around (pointed by DNF5 folks):
$ mock -r fedora-rawhide-i386 --forcearch i686 ...

Comment 9 Pavel Raiskup 2023-12-05 10:51:01 UTC
Using `setarch i386` may help to reproduce what Mock does.  DNF4 works
fine with: `sudo setarch i386 /bin/dnf-3 install --disablerepo='*' --repofrompath='hell,https://kojipkgs.fedoraproject.org/repos/rawhide/latest/$basearch/' --enablerepo hell --installroot /tmp/newroot bash --releasever=rawhide --nogpgcheck`

Comment 10 Panu Matilainen 2023-12-05 11:48:46 UTC
dnf5 appears to use %_host_cpu macro for arch detection but that does NOT expand to i686 on an x86_64 build of rpm. Only 32bit builds of rpm can detect 32bit CPUs, so the only place where %_host_cpu will be i686 (or such) on an x86_64 system is a 32bit chroot. Which is of course what is only being bootstrapped here, so...

Comment 11 Dale Turner 2023-12-05 14:35:44 UTC
Created attachment 2002828 [details]
Log of " mock --root=fedora-rawhide-i386 --init "

This is what my machine gives for:
mock --root=fedora-rawhide-i386 --init

Comment 12 Dale Turner 2023-12-05 14:37:06 UTC
Created attachment 2002829 [details]
Log of " mock --root=fedora-rawhide-i386 --init --yum "

This is what I get for:
mock --root=fedora-rawhide-i386 --init --yum

Comment 13 Marek Blaha 2023-12-05 14:45:00 UTC
There is PR https://github.com/rpm-software-management/dnf5/pull/1063 that should fix this issue.

Comment 14 Pavel Raiskup 2023-12-11 12:38:51 UTC
And this has been fixed by DNF5 in F40.

Comment 15 Pavel Raiskup 2023-12-11 13:00:10 UTC
Just make sure you `mock -r fedora-rawhide-i386 --scrub=bootstrap` again, otherwise you are using the old "cached" DNF5 package.

Thank you, Marek and the DNF5 team for fixing this so quickly!