> The `mock -r fedora-rawhide-x86_64 --scrub=all` always fails with the error message "ImportError: /lib64/libdnf.so.2: undefined symbol: g_once_init_enter_pointer ". Feel free to change the component to the proper one.

Sorry for my typo on my first comment. It's not `mock -r fedora-rawhide-x86_64 --scrub=all` but `mock -r fedora-rawhide-x86_64 --shell`.

The `mock -r fedora-rawhide-x86_64 --shell` always fails with the error message "ImportError: /lib64/libdnf.so.2: undefined symbol: g_once_init_enter_pointer ". Feel free to change the component to the proper one.

Hello Jun, thank you for the report, but I can not reproduce this.  Perhaps
something has changed in the meantime in the Rawhide repositories.

From the log you provided, it seems that the DNF in Mock's bootstrap chroot
doesn't work properly (build-time linked against a symbol that is not
provided at runtime).

What's weird though is that `fedora-rawhide-x86_64` is now a DNF5
chroot, not DNF4 - so Mock should be using /bin/dnf5 not /bin/dnf-3.
Can you check that your mock/mock-core-configs packages are not
outdated, and that you don't have some local configuration modifications?

The error says that /lib64/libdnf.so.2 of libdnf-0.73.0-1.fc40.x86_64 package in the bootstrap enviroment requires g_once_init_enter_pointer ELF symbol which is normally provided by /usr/lib64/libglib-2.0.so of glib2 package. According to Koji the libdnf package was built using glib2-0:2.79.1-1.fc40.x86_64. Jun, if you can still reproduce it, check what glib2 package you have installed in the bootstrap environment. In ideal world the package should be at that version or newer.

(In reply to Pavel Raiskup from comment #2)
> Hello Jun, thank you for the report, but I can not reproduce this.  Perhaps
> something has changed in the meantime in the Rawhide repositories.
> 
> From the log you provided, it seems that the DNF in Mock's bootstrap chroot
> doesn't work properly (build-time linked against a symbol that is not
> provided at runtime).
> 
> What's weird though is that `fedora-rawhide-x86_64` is now a DNF5
> chroot, not DNF4 - so Mock should be using /bin/dnf5 not /bin/dnf-3.
> Can you check that your mock/mock-core-configs packages are not
> outdated, and that you don't have some local configuration modifications?

Hello Raiskup,
Thank you for your reply.
For your question about the log with DNF4 and /bin/dnf-3, while the following commands show the Fedora rawhide config file /etc/mock/fedora-rawhide-x86_64.cfg belongs to the the RPM package mock-core-configs-40.1-1.fc39.noarch, I am using the customized /etc/mock/fedora-rawhide-x86_64.cfg before a age of starting the template feature: /etc/mock/templates/. I assume that's the reason why the log shows DNF4 and /bin/dnf-3.

```
$ rpm -qf /etc/mock/fedora-rawhide-x86_64.cfg
mock-core-configs-40.1-1.fc39.noarch

$ rpm -q mock-core-configs
mock-core-configs-40.1-1.fc39.noarch
```

I plan to download the mock-core-configs-40.1-1.fc39.noarch.rpm, extract and recover the /etc/mock/ with it.
https://koji.fedoraproject.org/koji/buildinfo?buildID=2403024

By the way, I posted this issue on the devel@ mailing list below too.
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/DPD7YCDCW2EG7DV7FBZ4FI47YERQ73W6/

(In reply to Petr Pisar from comment #3)
> The error says that /lib64/libdnf.so.2 of libdnf-0.73.0-1.fc40.x86_64
> package in the bootstrap enviroment requires g_once_init_enter_pointer ELF
> symbol which is normally provided by /usr/lib64/libglib-2.0.so of glib2
> package. According to Koji the libdnf package was built using
> glib2-0:2.79.1-1.fc40.x86_64. Jun, if you can still reproduce it, check what
> glib2 package you have installed in the bootstrap environment. In ideal
> world the package should be at that version or newer.

Hi,
Thank you for your reply, and the clue to find the root cause!
Yes, I can still reproduce the issue.

```
$ rpm -q libdnf
libdnf-0.72.0-1.fc39.x86_64

$ rpm -q glib2
glib2-2.78.3-1.fc39.x86_64

$ rpm -q libdnf --root /var/lib/mock/fedora-rawhide-x86_64-bootstrap/root/
libdnf-0.73.0-1.fc40.x86_64

$ rpm -q glib2 --root /var/lib/mock/fedora-rawhide-x86_64-bootstrap/root/
glib2-2.78.1-1.fc39.x86_64
```

Just in case, I removed the following directories.

````
$ pwd
/var/lib/mock

$ sudo rm -rf fedora-rawhide-x86_64 fedora-rawhide-x86_64-bootstrap/
```

And I executed the following command again.

```
$ mock -r fedora-rawhide-x86_64 --shell
INFO: mock.py version 5.5 starting (python version = 3.12.1, NVR = mock-5.5-1.fc39), args: /usr/libexec/mock/mock -r fedora-rawhide-x86_64 --shell
Start(bootstrap): init plugins
INFO: selinux enabled
Finish(bootstrap): init plugins
Start: init plugins
INFO: selinux enabled
Finish: init plugins
INFO: Signal handler active
Start: run
Start(bootstrap): chroot init
INFO: calling preinit hooks
INFO: enabled root cache
Start(bootstrap): unpacking root cache
Finish(bootstrap): unpacking root cache
INFO: enabled package manager cache
Start(bootstrap): cleaning package manager metadata
Finish(bootstrap): cleaning package manager metadata
INFO: Package manager dnf detected and used (fallback)
Finish(bootstrap): chroot init
Start: chroot init
INFO: calling preinit hooks
INFO: enabled root cache
INFO: enabled package manager cache
Start: cleaning package manager metadata
Finish: cleaning package manager metadata
INFO: enabled HW Info plugin
INFO: Package manager dnf detected and used (direct choice)
Start: installing minimal buildroot with dnf
Traceback (most recent call last):
  File "/usr/bin/dnf-3", line 61, in <module>
    from dnf.cli import main
  File "/usr/lib/python3.12/site-packages/dnf/__init__.py", line 30, in <module>
    import dnf.base
  File "/usr/lib/python3.12/site-packages/dnf/base.py", line 29, in <module>
    import libdnf.transaction
  File "/usr/lib64/python3.12/site-packages/libdnf/__init__.py", line 14, in <module>
    from . import conf
  File "/usr/lib64/python3.12/site-packages/libdnf/conf.py", line 10, in <module>
    from . import _conf
ImportError: /lib64/libdnf.so.2: undefined symbol: g_once_init_enter_pointer
ERROR: Command failed: 
 # /usr/bin/systemd-nspawn -q -M 0f7149625e3943c097acb4860c014194 -D /var/lib/mock/fedora-rawhide-x86_64-bootstrap/root -a --capability=cap_ipc_lock --bind=/tmp/mock-resolv.lk0jm5iu:/etc/resolv.conf --console=pipe --setenv=TERM=vt100 --setenv=SHELL=/bin/bash --setenv=HOME=/var/lib/mock/fedora-rawhide-x86_64/root/installation-homedir --setenv=HOSTNAME=mock --setenv=PATH=/usr/bin:/bin:/usr/sbin:/sbin '--setenv=PROMPT_COMMAND=printf "\033]0;<mock-chroot>\007"' '--setenv=PS1=<mock-chroot> \s-\v\$ ' --setenv=LANG=C.UTF-8 --setenv=LC_MESSAGES=C.UTF-8 --resolv-conf=off /usr/bin/dnf-3 --installroot /var/lib/mock/fedora-rawhide-x86_64/root/ --releasever 35 --setopt=deltarpm=False --setopt=allow_vendor_change=yes --allowerasing --disableplugin=local --disableplugin=spacewalk --disableplugin=versionlock install @buildsys-build --setopt=tsflags=nocontexts
Traceback (most recent call last):
  File "/usr/bin/dnf-3", line 61, in <module>
    from dnf.cli import main
  File "/usr/lib/python3.12/site-packages/dnf/__init__.py", line 30, in <module>
    import dnf.base
  File "/usr/lib/python3.12/site-packages/dnf/base.py", line 29, in <module>
    import libdnf.transaction
  File "/usr/lib64/python3.12/site-packages/libdnf/__init__.py", line 14, in <module>
    from . import conf
  File "/usr/lib64/python3.12/site-packages/libdnf/conf.py", line 10, in <module>
    from . import _conf
ImportError: /lib64/libdnf.so.2: undefined symbol: g_once_init_enter_pointer

```

Then the results of printing the RPM packages are same. The glib2 package is older than glib2-0:2.79.1-1.fc40.x86_64.

```
$ rpm -q libdnf --root /var/lib/mock/fedora-rawhide-x86_64-bootstrap/root/
libdnf-0.73.0-1.fc40.x86_64

$ rpm -q glib2 --root /var/lib/mock/fedora-rawhide-x86_64-bootstrap/root/
glib2-2.78.1-1.fc39.x86_64
```

I confirm that /usr/lib64/libglib-2.0.so.0 of glib2-2.78.1-1.fc39.x86_64 does not define g_once_init_enter_pointer symbol.

I suspect that what happens is that when mock creates the bootstrap environment from scratch, it downloads and extracts an fedora:latest image which contains the old glib2 package. Then mock does not update the content from YUM repositories (dnf upgrade). It instead directly installs libdnf as a dependency of python3-dnf-plugins-core. And that installs libdnf which was built against newer glib2.

As it was explained on the mailing list, libglib-2.0.so.0 does not version ELF symbols, hence no dependency on a new SONAME version appears on RPM level and thus there is nothing what would force DNF to upgrade glib2 package when installing libdnf.

We could add an ad-hoc dependency to libdnf on a newer glib2 to solve this. But that's not a systemic solution.
Another problem to investigate is why mock insists on fedora:latest image with glib2 from 2023-10-26. There were many glib2 build in between. E.g.latest glib2 is from 2024-01-26. Often reason is that this is indeed the latest image. I have no idea where mock downloads the images from.

Thank you, Jun,

> $ rpm -q mock-core-configs
> mock-core-configs-40.1-1.fc39.noarch

Can you please test with mock-core-configs 40.2?
https://rpm-software-management.github.io/mock/Release-Notes-Configs-40.2

Even though.. fedora-rawhide-x86_64 shouldn't be affected:

$ mock -r fedora-rawhide-x86_64 --debug-config 2>/dev/null | grep -e bootstrap -e package_manager 
config_opts['bootstrap_image'] = 'registry.fedoraproject.org/fedora:rawhide'
config_opts['package_manager'] = 'dnf5'

@Petr Pisar wrote:
> We could add an ad-hoc dependency to libdnf on a newer glib2 to solve this. But that's not a systemic solution.

Symbol removal should cause a soname bump, in turn, triggering
libdnf rebuild?

> Another problem to investigate is why mock insists on fedora:latest image with glib2 

This is the default image used by mock if `bootstrap_image` isn't set by
configuration.  Which really shouldn't be the case.  Jun, can you try to
analyze the `--debug-config` output?

(In reply to Pavel Raiskup from comment #7)
> Symbol removal should cause a soname bump, in turn, triggering
> libdnf rebuild?
> 
Yes, if glib2 maintainers notice it. But here the problem is adding a symbol. A symbol which is not used by libdnf C code. It gets there "secretly" through glib2 header files. Probably a changed macro or an in-lined function.

> > Another problem to investigate is why mock insists on fedora:latest image with glib2 
> 
> This is the default image used by mock if `bootstrap_image` isn't set by
> configuration.  Which really shouldn't be the case.  Jun, can you try to
> analyze the `--debug-config` output?

I think the problem is that Fedora moved container images from <https://registry.fedoraproject.org/repo/fedora/tags/> to <https://quay.io/repository/fedora/fedora?tab=tags> without telling anybody <https://docs.fedoraproject.org/en-US/containers/#_where_to_find_the_containers>, <https://pagure.io/ContainerSIG/container-sig/issue/60>.

On quay.io there is fedora:41 updated yesterday. There is fedora:latest too, but it points to fedora:39 updated 3 months ago. Maybe latest is supposed to point to the latest stable Fedora, i.e 39 now. Though, it could be updated frequently, ideally with every refresh of fedora-updates repository.

(In reply to Pavel Raiskup from comment #7)
> Thank you, Jun,
> 
> > $ rpm -q mock-core-configs
> > mock-core-configs-40.1-1.fc39.noarch
> Can you please test with mock-core-configs 40.2?
> https://rpm-software-management.github.io/mock/Release-Notes-Configs-40.2

```
$ rpm -q mock mock-core-configs
mock-5.5-1.fc39.noarch
mock-core-configs-40.2-1.fc39.noarch
```

Yes, I restored the /etc/mock from the `mock-core-configs-40.2-1.fc39.noarch`. There are no modified files in the `/etc/mock` any more. I am managing my custom in the `~/.config/mock.cfg`.

And the following command just works.

```
$ mock -r fedora-rawhide-x86_64 --shell
INFO: mock.py version 5.5 starting (python version = 3.12.1, NVR = mock-5.5-1.fc39), args: /usr/libexec/mock/mock -r fedora-rawhide-x86_64 --shell
Start(bootstrap): init plugins
INFO: selinux enabled
Finish(bootstrap): init plugins
Start: init plugins
INFO: selinux enabled
Finish: init plugins
INFO: Signal handler active
Start: run
Start(bootstrap): chroot init
WARNING: nosync is enabled but the library wasn't found on the system
INFO: calling preinit hooks
INFO: enabled root cache
INFO: enabled package manager cache
Start(bootstrap): cleaning package manager metadata
Finish(bootstrap): cleaning package manager metadata
INFO: Package manager dnf5 detected and used (fallback)
Finish(bootstrap): chroot init
Start: chroot init
WARNING: nosync is enabled but the library wasn't found on the system
INFO: calling preinit hooks
INFO: enabled root cache
INFO: enabled package manager cache
Start: cleaning package manager metadata
Finish: cleaning package manager metadata
INFO: enabled HW Info plugin
INFO: Package manager dnf5 detected and used (direct choice)
Finish: chroot init
Start: shell
```

I needed to run the following commands to make the command work.

```
$ sudo cp -p /etc/mock_20240226_bak/logging.ini /etc/mock/
$ mock --scrub=all -r fedora-rawhide-x86_64
```

> > Another problem to investigate is why mock insists on fedora:latest image with glib2 
> 
> This is the default image used by mock if `bootstrap_image` isn't set by
> configuration.  Which really shouldn't be the case.  Jun, can you try to
> analyze the `--debug-config` output?

Sure. Though I informed the `mock -r fedora-rawhide-x86_64 --shell` works with the config files in the `mock-core-configs-40.1-1.fc39.noarch`, I can still reproduce this error by using my used `/etc/mock` like this.

```
$ cd /etc/
$ mv mock mock_bak
$ sudo cp -rp mock_20240226_bak mock
```

```
$ mock --scrub=all -r fedora-rawhide-x86_64

$ mock -r fedora-rawhide-x86_64 --shell
INFO: mock.py version 5.5 starting (python version = 3.12.1, NVR = mock-5.5-1.fc39), args: /usr/libexec/mock/mock -r fedora-rawhide-x86_64 --shell
Start(bootstrap): init plugins
INFO: selinux enabled
Finish(bootstrap): init plugins
Start: init plugins
INFO: selinux enabled
Finish: init plugins
INFO: Signal handler active
Start: run
Start(bootstrap): chroot init
WARNING: nosync is enabled but the library wasn't found on the system
INFO: calling preinit hooks
INFO: enabled root cache
INFO: enabled package manager cache
Start(bootstrap): cleaning package manager metadata
Finish(bootstrap): cleaning package manager metadata
INFO: Guessed host environment type: unknown
INFO: Using bootstrap image: fedora:latest
INFO: Pulling image: fedora:latest
INFO: Copy content of container fedora:latest to /var/lib/mock/fedora-rawhide-x86_64-bootstrap/root
INFO: Checking that fedora:latest image matches host's architecture
INFO: mounting fedora:latest with podman image mount
INFO: image fedora:latest as /var/lib/containers/storage/overlay/8ff7ad910417a7b8a49019008335921d2aac0e3304a19ce258deabf431e59801/merged
INFO: umounting image fedora:latest (/var/lib/containers/storage/overlay/8ff7ad910417a7b8a49019008335921d2aac0e3304a19ce258deabf431e59801/merged) with podman image umount
INFO: Package manager dnf detected and used (fallback)
INFO: Bootstrap image not marked ready
Start(bootstrap): installing dnf tooling
No matches found for the following disable plugin patterns: local, spacewalk, versionlock
Fedora rawhide mirrors                               10 MB/s |  76 MB     00:07    
Fedora rawhide-debug                                6.5 MB/s |  34 MB     00:05    
Last metadata expiration check: 0:00:09 ago on Mon Feb 26 17:43:40 2024.
Package python3-dnf-4.18.1-2.fc39.noarch is already installed.
Dependencies resolved.
====================================================================================
 Package                       Arch        Version                Repository   Size
====================================================================================
Installing:
 python3-dnf-plugins-core      noarch      4.5.0-1.fc40           fedora      318 k
Upgrading:
 dnf                           noarch      4.19.0-1.fc40          fedora      509 k
 dnf-data                      noarch      4.19.0-1.fc40          fedora       40 k
 libdnf                        x86_64      0.73.0-1.fc40          fedora      697 k
 libstdc++                     x86_64      14.0.1-0.7.fc41        fedora      874 k
 python3-dnf                   noarch      4.19.0-1.fc40          fedora      591 k
 python3-hawkey                x86_64      0.73.0-1.fc40          fedora      105 k
 python3-libdnf                x86_64      0.73.0-1.fc40          fedora      847 k
 yum                           noarch      4.19.0-1.fc40          fedora       38 k
Installing dependencies:
 dbus-libs                     x86_64      1:1.14.10-3.fc40       fedora      156 k
 python3-dateutil              noarch      1:2.8.2-13.fc40        fedora      356 k
 python3-dbus                  x86_64      1.3.2-6.fc40           fedora      156 k
 python3-distro                noarch      1.9.0-3.fc40           fedora       50 k
 python3-six                   noarch      1.16.0-14.fc40         fedora       41 k
 python3-systemd               x86_64      235-9.fc40             fedora      106 k

Transaction Summary
====================================================================================
Install  7 Packages
Upgrade  8 Packages

Total download size: 4.8 M
Downloading Packages:
(1/15): dbus-libs-1.14.10-3.fc40.x86_64.rpm         606 kB/s | 156 kB     00:00    
(2/15): python3-dbus-1.3.2-6.fc40.x86_64.rpm        592 kB/s | 156 kB     00:00    
(3/15): python3-dateutil-2.8.2-13.fc40.noarch.rpm   1.2 MB/s | 356 kB     00:00    
(4/15): python3-distro-1.9.0-3.fc40.noarch.rpm      748 kB/s |  50 kB     00:00    
(5/15): python3-dnf-plugins-core-4.5.0-1.fc40.noarc 3.3 MB/s | 318 kB     00:00    
(6/15): python3-six-1.16.0-14.fc40.noarch.rpm       448 kB/s |  41 kB     00:00    
(7/15): python3-systemd-235-9.fc40.x86_64.rpm       1.3 MB/s | 106 kB     00:00    
(8/15): dnf-4.19.0-1.fc40.noarch.rpm                2.1 MB/s | 509 kB     00:00    
(9/15): dnf-data-4.19.0-1.fc40.noarch.rpm           167 kB/s |  40 kB     00:00    
(10/15): libdnf-0.73.0-1.fc40.x86_64.rpm            2.6 MB/s | 697 kB     00:00    
(11/15): libstdc++-14.0.1-0.7.fc41.x86_64.rpm       2.5 MB/s | 874 kB     00:00    
(12/15): python3-dnf-4.19.0-1.fc40.noarch.rpm       1.5 MB/s | 591 kB     00:00    
(13/15): python3-hawkey-0.73.0-1.fc40.x86_64.rpm    315 kB/s | 105 kB     00:00    
(14/15): python3-libdnf-0.73.0-1.fc40.x86_64.rpm    3.1 MB/s | 847 kB     00:00    
(15/15): yum-4.19.0-1.fc40.noarch.rpm               166 kB/s |  38 kB     00:00    
------------------------------------------------------------------------------------
Total                                               2.2 MB/s | 4.8 MB     00:02     
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Preparing        :                                                            1/1 
  Upgrading        : libstdc++-14.0.1-0.7.fc41.x86_64                          1/23 
  Upgrading        : libdnf-0.73.0-1.fc40.x86_64                               2/23 
  Upgrading        : python3-libdnf-0.73.0-1.fc40.x86_64                       3/23 
  Upgrading        : python3-hawkey-0.73.0-1.fc40.x86_64                       4/23 
  Upgrading        : dnf-data-4.19.0-1.fc40.noarch                             5/23 
  Upgrading        : python3-dnf-4.19.0-1.fc40.noarch                          6/23 
  Upgrading        : dnf-4.19.0-1.fc40.noarch                                  7/23 
  Running scriptlet: dnf-4.19.0-1.fc40.noarch                                  7/23 
  Installing       : python3-systemd-235-9.fc40.x86_64                         8/23 
  Installing       : python3-six-1.16.0-14.fc40.noarch                         9/23 
  Installing       : python3-dateutil-1:2.8.2-13.fc40.noarch                  10/23 
  Installing       : python3-distro-1.9.0-3.fc40.noarch                       11/23 
  Installing       : dbus-libs-1:1.14.10-3.fc40.x86_64                        12/23 
  Installing       : python3-dbus-1.3.2-6.fc40.x86_64                         13/23 
  Installing       : python3-dnf-plugins-core-4.5.0-1.fc40.noarch             14/23 
  Upgrading        : yum-4.19.0-1.fc40.noarch                                 15/23 
  Cleanup          : yum-4.18.1-2.fc39.noarch                                 16/23 
  Running scriptlet: dnf-4.18.1-2.fc39.noarch                                 17/23 
  Cleanup          : dnf-4.18.1-2.fc39.noarch                                 17/23 
  Running scriptlet: dnf-4.18.1-2.fc39.noarch                                 17/23 
  Cleanup          : python3-dnf-4.18.1-2.fc39.noarch                         18/23 
  Cleanup          : python3-hawkey-0.72.0-1.fc39.x86_64                      19/23 
  Cleanup          : dnf-data-4.18.1-2.fc39.noarch                            20/23 
  Cleanup          : python3-libdnf-0.72.0-1.fc39.x86_64                      21/23 
  Cleanup          : libdnf-0.72.0-1.fc39.x86_64                              22/23 
  Cleanup          : libstdc++-13.2.1-4.fc39.x86_64                           23/23 
  Running scriptlet: libstdc++-13.2.1-4.fc39.x86_64                           23/23 
  Verifying        : dbus-libs-1:1.14.10-3.fc40.x86_64                         1/23 
  Verifying        : python3-dateutil-1:2.8.2-13.fc40.noarch                   2/23 
  Verifying        : python3-dbus-1.3.2-6.fc40.x86_64                          3/23 
  Verifying        : python3-distro-1.9.0-3.fc40.noarch                        4/23 
  Verifying        : python3-dnf-plugins-core-4.5.0-1.fc40.noarch              5/23 
  Verifying        : python3-six-1.16.0-14.fc40.noarch                         6/23 
  Verifying        : python3-systemd-235-9.fc40.x86_64                         7/23 
  Verifying        : dnf-4.19.0-1.fc40.noarch                                  8/23 
  Verifying        : dnf-4.18.1-2.fc39.noarch                                  9/23 
  Verifying        : dnf-data-4.19.0-1.fc40.noarch                            10/23 
  Verifying        : dnf-data-4.18.1-2.fc39.noarch                            11/23 
  Verifying        : libdnf-0.73.0-1.fc40.x86_64                              12/23 
  Verifying        : libdnf-0.72.0-1.fc39.x86_64                              13/23 
  Verifying        : libstdc++-14.0.1-0.7.fc41.x86_64                         14/23 
  Verifying        : libstdc++-13.2.1-4.fc39.x86_64                           15/23 
  Verifying        : python3-dnf-4.19.0-1.fc40.noarch                         16/23 
  Verifying        : python3-dnf-4.18.1-2.fc39.noarch                         17/23 
  Verifying        : python3-hawkey-0.73.0-1.fc40.x86_64                      18/23 
  Verifying        : python3-hawkey-0.72.0-1.fc39.x86_64                      19/23 
  Verifying        : python3-libdnf-0.73.0-1.fc40.x86_64                      20/23 
  Verifying        : python3-libdnf-0.72.0-1.fc39.x86_64                      21/23 
  Verifying        : yum-4.19.0-1.fc40.noarch                                 22/23 
  Verifying        : yum-4.18.1-2.fc39.noarch                                 23/23 

Upgraded:
  dnf-4.19.0-1.fc40.noarch                 dnf-data-4.19.0-1.fc40.noarch           
  libdnf-0.73.0-1.fc40.x86_64              libstdc++-14.0.1-0.7.fc41.x86_64        
  python3-dnf-4.19.0-1.fc40.noarch         python3-hawkey-0.73.0-1.fc40.x86_64     
  python3-libdnf-0.73.0-1.fc40.x86_64      yum-4.19.0-1.fc40.noarch                
Installed:
  dbus-libs-1:1.14.10-3.fc40.x86_64                                                 
  python3-dateutil-1:2.8.2-13.fc40.noarch                                           
  python3-dbus-1.3.2-6.fc40.x86_64                                                  
  python3-distro-1.9.0-3.fc40.noarch                                                
  python3-dnf-plugins-core-4.5.0-1.fc40.noarch                                      
  python3-six-1.16.0-14.fc40.noarch                                                 
  python3-systemd-235-9.fc40.x86_64                                                 

Complete!
Finish(bootstrap): installing dnf tooling
Start(bootstrap): creating root cache
Finish(bootstrap): creating root cache
Finish(bootstrap): chroot init
Start: chroot init
WARNING: nosync is enabled but the library wasn't found on the system
INFO: calling preinit hooks
INFO: enabled root cache
INFO: enabled package manager cache
Start: cleaning package manager metadata
Finish: cleaning package manager metadata
INFO: enabled HW Info plugin
INFO: Package manager dnf detected and used (direct choice)
Start: installing minimal buildroot with dnf
Traceback (most recent call last):
  File "/usr/bin/dnf-3", line 61, in <module>
    from dnf.cli import main
  File "/usr/lib/python3.12/site-packages/dnf/__init__.py", line 30, in <module>
    import dnf.base
  File "/usr/lib/python3.12/site-packages/dnf/base.py", line 29, in <module>
    import libdnf.transaction
  File "/usr/lib64/python3.12/site-packages/libdnf/__init__.py", line 14, in <module>
    from . import conf
  File "/usr/lib64/python3.12/site-packages/libdnf/conf.py", line 10, in <module>
    from . import _conf
ImportError: /lib64/libdnf.so.2: undefined symbol: g_once_init_enter_pointer
ERROR: Command failed: 
 # /usr/bin/systemd-nspawn -q -M 7b0a6d54ab1a49f982ae057ef47875bc -D /var/lib/mock/fedora-rawhide-x86_64-bootstrap/root -a --capability=cap_ipc_lock --bind=/tmp/mock-resolv.rezogcva:/etc/resolv.conf --console=pipe --setenv=TERM=vt100 --setenv=SHELL=/bin/bash --setenv=HOME=/var/lib/mock/fedora-rawhide-x86_64/root/installation-homedir --setenv=HOSTNAME=mock --setenv=PATH=/usr/bin:/bin:/usr/sbin:/sbin '--setenv=PROMPT_COMMAND=printf "\033]0;<mock-chroot>\007"' '--setenv=PS1=<mock-chroot> \s-\v\$ ' --setenv=LANG=C.UTF-8 --setenv=LC_MESSAGES=C.UTF-8 --resolv-conf=off /usr/bin/dnf-3 --installroot /var/lib/mock/fedora-rawhide-x86_64/root/ --releasever 35 --setopt=deltarpm=False --setopt=allow_vendor_change=yes --allowerasing --disableplugin=local --disableplugin=spacewalk --disableplugin=versionlock install @buildsys-build --setopt=tsflags=nocontexts
Traceback (most recent call last):
  File "/usr/bin/dnf-3", line 61, in <module>
    from dnf.cli import main
  File "/usr/lib/python3.12/site-packages/dnf/__init__.py", line 30, in <module>
    import dnf.base
  File "/usr/lib/python3.12/site-packages/dnf/base.py", line 29, in <module>
    import libdnf.transaction
  File "/usr/lib64/python3.12/site-packages/libdnf/__init__.py", line 14, in <module>
    from . import conf
  File "/usr/lib64/python3.12/site-packages/libdnf/conf.py", line 10, in <module>
    from . import _conf
ImportError: /lib64/libdnf.so.2: undefined symbol: g_once_init_enter_pointer

```

I am going to upload the file `mock_broken.cfg` to this ticket.

```
$ mock --debug-config -r fedora-rawhide-x86_64 > mock_broken.cfg
```

Created attachment 2018968 [details]
mock_broken.cfg

The output of the following command.

```
$ mock --debug-config -r fedora-rawhide-x86_64 > mock_broken.cfg
```

Comment on attachment 2018968 [details]
mock_broken.cfg

>config_opts['additional_packages'] = []
>config_opts['build_log_fmt_str'] = '%(message)s'
>config_opts['cache_alterations'] = False
>config_opts['chroot_name'] = 'fedora-rawhide-x86_64'
>config_opts['chroot_setup_cmd'] = 'install @buildsys-build'
>config_opts['chrootuid'] = 1000
>config_opts['cleanup_on_failure'] = False
>config_opts['cleanup_on_success'] = False
>config_opts['config_file'] = '/etc/mock/fedora-rawhide-x86_64.cfg'
>config_opts['config_path'] = '/etc/mock'
>config_opts['config_paths'] = ['/home/jaruga/etc/my-config/home/.config/mock.cfg.thinkpad',
> '/etc/mock/chroot-aliases.cfg',
> '/etc/mock/site-defaults.cfg',
> '/home/jaruga/etc/my-config/etc/mock/fedora-rawhide-x86_64.cfg']
>config_opts['dist'] = 'rawhide'
>config_opts['dnf.conf'] = ('\n'
> '[main]\n'
> 'keepcache=1\n'
> 'debuglevel=2\n'
> 'reposdir=/dev/null\n'
> 'logfile=/var/log/yum.log\n'
> 'retries=20\n'
> 'obsoletes=1\n'
> 'gpgcheck=0\n'
> 'assumeyes=1\n'
> 'syslog_ident=mock\n'
> 'syslog_device=\n'
> '# Set `install_weak_deps=1` that is actual used config in production.\n'
> '# The default is `install_weak_deps=0`.\n'
> 'install_weak_deps=1\n'
> 'metadata_expire=0\n'
> 'best=1\n'
> '\n'
> '# repos\n'
> '\n'
> '[fedora]\n'
> 'name=Fedora rawhide mirrors\n'
> 'metalink=https://mirrors.fedoraproject.org/metalink?repo=rawhide&arch=$basearch\n'
> 'failovermethod=priority\n'
> 'enabled=1\n'
> '\n'
> '[fedora-latest]\n'
> 'name=Fedora rawhide latest\n'
> 'baseurl=https://kojipkgs.fedoraproject.org/repos/rawhide/latest/x86_64/\n'
> 'cost=2000\n'
> '# Disable usunally for performance. Enable to get the latest version.\n'
> 'enabled=0\n'
> '# enabled=1\n'
> '\n'
> '[fedora-debuginfo]\n'
> 'name=Fedora rawhide-debug\n'
> 'failovermethod=priority\n'
> 'metalink=https://mirrors.fedoraproject.org/metalink?repo=rawhide-debug&arch=$basearch\n'
> 'enabled=1\n'
> '\n'
> '[result]\n'
> 'name=result\n'
> 'baseurl=file:///var/lib/mock/fedora-rawhide-x86_64/result\n'
> '# Disable if using copr repo.\n'
> 'enabled=0\n'
> 'metadata_expire=0\n'
> '\n'
> '[jaruga-staging]\n'
> 'name=Copr repo for staging owned by jaruga\n'
> 'baseurl=https://copr-be.cloud.fedoraproject.org/results/jaruga/staging/fedora-rawhide-$basearch/\n'
> 'type=rpm-md\n'
> 'skip_if_unavailable=True\n'
> 'gpgcheck=1\n'
> 'gpgkey=https://copr-be.cloud.fedoraproject.org/results/jaruga/staging/pubkey.gpg\n'
> 'repo_gpgcheck=0\n'
> 'enabled=0\n'
> 'enabled_metadata=1\n'
> '\n'
> '[pvalena-ruby-on-rails]\n'
> 'name=Copr repo for ruby-on-rails owned by pvalena\n'
> 'baseurl=https://copr-be.cloud.fedoraproject.org/results/pvalena/ruby-on-rails/fedora-rawhide-$basearch/\n'
> 'type=rpm-md\n'
> 'skip_if_unavailable=True\n'
> 'gpgcheck=1\n'
> 'gpgkey=https://copr-be.cloud.fedoraproject.org/results/pvalena/ruby-on-rails/pubkey.gpg\n'
> 'repo_gpgcheck=0\n'
> 'enabled=0\n'
> 'enabled_metadata=1\n'
> '\n'
> '# bazel\n'
> '[vbatts-bazel]\n'
> 'name=Copr repo for bazel owned by vbatts\n'
> 'baseurl=https://copr-be.cloud.fedoraproject.org/results/vbatts/bazel/fedora-rawhide-$basearch/\n'
> 'type=rpm-md\n'
> 'skip_if_unavailable=True\n'
> 'gpgcheck=1\n'
> 'gpgkey=https://copr-be.cloud.fedoraproject.org/results/vbatts/bazel/pubkey.gpg\n'
> 'repo_gpgcheck=0\n'
> 'enabled=0\n'
> 'enabled_metadata=1\n')
>config_opts['dnf_warning'] = False
>config_opts['enable_disable_repos'] = []
>config_opts['extra_chroot_dirs'] = ['/run/lock']
>config_opts['legal_host_arches'] = ('x86_64',)
>config_opts['macros'] = {'%_buildhost': 'localhost.localdomain',
> '%_platform_multiplier': 1,
> '%_rpmfilename': '%%{NAME}-%%{VERSION}-%%{RELEASE}.%%{ARCH}.rpm',
> '%_topdir': '/builddir/build'}
>config_opts['no-config'] = {'epel-8': {'alternatives': {'alma+epel-8': {'description': ['Builds against '
>                                                             'AlmaLinux 8 '
>                                                             'repositories, '
>                                                             'together with '
>                                                             'the official '
>                                                             'EPEL '
>                                                             'repositories.',
>                                                             'Project page: '
>                                                             'https://almalinux.org/']},
>                             'centos-stream+epel-8': {'description': ['Builds '
>                                                                      'against '
>                                                                      'CentOS '
>                                                                      'Stream '
>                                                                      '8 '
>                                                                      'repositories '
>                                                                      '(some '
>                                                                      'packages '
>                                                                      'may be '
>                                                                      'a bit '
>                                                                      'ahead '
>                                                                      'the Red '
>                                                                      'Hat '
>                                                                      'Enterprise '
>                                                                      'Linux '
>                                                                      '8) '
>                                                                      'together '
>                                                                      'with '
>                                                                      'the '
>                                                                      'official '
>                                                                      'EPEL 8 '
>                                                                      'repositories.',
>                                                                      'Project '
>                                                                      'page: '
>                                                                      'https://www.centos.org/centos-stream/']},
>                             'circlelinux+epel-8': {'description': ['Builds '
>                                                                    'against '
>                                                                    'Circle '
>                                                                    'Linux 8 '
>                                                                    'repositories, '
>                                                                    'together '
>                                                                    'with the '
>                                                                    'official '
>                                                                    'EPEL '
>                                                                    'repositories.',
>                                                                    'Project '
>                                                                    'page: '
>                                                                    'https://cclinux.org/']},
>                             'rhel+epel-8': {'description': ['Builds against '
>                                                             'Red Hat '
>                                                             'Enterprise Linux '
>                                                             '8 repositories, '
>                                                             'together with '
>                                                             'the official '
>                                                             'EPEL '
>                                                             'repositories.',
>                                                             'This mimics what '
>                                                             'is done in the '
>                                                             'official EPEL '
>                                                             'build system, '
>                                                             'but you need a '
>                                                             'Red Hat '
>                                                             'subscription:',
>                                                             'https://rpm-software-management.github.io/mock/Feature-rhelchroots']},
>                             'rocky+epel-8': {'description': ['Builds against '
>                                                              'Rocky Linux 8 '
>                                                              'repositories, '
>                                                              'together with '
>                                                              'the official '
>                                                              'EPEL '
>                                                              'repositories.',
>                                                              'Project page: '
>                                                              'https://rockylinux.org/']}}},
> 'epel-9': {'alternatives': {'alma+epel-9': {'description': ['Builds against '
>                                                             'AlmaLinux 9 '
>                                                             'repositories, '
>                                                             'together with '
>                                                             'the official '
>                                                             'EPEL '
>                                                             'repositories.',
>                                                             'Project page: '
>                                                             'https://almalinux.org/']},
>                             'centos-stream+epel-9': {'description': ['Builds '
>                                                                      'against '
>                                                                      'CentOS '
>                                                                      'Stream '
>                                                                      '9 '
>                                                                      'repositories '
>                                                                      '(some '
>                                                                      'packages '
>                                                                      'may be '
>                                                                      'a bit '
>                                                                      'ahead '
>                                                                      'the Red '
>                                                                      'Hat '
>                                                                      'Enterprise '
>                                                                      'Linux '
>                                                                      '9) '
>                                                                      'together '
>                                                                      'with '
>                                                                      'the '
>                                                                      'official '
>                                                                      'EPEL '
>                                                                      'repositories.',
>                                                                      'Project '
>                                                                      'page: '
>                                                                      'https://www.centos.org/centos-stream/']},
>                             'rhel+epel-9': {'description': ['Builds against '
>                                                             'Red Hat '
>                                                             'Enterprise Linux '
>                                                             '9 repositories, '
>                                                             'together with '
>                                                             'the official '
>                                                             'EPEL '
>                                                             'repositories.',
>                                                             'This mimics what '
>                                                             'is done in the '
>                                                             'official EPEL '
>                                                             'build system, '
>                                                             'but you need a '
>                                                             'Red Hat '
>                                                             'subscription:',
>                                                             'https://rpm-software-management.github.io/mock/Feature-rhelchroots']},
>                             'rocky+epel-9': {'description': ['Builds against '
>                                                              'Rocky Linux 9 '
>                                                              'repositories, '
>                                                              'together with '
>                                                              'the official '
>                                                              'EPEL '
>                                                              'repositories.',
>                                                              'Project page: '
>                                                              'https://rockylinux.org/']}}}}
>config_opts['nosync'] = True
>config_opts['nosync_force'] = True
>config_opts['nspawn_args'] = ['--capability=cap_ipc_lock',
> '--bind=/tmp/mock-resolv.7os6_lfz:/etc/resolv.conf',
> '--bind=/dev/btrfs-control',
> '--bind=/dev/mapper/control',
> '--bind=/dev/fuse',
> '--bind=/dev/loop-control',
> '--bind=/dev/loop0',
> '--bind=/dev/loop1',
> '--bind=/dev/loop2',
> '--bind=/dev/loop3',
> '--bind=/dev/loop4',
> '--bind=/dev/loop5',
> '--bind=/dev/loop6',
> '--bind=/dev/loop7',
> '--bind=/dev/loop8',
> '--bind=/dev/loop9',
> '--bind=/dev/loop10',
> '--bind=/dev/loop11']
>config_opts['plugin_conf'] = {'__init___enable': False,
> '__init___opts': {'basedir': '/var/lib/mock/fedora-rawhide-x86_64',
>                   'cache_topdir': '/var/cache/mock',
>                   'cachedir': '/var/cache/mock/fedora-rawhide-x86_64',
>                   'resultdir': '/var/lib/mock/fedora-rawhide-x86_64/result',
>                   'root': 'fedora-rawhide-x86_64'},
> 'bind_mount_enable': True,
> 'bind_mount_opts': {'basedir': '/var/lib/mock/fedora-rawhide-x86_64',
>                     'cache_topdir': '/var/cache/mock',
>                     'cachedir': '/var/cache/mock/fedora-rawhide-x86_64',
>                     'create_dirs': False,
>                     'dirs': [('/home/jaruga/git', '/mnt/git')],
>                     'resultdir': '/var/lib/mock/fedora-rawhide-x86_64/result',
>                     'root': 'fedora-rawhide-x86_64'},
> 'ccache_enable': False,
> 'ccache_opts': {'basedir': '/var/lib/mock/fedora-rawhide-x86_64',
>                 'cache_topdir': '/var/cache/mock',
>                 'cachedir': '/var/cache/mock/fedora-rawhide-x86_64',
>                 'compress': None,
>                 'dir': '/var/cache/mock/fedora-rawhide-x86_64/ccache/u1000/',
>                 'max_cache_size': '4G',
>                 'resultdir': '/var/lib/mock/fedora-rawhide-x86_64/result',
>                 'root': 'fedora-rawhide-x86_64'},
> 'chroot_scan_enable': False,
> 'chroot_scan_opts': {'basedir': '/var/lib/mock/fedora-rawhide-x86_64',
>                      'cache_topdir': '/var/cache/mock',
>                      'cachedir': '/var/cache/mock/fedora-rawhide-x86_64',
>                      'only_failed': True,
>                      'regexes': ['^[^k]?core(\\.\\d+)?$', '\\.log$'],
>                      'resultdir': '/var/lib/mock/fedora-rawhide-x86_64/result',
>                      'root': 'fedora-rawhide-x86_64',
>                      'write_tar': False},
> 'compress_logs_enable': False,
> 'compress_logs_opts': {'basedir': '/var/lib/mock/fedora-rawhide-x86_64',
>                        'cache_topdir': '/var/cache/mock',
>                        'cachedir': '/var/cache/mock/fedora-rawhide-x86_64',
>                        'command': 'gzip',
>                        'resultdir': '/var/lib/mock/fedora-rawhide-x86_64/result',
>                        'root': 'fedora-rawhide-x86_64'},
> 'hw_info_enable': True,
> 'hw_info_opts': {'basedir': '/var/lib/mock/fedora-rawhide-x86_64',
>                  'cache_topdir': '/var/cache/mock',
>                  'cachedir': '/var/cache/mock/fedora-rawhide-x86_64',
>                  'resultdir': '/var/lib/mock/fedora-rawhide-x86_64/result',
>                  'root': 'fedora-rawhide-x86_64'},
> 'lvm_root_enable': False,
> 'lvm_root_opts': {'basedir': '/var/lib/mock/fedora-rawhide-x86_64',
>                   'cache_topdir': '/var/cache/mock',
>                   'cachedir': '/var/cache/mock/fedora-rawhide-x86_64',
>                   'pool_name': 'mockbuild',
>                   'resultdir': '/var/lib/mock/fedora-rawhide-x86_64/result',
>                   'root': 'fedora-rawhide-x86_64'},
> 'mount_enable': True,
> 'mount_opts': {'basedir': '/var/lib/mock/fedora-rawhide-x86_64',
>                'cache_topdir': '/var/cache/mock',
>                'cachedir': '/var/cache/mock/fedora-rawhide-x86_64',
>                'dirs': [],
>                'resultdir': '/var/lib/mock/fedora-rawhide-x86_64/result',
>                'root': 'fedora-rawhide-x86_64'},
> 'overlayfs_enable': False,
> 'overlayfs_opts': {'basedir': '/var/lib/mock/fedora-rawhide-x86_64',
>                    'cache_topdir': '/var/cache/mock',
>                    'cachedir': '/var/cache/mock/fedora-rawhide-x86_64',
>                    'resultdir': '/var/lib/mock/fedora-rawhide-x86_64/result',
>                    'root': 'fedora-rawhide-x86_64'},
> 'package_state_enable': True,
> 'package_state_opts': {'available_pkgs': False,
>                        'basedir': '/var/lib/mock/fedora-rawhide-x86_64',
>                        'cache_topdir': '/var/cache/mock',
>                        'cachedir': '/var/cache/mock/fedora-rawhide-x86_64',
>                        'installed_pkgs': True,
>                        'resultdir': '/var/lib/mock/fedora-rawhide-x86_64/result',
>                        'root': 'fedora-rawhide-x86_64'},
> 'pm_request_enable': False,
> 'pm_request_opts': {'basedir': '/var/lib/mock/fedora-rawhide-x86_64',
>                     'cache_topdir': '/var/cache/mock',
>                     'cachedir': '/var/cache/mock/fedora-rawhide-x86_64',
>                     'resultdir': '/var/lib/mock/fedora-rawhide-x86_64/result',
>                     'root': 'fedora-rawhide-x86_64'},
> 'procenv_enable': False,
> 'procenv_opts': {'basedir': '/var/lib/mock/fedora-rawhide-x86_64',
>                  'cache_topdir': '/var/cache/mock',
>                  'cachedir': '/var/cache/mock/fedora-rawhide-x86_64',
>                  'resultdir': '/var/lib/mock/fedora-rawhide-x86_64/result',
>                  'root': 'fedora-rawhide-x86_64'},
> 'root_cache_enable': True,
> 'root_cache_opts': {'age_check': True,
>                     'basedir': '/var/lib/mock/fedora-rawhide-x86_64',
>                     'cache_topdir': '/var/cache/mock',
>                     'cachedir': '/var/cache/mock/fedora-rawhide-x86_64',
>                     'compress_program': 'pigz',
>                     'decompress_program': None,
>                     'dir': '/var/cache/mock/fedora-rawhide-x86_64/root_cache/',
>                     'exclude_dirs': ['./proc',
>                                      './sys',
>                                      './dev',
>                                      './tmp/ccache',
>                                      './var/cache/yum',
>                                      './var/cache/dnf',
>                                      './var/log'],
>                     'extension': '.gz',
>                     'max_age_days': 15,
>                     'resultdir': '/var/lib/mock/fedora-rawhide-x86_64/result',
>                     'root': 'fedora-rawhide-x86_64'},
> 'rpkg_preprocessor_enable': False,
> 'rpkg_preprocessor_opts': {'basedir': '/var/lib/mock/fedora-rawhide-x86_64',
>                            'cache_topdir': '/var/cache/mock',
>                            'cachedir': '/var/cache/mock/fedora-rawhide-x86_64',
>                            'cmd': '/usr/bin/preproc-rpmspec %(source_spec)s '
>                                   '--output %(target_spec)s',
>                            'requires': ['preproc-rpmspec'],
>                            'resultdir': '/var/lib/mock/fedora-rawhide-x86_64/result',
>                            'root': 'fedora-rawhide-x86_64'},
> 'rpmautospec_enable': False,
> 'rpmautospec_opts': {'basedir': '/var/lib/mock/fedora-rawhide-x86_64',
>                      'cache_topdir': '/var/cache/mock',
>                      'cachedir': '/var/cache/mock/fedora-rawhide-x86_64',
>                      'cmd_base': ['/usr/bin/rpmautospec', 'process-distgit'],
>                      'requires': ['rpmautospec'],
>                      'resultdir': '/var/lib/mock/fedora-rawhide-x86_64/result',
>                      'root': 'fedora-rawhide-x86_64'},
> 'selinux_enable': True,
> 'selinux_opts': {'basedir': '/var/lib/mock/fedora-rawhide-x86_64',
>                  'cache_topdir': '/var/cache/mock',
>                  'cachedir': '/var/cache/mock/fedora-rawhide-x86_64',
>                  'resultdir': '/var/lib/mock/fedora-rawhide-x86_64/result',
>                  'root': 'fedora-rawhide-x86_64'},
> 'showrc_enable': False,
> 'showrc_opts': {'basedir': '/var/lib/mock/fedora-rawhide-x86_64',
>                 'cache_topdir': '/var/cache/mock',
>                 'cachedir': '/var/cache/mock/fedora-rawhide-x86_64',
>                 'resultdir': '/var/lib/mock/fedora-rawhide-x86_64/result',
>                 'root': 'fedora-rawhide-x86_64'},
> 'sign_enable': False,
> 'sign_opts': {'basedir': '/var/lib/mock/fedora-rawhide-x86_64',
>               'cache_topdir': '/var/cache/mock',
>               'cachedir': '/var/cache/mock/fedora-rawhide-x86_64',
>               'cmd': 'rpmsign',
>               'opts': '--addsign %(rpms)s',
>               'resultdir': '/var/lib/mock/fedora-rawhide-x86_64/result',
>               'root': 'fedora-rawhide-x86_64'},
> 'tmpfs_enable': False,
> 'tmpfs_opts': {'basedir': '/var/lib/mock/fedora-rawhide-x86_64',
>                'cache_topdir': '/var/cache/mock',
>                'cachedir': '/var/cache/mock/fedora-rawhide-x86_64',
>                'keep_mounted': False,
>                'max_fs_size': None,
>                'mode': '0755',
>                'required_ram_mb': 900,
>                'resultdir': '/var/lib/mock/fedora-rawhide-x86_64/result',
>                'root': 'fedora-rawhide-x86_64'},
> 'yum_cache_enable': True,
> 'yum_cache_opts': {'basedir': '/var/lib/mock/fedora-rawhide-x86_64',
>                    'cache_topdir': '/var/cache/mock',
>                    'cachedir': '/var/cache/mock/fedora-rawhide-x86_64',
>                    'max_age_days': 30,
>                    'max_metadata_age_days': 30,
>                    'online': True,
>                    'resultdir': '/var/lib/mock/fedora-rawhide-x86_64/result',
>                    'root': 'fedora-rawhide-x86_64'}}
>config_opts['print_main_output'] = True
>config_opts['releasever'] = '35'
>config_opts['repo_arch'] = 'x86_64'
>config_opts['root'] = 'fedora-rawhide-x86_64'
>config_opts['root_log_fmt_str'] = '%(levelname)s %(filename)s:%(lineno)d:  %(message)s'
>config_opts['rpmbuild_arch'] = 'x86_64'
>config_opts['rpmbuild_networking'] = True
>config_opts['state_log_fmt_str'] = '%(asctime)s - %(message)s'
>config_opts['target_arch'] = 'x86_64'
>config_opts['use_host_resolv'] = True
>config_opts['verbose'] = 1

Thank you for the details, Petr and Jun Aruga.

@Jun Aruga wrote:
> config_opts['config_paths'] = ['/home/jaruga/etc/my-config/home/.config/mock.cfg.thinkpad',
> '/etc/mock/chroot-aliases.cfg',
> '/etc/mock/site-defaults.cfg',
> '/home/jaruga/etc/my-config/etc/mock/fedora-rawhide-x86_64.cfg']

This indeed seems like you are using a custom fedora-rawhide-x86_64 config
that is probably broken, because `/etc/mock/templates/fedora-rawhide.tpl`
should be included somehow.

From the Mock perspective, using `fedora:latest` seems expected.

Side note, we are working on a better `default.cfg` documentation in
https://github.com/rpm-software-management/mock/pull/1339 (this is likely
related one).

(In reply to Petr Pisar from comment #8)
> (In reply to Pavel Raiskup from comment #7)
> > Symbol removal should cause a soname bump, in turn, triggering
> > libdnf rebuild?
>
> Yes, if glib2 maintainers notice it. But here the problem is adding a
> symbol. A symbol which is not used by libdnf C code. It gets there
> "secretly" through glib2 header files. Probably a changed macro or an
> in-lined function.

I see, this would be automatically resolved by symbol versioning in glib2, but
that doesn't exist now (per `rpm -q --provides glib2`).  I agree that we can
not do anything else than explicitly `Requires: glib2 >= XYZ` in libdnf.
Switching against libdnf, as I don't see anything we can do in Mock about
this (but building Rawhide with DNF4 seems like a corner case nowadays,
triggered by outdated mock-core-configs and user custom configurations,
so I doubt this deserves too much attention ATM).

> > > Another problem to investigate is why mock insists on fedora:latest image with glib2 
> > 
> > This is the default image used by mock if `bootstrap_image` isn't set by
> > configuration.  Which really shouldn't be the case.  Jun, can you try to
> > analyze the `--debug-config` output?
> 
> I think the problem is that Fedora moved container images from
> <https://registry.fedoraproject.org/repo/fedora/tags/> to
> <https://quay.io/repository/fedora/fedora?tab=tags> without telling anybody
> <https://docs.fedoraproject.org/en-US/containers/
> #_where_to_find_the_containers>,
> <https://pagure.io/ContainerSIG/container-sig/issue/60>.
> 
> On quay.io there is fedora:41 updated yesterday. There is fedora:latest too,
> but it points to fedora:39 updated 3 months ago. Maybe latest is supposed to
> point to the latest stable Fedora, i.e 39 now. Though, it could be updated
> frequently, ideally with every refresh of fedora-updates repository.

Are the quay.io images tested and in "production" quality?  Were the
branched tags ever updated after the initial release?

The Fedora registry change (related?) is discussed here, but the change
doesn't seem to be implemented yet:

  https://pagure.io/fedora-infrastructure/issue/11543

I mean, if the `fedora:rawhide` in Fedora registry gets the needed update
in the future (I assume yes) the glib2 issue would resolve itself then.
I propose to close this bug as NEXTRELEASE.

(In reply to Pavel Raiskup from comment #12)
> Thank you for the details, Petr and Jun Aruga.
> 
> @Jun Aruga wrote:
> > config_opts['config_paths'] = ['/home/jaruga/etc/my-config/home/.config/mock.cfg.thinkpad',
> > '/etc/mock/chroot-aliases.cfg',
> > '/etc/mock/site-defaults.cfg',
> > '/home/jaruga/etc/my-config/etc/mock/fedora-rawhide-x86_64.cfg']
> 
> This indeed seems like you are using a custom fedora-rawhide-x86_64 config
> that is probably broken, because `/etc/mock/templates/fedora-rawhide.tpl`
> should be included somehow.
> 
> From the Mock perspective, using `fedora:latest` seems expected.
> 
> Side note, we are working on a better `default.cfg` documentation in
> https://github.com/rpm-software-management/mock/pull/1339 (this is likely
> related one).

Pavel, thank you for checking my config items. I was using a `/etc/mock/fedora-rawhide-x86_64.cfg` that I modified directly before the age of the mock templates feature (`/etc/mock/templates`) was started. I was also modifying the `/etc/mock/default.cfg` like this.

```
ln -sf /etc/mock/default.cfg /etc/mock/fedora-rawhide-x86_64.cfg
```

But now I learned a lesson through this issue. I am not modifying any files in the `/etc/mock` directory after restoring files in the `/etc/mock` from the `mock-core-configs-40.2-1.fc39.noarch`. I am managing my customized files below.

```
~/.config/mock.cfg
~/.config/mock/*.cfg (including the `~/.config/mock/default.cfg`)
```

glib2 in Fedora < 40 does not have g_once_init_enter_pointer symbol.
The symbol was added in glib2-2.79.0-2.fc40.

The affected package is libdnf-0.73.0-1.fc40.x86_64 in Fedora 40. And in 41 if a user upgrades from Fedora 39 or not fully upgraded Fedora 40.

The same problem has also libmodulemd.so (libmodulemd-2.15.0-8.fc40.x86_64). Provided libdnf does not explicitly uses glib2, but it uses libmodulemd, and libmodulemd is used by dnf5 (which also does not explicitly uses glib2), I would rather place the workaround to libmodulemd package.

Adding the dependency to libmodulemd won't help in the mock case. There is already installed old libmodulemd without the dependency and installing libdnf won't upgrade libmodulemd. Moving it back to libdnf.

I proposed a fix to the upstream <https://github.com/rpm-software-management/libdnf/pull/1645>. Let's see whether other maintainers will want to fix it or not.