Bug 1118719

Hi Vit,

This has few technical problems which price of overcoming can not be justified by 'faster adoption of new RPM features'. What is the real use case you are after then?

Yes, I understand that it might not be simple.

Traditionally, Fedora builders were hosted on RHEL6, so that meant that RPM used to prepare build root was rather old. Although now the builders are hosted on F20 to my knowledge, still, I can't use the latest and brightest features of RPM.

For example, I am not sure if I can use the newly introduced %load macro on Koji, during the SRPM build step.

Actually, this RFE was triggered by recent FPC discussion with regards of SCL, repeating several times that SCL is pile of hacks, which is actually true. And true is also, that we never asked RPM for any modifications, since the RPM used to prepare the buildroot was always older then the RPM which features we would like to use.

So to say, I probably don't have any better/specific usecase at my hand ATM, but I have to ask, since everybody else probably trashed this idea even before asking :)

OK, thank you for a more elaborate description, we'll see what we can do.

Thanks. I am decreasing the priority if that makes any difference :)

This request is not too far fetched and pretty reasonable. Having a more recent version of yum/dnf/rpm on the builders would be a great thing.

But this has do be done on the builders and cannot be solved by the dnf (or any other of the above) component(s). Reassigning to "distribution" as the proper solution needs to be tightly integrated with the build system. There are several possible solutions for this problems and only rel-eng has a chance of figuring out with is actually going to work and if it is really worth the hassle.

Florian

(In reply to Florian Festi from comment #5)
Actually, you are right that solving this on "distribution" level is also one possibility. On the other hand, I feel it makes it somehow harder for me as a packager, since if I can speculate about what you meant by "several possible solutions for this", this would require updated version of RPM on builders, which differs from the standard version of RPM I have available on my system by default.

Could you please elaborate more about this?

builders are currently Fedora 20 and will be moved to Fedora 21 when it is stable. I think this is more about being able to have a newer rpm on the local machine? not really sure though as its all a bit vague and handwavy.

The point is:

*As soon as new tool from RPM/DNF/YUM chain is available, we should be able to use its features ASAP*

Since mock buildroots are prepared by RPM/DNF/YUM from the builders, we are lagging behind. We should be able to use Rawhide's RPM/DNF/YUM instead.

May be we need something like mock in mock? The first level build root would be prepared by current system RPM/DNF/YUM and it would contain Rawhide versions of the same set of packages. This first level buildroot in turn would be used to create the second level mock build root, as we know it today.

Since there is now ongoing some development on Mock, I'm adding Mikolaj on CC. May be he can have some interesting ideas as well.

The problem could be solved in several ways.

1) An SCL containing latest mock with latest dependencies like RPM and DNF.  It shouldn't be to hard to make them available even on RHEL 6, if needed.

2) Chroot with minimal rawhide installation containing mock with deps. The chroot could be maintained with system mock, resulting it two or more layers of nested mocks.

3) It should be easy to make paths to rpm and dnf configurable in mock, so that it could use newer versions of these tools, for example from user's $HOME. I personally did that a few days ago when I was trying to use latest upstream RPM, which has the ability to generate weak auto-requires, but is not yet in Fedora, not even in rawhide. I solved this by compiling RPM myself and pointing mock to it.

Michael, could you make rpm/dnf paths configurable in mock config? Additionally please make room for specifying custom args to these tools. (Example use case: Passing --rpmfcdebug to rpmbuild as a nice way of debugging auto-requires generators.) I opened issue for this: https://github.com/msimacek/mock/issues/7

I think this is something what can be done in Mock. Therefore switching component to mock.

*** Bug 1425446 has been marked as a duplicate of this bug. ***

This makes impossible to have new features in RPM (in rawhide) and use them from Fedora because koji builders use stable fedora.

Same applies to COPR. If I have patched RPM or dnf, it won't use it, hence build will fail.

Oh, I wouldn't complain too much about the current situation. In not so distant past, rpm features available to Fedora rawhide were limited to those of latest released RHEL. To us old-timers, being only limited to latest Fedora stable is almost like a life-long dream come true :)

See https://github.com/rpm-software-management/mock/issues/15 for another use case (duplicate of this one).

I've had a crack at implementing a feature to first build an outer chroot, then use that to assemble the main build chroot - effectively making the chroot built by its own version of yum/dnf

https://github.com/rpm-software-management/mock/pull/51

@Michael would you mind to look at bug 1447627 to polish this feature a bit?

Hmm I probably should take a second pass at this :-)

Frankly, except the bug 1447627, I am not sure the feature in its current state is usable. The longer I am using it, the more I think it should not be possible to enable it globally, but it should be set per buildroot.

E.g. I am using Fedora Rawhide and I want to build RHEL6 packages. That means I'd like to be able to use Fedora's DNF to setup bootstrap buildroot, which contains YUM. What I really don't want actually is to have YUM installed on my system.

Now the question is where the YUM is coming from, if it is RHEL6 version or Fedora version. Possibly I might choose different one for different reasons.

And we are speaking here just about DNF/YUM from Fedora/RHEL, but if I want to use the bootstrap buildroot for lets say OpenSuse, I probably still want to use Fedora's DNF to setup the bootstrap buildroot, but I want to point it to OpenSuse repositories and install Zypper from there, because Zypper might not be in Fedora (Is it? I don't know. If it is, is it well maintained?).

Well you can have:
config_opts['bootstrap_package_manager'] = 'dnf'
config_opts['package_manager'] = 'zypper'

and if you implement the functionality of zypper package manager (just few lines)
then bootstrap will be installed using dnf and target chroot using zypper from the bootstrap chroot (i.e. the suse version).

(In reply to Miroslav Suchý from comment #19)
> Well you can have:
> config_opts['bootstrap_package_manager'] = 'dnf'
> config_opts['package_manager'] = 'zypper'
> 
> and if you implement the functionality of zypper package manager (just few
> lines)
> then bootstrap will be installed using dnf and target chroot using zypper
> from the bootstrap chroot (i.e. the suse version).

Nice, but this does not answer if DNF is installing Zypper from Fedora or Suse repositories.

In this situation Mock would install Zypper from Suse (because it will use repository specified in mock config). First it will install it into bootstrap chroot using host's DNF, then it will install target chroot using the Zypper. But in both situation it will use suse repository.

Ok, this is too artificial discussion. Here is real situation. I am using Fedora Rawhide and I don't have YUM installed, since I don't need it for anything. But I want to build package for EPEL. Here are the changes to the config file I did:

~~~
# diff /etc/mock/epel-7-x86_64.cfg{.orig,} -u
--- /etc/mock/epel-7-x86_64.cfg.orig	2017-12-12 10:15:46.788617613 +0100
+++ /etc/mock/epel-7-x86_64.cfg	2017-12-12 10:16:57.062265280 +0100
@@ -5,6 +5,11 @@
 config_opts['dist'] = 'el7'  # only useful for --resultdir variable subst
 config_opts['releasever'] = '7'
 
+config_opts['use_bootstrap_container'] = True
+
+config_opts['bootstrap_package_manager'] = 'dnf'
+config_opts['package_manager'] = 'yum'
+
 config_opts['yum.conf'] = """
 [main]
 keepcache=1
~~~

And this is mock output:

~~~
$ mock -r epel-7-x86_64 --init
INFO: mock.py version 1.4.7 starting (python version = 3.6.3)...
ERROR: Command /usr/bin/yum is not available. Either install package containing this command
or run mock with --yum or --dnf to overwrite config value. However this may
lead to different dependency solving!
Traceback (most recent call last):
  File "/usr/libexec/mock/mock", line 949, in <module>
    main()
  File "/usr/lib/python3.6/site-packages/mockbuild/trace_decorator.py", line 96, in trace
    result = func(*args, **kw)
  File "/usr/libexec/mock/mock", line 706, in main
    is_bootstrap=True)
  File "/usr/lib/python3.6/site-packages/mockbuild/trace_decorator.py", line 96, in trace
    result = func(*args, **kw)
  File "/usr/lib/python3.6/site-packages/mockbuild/buildroot.py", line 62, in __init__
    self.pkg_manager = package_manager(config, self, plugins, bootstrap_buildroot)
  File "/usr/lib/python3.6/site-packages/mockbuild/package_manager.py", line 21, in package_manager
    return Yum(config_opts, chroot, plugins, bootstrap_buildroot)
  File "/usr/lib/python3.6/site-packages/mockbuild/package_manager.py", line 191, in __init__
    self._check_command()
  File "/usr/lib/python3.6/site-packages/mockbuild/package_manager.py", line 171, in _check_command
    lead to different dependency solving!""".format(self.command))
Exception: Command /usr/bin/yum is not available. Either install package containing this command
or run mock with --yum or --dnf to overwrite config value. However this may
lead to different dependency solving!
~~~

I am closing this RFE as done. We have the bootstrap feature. Yes, it has issues. But address them one by one. The problem in #22 seems to be dupe of bug 1467314, so lets continue there.