+++ This bug was initially created as a clone of Bug #974372 +++
Description of problem:
When cloning errata using spacewalk-clone-by-date when using the -o option to only clone security errata the dependencies for the security errata are not fully resolved. However when the -o option is not specified the all dependencies are taken care of.
Version-Release number of selected component (if applicable):
RHN Satellite 5.5
How reproducible:
Very reproducible
Steps to Reproduce:
1. Create custom channel
2. Use spacewalk-clone-by-date to clone security errata to custom channel with the '-o' (--security_only) option.
3. Subscribe client to the channel and run yum update
(for my test i used a RHEL6.2)
Actual results:
Multiple dependencies errors
Expected results:
No dependency errors.
--- Additional comment from Xixi on 2013-08-06 01:38:39 EDT ---
...
Update #4 (5/29/2013):
...
I did some more digging and found that spacewalk actually attempts to take care of this very issue already with "spacewalk-clone-by-date". I tried that out. A preliminary test revealed that there were 130 security errata available to clone.
I ran the command :
# spacewalk-clone-by-date -l rhel-x86_64-server-6 it-rhel-x86_64-server-6 -d 2013-05-29 -o
Reading repository information.
By continuing the following will be cloned:
rhel-x86_64-server-6 -> it-rhel-x86_64-server-6 (130/499 Errata)
Continue with clone (y/n)?y
Cloning Errata into it-rhel-x86_64-server-6 (130):
________________________________________
######################################## - complete
Copying repodata, please wait.
Solving Dependencies (742):
________________________________________
######################################## - complete
Processing Dependencies:
________________________________________
######################################## - complete
An it looks like it caught all 130. As you can see it does attempt to fix deps as well.
But it appears to have missed the latest version of "bfa-firmware" and "selinux-policy"
--> Processing Conflict: kernel-2.6.32-358.6.2.el6.x86_64 conflicts bfa-firmware < 3.0.3.1
--> Finished Dependency Resolution
Error: kernel conflicts with bfa-firmware
Error: Package: selinux-policy-targeted-3.7.19-155.el6_3.noarch (@it-rhel-x86_64-server-6)
Requires: selinux-policy = 3.7.19-155.el6_3
Removing: selinux-policy-3.7.19-155.el6_3.noarch (@it-rhel-x86_64-server-6)
selinux-policy = 3.7.19-155.el6_3
Updated By: selinux-policy-3.7.19-195.el6.noarch (it-rhel-x86_64-server-6)
selinux-policy = 3.7.19-195.el6
Available: selinux-policy-3.7.19-54.el6.noarch (it-rhel-x86_64-server-6)
selinux-policy = 3.7.19-54.el6
Available: selinux-policy-3.7.19-54.el6_0.3.noarch (it-rhel-x86_64-server-6)
selinux-policy = 3.7.19-54.el6_0.3
Available: selinux-policy-3.7.19-54.el6_0.5.noarch (it-rhel-x86_64-server-6)
selinux-policy = 3.7.19-54.el6_0.5
Available: selinux-policy-3.7.19-93.el6.noarch (it-rhel-x86_64-server-6)
selinux-policy = 3.7.19-93.el6
Available: selinux-policy-3.7.19-93.el6_1.2.noarch (it-rhel-x86_64-server-6)
selinux-policy = 3.7.19-93.el6_1.2
Available: selinux-policy-3.7.19-93.el6_1.7.noarch (it-rhel-x86_64-server-6)
selinux-policy = 3.7.19-93.el6_1.7
Available: selinux-policy-3.7.19-126.el6.noarch (it-rhel-x86_64-server-6)
selinux-policy = 3.7.19-126.el6
Available: selinux-policy-3.7.19-126.el6_2.3.noarch (it-rhel-x86_64-server-6)
selinux-policy = 3.7.19-126.el6_2.3
Available: selinux-policy-3.7.19-126.el6_2.4.noarch (it-rhel-x86_64-server-6)
selinux-policy = 3.7.19-126.el6_2.4
Available: selinux-policy-3.7.19-126.el6_2.6.noarch (it-rhel-x86_64-server-6)
selinux-policy = 3.7.19-126.el6_2.6
Available: selinux-policy-3.7.19-126.el6_2.9.noarch (it-rhel-x86_64-server-6)
selinux-policy = 3.7.19-126.el6_2.9
Available: selinux-policy-3.7.19-126.el6_2.10.noarch (it-rhel-x86_64-server-6)
selinux-policy = 3.7.19-126.el6_2.10
Available: selinux-policy-3.7.19-154.el6.noarch (it-rhel-x86_64-server-6)
selinux-policy = 3.7.19-154.el6
Is this a bug that needs to be filed ?
...
--- Additional comment from Perry Clegg on 2013-08-08 00:35:51 EDT ---
The goal of the customer was to create a custom satellite channel that only contained security errata updates, similar to when subscribing to the RHEL6 base channel and running 'yum update --security'. The reasoning behind being to negate the risk of non security packages being applied as the custom channel will only contain the applicable security patches.
After running the clone by date with the -o option and running yum update on the clone channel there were missing packages that are found when running 'yum update --security' leading to the dependency issues.
Cheers,
Perry
--- Additional comment from on 2013-12-27 10:05:50 EST ---
Was able to reproduce this and here are my notes :
Satellite 5.5 - (5.5.0.20-1.el5sat)
Ran the following to create the cloned channel :
# spacewalk-clone-by-date --channels=rhel-x86_64-server-6 rhel-x86_64-server-6-errata --to_date=2013-12-13 --security_only -u admin -p redhat
Started with this client, fresh installation of RHEL 6.2 :
Red Hat Enterprise Linux Server release 6.2 (Santiago)
Registered the client to the Satellite, changed the base channel "rhel-x86_64-server-6-errata".
Ran "yum update" and "yum update --security" and received dependency errors on both. I'll upload the files to the case with the output of each so you can reference.
--- Additional comment from Neha on 2014-05-08 08:42:54 EDT ---
One of the customer is facing similar issue where dependency resolution is not working as expected with -o option.
Its reproducible on both satellite 5.5/5.6
Looks like looks dependency resolver for spacewalk-clone-by-date is not adding Include dependencies.
[root@talpona ~]# spacewalk-clone-by-date --username=admin --password=redhat --to_date=2014-03-31 --channels=rhel-x86_64-server-6 rhel-x86_64-server-6-se --security_only --assumeyes --validate
Reading repository information.
Added rhel-x86_64-server-6-se repo from /tmp/tmpOSSxh9
Reading in repository metadata - please wait....
Checking Dependencies
Repos looked at: 1
rhel-x86_64-server-6-se
Num Packages in Repos: 6173
package: elfutils-libs-0.152-1.el6.x86_64 from rhel-x86_64-server-6-se
unresolved deps:
elfutils-libelf(x86-64) = 0:0.152-1.el6
package: gtk2-2.20.1-4.el6.i686 from rhel-x86_64-server-6-se
unresolved deps:
atk >= 0:1.29.4-2
package: gtk2-devel-2.20.1-4.el6.i686 from rhel-x86_64-server-6-se
unresolved deps:
atk-devel >= 0:1.29.4-2
glib2-devel >= 0:2.23.6-1
package: libini_config-0.6.1-6.el6.x86_64 from rhel-x86_64-server-6-se
unresolved deps:
libref_array = 0:0.1.1-6.el6
libpath_utils = 0:0.2.1-6.el6
libcollection = 0:0.6.0-6.el6
libpath_utils.so.1()(64bit)
libref_array.so.1()(64bit)
package: libldb-devel-1.1.13-3.el6.i686 from rhel-x86_64-server-6-se
unresolved deps:
pkgconfig(tdb)
libldb.so.1
libtdb-devel >= 0:1.2.10
package: libwacom-0.5-3.el6.i686 from rhel-x86_64-server-6-se
unresolved deps:
libwacom-data
package: libwacom-0.5-3.el6.x86_64 from rhel-x86_64-server-6-se
unresolved deps:
libwacom-data
package: libxcb-devel-1.8.1-1.el6.i686 from rhel-x86_64-server-6-se
unresolved deps:
libxcb = 0:1.8.1-1.el6
package: perl-Term-ProgressBar-2.09-10.el6.noarch from rhel-x86_64-server-6-se
unresolved deps:
perl(Class::MethodMaker) >= 0:1.02
perl(Term::ReadKey) >= 0:2.14
package: python-saslwrapper-0.10-2.el6.x86_64 from rhel-x86_64-server-6-se
unresolved deps:
saslwrapper = 0:0.10-2.el6
package: rhn-client-tools-1.0.0.1-8.el6.noarch from rhel-x86_64-server-6-se
unresolved deps:
rhnlib >= 0:2.5.22-13
python-ethtool >= 0:0.4
package: rhn-setup-gnome-1.0.0-87.el6.noarch from rhel-x86_64-server-6-se
unresolved deps:
rhn-setup = 0:1.0.0-87.el6
rhn-client-tools = 0:1.0.0-87.el6
tasos++ as usual for analysis.
For example its adding gtk2-2.20.1-4.el6.i686, but its missing its dependency
atk >= 1.29.4-2
In clone channel package available:
atk-1.28.0-2.el6.i686
atk-1.28.0-2.el6.x86_64
Same way its adding python-saslwrapper as part of dependency but missing saslwrapper = 0:0.10-2.el6
clone channel contains
python-saslwrapper-0.1.934605-2.el6.x86_64
python-saslwrapper-0.10-2.el6.x86_64
saslwrapper-0.1.934605-2.el6.i686
saslwrapper-0.1.934605-2.el6.x86_64
~ Neha
--- Additional comment from Neha on 2014-05-08 08:51:34 EDT ---
Uploaded errata-clone.log for above cloning.
Cloning rhel-x86_64-server-6 to rhel-x86_64-server-6-se with original package set.
Adding 5 needed dependencies to rhel-x86_64-server-6-se
polkit-0.96-5.el6_4.i686
numad-0.5-4.20120522git.el6.x86_64
netcf-libs-0.1.9-2.el6.x86_64
rhn-client-tools-1.0.0.1-8.el6.noarch
kernel-2.6.32-71.7.1.el6.x86_64
Cloning Errata into rhel-x86_64-server-6-se (477):
RHSA-2011:0373 - Important: firefox security update
RHSA-2011:0013 - Moderate: wireshark security update
.................................................
Synchronizing Errata in rhel-x86_64-server-6-se with originals
2621 packages were added to rhel-x86_64-server-6-se as a result of clone:
nspr-4.9.2-0.el6_3.1.i686
nspr-devel-4.9.2-0.el6_3.1.i686
nspr-4.9.2-0.el6_3.1.x86_64
............................
Adding 136 needed dependencies to rhel-x86_64-server-6-se
libldb-1.1.13-3.el6.x86_64
vgabios-0.6b-3.6.el6.noarch
mod_auth_kerb-5.4-9.el6.x86_64
mvapich2-psm-1.8-1.el6.x86_64
tomcatjss-2.1.0-1.el6.noarch
perl-Mozilla-LDAP-1.5.3-4.el6.x86_64
libesmtp-1.0.4-15.el6.x86_64
..............................
--- Additional comment from Xixi on 2014-06-11 20:02:03 EDT ---
(In reply to Neha from comment #14)
> tasos++ as usual for analysis.
copying Taso's notes to BZ -
"#14 Created By: Tasos Papaioannou (5/2/2014)
The following security advisory:
RHSA-2013:1540
Low: evolution security, bug fix, and enhancement update
https://rhn.redhat.com/errata/RHSA-2013-1540.html
contains the package:
evolution-data-server-2.32.3-18.el6.x86_64.rpm
evolution-data-server-2.32.3-18.el6.x86_64.rpm
which require:
gtk2 >= 2.20.1
spacewalk-clone-by-date includes the package to resolve this:
gtk2-2.20.1-4.el6.i686
It didn't also include the x86_64 arch.
But this package itself requires:
atk >= 1.29.4-2
So, it looks like the dependency resolver for spacewalk-clone-by-date needs to:
1.) Include the x86_64 arch of a dependency, when required.
2.) Include dependencies of dependencies."
--- Additional comment from Neha on 2014-06-12 02:07:45 EDT ---
Reproducer detail:
Satellite 5.6
Channel:
rhel-x86_64-server-6-se
Client:
dhcp210-204.gsslab.pnq.redhat.com (RHEL 6.4)
Steps to reproduce:
1] clone a channel
spacewalk-clone-by-date --username=admin --password=redhat --to_date=2014-03-31 --channels=rhel-x86_64-server-6 rhel-x86_64-server-6-se --security_only --assumeyes
errata-clone.log attached
2] run validate on cloned channel
spacewalk-clone-by-date --username=admin --password=redhat --to_date=2014-03-31 --channels=rhel-x86_64-server-6 rhel-x86_64-server-6-se --security_only --assumeyes --validate
output listed in comment #14
3] run yum update on client
failed with dep issue, yum_update.txt attached
4] run yum update --security
failed with dep issue, yum_update_security.txt attached
--- Additional comment from Stephen Herr on 2014-07-24 11:53:36 EDT ---
In addition to the problems mentioned in comment 19 there is another problem that connot be solved by normal dependency resolution. Consider:
1) package selinux-policy-targeted-3.7.19-54.el6_0.5 (installed on client) requires exactly selinux-policy-3.7.19-54.el6_0.5 (also installed)
2) a security erratum updates selinux-policy-3.7.19-195.el6, but not selinux-policy-targeted. We are cloning only security errata, so we only clone selinux-policy.
3) during dependency resolution the channel appears to be fine; there are no missing dependencies. selinux-policy-targeted-3.7.19-54.el6_0.5 has all of its dependencies present, as does selinux-policy-3.7.19-54.el6_0.5 and selinux-policy-3.7.19-195.el6.
4) client subscribes to cloned channel, upgrades all things security related. selinux-policy needs to be updated, but that breaks selinux-policy-targeted's dependencies. There is not equivalent update available for selinux-policy-targeted.
I am investigating potential fixes.
--- Additional comment from Stephen Herr on 2014-07-24 13:24:50 EDT ---
It's important to not that step 2 in comment 27 is not the only way this can happen. If someone released a security erratum that released only selinux-policy and not a version-locked selinux-policy-targeted, I would say that's a problem with the erratum. But that's not what's actually happening in this case.
What is happening is that there's some third rpm that is delivered by a security erratum that requires an updated selinux-policy. The new selinux-policy gets pulled in as a dependency, but there's nothing anywhere that says we require a new selinux-policy-targeted, so it doesn't get pulled in, and the breakage happens.
Note too that there's nothing magical about security errata, the clone-by-date utility allows you to specify specific errata that you want to clone and if you do you will run into this type of problem as well.
A channel is almost certianly dependency-complete at any specific point in time, which is why users just using --to_date don't run into this problem. Any time you start cherry-picking "this erratum but not those" you're going to run into this problem. Clone-by-date is really too permissive, because not only can you select specific errata (or just security errata), you can also blacklist packages that match some regular expression, in which case you are almost certainly breaking something. If we were going for guaranteed-to-work then these options should not exist, they can only possibly be a best-effort type of thing. However, I think that in the case of security errata our best effort could be a little better than it is now.
The best idea I have for resolving this problem is to instead of cloning only packages that solve package-level dependencies, clone whole errata that solve the package dependencies. So for example, the some package that we're cloning requires an updated selinux-policy rpm. Fine, but instead of just cloning selinux-policy we have to instead clone every package delivered by the erratum that delivered the updated selinux-policy.
This should work much better (at least for Red Hat provided channels) because errata are the atomic units of change for channels, not individual packages. If the original channel is a custom channel then all bets are off, because who knows how well ordered (if they even exist) the custom errata in that channel is; we'd have to fall back to cloning packages in that case.
I also would not expect too many unnecessary packages to be cloned as a result of this change, errata tend to be very focused and only contain packages that must be delivered together. It may result in some packages that are not otherwise strickly necessary (like say updated kernel-devel rpms to go with updated kernel), but there's no way to tell programatically at clone time which of those a client may need and which it won't. This should also solve the x86_64 vs i386 issue that comment 19 mentions, but we still need to do multiple rounds of dependency resolution to resolve dependencies-of-dependencies.
Channel validity has to be the #1 concern of clone-by-date, it makes no sense to clone errata or packages that you can't install because of dependency errors. If that means we have to pull in a few packages that may not stricly be required into your security-update-only channel, well sorry, that's the price that must be paid for correct channel behaivor.
--- Additional comment from Stephen Herr on 2014-07-25 14:25:53 EDT ---
I have updated clone-by-date to clone entire errata required by dependencies (instead of individual packages) and to recursively resolve dependencies in order to catch and pull in dependencies-of-dependencies. This should solve 90% of current dependency problems.
The remaining 10% of dependency problems are unresolvable. Dependency problems can still exist because the original channel is not complete, the orriginal errata information is missing / incomplete / inaccurate, or because the packages in question are not actually dependencies of each other but instead they conflict with a specific version. This last case is a reasonable thing for the package maintainer to do in some cases. For example, the kernel-2.6.32-431.1.2.el6.x86_64 conflicts with bfa-firmware < 3.2.21.1-2. You are not required to have bfa-firmware installed in order to install kernel, so there is no dependency, however if you have bfa-firmware installed it must be above that version. The package definitions are correct, however because there is no direct dependency there is nothing to tell clone-by-date that it needs to pull in updated bfa-firmware packages. There is no way to predict which of these "soft dependencies" will be required by a particular client until that client tries to upgrade. The channel administrators will have to test and resolve these issues themselves.
Before these changes:
spacewalk-clone-by-date --channels=rhel-x86_64-server-6 rhel-x86_64-server-6-errata --to_date=2013-12-13 --security_only
Resulted in:
6236 packages and 487 errata in the cloned channel.
After these changes the same commands results in:
6528 packages and 559 errata in the cloned channel.
This is a 5% increase in the number of packages cloned and a 15% increase in the number of errata cloned, but it leads to much more correct channel behavior.
After these changes and the above command, running with --validate results in:
Num Packages in Repos: 6528
package: 1:libguestfs-1.2.7-1.24.el6.x86_64 from rhel-x86_64-server-6-errata
unresolved deps:
gfs2-utils
There is no gfs2-utils in the original channel, so this cannot possibly be a clone-by-date problem.
Subscribing a client that was kickstarted from the RHEL 6.0 kickstart tree and has yum-plugin-security installed and doing a 'yum update --security' results in:
--> Processing Conflict: kernel-2.6.32-431.1.2.el6.x86_64 conflicts bfa-firmware < 3.2.21.1-2
This is because of the "soft dependency" problem described above. Manually cloning the RHBA-2013:1549 erratum which provides updated bfa-firmware packages resolves this issue and allows the update to proceed. Doing a bare "yum update" also completes successfully.
These two paragraphs have also been added to the man page to clarify the intent and ability of clone-by-date:
"""
Spacewalk-clone-by-date is tool that provides a best-effort attempt at cloning valid and dependency-complete channels. However, there is not enough information available to guarantee that the results will always be completely dependency-complete, espcecially if used with the --errata, --security_only, --blacklist, or --removelist options. This is a tool to assist Administrators with channel creation, it cannot replace them. Cases that spacewalk-clone-by-date cannot completely resolve all potential dependencies include but are not limited to: if the original channel is not dependency-complete, if the original channel does not have or contains incomplete errata information, or if packages are not strictly dependent on each other but conflict with specific versions.
Note: Unlike pervious versions spacewalk-clone-by-date now clones entire errata that are determined to be required when doing dependency resolution, not just single packages. This solves a number of dependency-completeness problems but is still not a perfect solution. This may mean that errata end up being cloned that do not meet you specifications (for example, that were added after the --to_date or are not security errata when used with --security_only). This is expected behavior. The only way to avoid this is to run with --skip_depsolve and manually resolve any dependency cloning that may be required.
"""