Bug 1280167

Summary: Kickstart wizard uses activation key base and child registration RPMS instead of the kickstart base and child channels.
Product: [Community] Spacewalk Reporter: whtenben
Component: ClientsAssignee: Michael Mráka <mmraka>
Status: CLOSED EOL QA Contact: Red Hat Satellite QA List <satqe-list>
Severity: high Docs Contact:
Priority: unspecified    
Version: 2.3CC: avi.miller, whtenben
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-07-12 11:58:24 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description whtenben 2015-11-11 05:02:13 UTC
Description of problem: 
Unable to register clients via rhnreg_ks due to newer version of rhnlib being installed in the kickstart as it is conflicting with rhn-setup provided by older child channel.  

The kickstart has specified the child channel with the expected rhn registration versions to be installed.  However since an activation key is used, which has newer versions of the rpms, these are used and is causing registration failures.


Version-Release number of selected component (if applicable):
Base and child channels of the kickstart have the expected and installed versions rhn-setup v 2.2.7-1 and rhnlib 2.5.72-1.  The activation key base channel and child channels have rhnlib v 2.5.77-1.

How reproducible:
100%

Steps to Reproduce:
1.Create a kickstart -> Operating system -> Base and spacewalk client child channels selected. Spacewalk client 2.3 channel being one of them.

2. In the kickstart add these to the rpms to be installed, which will be installed from the kickstart base and child channels. 

%packages 
@ Base
# below pkg needed for rhn registration
rhn-check
rhnsd
rhn-setup
rhn-client-tools
yum-rhn-plugin
libidn
rhncfg
rhncfg-client
rhncfg-actions
pyOpenSSL
libxml2-python
rhnlib

3. Create an activation key with an updated base channel and child channel of Spacewalk client 2.4.

4. The kickstart wizard selects the new client 2.4 rhnlib, libxml2, etc vs using the base and child channel versions from the kickstart.  This is found in the kickstart at:

mkdir -p /tmp/rhn_rpms/optional
cd /tmp/rhn_rpms/optional 
wget -P /tmp/rhn_rpms/optional http://satellite_server/download/package/xxx/rhnlib-2.5.77-1.el6.noarch.rpm http://satellite_server/download/package/xxx/pyOpenSSL-0.13.1-2.el6.x86_64.rpm http://satellite_server/download/package/xxx/libxml2-2.7.6-20.el6.x86_64.rpm http://satellite_server/download/package/xxx/libxml2-python-2.7.6-20.el6.x86_64.rpm 
rpm -Uvh --replacepkgs --replacefiles /tmp/rhn_rpms/optional/pyOpenSSL* /tmp/rhn_rpms/optional/rhnlib* /tmp/rhn_rpms/optional/libxml2-python* /tmp/rhn_rpms/optional/libxml2* 

Actual results: Client does not register with the spacewalk.  Refer to errors below.


Expected results: Registration to the spacewalk with the kickstart base and child channel version of the RPMs.  This allows for version control and also update at a later time specified by the Admin.


Additional info:

Installed ->

rhn-setup v 2.2.7-1 
rhnlib v 2.5.77-1

Error Result ->
rhnreg_ks --serverUrl=https://satellite_server/XMLRPC --sslCACert=/usr/share/rhn/RHN-ORG-TRUSTED-SSL-CERT --activationkey=$key
Traceback (most recent call last):
  File "/usr/sbin/rhnreg_ks", line 25, in <module>
    from rhn.connections import idn_pune_to_unicode
ImportError: cannot import name idn_pune_to_unicode
Traceback (most recent call last):
  File "/usr/sbin/rhn_check", line 43, in <module>
    from up2date_client import up2dateErrors
  File "/usr/share/rhn/up2date_client/up2dateErrors.py", line 23, in <module>
    import up2dateLog
  File "/usr/share/rhn/up2date_client/up2dateLog.py", line 4, in <module>
    import config
  File "/usr/share/rhn/up2date_client/config.py", line 17, in <module>
    from rhn.connections import idn_ascii_to_pune, idn_pune_to_unicode
ImportError: cannot import name idn_ascii_to_pune

Comment 1 whtenben 2015-11-11 15:02:59 UTC
Workaround -> Tested this AM.

Add the following to %packages section of "# below pkg needed for rhn registration:" 
libxml2
perl

Under the kickstart, go to scripts and add new script.

Name: Whatever you want it to be
Script contents:  

You can put the following in OR create a snippet with the following and reference the snippet as -> $SNIPPET('spacewalk/1/xxxx').

mkdir -p /mnt/sysimage/tmp/rhn_rpms/optional
chattr +i /mnt/sysimage/tmp/rhn_rpms/optional


Ensure these are checked:
nochroot
Template

Save the kickstart.

Then, make sure the new Nochroot Post script comes before the Post Script types.

Comment 2 whtenben 2015-11-11 19:54:43 UTC
The above workaround is causing issues with WEB TRACEBACK emails for every rpm that can not be "wget" to the /tmp/rhn_rpms/optional for every server that is being kickstarted.  This work around has been removed due to the flood of emails.

Please research, address, and/or provided feed back ASAP on how to bypass this Automated installation of newer packages.

- Thanks

Comment 3 whtenben 2015-11-12 15:42:00 UTC
Under steps to reproduce it should be spacewalk 2.2 client. The new channel has Spacewalk 2.4 client.  

Steps to Reproduce:
1.Create a kickstart -> Operating system -> Base and spacewalk client child channels selected. Spacewalk client 2.2 channel being one of them.

Comment 4 Avi Miller 2015-11-18 19:38:05 UTC
Another workaround option for you (untested, just theory in my head):

1. Reconfigure the activation key to reference the same channels as the kickstart
2. Create a kickstart script that changes the channel registration on the server using the rhn-channel utility
3. Enable the script in the post-install section so it happens after Spacewalk's automatic registration

This isn't a resolution, but it is a workaround that should avoid the email flood.

Comment 5 whtenben 2015-11-19 18:45:25 UTC
Updating all of the activation keys, kickstarts, etc across multiple spacewalks is a lot of man hours.  Then once the fix is applied, then have to remember to undo all of this.

Please be kind and gentle as I know that this is a horrible, horrible, work around:

/var/www/html/pub/whateverinameit
#!/bin/bash

#This script is to prevent newer versions of rhn, libxml, pyOpenSSL from being installed
#Hack around https://bugzilla.redhat.com/show_bug.cgi?id=1280167
#The post nochroot script, will mv /mnt/sysimage/bin/rpm to /mnt/sysimage/bin/origrpm and then "wget" whateverinameit.
#Then in the first post snippet, mv -f /bin/origrpm /bin/rpm.

if [[ "$@" =~ "Uvh" ]]; then
        echo "Not installing \"$@\""
else
        /bin/origrpm "$@"
fi


In the nochroot script

if [ -f /mnt/sysimage/bin/rpm ]; then
	mv -f /mnt/sysimage/bin/rpm /mnt/sysimage/bin/origrpm
	wget http://satellite_server/pub/whateverinameit -O /mnt/sysimage/bin/rpm
	chmod 755 /mnt/sysimage/bin/rpm
fi


Update your first snippet right after registration:

if [ -f /bin/origrpm ]; then
	mv -f /bin/origrpm /bin/rpm
fi

Comment 6 Avi Miller 2015-11-19 20:09:41 UTC
To be honest, I'm not sure I agree that this is a bug. You've requested that channels be subscribed according the activation key, but then don't want packages from those channels installed during installation. 

I guess I just don't understand the use-case here. Why would your activation key have a different set of channels to your kickstart? Surely that means that the first time you update after installation, everything will break anyway?

To my mind, Spacewalk is working as intended here, because it's activating the system as you've configured which subscribed to the channels you've selected in the key itself.

Note that using API automation means that updating your key or kickstart across multiple servers should take minutes not hours.

Comment 7 whtenben 2015-11-23 19:03:40 UTC
To reproduce, using distribution DVDs of CentOS7.0/7.1/kickstarts using Spacewalk repos.

	7.0 inlcludes:
		rhnlib-2.5.65-2.el7.noarch.rpm
		rhn-setup-2.0.2-5.el7.noarch.rpm

	7.1 includes:
		rhnlib-2.5.65-2.el7.noarch.rpm
		rhn-setup-2.0.2-6.el7.noarch.rpm

	Any kickstart that installs Spacewalk < 2.2 via a repository or a Spacwalk channel.


Any of these 3 modifications of a a kickstart will produce the same results.

1. Update the activation key channel for Spacewalk to 2.3 or 2.4, then create a kickstart.
2. Update the activation key channel for Spacewalk to 2.3 or 2.4, then update any of the kickstarts.
3. Create a clone of a kickstart in which the channel was created on the activation key for Spacewalk 2.2 and the activation key Spacewalk channel is updated to 2.4 already.  Then update the clone by pressing update. (They are a clones, the expectation is that they should be almost identical correct?)


We will use Spacewalk 2.4 in the following example, which includes rhnlib-2.5.77-1.

Example:

Created a clone of kickstart dev66 (created when Spacewalk 2.2 was in the activation key) to dev66bug (Spacewalk 2.4 curretnly in the activation key). 

[root]# cd /var/lib/rhn/kickstarts/wizard
[root]# diff dev66bug--1.cfg dev66--1.cfg  |grep wget
< wget -P /tmp/rhn_rpms/optional http://$redhat_management_server/pyOpenSSL-0.13.1-2.el6.x86_64.rpm http://$redhat_management_server/libxml2-2.7.6-20.el6.x86_64.rpm http://$redhat_management_server/libxml2-python-2.7.6-20.el6.x86_64.rpm http://$redhat_management_server/rhnlib-2.5.77-1.el6.noarch.rpm

> wget -P /tmp/rhn_rpms/optional http://$redhat_management_server/libxml2-2.7.6-17.el6_6.1.x86_64.rpm http://$redhat_management_server/pyOpenSSL-0.10-2.el6.x86_64.rpm http://$redhat_management_server/rhnlib-2.5.72-1.el6.noarch.rpm http://$redhat_management_server/libxml2-python-2.7.6-17.el6_6.1.x86_64.rpm


Then after the wget and rpm -Uvh --replacepkgs --replacefiles are ran.  Comes the cobbler snippet with the registration piece.

Now, because the auto generated kickstart has made the descision to install/update just rhnlib-2.5.77-1 and none of the other registration pieces the registration of the server will break.

Reason: rhnlib v 2.5.77-1 changed idn_ascii_to_puny where the rhnreg_ks provided by rhn-setup is looking for idn_ascii_to_pune.


This has been documented in another bug: https://bugzilla.redhat.com/show_bug.cgi?id=1247773

Maybe a fix by RedHat in spacewalk 2.3:

https://www.redhat.com/archives/spacewalk-list/2015-May/msg00117.html

Will test the above fix out in the Development environment and update the bug.

Comment 8 whtenben 2016-12-12 20:01:52 UTC
Upgraded to Spacewalk 2.4 and upgraded to another Spacewalk Client in the channels activation keys.  Now the older kickstarts are receiving invalid hash errors in catalina.out when servers are getting built. 

The only solution is to go through each kickstart and update/touch them.  This is because the same issue above: The Spacewalk auto generates the kickstart base on a snapshot of what was in the channel.  If that ever changes then the kickstart is broken.

grep hash catalina.out 
2016-10-27 09:36:39,201 [TP-Processor10] ERROR com.redhat.rhn.frontend.action.common.DownloadFile - Invalid hash on file download url: package/33091d86af0dd4f1df6c698638bf6e995816a438/0/1/181889/rhnlib-2.5.77-1.el6.noarch.rpm 
2016-10-27 09:36:39,230 [TP-Processor5] ERROR com.redhat.rhn.frontend.action.common.DownloadFile - Invalid hash on file download url: package/ce700d7cc65f169f81f940eac07525cbb94c0772/0/1/171334/pyOpenSSL-0.13.1-2.el6.x86_64.rpm 
2016-10-27 09:36:39,258 [TP-Processor3] ERROR com.redhat.rhn.frontend.action.common.DownloadFile - Invalid hash on file download url: package/de667ff538e240e7f76e911a43c9f0036922c5d8/0/1/171287/libxml2-2.7.6-20.el6.x86_64.rpm 
2016-10-27 09:36:39,284 [TP-Processor9] ERROR com.redhat.rhn.frontend.action.common.DownloadFile - Invalid hash on file download url: package/6ce7b5017937e0f252e0a4c5bd8a0dab7ce32c1c/0/1/171185/libxml2-python-2.7.6-20.el6.x86_64.rpm 
2016-10-27 09:37:12,411 [TP-Processor3] ERROR com.redhat.rhn.frontend.action.common.DownloadFile - Invalid hash on file download url: package/33091d86af0dd4f1df6c698638bf6e995816a438/0/1/181889/rhnlib-2.5.77-1.el6.noarch.rpm 
2016-10-27 09:37:12,439 [TP-Processor11] ERROR com.redhat.rhn.frontend.action.common.DownloadFile - Invalid hash on file download url: package/ce700d7cc65f169f81f940eac07525cbb94c0772/0/1/171334/pyOpenSSL-0.13.1-2.el6.x86_64.rpm 
2016-10-27 09:37:12,469 [TP-Processor5] ERROR com.redhat.rhn.frontend.action.common.DownloadFile - Invalid hash on file download url: package/de667ff538e240e7f76e911a43c9f0036922c5d8/0/1/171287/libxml2-2.7.6-20.el6.x86_64.rpm 
2016-10-27 09:37:12,498 [TP-Processor10] ERROR com.redhat.rhn.frontend.action.common.DownloadFile - Invalid hash on file download url: package/6ce7b5017937e0f252e0a4c5bd8a0dab7ce32c1c/0/1/171185/libxml2-python-2.7.6-20.el6.x86_64.rpm

Comment 9 whtenben 2017-06-08 19:22:02 UTC
Could the "Registration and server actions" be broken out, and allow the Spacewalk admin to uncheck the "download/install rpms" per kickstart?  It could be made a default, but would be beneficial for admins that are doing #1 and not have to worry about a process that is not doing dependency checks and is hard coded to specific versions.

Reason: 
1. Admins (like myself), are specifying a Spacewalk channel which is adding the repo available during the initial software install.  Then the admin in the Software section the admin can specify the rhn software they want installed.

# below pkg needed for rhn registration
rhn-check
rhnsd
rhn-setup
rhn-client-tools
yum-rhn-plugin
libidn
rhncfg
rhncfg-client
rhncfg-actions
libxml2
pyOpenSSL
libxml2-python
rhnlib
perl


2. New distributions are providing the rhn packages.  If another pune to puny or anything similar could break the registration process again for spacewalk admins.


- Thanks for the time.

Comment 10 Michael Mráka 2019-07-12 11:58:24 UTC
Spacewalk 2.8 (and older) has already reached it's End Of Life.

Thank you for reporting this issue and we are sorry that we were not
able to fix it before end of life. If you would still like
to see this bug fixed and are able to reproduce it against current version
of Spacewalk 2.9, you are encouraged change the 'version' and re-open it.