Bug 1220309

Summary: [upgrade] capsule-upgrade [6.0.8 to 6.1 GA snap3] fails with error: Upgrade step stop_services failed
Product: Red Hat Satellite Reporter: Sachin Ghai <sghai>
Component: UpgradesAssignee: Stephen Benjamin <stbenjam>
Status: CLOSED CURRENTRELEASE QA Contact: Sachin Ghai <sghai>
Severity: high Docs Contact:
Priority: unspecified    
Version: 6.1.0CC: bkearney, jhutar, mmccune, stbenjam
Target Milestone: Unspecified   
Target Release: Unused   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-08-12 13:59:43 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1102878, 1195450    
Attachments:
Description Flags
foreman-debug logs from capsule
none
capsule_upgrade_success none

Description Sachin Ghai 2015-05-11 09:44:58 UTC
Created attachment 1024141 [details]
foreman-debug logs from capsule

Description of problem:
capsule upgrade fails with below error:

[root@sat6-capsule ~]# capsule-installer --upgrade
Upgrading...
Upgrade Step: stop_services...
Upgrade step stop_services failed. Check logs for more information.

installer.log:
==============
[ERROR 2015-05-11 05:26:03 main] sh: katello-service: command not found

[ERROR 2015-05-11 05:26:03 main] Upgrade step stop_services failed. Check logs for more information.
[DEBUG 2015-05-11 05:26:03 main] Exit with status code: 1 (signal was 1)
[ERROR 2015-05-11 05:26:03 main] Repeating errors encountered during run:
[ERROR 2015-05-11 05:26:03 main] sh: katello-service: command not found

[ERROR 2015-05-11 05:26:03 main] Upgrade step stop_services failed. Check logs for more information.


Version-Release number of selected component (if applicable):
upgrade from sat6.0.8 to sat6.1 GA snap3 c2

How reproducible:


Steps to Reproduce:

# set sat6.1 capsule repo and disable sat6.0.8 capsule repo
# yum update -y
# yum install capsule-installer
# cp /etc/katello-installer/answers.capsule-installer.yaml.rpmsave /etc/capsule-installer/answers.capsule-installer.yaml
# capsule-installer --upgrade


Actual results:
[ERROR 2015-05-11 05:26:03 main] sh: katello-service: command not found

Expected results:
upgrade should succeed

Additional info:

Comment 1 Stephen Benjamin 2015-05-11 09:52:34 UTC
*** Bug 1220313 has been marked as a duplicate of this bug. ***

Comment 2 Stephen Benjamin 2015-05-11 09:53:12 UTC
Upstream fix from https://github.com/Katello/katello/pull/5217

katello|95f1e0cc6e2b3ff9877ce2d3f8b2203512ab2296

Comment 6 Sachin Ghai 2015-05-13 11:39:31 UTC
This is still reproducible with snap4

[root@sat6-capsule-rhel66 ~]#  capsule-installer --upgrade
Upgrading...
Upgrade Step: stop_services...
Upgrade step stop_services failed. Check logs for more information.


==> /var/log/capsule-installer/capsule-installer.log <==
[ERROR 2015-05-13 07:33:39 main] sh: katello-service: command not found

[ERROR 2015-05-13 07:33:39 main] Upgrade step stop_services failed. Check logs for more information.
[DEBUG 2015-05-13 07:33:39 main] Exit with status code: 1 (signal was 1)
[ERROR 2015-05-13 07:33:39 main] Repeating errors encountered during run:
[ERROR 2015-05-13 07:33:39 main] sh: katello-service: command not found

installed -packages:
=====================
katello-service-2.2.0.9-1.el6_6sat.noarch
capsule-installer-2.3.9-1.el6_6sat.noarch


]# rpm -ql katello-service-2.2.0.9-1.el6_6sat.noarch
/etc/sysconfig/service-wait
/usr/sbin/service-wait
/usr/share/katello/script/service-wait
/usr/share/man/man8/katello-service.8.gz

Comment 8 Sachin Ghai 2015-05-14 06:30:23 UTC
Guys..its still failing with snap4-c2

[root@sat6-capsule-rhel66 ~]# capsule-installer --upgrade
Upgrading...
Upgrade Step: stop_services...
Upgrade step stop_services failed. Check logs for more information.


[root@sat6-capsule-rhel66 ~]# rpm -ql katello-service-2.2.0.10-1.el6_6sat.noarch
/etc/sysconfig/service-wait
/usr/bin/katello-service
/usr/sbin/service-wait
/usr/share/katello/script/service-wait
/usr/share/man/man8/katello-service.8.gz

Looks like sift link is broken.. we need katello-service under /usr/share/katello/script/ ?

[root@sat6-capsule-rhel66 ~]# ll /usr/bin/katello-service
lrwxrwxrwx. 1 root root 41 May 14 02:23 /usr/bin/katello-service -> /usr/share/katello/script/katello-service
[root@sat6-capsule-rhel66 ~]# cd /usr/share/katello/script/
[root@sat6-capsule-rhel66 script]# ll
total 8
-rwxr-xr-x. 1 root root 5601 May 13 11:56 service-wait


[root@sat6-capsule-rhel66 script]# rpm -qa | grep katello-service
katello-service-2.2.0.10-1.el6_6sat.noarch
[root@sat6-capsule-rhel66 script]#

Comment 9 Jan Hutař 2015-05-19 19:59:39 UTC
*** Bug 1222910 has been marked as a duplicate of this bug. ***

Comment 10 Jan Hutař 2015-05-19 20:03:32 UTC
Some info:

When you (manually, it does not seem to be in dependencies) install katello-service package, you are step further:

# yum info katello-service | grep 'From repo'
From repo   : sat6-Satellite-6.1.0-RHEL-7-20150513.0-Capsule-x86_64-dvd1.iso
# ls -al `type -p katello-service`
lrwxrwxrwx. 1 root root 41 May 19 07:10 /usr/bin/katello-service -> /usr/share/katello/script/katello-service
# yum install /usr/share/katello/script/katello-service
Loaded plugins: package_upload, product-id, subscription-manager
[...]
No package /usr/share/katello/script/katello-service available.
Error: Nothing to do

When I attempt to install katello-common package which owns the target script in Satellite repo, due to dependencies it wants to install additional 406 dependent packages with it.

I have workarounded this by creating mine own dumb katello-service script (based on bug 1195618 comment #17):

# cat /usr/share/katello/script/katello-service
#!/bin/bash
if [ "$1" == 'start' ]; then
  for app in pulp_workers pulp_celerybeat pulp_resource_manager; do service $app start; done
elif [ "$1" == 'stop' ]; then
  for app in pulp_workers pulp_celerybeat pulp_resource_manager; do service $app stop; done
elif [ "$1" == 'restart' ]; then
  for app in pulp_workers pulp_celerybeat pulp_resource_manager; do service $app stop; done
  for app in pulp_workers pulp_celerybeat pulp_resource_manager; do service $app start; done
elif [ "$1" == 'status' ]; then
  return 0
fi

Comment 13 Sachin Ghai 2015-06-04 08:36:55 UTC
Verified with upgrade from snap7.

Now capsule-upgrade succeed and I didn;t get the stop services issue with:

#capsule-installer --upgrade --certs-tar <tar loc>

Please note that now user needs to re-run `capsule-certs-generate` for your capsule on the Satellite, and copy the new tarball over, and then specify `capsule-installer --upgrade --certs-tar <path to tar>`

Comment 14 Sachin Ghai 2015-06-04 08:37:27 UTC
Created attachment 1034609 [details]
capsule_upgrade_success

Comment 15 Bryan Kearney 2015-08-11 13:24:57 UTC
This bug is slated to be released with Satellite 6.1.

Comment 16 Bryan Kearney 2015-08-12 13:59:43 UTC
This bug was fixed in version 6.1.1 of Satellite which was released on 12 August, 2015.