Bug 1323726

Summary: Installing errata from satellite UI failed
Product: Red Hat Satellite Reporter: sthirugn <sthirugn>
Component: Errata ManagementAssignee: Justin Sherrill <jsherril>
Status: CLOSED ERRATA QA Contact: jcallaha
Severity: urgent Docs Contact:
Priority: unspecified    
Version: 6.2.0CC: bbuckingham, bkearney, ehelms, jcallaha, jortel, jsherril, mmccune, sauchter, sthirugn
Target Milestone: UnspecifiedKeywords: Regression, Triaged
Target Release: Unused   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: gofer-2.7.6-1 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-07-27 09:30:31 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1277292, 1315278, 1320575, 1327236    

Description sthirugn@redhat.com 2016-04-04 14:04:49 UTC
Description of problem:
Installing errata from satellite UI failed

Version-Release number of selected component (if applicable):
Satellite 6.2-beta-snap-6

How reproducible:
Always

Steps to Reproduce:
1. Register/subscribe a content host, install katello-agent
2. Attempt to install one errata from Satellite UI -> content host -> errata tab


Actual results:
the task did not complete and it eventually timed after 3600 seconds (1 hour) 

Expected results:
the errata apply should succeed

Additional info:

Comment 3 Mike McCune 2016-04-05 04:53:34 UTC
Did you check your firewall settings? I was able to install errata on a managed system after ensuring my firewall on the Satellite was either disabled or had the proper ports open.

That said, I did get an orange warning, message but the install completed:

Id: be8b2c57-9186-4db3-a67d-def9b8e40a87
Label: Actions::Katello::Host::Erratum::Install
Name: Install erratum
Owner: admin 

 Output:

openssh-clients-5.3p1-112.el6_7.x86_64
openssh-5.3p1-112.el6_7.x86_64
openssh-server-5.3p1-112.el6_7.x86_64

can you share your host info and we can diagnose further as I don't think it is fundamentally broken..

Comment 4 Mike McCune 2016-04-06 05:42:21 UTC
further diagnosis over IRC shows this was a firewall issue then coupled with a lazy sync issue.

Moving this back ON_QA and feel free to re-fail if a test with Immediate + firewall fixes still fails.

Comment 5 sthirugn@redhat.com 2016-04-07 13:18:05 UTC
This same error occurs with satellite-6.2.0-6.2.beta.el7sat.noarch



I have all the right ports open and firewalld enabled/running:

# netstat -tanp | grep LISTEN
tcp        0      0 0.0.0.0:5672            0.0.0.0:*               LISTEN      9302/qpidd          
tcp        0      0 127.0.0.1:27017         0.0.0.0:*               LISTEN      9159/mongod         
tcp        0      0 127.0.0.1:59791         0.0.0.0:*               LISTEN      30644/Passenger Rac 
tcp        0      0 0.0.0.0:5647            0.0.0.0:*               LISTEN      10730/qdrouterd     
tcp        0      0 127.0.0.1:8751          0.0.0.0:*               LISTEN      6205/python         
tcp        0      0 127.0.0.1:50611         0.0.0.0:*               LISTEN      1304/Passenger Rack 
tcp        0      0 172.17.0.1:53           0.0.0.0:*               LISTEN      2509/named          
tcp        0      0 10.16.4.18:53           0.0.0.0:*               LISTEN      2509/named          
tcp        0      0 127.0.0.1:53            0.0.0.0:*               LISTEN      2509/named          
tcp        0      0 192.168.100.1:53        0.0.0.0:*               LISTEN      31780/dnsmasq       
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      8712/sshd           
tcp        0      0 127.0.0.1:5432          0.0.0.0:*               LISTEN      9202/postgres       
tcp        0      0 127.0.0.1:25            0.0.0.0:*               LISTEN      8311/sendmail: acce 
tcp        0      0 127.0.0.1:953           0.0.0.0:*               LISTEN      2509/named          
tcp        0      0 0.0.0.0:16509           0.0.0.0:*               LISTEN      31531/libvirtd      
tcp        0      0 0.0.0.0:8000            0.0.0.0:*               LISTEN      9422/ruby           
tcp        0      0 127.0.0.1:38306         0.0.0.0:*               LISTEN      12085/Passenger Rac 
tcp        0      0 0.0.0.0:9090            0.0.0.0:*               LISTEN      9422/ruby           
tcp        0      0 0.0.0.0:5671            0.0.0.0:*               LISTEN      9302/qpidd          
tcp        0      0 0.0.0.0:7911            0.0.0.0:*               LISTEN      8341/dhcpd          
tcp6       0      0 :::5000                 :::*                    LISTEN      2409/httpd          
tcp6       0      0 :::5672                 :::*                    LISTEN      9302/qpidd          
tcp6       0      0 :::8009                 :::*                    LISTEN      9342/java           
tcp6       0      0 :::8140                 :::*                    LISTEN      2409/httpd          
tcp6       0      0 :::5646                 :::*                    LISTEN      10730/qdrouterd     
tcp6       0      0 :::80                   :::*                    LISTEN      2409/httpd          
tcp6       0      0 :::8080                 :::*                    LISTEN      9342/java           
tcp6       0      0 :::12432                :::*                    LISTEN      2089/python         
tcp6       0      0 :::12434                :::*                    LISTEN      2089/python         
tcp6       0      0 :::53                   :::*                    LISTEN      2509/named          
tcp6       0      0 :::22                   :::*                    LISTEN      8712/sshd           
tcp6       0      0 ::1:7094                :::*                    LISTEN      9158/python         
tcp6       0      0 ::1:5432                :::*                    LISTEN      9202/postgres       
tcp6       0      0 :::3128                 :::*                    LISTEN      740/(squid-1)       
tcp6       0      0 :::443                  :::*                    LISTEN      2409/httpd          
tcp6       0      0 :::8443                 :::*                    LISTEN      9342/java           
tcp6       0      0 :::16509                :::*                    LISTEN      31531/libvirtd      
tcp6       0      0 127.0.0.1:8005          :::*                    LISTEN      9342/java           
tcp6       0      0 :::2375                 :::*                    LISTEN      1234/docker         
tcp6       0      0 :::5671                 :::*                    LISTEN      9302/qpidd          


It looks like the client got the errata update, but satellite task never finished

# yum history info 7
Loaded plugins: package_upload, product-id, search-disabled-repos, subscription-manager
Transaction ID : 7
Begin time     : Thu Apr  7 09:09:17 2016
Begin rpmdb    : 651:1f11683117a0c641d288f469d2c0a2176d5618b2
End time       :            09:09:42 2016 (25 seconds)
End rpmdb      : 651:202b78e80db8adee9258702452e03c24f26ed857
User           : System <unset>
Return-Code    : Success
Transaction performed with:
    Installed     rpm-4.11.3-17.el7.x86_64                  @beaker-Server/7.2
    Installed     subscription-manager-1.15.9-15.el7.x86_64 @beaker-Server/7.2
    Installed     yum-3.4.3-132.el7.noarch                  @beaker-Server/7.2
    Installed     yum-metadata-parser-1.1.4-10.el7.x86_64   @beaker-Server/7.2
Packages Altered:
    Updated grub2-1:2.02-0.29.el7.x86_64         @beaker-Server/7.2
    Update        1:2.02-0.34.el7_2.x86_64       @rhel-7-server-rpms
    Updated grub2-tools-1:2.02-0.29.el7.x86_64   @beaker-Server/7.2
    Update              1:2.02-0.34.el7_2.x86_64 @rhel-7-server-rpms

Comment 6 Mike McCune 2016-04-08 00:20:43 UTC
your box had no pulp processes running, was being hit with:

https://bugzilla.redhat.com/show_bug.cgi?id=1321644

"Pulp celery_beat and resource_manager no longer running "


# ps -e f |grep resource_manager
 7657 pts/0    S+     0:00          \_ grep --color=auto resource_manager

should show:

# ps -ef |grep resource_manager
apache   10877     1  0 18:31 ?        00:00:02 /usr/bin/python /usr/bin/celery worker -A pulp.server.async.app -n resource_manager@%h -Q resource_manager -c 1 --events --umask 18 --pidfile=/var/run/pulp/resource_manager.pid --heartbeat-interval=30
apache   11066 10877  0 18:31 ?        00:00:01 /usr/bin/python /usr/bin/celery worker -A pulp.server.async.app -n resource_manager@%h -Q resource_manager -c 1 --events --umask 18 --pidfile=/var/run/pulp/resource_manager.pid --heartbeat-interval=30
root     22860 32383  0 20:20 pts/7    00:00:00 grep --color=auto resource_manager


I recommend upgrading to  SNAP 7 to resolve this.

Comment 7 Mike McCune 2016-04-08 00:21:06 UTC
sorry to flip this back ON_QA again but this issue is unrelated to the agent :)

Comment 8 sthirugn@redhat.com 2016-04-08 21:08:34 UTC
As per the irc conversation with Mike Mccune, this bug is valid

Comment 9 Jeff Ortel 2016-04-13 22:33:53 UTC
Fixed in gofer-proton 2.7.6 built in koji.

Comment 11 Mike McCune 2016-04-16 22:37:33 UTC
we are now at python-gofer-2.7.6

Comment 12 jcallaha 2016-04-20 19:56:43 UTC
Verified in Satellite 6.2 Beta Snap 9. While the UI shows an error, the errata is actually installed on the target host.

Comment 13 Justin Sherrill 2016-04-21 13:19:00 UTC
Since the task didn't actually succeed, moving back to assigned

Comment 14 Justin Sherrill 2016-04-21 13:52:10 UTC
Moving back to on_qa

NOTE: You may need to resync the tools repo.  Your *gofer* packages on the client should be 2.7.6-1

Comment 15 jcallaha 2016-04-21 15:38:29 UTC
Verified in Satellite 6.2 Beta Snap 9 with the following packages installed on the client.

python-gofer-proton-2.7.6-1.el7sat.noarch
gofer-2.7.6-1.el7sat.noarch
python-gofer-2.7.6-1.el7sat.noarch

Comment 17 errata-xmlrpc 2016-07-27 09:30:31 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2016:1501