Bug 1365542

Summary: Upgrades fail if a proxy is set on the system
Product: Red Hat Satellite Reporter: David Davis <daviddavis>
Component: InstallationAssignee: David Davis <daviddavis>
Status: CLOSED ERRATA QA Contact: jcallaha
Severity: urgent Docs Contact:
Priority: high    
Version: 6.2.0CC: bbuckingham, chrobert, jcallaha, ktordeur, mmccune, nshaik, pmoravec, xdmoon, zhunting
Target Milestone: UnspecifiedKeywords: Triaged
Target Release: Unused   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: katello-3.0.0-11 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-08-22 06:36:18 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description David Davis 2016-08-09 13:47:40 UTC
Description of problem:

We're seeing 6.2 systems where tomcat fails to start if $https_proxy is set.


Version-Release number of selected component (if applicable):

6.2.0


How reproducible:

100%


Steps to Reproduce:
1. On a 6.1 system, set $https_proxy
2. Perform upgrade


Actual results:

Installer fails with:

[ INFO 2016-08-03 23:18:08 main] Upgrade Step: start_tomcat...
[ERROR 2016-08-03 23:20:10 main] Redirecting to /bin/systemctl start  tomcat.service

Some services failed to start: tomcat


Expected results:

Upgrade ignores proxy and succeeds.


Additional info:

I found this commit:

https://github.com/Katello/puppet-service_wait/commit/c641a1112aea1e4ffd01a5d39aff0f4c5d32fb4f

Looks like we need to make the same change here:

https://github.com/Katello/katello-packaging/blob/b56fc78b6144ccbb7581944f705514da70ef53c5/katello/service-wait#L46

Comment 1 Partha Aji 2016-08-09 13:49:48 UTC
Created redmine issue http://projects.theforeman.org/issues/16028 from this bug

Comment 2 David Davis 2016-08-09 13:58:07 UTC
Upstream PR:

https://github.com/Katello/katello-packaging/pull/267

Comment 4 Pavel Moravec 2016-08-10 08:49:06 UTC
For user scenario "http_proxy env. variable set, running installer / upgrade fails", the fix above isn't sufficient. Installer then raises an error during foreman-rake db:seed:


[DEBUG 2016-08-09 15:46:49 main]  Executing '/usr/sbin/foreman-rake db:seed'
[ WARN 2016-08-09 15:47:19 main]  /Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]/returns: rake aborted!
[ WARN 2016-08-09 15:47:19 main]  /Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]/returns: Net::HTTPServerException: 407 "Proxy Authentication Required"
[ WARN 2016-08-09 15:47:19 main]  /Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]/returns: /opt/theforeman/tfm/root/usr/share/gems/gems/rbovirt-0.0.37/lib/restclient_ext/request.rb:44:in `transmit'
[ WARN 2016-08-09 15:47:19 main]  /Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]/returns: /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-1.6.7/lib/restclient/request.rb:64:in `execute'
[ WARN 2016-08-09 15:47:19 main]  /Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]/returns: /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-1.6.7/lib/restclient/request.rb:33:in `execute'
[ WARN 2016-08-09 15:47:19 main]  /Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]/returns: /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-1.6.7/lib/restclient/resource.rb:51:in `get'
[ WARN 2016-08-09 15:47:19 main]  /Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]/returns: /opt/theforeman/tfm/root/usr/share/gems/gems/runcible-1.7.0/lib/runcible/base.rb:78:in `get_response'
[ WARN 2016-08-09 15:47:19 main]  /Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]/returns: /opt/theforeman/tfm/root/usr/share/gems/gems/runcible-1.7.0/lib/runcible/base.rb:66:in `call'
[ WARN 2016-08-09 15:47:19 main]  /Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]/returns: /opt/theforeman/tfm/root/usr/share/gems/gems/runcible-1.7.0/lib/runcible/resources/event_notifier.rb:48:in `list'
[ WARN 2016-08-09 15:47:19 main]  /Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]/returns: /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.68/app/models/katello/glue/pulp/repo.rb:32:in `ensure_sync_notification'
[ WARN 2016-08-09 15:47:19 main]  /Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]/returns: /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.68/db/seeds.d/108-ensure_sync_notification.rb:9:in `<top (required)>'
..

The cause is, 108-ensure_sync_notification.rb makes a ruby restclient call to pulp:

client=https://satellite.example.com, path=/pulp/api/v2/events/, args=[:get, {"content_type"=>"application/json", "accept"=>"application/json"}]

that fails. No idea ATM why, if it makes the call against FQDN of Satellite with no proxy set (RestClient.proxy isnt apparently set, I expect).


Consequence: activation keys disappear in WebUI (huh? why? but this has been seen at 2 customers already).


Not sure if it is worth filing extra BZ or to fix this flaw under the same one since same basic user story is failing.

Comment 5 David Davis 2016-08-10 15:13:54 UTC
Pavel, it's probably going to require a separate fix. Moreover, I tried to reproduce the error by setting $http_proxy and $https_proxy and running foreman-rake db:seed but couldn't. I'll need to dig into it some more.

Comment 6 Bryan Kearney 2016-08-10 16:17:51 UTC
Moving to POST since upstream bug http://projects.theforeman.org/issues/16028 has been closed

Comment 7 Pavel Moravec 2016-08-11 07:52:53 UTC
(In reply to David Davis from comment #5)
> Pavel, it's probably going to require a separate fix. Moreover, I tried to
> reproduce the error by setting $http_proxy and $https_proxy and running
> foreman-rake db:seed but couldn't. I'll need to dig into it some more.

Raised as https://bugzilla.redhat.com/show_bug.cgi?id=1366139

Comment 8 jcallaha 2016-08-19 19:22:00 UTC
Verified in Satellite 6.2.1 Snap 1.3. Upgrade from Satellite 6.1.8 executed without issue.

Proxy variables were set before upgrade and remained after, but did not affect the upgrade itself.

[root@rhsm-qe-1 ~]# echo $http_proxy
120.26.142.73
[root@rhsm-qe-1 ~]# echo $https_proxy
120.26.142.73

Comment 10 errata-xmlrpc 2016-08-22 06:36:18 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2016:1643