Hide Forgot
Description of problem: Hi, The satellite is not a production server, but a testing instance. I executed 'katello-backup /foo' and hit CTRL+C at the moment that foreman-proxy was being shutdown. After that it is no longer possible to start the service back. katello-service restart gives exit code 1 systemcl restart foreman-proxy fails rebooting the system does not fix it katello-service stop and katello-service start does not fix it katello-installer --upgrade does not fix it Version-Release number of selected component (if applicable): Sat 6.1.8 RHEL 7.2 How reproducible: unknown, i did not try to reproduce on another satellite. Steps to Reproduce: 1. mkdir /foo 2. katello-installer /foo 3. wait 1 second 4. ctrl+c it 5. katello-service start foreman-proxy service does not start Actual results: foreman-proxy service unit permanently broken Expected results: being able to start the sat6 service back without error Additional info: # systemctl restart foreman-proxy Job for foreman-proxy.service failed because the control process exited with error code. See "systemctl status foreman-proxy.service" and "journalctl -xe" for details. # journalctl -u foreman-proxy -n 20 -- Logs begin at mié 2015-09-30 10:45:25 ART, end at vie 2016-04-15 15:23:00 ART. -- abr 15 15:07:24 sixthsat2.example.com systemd[1]: foreman-proxy.service failed. abr 15 15:22:59 sixthsat2.example.com systemd[1]: Starting Foreman Proxy... abr 15 15:23:00 sixthsat2.example.com smart-proxy[3731]: /usr/share/gems/gems/bundler-1.7.8/lib/bundler.rb:327:in `initialize': Permission denied - foo (Errno::EACCES) abr 15 15:23:00 sixthsat2.example.com smart-proxy[3731]: from /usr/share/gems/gems/bundler-1.7.8/lib/bundler.rb:327:in `open' abr 15 15:23:00 sixthsat2.example.com smart-proxy[3731]: from /usr/share/gems/gems/bundler-1.7.8/lib/bundler.rb:327:in `read_file' abr 15 15:23:00 sixthsat2.example.com smart-proxy[3731]: from /usr/share/gems/gems/bundler-1.7.8/lib/bundler/definition.rb:56:in `initialize' abr 15 15:23:00 sixthsat2.example.com smart-proxy[3731]: from /usr/share/gems/gems/bundler-1.7.8/lib/bundler/dsl.rb:152:in `new' abr 15 15:23:00 sixthsat2.example.com smart-proxy[3731]: from /usr/share/gems/gems/bundler-1.7.8/lib/bundler/dsl.rb:152:in `to_definition' abr 15 15:23:00 sixthsat2.example.com smart-proxy[3731]: from /usr/share/gems/gems/bundler-1.7.8/lib/bundler/dsl.rb:11:in `evaluate' abr 15 15:23:00 sixthsat2.example.com smart-proxy[3731]: from /usr/share/gems/gems/bundler_ext-0.3.0/lib/bundler_ext/bundler_ext.rb:14:in `parse_from_gemfile' abr 15 15:23:00 sixthsat2.example.com smart-proxy[3731]: from /usr/share/gems/gems/bundler_ext-0.3.0/lib/bundler_ext/bundler_ext.rb:35:in `system_require' abr 15 15:23:00 sixthsat2.example.com smart-proxy[3731]: from /usr/share/foreman-proxy/lib/bundler_helper.rb:22:in `require_groups' abr 15 15:23:00 sixthsat2.example.com smart-proxy[3731]: from /usr/share/foreman-proxy/lib/smart_proxy.rb:23:in `<top (required)>' abr 15 15:23:00 sixthsat2.example.com smart-proxy[3731]: from /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:55:in `require' abr 15 15:23:00 sixthsat2.example.com smart-proxy[3731]: from /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:55:in `require' abr 15 15:23:00 sixthsat2.example.com smart-proxy[3731]: from /usr/share/foreman-proxy/bin/smart-proxy:5:in `<main>' abr 15 15:23:00 sixthsat2.example.com systemd[1]: foreman-proxy.service: control process exited, code=exited status=1 abr 15 15:23:00 sixthsat2.example.com systemd[1]: Failed to start Foreman Proxy. abr 15 15:23:00 sixthsat2.example.com systemd[1]: Unit foreman-proxy.service entered failed state. abr 15 15:23:00 sixthsat2.example.com systemd[1]: foreman-proxy.service failed. # cat /usr/lib/systemd/system/foreman-proxy.service [Unit] Description=Foreman Proxy Wants=basic.target After=basic.target network.target [Service] Type=forking User=foreman-proxy ExecStart=/usr/share/foreman-proxy/bin/smart-proxy PIDFile=/run/foreman-proxy/foreman-proxy.pid [Install] WantedBy=multi-user.target * The unit does not seem to be overriden. ( systemd-delta does not show it) # cat /usr/share/foreman-proxy/bin/smart-proxy #!/usr/bin/env ruby $LOAD_PATH.unshift(*Dir[File.expand_path("../../lib", __FILE__), File.expand_path("../../modules", __FILE__)]) require 'smart_proxy' Proxy::Launcher.new.launch WORKAROUND: Ejecute '/usr/share/foreman-proxy/bin/smart-proxy' directly, it is executed without error. The capsule status becomes 'green' in the Sat6 UI. (otherwise it is in Red:Error). For systemct it will remain failed. Cheers.
Ok, i found what seems to be happening: # mkdir /foo # katello-service restart * this causes foreman-proxy unit file to fail. FIX: # rm /foo # katello-service restart * It looks like it is not related to katello-backup * By just EXISTING "/foo" will cause the Satellite 6 services to fail to start. I confirmed it by testing in another Satellite 6.1.8 instance. Could anybody reproduce this?
Moving 6.2 bugs out to sat-backlog.
Still happens on 6.2.8 (foreman-proxy-1.11.0.7-1.el7sat.noarch) Apr 03 17:05:36 sat62.egolov.example.com systemd[1]: Starting Foreman Proxy... Apr 03 17:05:36 sat62.egolov.example.com smart-proxy[23326]: /usr/share/gems/gems/bundler-1.7.8/lib/bundler.rb:327:in `read': Is a directory - foo (Errno::EISDIR) Apr 03 17:05:36 sat62.egolov.example.com smart-proxy[23326]: from /usr/share/gems/gems/bundler-1.7.8/lib/bundler.rb:327:in `block in read_file' Apr 03 17:05:36 sat62.egolov.example.com smart-proxy[23326]: from /usr/share/gems/gems/bundler-1.7.8/lib/bundler.rb:327:in `open' Apr 03 17:05:36 sat62.egolov.example.com smart-proxy[23326]: from /usr/share/gems/gems/bundler-1.7.8/lib/bundler.rb:327:in `read_file' Apr 03 17:05:36 sat62.egolov.example.com smart-proxy[23326]: from /usr/share/gems/gems/bundler-1.7.8/lib/bundler/definition.rb:56:in `initialize' Apr 03 17:05:36 sat62.egolov.example.com smart-proxy[23326]: from /usr/share/gems/gems/bundler-1.7.8/lib/bundler/dsl.rb:152:in `new' Apr 03 17:05:36 sat62.egolov.example.com smart-proxy[23326]: from /usr/share/gems/gems/bundler-1.7.8/lib/bundler/dsl.rb:152:in `to_definition' Apr 03 17:05:36 sat62.egolov.example.com smart-proxy[23326]: from /usr/share/gems/gems/bundler-1.7.8/lib/bundler/dsl.rb:11:in `evaluate' Apr 03 17:05:36 sat62.egolov.example.com smart-proxy[23326]: from /usr/share/gems/gems/bundler_ext-0.3.0/lib/bundler_ext/bundler_ext.rb:14:in `parse_from_gemfile' Apr 03 17:05:36 sat62.egolov.example.com smart-proxy[23326]: from /usr/share/gems/gems/bundler_ext-0.3.0/lib/bundler_ext/bundler_ext.rb:35:in `system_require' Apr 03 17:05:36 sat62.egolov.example.com smart-proxy[23326]: from /usr/share/foreman-proxy/lib/bundler_helper.rb:22:in `require_groups' Apr 03 17:05:36 sat62.egolov.example.com smart-proxy[23326]: from /usr/share/foreman-proxy/lib/smart_proxy_main.rb:32:in `<top (required)>' Apr 03 17:05:36 sat62.egolov.example.com smart-proxy[23326]: from /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:55:in `require' Apr 03 17:05:36 sat62.egolov.example.com smart-proxy[23326]: from /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:55:in `require' Apr 03 17:05:36 sat62.egolov.example.com smart-proxy[23326]: from /usr/share/foreman-proxy/bin/smart-proxy:5:in `<main>' Apr 03 17:05:36 sat62.egolov.example.com systemd[1]: foreman-proxy.service: control process exited, code=exited status=1 Apr 03 17:05:36 sat62.egolov.example.com systemd[1]: Failed to start Foreman Proxy. Apr 03 17:05:36 sat62.egolov.example.com systemd[1]: Unit foreman-proxy.service entered failed state. Apr 03 17:05:36 sat62.egolov.example.com systemd[1]: foreman-proxy.service failed.
This is fixed upstream in bundler_exec since https://github.com/bundlerext/bundler_ext/commit/b80372c2b9ea5f10046b62a0a0f900225a3869a8 Foreman/Katello don't have this issue, as they are using a self-built rubygem-bundler_ext-0.4.1-1.el7.noarch.rpm Can we also rebase the bundler_exec we ship in Satellite, please? The trivial patch that fixes this issue is: --- a/lib/bundler_ext/gemfile.rb +++ b/lib/bundler_ext/gemfile.rb @@ -35,7 +35,7 @@ def self.process(bundler_gemfile, env) def self.parse(gemfile, *groups) setup_env(gemfile) env = self.parse_env(groups) - gemfile = Bundler::Dsl.evaluate(gemfile,'foo',true) + gemfile = Bundler::Dsl.evaluate(gemfile, nil, true) process(gemfile, env) end end
I'm closing this a duplicate of 1438501, as both issues have the same root cause and will have the same resolution in updating the bundler_ext package *** This bug has been marked as a duplicate of bug 1438501 ***