Bug 1130225
| Summary: | Can't log in to Satellite 6 on RHEL 7 after reboot | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Product: | Red Hat Satellite | Reporter: | Jason Montleon <jmontleo> | ||||||
| Component: | Other | Assignee: | Jason Montleon <jmontleo> | ||||||
| Status: | CLOSED CURRENTRELEASE | QA Contact: | Og Maciel <omaciel> | ||||||
| Severity: | high | Docs Contact: | |||||||
| Priority: | high | ||||||||
| Version: | 6.0.4 | CC: | bbuckingham, bkearney, omaciel, sthirugn | ||||||
| Target Milestone: | Unspecified | ||||||||
| Target Release: | Unused | ||||||||
| Hardware: | x86_64 | ||||||||
| OS: | Linux | ||||||||
| Whiteboard: | |||||||||
| Fixed In Version: | Doc Type: | Bug Fix | |||||||
| Doc Text: | Story Points: | --- | |||||||
| Clone Of: | Environment: | ||||||||
| Last Closed: | 2014-09-11 12:28:44 UTC | Type: | Bug | ||||||
| Regression: | --- | Mount Type: | --- | ||||||
| Documentation: | --- | CRM: | |||||||
| Verified Versions: | Category: | --- | |||||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||
| Embargoed: | |||||||||
| Attachments: |
|
||||||||
Created attachment 926851 [details]
logs
I turned on debugging and got this stack trace:
Operation FAILED: Connection refused - connect(2)
/opt/rh/ruby193/root/usr/share/ruby/net/http.rb:763:in `initialize'
/opt/rh/ruby193/root/usr/share/ruby/net/http.rb:763:in `open'
/opt/rh/ruby193/root/usr/share/ruby/net/http.rb:763:in `block in connect'
/opt/rh/ruby193/root/usr/share/ruby/timeout.rb:55:in `timeout'
/opt/rh/ruby193/root/usr/share/ruby/timeout.rb:100:in `timeout'
/opt/rh/ruby193/root/usr/share/ruby/net/http.rb:763:in `connect'
/opt/rh/ruby193/root/usr/share/ruby/net/http.rb:756:in `do_start'
/opt/rh/ruby193/root/usr/share/ruby/net/http.rb:745:in `start'
/opt/rh/ruby193/root/usr/share/gems/gems/rbovirt-0.0.29/lib/restclient_ext/request.rb:44:in `transmit'
/opt/rh/ruby193/root/usr/share/gems/gems/rest-client-1.6.7/lib/restclient/request.rb:64:in `execute'
/opt/rh/ruby193/root/usr/share/gems/gems/rest-client-1.6.7/lib/restclient/request.rb:33:in `execute'
/opt/rh/ruby193/root/usr/share/gems/gems/rest-client-1.6.7/lib/restclient.rb:72:in `post'
/opt/rh/ruby193/root/usr/share/gems/gems/tire-0.6.2/lib/tire/http/client.rb:19:in `post'
/opt/rh/ruby193/root/usr/share/gems/gems/tire-0.6.2/lib/tire/index.rb:146:in `store'
/opt/rh/ruby193/root/usr/share/gems/gems/tire-0.6.2/lib/tire/model/search.rb:148:in `block in update_index'
/opt/rh/ruby193/root/usr/share/gems/gems/activesupport-3.2.8/lib/active_support/callbacks.rb:403:in `_run__252718943274004389__update_elasticsearch_index__1536697697987125702__callbacks'
/opt/rh/ruby193/root/usr/share/gems/gems/activesupport-3.2.8/lib/active_support/callbacks.rb:405:in `__run_callback'
/opt/rh/ruby193/root/usr/share/gems/gems/activesupport-3.2.8/lib/active_support/callbacks.rb:385:in `_run_update_elasticsearch_index_callbacks'
/opt/rh/ruby193/root/usr/share/gems/gems/activesupport-3.2.8/lib/active_support/callbacks.rb:81:in `run_callbacks'
/opt/rh/ruby193/root/usr/share/gems/gems/tire-0.6.2/lib/tire/model/search.rb:144:in `update_index'
/opt/rh/ruby193/root/usr/share/gems/gems/tire-0.6.2/lib/tire/model/callbacks.rb:21:in `block in included'
/opt/rh/ruby193/root/usr/share/gems/gems/activesupport-3.2.8/lib/active_support/callbacks.rb:468:in `_run__252718943274004389__save__1536697697987125702__callbacks'
/opt/rh/ruby193/root/usr/share/gems/gems/activesupport-3.2.8/lib/active_support/callbacks.rb:405:in `__run_callback'
/opt/rh/ruby193/root/usr/share/gems/gems/activesupport-3.2.8/lib/active_support/callbacks.rb:385:in `_run_save_callbacks'
/opt/rh/ruby193/root/usr/share/gems/gems/activesupport-3.2.8/lib/active_support/callbacks.rb:81:in `run_callbacks'
/opt/rh/ruby193/root/usr/share/gems/gems/activerecord-3.2.8/lib/active_record/callbacks.rb:264:in `create_or_update'
/opt/rh/ruby193/root/usr/share/gems/gems/activerecord-3.2.8/lib/active_record/persistence.rb:84:in `save'
/opt/rh/ruby193/root/usr/share/gems/gems/activerecord-3.2.8/lib/active_record/validations.rb:50:in `save'
/opt/rh/ruby193/root/usr/share/gems/gems/activerecord-3.2.8/lib/active_record/attribute_methods/dirty.rb:22:in `save'
/opt/rh/ruby193/root/usr/share/gems/gems/activerecord-3.2.8/lib/active_record/transactions.rb:241:in `block (2 levels) in save'
/opt/rh/ruby193/root/usr/share/gems/gems/activerecord-3.2.8/lib/active_record/transactions.rb:295:in `block in with_transaction_returning_status'
/opt/rh/ruby193/root/usr/share/gems/gems/activerecord-3.2.8/lib/active_record/connection_adapters/abstract/database_statements.rb:192:in `transaction'
/opt/rh/ruby193/root/usr/share/gems/gems/activerecord-3.2.8/lib/active_record/transactions.rb:208:in `transaction'
/opt/rh/ruby193/root/usr/share/gems/gems/activerecord-3.2.8/lib/active_record/transactions.rb:293:in `with_transaction_returning_status'
/opt/rh/ruby193/root/usr/share/gems/gems/activerecord-3.2.8/lib/active_record/transactions.rb:241:in `block in save'
/opt/rh/ruby193/root/usr/share/gems/gems/activerecord-3.2.8/lib/active_record/transactions.rb:252:in `rollback_active_record_state!'
/opt/rh/ruby193/root/usr/share/gems/gems/activerecord-3.2.8/lib/active_record/transactions.rb:240:in `save'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-1.5.0/app/lib/katello/lazy_accessor.rb:96:in `save'
/opt/rh/ruby193/root/usr/share/gems/gems/foreman-tasks-0.6.6/app/models/foreman_tasks/concerns/action_triggering.rb:26:in `block in save'
/opt/rh/ruby193/root/usr/share/gems/gems/foreman-tasks-0.6.6/app/models/foreman_tasks/concerns/action_triggering.rb:106:in `dynflow_task_wrap'
/opt/rh/ruby193/root/usr/share/gems/gems/foreman-tasks-0.6.6/app/models/foreman_tasks/concerns/action_triggering.rb:26:in `save'
/opt/rh/ruby193/root/usr/share/gems/gems/activerecord-3.2.8/lib/active_record/persistence.rb:184:in `update_attribute'
/usr/share/foreman/app/models/user.rb:209:in `block in post_successful_login'
/usr/share/foreman/app/models/concerns/foreman/thread_session.rb:73:in `as'
/usr/share/foreman/app/models/concerns/foreman/thread_session.rb:79:in `as_anonymous_admin'
/usr/share/foreman/app/models/user.rb:208:in `post_successful_login'
/usr/share/foreman/app/models/user.rb:199:in `try_to_login'
/usr/share/foreman/app/controllers/users_controller.rb:72:in `login'
It looks like elasticsearch is down, and I can't restart it using the init script/systemd - it exits immediately. After some debugging, it seems /var/run/elasticsearch isn't present - the package hasn't been updated properly for EL7 to use tmpfiles.
[root@satellite1 ~]# runuser -s /bin/bash elasticsearch -c 'ulimit -S -c 0 >/dev/null 2>&1 ; ES_HOME=/usr/share/java/elasticsearch ES_INCLUDE=/usr/share/java/elasticsearch/bin/elasticsearch.in.sh ES_MIN_MEM=256m ES_MAX_MEM=256m /usr/share/java/elasticsearch/bin/elasticsearch -p /var/run/elasticsearch/elasticsearch.pid -Des.config=/etc/elasticsearch/elasticsearch.yml -Des.path.conf=/etc/elasticsearch -Des.path.home=/usr/share/java/elasticsearch -Des.path.logs=/var/log/elasticsearch -Des.path.data=/var/lib/elasticsearch -Des.path.work=/tmp/elasticsearch -Des.path.plugins=/usr/share/java/elasticsearch/plugins'
[root@satellite1 ~]# {0.90.10}: pid Failed ...
- FileNotFoundException[/var/run/elasticsearch/elasticsearch.pid (No such file or directory)]
Something like this is needed in /usr/lib/tmpfiles.d/elasticsearch.conf:
d /run/elasticsearch 0755 elasticsearch elasticsearch -
(I can't tell what the upstream is for the package to check if one already exists.)
Verified: Browser: ===== * Firefox 31.0 (MacOS) Build: ==== * Satellite/Satellite-6.0.4-RHEL-7-20140829.0 Packages: ====== * candlepin-0.9.23-1.el7.noarch * candlepin-common-1.0.1-1.el7.noarch * candlepin-guice-3.0-2_redhat_1.el7.noarch * candlepin-scl-1-5.el7.noarch * candlepin-scl-quartz-2.1.5-6.el7.noarch * candlepin-scl-rhino-1.7R3-3.el7.noarch * candlepin-scl-runtime-1-5.el7.noarch * candlepin-selinux-0.9.23-1.el7.noarch * candlepin-tomcat-0.9.23-1.el7.noarch * elasticsearch-0.90.10-6.el7sat.noarch * foreman-1.6.0.42-1.el7sat.noarch * foreman-compute-1.6.0.42-1.el7sat.noarch * foreman-gce-1.6.0.42-1.el7sat.noarch * foreman-libvirt-1.6.0.42-1.el7sat.noarch * foreman-ovirt-1.6.0.42-1.el7sat.noarch * foreman-postgresql-1.6.0.42-1.el7sat.noarch * foreman-proxy-1.6.0.30-1.el7sat.noarch * foreman-selinux-1.6.0.14-1.el7sat.noarch * foreman-vmware-1.6.0.42-1.el7sat.noarch * katello-1.5.0-30.el7sat.noarch * katello-certs-tools-1.5.6-1.el7sat.noarch * katello-default-ca-1.0-1.noarch * katello-installer-0.0.64-1.el7sat.noarch * katello-server-ca-1.0-1.noarch * openldap-2.4.39-3.el7.x86_64 * pulp-katello-0.3-4.el7sat.noarch * pulp-nodes-common-2.4.1-0.5.rc1.el7sat.noarch * pulp-nodes-parent-2.4.1-0.5.rc1.el7sat.noarch * pulp-puppet-plugins-2.4.1-0.5.rc1.el7sat.noarch * pulp-puppet-tools-2.4.1-0.5.rc1.el7sat.noarch * pulp-rpm-plugins-2.4.1-0.6.beta.el7sat.noarch * pulp-selinux-2.4.1-0.5.rc1.el7sat.noarch * pulp-server-2.4.1-0.5.rc1.el7sat.noarch * python-ldap-2.4.6-6.el7.x86_64 * ruby193-rubygem-net-ldap-0.3.1-3.el7sat.noarch * ruby193-rubygem-runcible-1.1.0-2.el7sat.noarch * rubygem-hammer_cli-0.1.1-12.el7sat.noarch * rubygem-hammer_cli_foreman-0.1.1-16.el7sat.noarch * rubygem-hammer_cli_foreman_tasks-0.0.3-3.el7sat.noarch * rubygem-hammer_cli_import-0.10.2-1.2.el7sat.noarch * rubygem-hammer_cli_katello-0.0.4-14.el7sat.noarch This was delivered with Satellite 6.0 which was released on 10 September 2014. |
Created attachment 926850 [details] foreman-debug Description of problem: It's possible to log into Satellite 7 until you reboot. After that you get a connection refused error upon logging in Version-Release number of selected component (if applicable): candlepin-0.9.23-1.el7.noarch pulp-server-2.4.0-0.30.beta.el7sat.noarch foreman-1.6.0.40-1.el7sat.noarch katello-1.5.0-28.el7sat.noarch ruby193-rubygem-katello-1.5.0-82.el7sat.noarch How reproducible: Seems always Steps to Reproduce: 1. Install Satellite 6.0.4 on RHEL 7 2. Reboot 3. Log in Actual results: Error Connection refused - connect(2) Errno::ECONNREFUSED Connection refused - connect(2) app/models/user.rb:209:in `block in post_successful_login' app/models/concerns/foreman/thread_session.rb:73:in `as' app/models/concerns/foreman/thread_session.rb:79:in `as_anonymous_admin' app/models/user.rb:208:in `post_successful_login' app/models/user.rb:199:in `try_to_login' app/controllers/users_controller.rb:72:in `login' app/models/concerns/foreman/thread_session.rb:33:in `clear_thread' lib/middleware/catch_json_parse_errors.rb:9:in `call' Expected results: Should be able to log in without an error. Additional info: