Bug 1751967
| Summary: | Candlepin crash after updating java to latest | ||||||
|---|---|---|---|---|---|---|---|
| Product: | [Community] Candlepin | Reporter: | Birkir Freyr <birkir.freyr.hjartarson> | ||||
| Component: | candlepin | Assignee: | candlepin-bugs | ||||
| Status: | CLOSED CURRENTRELEASE | QA Contact: | |||||
| Severity: | unspecified | Docs Contact: | |||||
| Priority: | unspecified | ||||||
| Version: | 2.6 | CC: | baptiste.agasse, bcourt, csnyder, ekohlvan, jkrajice, kkohli, nmoumoul, redakkan, skallesh | ||||
| Target Milestone: | --- | ||||||
| Target Release: | --- | ||||||
| Hardware: | x86_64 | ||||||
| OS: | Linux | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | candlepin-2.6.8 | Doc Type: | If docs needed, set a value | ||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | Environment: | ||||||
| Last Closed: | 2019-09-20 16:14:14 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Attachments: |
|
||||||
|
Description
Birkir Freyr
2019-09-13 10:05:25 UTC
Are there any logs you can provide so that we can get a bit more detail on what candlepin was doing leading up to the crash? The following would be helpful! Thank you! /var/log/tomcat/catalina*.log /var/log/candlepin/candlepin.log Created attachment 1616448 [details] sosreport It looks like we can reproduce this in our CI. I'm attaching the sosreport from the run at https://ci.centos.org/job/foreman-katello-nightly-test/568/ The last line from candlepin.log is: 2019-09-18 16:48:06,734 [thread=http-bio-8443-exec-2] [req=4a7ae4e4-097c-4528-941e-77cbcf7933fe, org=, csid=] INFO org.candlepin.common.filter.LoggingFilter - Request: verb=POST, uri=/candlepin/owners/Default_Organization/uebercert In the Foreman installer we can also see this is where it dies. During database seeding it disconnects: [ WARN 2019-09-18T16:48:08 verbose] /Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]/returns: ForemanTasks::TaskError: Task 027ebb30-7f35-49f0-9cc5-4cbfaeb8326a: RestClient::ServerBrokeConnection: Katello::Resources::Candlepin::Owner: Server broke connection (POST /candlepin/owners/Default_Organization/uebercert) In /var/log/messages: Sep 18 16:47:59 pipeline-katello-server-nightly-centos7 server: INFO: Server startup in 37227 ms Sep 18 16:48:08 pipeline-katello-server-nightly-centos7 server: # Sep 18 16:48:08 pipeline-katello-server-nightly-centos7 server: # A fatal error has been detected by the Java Runtime Environment: Sep 18 16:48:08 pipeline-katello-server-nightly-centos7 server: # Sep 18 16:48:08 pipeline-katello-server-nightly-centos7 server: # SIGSEGV (0xb) at pc=0x00007f20f98cda8f, pid=2334, tid=0x00007f2115633700 Sep 18 16:48:08 pipeline-katello-server-nightly-centos7 server: # Sep 18 16:48:08 pipeline-katello-server-nightly-centos7 server: # JRE version: OpenJDK Runtime Environment (8.0_222-b10) (build 1.8.0_222-b10) Sep 18 16:48:08 pipeline-katello-server-nightly-centos7 server: # Java VM: OpenJDK 64-Bit Server VM (25.222-b10 mixed mode linux-amd64 compressed oops) Sep 18 16:48:08 pipeline-katello-server-nightly-centos7 server: # Problematic frame: Sep 18 16:48:08 pipeline-katello-server-nightly-centos7 server: # C [libplds4.so+0x1a8f] PL_HashTableLookupConst+0xf Sep 18 16:48:08 pipeline-katello-server-nightly-centos7 server: # Sep 18 16:48:08 pipeline-katello-server-nightly-centos7 server: # Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again Sep 18 16:48:08 pipeline-katello-server-nightly-centos7 server: # Sep 18 16:48:08 pipeline-katello-server-nightly-centos7 server: # An error report file with more information is saved as: Sep 18 16:48:08 pipeline-katello-server-nightly-centos7 server: # /usr/share/tomcat/hs_err_pid2334.log Sep 18 16:48:08 pipeline-katello-server-nightly-centos7 server: # Sep 18 16:48:08 pipeline-katello-server-nightly-centos7 server: # If you would like to submit a bug report, please visit: Sep 18 16:48:08 pipeline-katello-server-nightly-centos7 server: # http://bugreport.java.com/bugreport/crash.jsp Sep 18 16:48:08 pipeline-katello-server-nightly-centos7 server: # The crash happened outside the Java Virtual Machine in native code. Sep 18 16:48:08 pipeline-katello-server-nightly-centos7 server: # See problematic frame for where to report the bug. Sep 18 16:48:08 pipeline-katello-server-nightly-centos7 server: # Sep 18 16:48:08 pipeline-katello-server-nightly-centos7 systemd: tomcat.service: main process exited, code=killed, status=6/ABRT Sep 18 16:48:08 pipeline-katello-server-nightly-centos7 systemd: Unit tomcat.service entered failed state. Sep 18 16:48:08 pipeline-katello-server-nightly-centos7 systemd: tomcat.service failed. It looks like we don't capture the hs_err_pid2334.log file because it's in /usr/share/tomcat. This looks very similar to https://bugzilla.redhat.com/show_bug.cgi?id=1724115 which was already fixed in candlepin-2.6.8. |