Bug 1842281
| Summary: | sshd[5434]: segfault at 258 ip 0000562ae1527349 sp 00007ffc250e46a0 error 4 in sshd[562ae14d5000+85000] | ||
|---|---|---|---|
| Product: | [Fedora] Fedora | Reporter: | Steven Haigh <netwiz> |
| Component: | openssh | Assignee: | Jakub Jelen <jjelen> |
| Status: | CLOSED ERRATA | QA Contact: | Fedora Extras Quality Assurance <extras-qa> |
| Severity: | unspecified | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 32 | CC: | crypto-team, dwalsh, jfch, jjelen, lkundrak, mattias.ellert, netwiz, plautrba, tmraz |
| Target Milestone: | --- | ||
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | openssh-8.3p1-2.fc32 | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2020-06-03 01:49:48 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
Looking at the files in /var/lib/systemd/coredump, it seems that this happens every 5 minutes - so probably being caused by the nagios check_ssh command. -rw-r----- 1 root root 452509 Jun 1 00:24 core.sshd.0.e067d5b1071f40c8bbc6dfdf9f146b99.3851.1590935063000000000000.lz4 -rw-r----- 1 root root 452710 Jun 1 00:34 core.sshd.0.e067d5b1071f40c8bbc6dfdf9f146b99.5434.1590935664000000000000.lz4 -rw-r----- 1 root root 451756 Jun 1 00:39 core.sshd.0.e067d5b1071f40c8bbc6dfdf9f146b99.5656.1590935963000000000000.lz4 -rw-r----- 1 root root 451894 Jun 1 00:44 core.sshd.0.e067d5b1071f40c8bbc6dfdf9f146b99.6384.1590936264000000000000.lz4 -rw-r----- 1 root root 452283 Jun 1 00:49 core.sshd.0.e067d5b1071f40c8bbc6dfdf9f146b99.6734.1590936563000000000000.lz4 -rw-r----- 1 root root 451923 Jun 1 00:54 core.sshd.0.e067d5b1071f40c8bbc6dfdf9f146b99.7371.1590936864000000000000.lz4 Nothing in check_ssh should be able to cause the sshd server to core dump! Can you send the coredump or install a debuginfo packages to get more usable backtraces? Sent a link to download the coredump direct - as not sure if it'd contain anything sensitive. The full backtrace (not containing anything sensitive):
(gdb) bt
#0 cipher_free (cc=0x55f2726b0d00) at cipher.c:410
#1 0x000055f27162b169 in ssh_packet_close_internal (ssh=0x55f2726b0350, do_close=1) at packet.c:649
#2 0x000055f2715da875 in main (ac=<optimized out>, av=<optimized out>) at sshd.c:2391
(gdb) l
405 void
406 cipher_free(struct sshcipher_ctx *cc)
407 {
408 if (cc == NULL || cc->cipher == NULL)
409 return;
410 if ((cc->cipher->flags & CFLAG_CHACHAPOLY) != 0) {
411 chachapoly_free(cc->cp_ctx);
412 cc->cp_ctx = NULL;
413 } else if ((cc->cipher->flags & CFLAG_AESCTR) != 0)
414 explicit_bzero(&cc->ac_ctx, sizeof(cc->ac_ctx));
The crash happens because of the memory in cc->cipher is no longer valid
(gdb) p *cc->cipher
Cannot access memory at address 0x260
It looks like it is already freed by previous cleanup, which is causing the crash. Can you check with the following scratch build if it will solve the issue:
https://koji.fedoraproject.org/koji/taskinfo?taskID=45277283
It is just the master with the following commit:
https://src.fedoraproject.org/fork/jjelen/rpms/openssh/c/7f87bd9cc976245cfea7a180dfac5964a9f313d3?branch=master
Have installed, restarted sshd - now just waiting for a while to see if it triggers a core dump still.... Ok, good news - no coredump in the time since my last comment. I'd say that means we're good to go. Thanks for testing. I will submit update soon. FEDORA-2020-3db2c9e406 has been submitted as an update to Fedora 32. https://bodhi.fedoraproject.org/updates/FEDORA-2020-3db2c9e406 FEDORA-2020-3db2c9e406 has been pushed to the Fedora 32 testing repository. In short time you'll be able to install the update with the following command: `sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2020-3db2c9e406` You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-3db2c9e406 See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates. FEDORA-2020-3db2c9e406 has been pushed to the Fedora 32 stable repository. If problem still persists, please make note of it in this bug report. |
It seems that ssh keeps segfaulting on my system. I removed a few hundred coredumps from /var/lib/systemd/coredump/ Crash message shows: kernel: sshd[5434]: segfault at 258 ip 0000562ae1527349 sp 00007ffc250e46a0 error 4 in sshd[562ae14d5000+85000] kernel: Code: c3 48 8b 7f 10 e9 b7 79 01 00 b8 fd ff ff ff c3 90 f3 0f 1e fa 48 85 ff 74 77 55 48 8b 87 20 01 00 00 48 89 fd 48 85 c0 74 47 <8b> 40 18 a8 02 75 28 a8 04 75 44 48 8b 7d 08 e8 53 14 fb ff 48 c7 systemd-coredump[5460]: Process 5434 (sshd) of user 0 dumped core. Stack trace of thread 5434: #0 0x0000562ae1527349 cipher_free (sshd + 0x5e349) #1 0x0000562ae152c169 ssh_packet_close_internal (sshd + 0x63169) #2 0x0000562ae14db875 main (sshd + 0x12875) #3 0x00007f04f2f02042 __libc_start_main (libc.so.6 + 0x27042) #4 0x0000562ae14dc07e _start (sshd + 0x1307e) Versions: # rpm -qa | grep openssh | sort openssh-8.3p1-1.fc32.x86_64 openssh-clients-8.3p1-1.fc32.x86_64 openssh-server-8.3p1-1.fc32.x86_64