Bug 105710
| Summary: | sshd not accepting connections | ||||||
|---|---|---|---|---|---|---|---|
| Product: | [Retired] Red Hat Linux | Reporter: | Ted Kaczmarek <tedkaz> | ||||
| Component: | openssh | Assignee: | Tomas Mraz <tmraz> | ||||
| Status: | CLOSED WORKSFORME | QA Contact: | Brian Brock <bbrock> | ||||
| Severity: | high | Docs Contact: | |||||
| Priority: | high | ||||||
| Version: | 9 | CC: | mark.l.huang | ||||
| Target Milestone: | --- | ||||||
| Target Release: | --- | ||||||
| Hardware: | All | ||||||
| OS: | Linux | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | Doc Type: | Bug Fix | |||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | Environment: | ||||||
| Last Closed: | 2005-02-05 16:39:41 UTC | Type: | --- | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Attachments: |
|
||||||
Reviewing my monitoring logs shows that is goes down between every 30-40 minutes for between 30-60 minutes, OK, every ssh daemon running 3.5X is affected, not all the same way. Any sshd daemon that is 3.4X is not experiencing this issue. I have some running on port 2022 in debug mode, but I suspect these will not have an issue. Created attachment 94833 [details]
Debug of sshd hosing
Why would not receiving an identification string cause the daemon to hose?
One would suspect it would just reject the connection. Anyway I have a few
workaround options but am amazed that this was let loose. Talk about a simple
way to DOS someone.
Ted
it seems that opennms launches on ssh poller invocation abackground-task.If so , the check if I/O-channels STDIN,STDOUT and STDERR are linked toa local file/device.e.g. ssh ares4 '/opt/opennms/doit_as_backgroundtask.sh </dev/null+>/var/log/doit_as_backgroundtask.log 2>&1'If one of these I/O channels are bound to the console then the sshd will not terminate correctly. That's the reason I reach the redhat defaultlimit of maximum 10 sshd processes. raising this limit to <30 helps. Forthe archive. I'm also unable to connect to one server using openssh-server-3.5p1-11. This has now happened for the second time within one month. Only a restart of the daemon fixes this: # ssh -vvv web3 OpenSSH_3.5p1, SSH protocols 1.5/2.0, OpenSSL 0x0090701f debug1: Reading configuration data /etc/ssh/ssh_config debug1: Applying options for * debug1: Rhosts Authentication disabled, originating port will not be trusted. debug1: ssh_connect: needpriv 0 debug1: Connecting to web3 [172.16.1.3] port 22. debug1: Connection established. debug1: identity file /root/.ssh/identity type -1 debug3: Not a RSA1 key file /root/.ssh/id_rsa. debug2: key_type_from_name: unknown key type '-----BEGIN' debug3: key_read: no key found debug3: key_read: no space ... debug3: key_read: no space debug2: key_type_from_name: unknown key type '-----END' debug3: key_read: no key found debug1: identity file /root/.ssh/id_rsa type 1 debug1: identity file /root/.ssh/id_dsa type -1 ssh_exchange_identification: Connection closed by remote host debug1: Calling cleanup 0x80674e0(0x0) How may sshd's do you see running when this happens? You will need console access or enable another daemon that you can remotely access host with. You may want to increase the amount of sshd's allowed to 30, that so far has prevented my problem, it will get up to about 24-26 typically in my setup. Does it still happen for you with the current Fedora Core? Wouldn't be able to tell you for sure, the OpenNMS guys accidentally created the DOS that did this with one version of their ssh poller. I haven't seen any problems with this since I upgraded the poller code. That was quite some time ago. I would say close it at this time. It's possible that it could be a problem of syslog deadlock which was reported in another bug report. Closing for now. I believe that the root cause of both this bug as well as <a href="https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=144799">144799</a> is a bug in OpenSSH 3.6.x and 3.7.x that only shows up under privilege separation: <a href="http://marc.theaimsgroup.com/?l=openssh-unix-dev&m=107520317020444&w=2">http://marc.theaimsgroup.com/?l=openssh-unix-dev&m=107520317020444&w=2</a> Basically, the MaxStartups (10) and LoginGraceTime (120 seconds) settings are supposed to defend against DoS, but when run in privilege-separated mode (the default), OpenSSH server does not properly kill its children after LoginGraceTime expires. Thus, sshd processes that are still in authentication phase after LoginGraceTime expires can hang around forever, and you can DoS an FC2 OpenSSH server with default settings by simply opening up 10 connections to the server, specifying an SSH key with a passphrase, and never typing the passphrase. We recently upgraded all 500 of our <a href="http://www.planet-lab.org/">PlanetLab</a> nodes, which run a modified FC2, to the latest FC3 version of OpenSSH server (3.9ish), which seems to have fixed the problem. The latest FC3 version of OpenSSH server does have some annoying New Requirements ("locked" accounts (those with "!!" passwords in /etc/shadow) must be "unlocked" ("!!" replaced with "*"), the server refuses to start unless argv[0] is an absolute path (i.e. you cannot rename the process)), but works. |
From Bugzilla Helper: User-Agent: Mozilla/5.0 (X11; U; Linux i686) Gecko/20030701 Galeon/1.3.7 Description of problem: Clients get ssh_exchange_identification error, restrting daemon will fix for about 30 minutes. Will also start working by itself after some undetermined period of time. Version-Release number of selected component (if applicable): openssh-server-3.5p1-11 How reproducible: Always Steps to Reproduce: 1.up2date openssh 2.ssh to up2date sshd 3. Actual Results: ssh_exchange_identification: Connection closed by remote host Expected Results: ssh connection establishes Additional info: I am able to reproduce this at will.