Bug 890376

Summary: Use KillMode=process for the sshd.service
Product: [Fedora] Fedora Reporter: GV <rhel>
Component: opensshAssignee: Petr Lautrbach <plautrba>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: urgent Docs Contact:
Priority: unspecified    
Version: 18CC: i, johannbg, lnykryn, mattias.ellert, metherid, mgrepl, mschmidt, msekleta, notting, plautrba, systemd-maint, tmraz, vpavlin
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-04-26 00:57:06 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description GV 2012-12-26 19:19:29 UTC
# ssh root.2.1
Last login: Wed Dec 26 20:44:25 2012 from 10.0.2.2
[root@localhost ~]# yum update
Loaded plugins: langpacks, presto, remove-with-leaves, show-leaves, verify
Resolving Dependencies
--> Running transaction check
---> Package openssh.x86_64 0:6.1p1-2.fc18 will be updated
---> Package openssh.x86_64 0:6.1p1-4.fc18 will be an update
---> Package openssh-clients.x86_64 0:6.1p1-2.fc18 will be updated
---> Package openssh-clients.x86_64 0:6.1p1-4.fc18 will be an update
---> Package openssh-server.x86_64 0:6.1p1-2.fc18 will be updated
---> Package openssh-server.x86_64 0:6.1p1-4.fc18 will be an update
--> Finished Dependency Resolution

Dependencies Resolved

=======================================================================================================================================================================================
 Package                                          Arch                                    Version                                        Repository                               Size
=======================================================================================================================================================================================
Updating:
 openssh                                          x86_64                                  6.1p1-4.fc18                                   fedora                                  294 k
 openssh-clients                                  x86_64                                  6.1p1-4.fc18                                   fedora                                  436 k
 openssh-server                                   x86_64                                  6.1p1-4.fc18                                   fedora                                  337 k

Transaction Summary
=======================================================================================================================================================================================
Upgrade  3 Packages

Total download size: 1.0 M
Is this ok [y/N]: y
Downloading Packages:
Setting up and reading Presto delta metadata
(1/3): openssh-6.1p1-4.fc18.x86_64.rpm                                                                                                                          | 294 kB  00:00:00     
(2/3): openssh-clients-6.1p1-4.fc18.x86_64.rpm                                                                                                                  | 436 kB  00:00:00     
(3/3): openssh-server-6.1p1-4.fc18.x86_64.rpm                                                                                                                   | 337 kB  00:00:00     
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                                                                  746 kB/s | 1.0 MB     00:01     
Running Transaction Check
Running Transaction Test
Transaction Test Succeeded
Running Transaction
  Updating   : openssh-6.1p1-4.fc18.x86_64                                                                                                                                         1/6 
  Updating   : openssh-server-6.1p1-4.fc18.x86_64                                                                                                                                  2/6 
  Updating   : openssh-clients-6.1p1-4.fc18.x86_64                                                                                                                                 3/6 
  Cleanup    : openssh-clients-6.1p1-2.fc18.x86_64                                                                                                                                 4/6 
  Cleanup    : openssh-server-6.1p1-2.fc18.x86_64                                                                                                                                  5/6 
Connection to 10.0.2.1 closed by remote host.
Connection to 10.0.2.1 closed.
# ssh root.2.1
Last login: Wed Dec 26 21:04:34 2012 from 10.0.2.2
[root@localhost ~]# yum-complete-transaction 
Loaded plugins: presto, remove-with-leaves
There are 1 outstanding transactions to complete. Finishing the most recent one
The remaining transaction had 1 elements left to run
--> Running transaction check
---> Package openssh.x86_64 0:6.1p1-2.fc18 will be erased
--> Processing Dependency: openssh = 6.1p1-2.fc18 for package: openssh-server-6.1p1-2.fc18.x86_64
--> Running transaction check
---> Package openssh-server.x86_64 0:6.1p1-2.fc18 will be erased
--> Finished Dependency Resolution

Dependencies Resolved


Transaction size changed - this means we are not doing the
same transaction as we were before. Aborting and disabling
this transaction.

You could try running: package-cleanup --problems
                       package-cleanup --dupes
                       rpm -Va --nofiles --nodigest

Transaction files renamed to:
  /var/lib/yum/transaction-all.2012-12-26.21:04.49.disabled
  /var/lib/yum/transaction-done.2012-12-26.21:04.49.disabled

[root@localhost ~]# yum check
Loaded plugins: langpacks, presto, remove-with-leaves, show-leaves, verify
openssh-6.1p1-4.fc18.x86_64 is a duplicate with openssh-6.1p1-2.fc18.x86_64
openssh-server-6.1p1-4.fc18.x86_64 is a duplicate with openssh-server-6.1p1-2.fc18.x86_64
Error: check all

I had 300+ packages to be updated and because of this I had to revert the snapshot of the virtual machine. That's OK since Fedora 18 is still beta and this was a test machine but this will be unacceptable after release of Fedora 18.

Please fix-it before the release of Fedora 18!

Comment 1 Tomas Mraz 2013-01-02 09:39:16 UTC
Do you have pam_systemd.so in /etc/pam.d/system-auth and /etc/pam.d/password-auth configuration files?

Comment 2 GV 2013-01-02 10:06:32 UTC
Yes, I have.

The problem was dbus. It was not started. And because of this systemd-logind was not running. Once I start the dbus service the update worked fine.

Still, I don't think is normal that openssh + yum update to behave like this if dbus/systemd-logind is not running.

Comment 3 Tomas Mraz 2013-01-02 10:33:46 UTC
We can't do much about this in openssh (well we could remove the restart of sshd in the openssh-server %post, but that would break years long practice of restarting daemons after it is updated).

So this should be made more robust in systemd I suppose.

Comment 4 GV 2013-01-02 10:40:34 UTC
No. Please don't do that. That would be madness!

And I agree, systemd should handle this.

Comment 5 Michal Schmidt 2013-01-07 14:10:43 UTC
(In reply to comment #2)
> The problem was dbus. It was not started.

The dbus package is required by systemd and dbus.service is enabled statically.
Do you know why it was not started? Was the unit masked?

(In reply to comment #3)
> We can't do much about this in openssh (well we could remove the restart of
> sshd in the openssh-server %post, but that would break years long practice
> of restarting daemons after it is updated).

One option could be to set KillMode=process in the [Service] section of sshd.service (see man systemd.kill). 

> So this should be made more robust in systemd I suppose.

It's not obvious to me how. logind is needed to move the process to the /user/$USER/$SESSION_ID cgroup in the systemd hierarchy. This way the process ceases to be a part of sshd.service. Suppose that pam_systemd detects a failure to communicate with logind. Currently it just logs the error and returns an error code to the caller. What action should it take to make it more robust? Should it move the process by itself to some fallback cgroup? I think that would be weird behaviour.

Comment 6 GV 2013-01-07 16:13:41 UTC
> The dbus package is required by systemd and dbus.service is enabled
> statically.
dbus may not run for various reasons. It could crash. Being required to run not necessarily mean it will run all the time.

> Do you know why it was not started?
Because it was masked for some test I forget about when I filled the bug. Because those days X start most of the time before every service is started I was unable to see the error message from terminal.

> It's not obvious to me how.
Well, before systemd this was not an issue. Why is this a problem now?

Comment 7 Michal Schmidt 2013-01-07 17:09:31 UTC
(In reply to comment #6)
> Well, before systemd this was not an issue. Why is this a problem now?

It's due to the way systemd stops services by default (by signalling the whole cgroup). See the man page I referenced in my previous comment.

Comment 8 Tomas Mraz 2013-01-07 17:11:54 UTC
(In reply to comment #5)
> One option could be to set KillMode=process in the [Service] section of
> sshd.service (see man systemd.kill). 

Yes, that seems like an obvious workaround which should be applied. I'm moving this bug back to openssh.

Comment 9 Paul P Komkoff Jr 2013-02-25 15:11:09 UTC
There's a scenario for this to happen not just when dbus is dead.
You just need to use non-default DefaultControllers= in systemd/system.conf
The default value is cpu, but I'm using cpu memory blkio
Then, for the default values in logind.conf (ResetControllers=cpu, Controllers=) we have logind moving children to / in cpu:, but leaving them in sshd.service in memory: and blkio:

But I will probably agree with you when you'll tell me that such setup is wrong, and [Reset]Controllers in logind.conf shall match [Default]Controllers in systemd/sshd.service.

Comment 10 Michal Schmidt 2013-02-27 16:31:57 UTC
(In reply to comment #9)
> There's a scenario for this to happen not just when dbus is dead.
> You just need to use non-default DefaultControllers= in systemd/system.conf

I don't know why this influences what gets killed. Looks like a systemd bug.

Comment 11 Fedora Update System 2013-04-16 18:15:50 UTC
openssh-6.2p1-3.fc19 has been submitted as an update for Fedora 19.
https://admin.fedoraproject.org/updates/openssh-6.2p1-3.fc19

Comment 12 Fedora Update System 2013-04-17 16:19:24 UTC
openssh-6.1p1-7.fc18 has been submitted as an update for Fedora 18.
https://admin.fedoraproject.org/updates/openssh-6.1p1-7.fc18

Comment 13 Fedora Update System 2013-04-18 02:28:25 UTC
Package openssh-6.1p1-7.fc18:
* should fix your issue,
* was pushed to the Fedora 18 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing openssh-6.1p1-7.fc18'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2013-5918/openssh-6.1p1-7.fc18
then log in and leave karma (feedback).

Comment 14 Fedora Update System 2013-04-23 03:39:45 UTC
openssh-6.2p1-4.fc19 has been pushed to the Fedora 19 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 15 Fedora Update System 2013-04-26 00:57:09 UTC
openssh-6.1p1-8.fc18 has been pushed to the Fedora 18 stable repository.  If problems still persist, please make note of it in this bug report.