Bug 1984980 - User logins are super slow when system has tons of disks attached [NEEDINFO]
Summary: User logins are super slow when system has tons of disks attached
Keywords:
Status: NEW
Alias: None
Product: Red Hat Enterprise Linux 8
Classification: Red Hat
Component: systemd
Version: 8.4
Hardware: All
OS: Linux
high
high
Target Milestone: rc
: ---
Assignee: Michal Sekletar
QA Contact: Frantisek Sumsal
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-07-22 15:08 UTC by Renaud Métrich
Modified: 2023-08-06 01:10 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:
Type: Bug
Target Upstream Version:
Embargoed:
apmukher: needinfo? (msekleta)


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Knowledge Base (Solution) 5211801 0 None None None 2022-01-18 10:52:54 UTC

Internal Links: 2041815

Description Renaud Métrich 2021-07-22 15:08:14 UTC
Description of problem:

When a system has tons of disks attached (~7000), user logins are slow due to "systemd --user" instance reading all the disks.

A customer reported 11 seconds to log in:
-------- 8< ---------------- 8< ---------------- 8< ---------------- 8< --------
$ time ssh system date
Tue Jul  6 06:39:59 GMT 2021

real    0m11.493s
user    0m0.012s
sys     0m0.004s
-------- 8< ---------------- 8< ---------------- 8< ---------------- 8< --------

Executing strace, we can see that all block devices (/sys/devices/virtual/block/<device>) are opened and stat'ed, which takes a lot of time in the end.

IMO "systemd --user" needs to be optimized to either not retrieve this information or get it from "systemd --system" upon start.


Version-Release number of selected component (if applicable):

systemd-239-41.el8_3.2.x86_64


How reproducible:

Always on customer site having VxVM volumes.
Cannot reproduce locally using either iscsi or loopback devices

Comment 1 François Rigault 2023-06-25 16:39:35 UTC
> Cannot reproduce locally using either iscsi or loopback devices

there is https://github.com/systemd/systemd/issues/9518 in systemd 239 that prevents systemd --user to list every device, which makes it more difficult to reproduce on RHEL 8, but it's easy to reproduce on any other version (eg RHEL 9.2, Fedora Rawhide)


Note You need to log in before you can comment on or make changes to this bug.