Bug 1862056 - usermod does not fsync() .lock file and after reset it fails to read PID, since file become empty
Summary: usermod does not fsync() .lock file and after reset it fails to read PID, sin...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: shadow-utils
Version: rawhide
Hardware: All
OS: Linux
unspecified
unspecified
Target Milestone: ---
Assignee: Iker Pedrosa
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard: sync-to-jira
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-07-30 10:13 UTC by Коренберг Марк
Modified: 2020-12-03 01:41 UTC (History)
3 users (show)

Fixed In Version: shadow-utils-4.8.1-5.fc33 shadow-utils-4.8.1-3.fc32
Clone Of:
Environment:
Last Closed: 2020-11-24 02:51:01 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Коренберг Марк 2020-07-30 10:13:56 UTC
`usermod` does not `fsync()` `passwd.lock` file.

After hard system reset (power loss) and repeating `usermod` command it fails to read PID from the file since the file becomes empty.

Error message:

usermod: cannot lock /etc/passwd; try again later.

Comment 1 Коренберг Марк 2020-07-30 10:18:32 UTC
I suggest using flock() instead

Comment 2 Ben Cotton 2020-11-03 16:44:35 UTC
This message is a reminder that Fedora 31 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora 31 on 2020-11-24.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
Fedora 'version' of '31'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 31 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 3 Коренберг Марк 2020-11-04 07:25:36 UTC
Still actual in Fedora 32. Strace:



openat(AT_FDCWD, "/etc/passwd.742053", O_WRONLY|O_CREAT|O_TRUNC, 0600) = 7
getpid()                                = 742053
write(7, "742053\0", 7)                 = 7
close(7)                                = 0
link("/etc/passwd.742053", "/etc/passwd.lock") = 0
unlink("/etc/passwd.742053")            = 0


As you can see, no fsync() / fadatasync() after writing pid. After reboot file may become empty. To test you can make empty file /etc/passwd.lock and test that usermod will fail.

Comment 4 Iker Pedrosa 2020-11-04 08:22:40 UTC
Moving the version to rawhide, since the problem still exists.

As for the solution. Flock file locking can't be used because it doesn't assure process synchronization. So, I've proposed a PR upstream to force file sync after write in lock file: https://github.com/shadow-maint/shadow/pull/293. If you'd like to test it I can create a scratch build, but please tell me which fedora version you are using.

Comment 5 Коренберг Марк 2020-11-05 07:04:41 UTC
Huge thanks.

One more thing (another bug)

According to the strace, it opens /etc/passwd with O_TRUNC and writes new contents. If power outage happened at this point, FS may reveal en empty file. In order to overcome, it MUST:

1. open temporary file in the same dir (i.e. in /etc) in secure way like mkostemp()
2. copy owner, group, permissions, possibly some attributes (selinux? extended?)
3. fill it with new data
4. fdatasync()
5. close(tmpfile)
5. move("/etc/passwd.tmp", "/etc/passwd")
6. dirfd = open("/etc", O_DIRECTIRY|O_RDONLY)
7. fdatasync(dirfd)
8. close(dirfd)

This is the only way to securely replace a file.

Comment 6 Iker Pedrosa 2020-11-05 16:21:32 UTC
Since this is another bug I'd recommend you to open a new ticket. Indeed, I'd recommend you to open that ticket upstream as it'll get more attention there than in bugzilla (unlikely issue with low impact).

Comment 7 Коренберг Марк 2020-11-05 16:35:14 UTC
> Flock file locking can't be used because it doesn't assure process synchronization.

It's not true. flock() syscall DOES assure inter-process synchronisation.

Comment 8 Iker Pedrosa 2020-11-06 08:40:52 UTC
(In reply to Коренберг Марк from comment #7)
> > Flock file locking can't be used because it doesn't assure process synchronization.
> 
> It's not true. flock() syscall DOES assure inter-process synchronisation.

Yes, sorry, I misunderstood the explanation in https://gavv.github.io/articles/file-locks/ about flock().

Comment 9 Tomas Mraz 2020-11-06 08:51:24 UTC
It is irrelevant that flock does it. We cannot change the locking mechanism unless we ensure that anything including third party apps that might be writing to /etc/passwd uses the same mechanism.

Comment 10 Iker Pedrosa 2020-11-09 08:17:41 UTC
* master
    599cc003daf833bffdc9cbe0d33dc8b3e7ec74c8 - commonio: force lock file sync

Comment 11 Коренберг Марк 2020-11-09 08:37:27 UTC
Thanks!

About file truncation: https://bugzilla.redhat.com/show_bug.cgi?id=1895831

Comment 12 Fedora Update System 2020-11-16 09:21:02 UTC
FEDORA-2020-8425d61a06 has been submitted as an update to Fedora 33. https://bodhi.fedoraproject.org/updates/FEDORA-2020-8425d61a06

Comment 13 Fedora Update System 2020-11-17 02:34:30 UTC
FEDORA-2020-8425d61a06 has been pushed to the Fedora 33 testing repository.
In short time you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2020-8425d61a06`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-8425d61a06

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 14 Fedora Update System 2020-11-23 14:44:54 UTC
FEDORA-2020-c3d81fda83 has been submitted as an update to Fedora 32. https://bodhi.fedoraproject.org/updates/FEDORA-2020-c3d81fda83

Comment 15 Fedora Update System 2020-11-24 02:20:38 UTC
FEDORA-2020-c3d81fda83 has been pushed to the Fedora 32 testing repository.
In short time you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2020-c3d81fda83`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-c3d81fda83

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 16 Fedora Update System 2020-11-24 02:51:01 UTC
FEDORA-2020-8425d61a06 has been pushed to the Fedora 33 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 17 Fedora Update System 2020-12-03 01:41:18 UTC
FEDORA-2020-c3d81fda83 has been pushed to the Fedora 32 stable repository.
If problem still persists, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.