I have pushed a downstream change for now to avoid breakage of existing software.  On the other hand, coreutils/gnulib/glibc upstream developers are not going to fix this anytime soon.  So the long term fix needs to go to lorax: it needs to mount a proc file system on /var/tmp/lorax/lorax.7xmi9p_a/installtree/proc before running mknod in the chroot.


+++ This bug was initially created as a clone of Bug #1811038 +++

Description of problem:
coreutils-8.32-1 failed the rawhide compose today.

DEBUG util.py:598:  /usr/bin/mknod: cannot set permissions of '/dev/random': Operation not supported
DEBUG util.py:598:  command output:
DEBUG util.py:598:  /usr/bin/mknod: cannot set permissions of '/dev/random': Operation not supported
DEBUG util.py:598:  2020-03-06 06:06:21,843: command returned failure (1)
DEBUG util.py:598:  command returned failure (1)
DEBUG util.py:598:  2020-03-06 06:06:21,843: template command error in runtime-postinstall.tmpl:
DEBUG util.py:598:  template command error in runtime-postinstall.tmpl:
DEBUG util.py:598:  2020-03-06 06:06:21,843:   runcmd chroot /var/tmp/lorax/lorax.7xmi9p_a/installtree /usr/bin/mknod -m 666 /dev/random c 1 8
DEBUG util.py:598:    runcmd chroot /var/tmp/lorax/lorax.7xmi9p_a/installtree /usr/bin/mknod -m 666 /dev/random c 1 8
DEBUG util.py:598:  2020-03-06 06:06:21,845:   subprocess.CalledProcessError: Command '['chroot', '/var/tmp/lorax/lorax.7xmi9p_a/installtree', '/usr/bin/mknod', '-m', '666', '/dev/random', 'c', '1', '8']' returned non-zero exit status 1.
DEBUG util.py:598:    subprocess.CalledProcessError: Command '['chroot', '/var/tmp/lorax/lorax.7xmi9p_a/installtree', '/usr/bin/mknod', '-m', '666', '/dev/random', 'c', '1', '8']' returned non-zero exit status 1.
DEBUG util.py:598:  Traceback (most recent call last):
DEBUG util.py:598:    File "/usr/sbin/lorax", line 222, in <module>
DEBUG util.py:598:      main()
DEBUG util.py:598:    File "/usr/sbin/lorax", line 204, in main
DEBUG util.py:598:      lorax.run(dnfbase, opts.product, opts.version, opts.release,
DEBUG util.py:598:    File "/usr/lib/python3.8/site-packages/pylorax/__init__.py", line 283, in run
DEBUG util.py:598:      rb.postinstall()
DEBUG util.py:598:    File "/usr/lib/python3.8/site-packages/pylorax/treebuilder.py", line 145, in postinstall
DEBUG util.py:598:      self._runner.run("runtime-postinstall.tmpl", configdir=configdir_path)
DEBUG util.py:598:    File "/usr/lib/python3.8/site-packages/pylorax/ltmpl.py", line 149, in run
DEBUG util.py:598:      self._run(commands)
DEBUG util.py:598:    File "/usr/lib/python3.8/site-packages/pylorax/ltmpl.py", line 168, in _run
DEBUG util.py:598:      f(*args)
DEBUG util.py:598:    File "/usr/lib/python3.8/site-packages/pylorax/ltmpl.py", line 528, in runcmd
DEBUG util.py:598:      stdout = runcmd_output(cmd)
DEBUG util.py:598:    File "/usr/lib/python3.8/site-packages/pylorax/executils.py", line 349, in runcmd_output
DEBUG util.py:598:      return execWithCapture(cmd[0], cmd[1:], **kwargs)
DEBUG util.py:598:    File "/usr/lib/python3.8/site-packages/pylorax/executils.py", line 249, in execWithCapture
DEBUG util.py:598:      return _run_program(argv, stdin=stdin, root=root, log_output=log_output, filter_stderr=filter_stderr,
DEBUG util.py:598:    File "/usr/lib/python3.8/site-packages/pylorax/executils.py", line 203, in _run_program
DEBUG util.py:598:      raise subprocess.CalledProcessError(proc.returncode, argv, output)
DEBUG util.py:598:  subprocess.CalledProcessError: Command '['chroot', '/var/tmp/lorax/lorax.7xmi9p_a/installtree', '/usr/bin/mknod', '-m', '666', '/dev/random', 'c', '1', '8']' returned non-zero exit status 1.

koji rawhide failure task link : https://koji.fedoraproject.org/koji/taskinfo?taskID=42239572



Version-Release number of selected component (if applicable):
coreutils-8.32-1.fc33

--- Additional comment from Kamil Dudka on 2020-03-10 15:33:08 CET ---

Minimal example:

# yum install -y \
    https://kojipkgs.fedoraproject.org/packages/coreutils/8.32/1.fc33/x86_64/coreutils-8.32-1.fc33.x86_64.rpm \
    https://kojipkgs.fedoraproject.org/packages/coreutils/8.32/1.fc33/x86_64/coreutils-common-8.32-1.fc33.x86_64.rpm

# mkdir -p /var/tmp/chroot/{bin,dev,lib64,proc}
# cp -L /lib64/{ld-linux-x86-64.so.2,libselinux.so.1,libc.so.6,libpcre2-8.so.0,libdl.so.2,libpthread.so.0} /var/tmp/chroot/lib64
# cp /bin/mknod /var/tmp/chroot/bin
# chroot /var/tmp/chroot /bin/mknod /dev/random -m0666 c 1 8
/bin/mknod: cannot set permissions of '/dev/random': Operation not supported
# echo $?
1

# rm -fv /var/tmp/chroot/dev/random 
removed '/var/tmp/chroot/dev/random'

# strace chroot /var/tmp/chroot /bin/mknod /dev/random -m0666 c 1 8
[...]
umask(000)                              = 022
umask(022)                              = 000
mknod("/dev/random", S_IFCHR|0666, makedev(0x1, 0x8)) = 0
openat(AT_FDCWD, "/dev/random", O_RDONLY|O_NOFOLLOW|O_CLOEXEC|O_PATH) = 3
newfstatat(3, "", {st_mode=S_IFCHR|0644, st_rdev=makedev(0x1, 0x8), ...}, AT_EMPTY_PATH) = 0
chmod("/proc/self/fd/3", 0666)          = -1 ENOENT (No such file or directory)
close(3)                                = 0
write(2, "/bin/mknod: ", 12/bin/mknod: )            = 12
write(2, "cannot set permissions of '/dev/"..., 39cannot set permissions of '/dev/random') = 39
write(2, ": Operation not supported", 25: Operation not supported) = 25
write(2, "\n", 1
)                       = 1
close(1)                                = 0
close(2)                                = 0
exit_group(1)                           = ?

# rm -fv /var/tmp/chroot/dev/random 
removed '/var/tmp/chroot/dev/random'
# mount -t proc proc /var/tmp/chroot/proc
# strace chroot /var/tmp/chroot /bin/mknod /dev/random -m0666 c 1 8
umask(000)                              = 022
umask(022)                              = 000
mknod("/dev/random", S_IFCHR|0666, makedev(0x1, 0x8)) = 0
openat(AT_FDCWD, "/dev/random", O_RDONLY|O_NOFOLLOW|O_CLOEXEC|O_PATH) = 3
newfstatat(3, "", {st_mode=S_IFCHR|0644, st_rdev=makedev(0x1, 0x8), ...}, AT_EMPTY_PATH) = 0
chmod("/proc/self/fd/3", 0666)          = 0
close(3)                                = 0
close(1)                                = 0
close(2)                                = 0
exit_group(0)                           = ?

--- Additional comment from Kamil Dudka on 2020-03-10 17:35:36 CET ---

It turns out that the observed behavior is caused by the implementation of lchmod() in glibc.  A pure rebuild of coreutils-8.31-10.fc32 on a rawhide system triggers the same bug.

--- Additional comment from Pádraig Brady on 2020-03-10 20:29:00 CET ---

You can force the gnulib implementation of lchmod with `configure ac_cv_func_lchmod=no`.
That implementation in 8.32 is similar to the glibc implementation,
but it also falls back to chmod() without /proc,
and so does not have this problem.

--- Additional comment from Kamil Dudka on 2020-03-11 12:05:43 CET ---

Thanks!  I will use it as a workaround until the issue is resolved upstream.

--- Additional comment from Kamil Dudka on 2020-03-11 13:23:55 CET ---

dist-git commit: https://src.fedoraproject.org/rpms/coreutils/c/acfa9e81

--- Additional comment from Kamil Dudka on 2020-03-12 14:02:14 CET ---

I have pushed a downstream change for now to avoid breakage of existing software.  On the other hand, coreutils/gnulib/glibc upstream developers are not going to fix this anytime soon.  So the long term fix needs to go to lorax: it needs to mount a proc file system on /var/tmp/lorax/lorax.7xmi9p_a/installtree/proc before running mknod in the chroot.