Bug 1795576
| Summary: | kernel: VFS and/or XFS filesystem bug with timestamp larger than 32-bit Epoch (year 2038 bug) | ||
|---|---|---|---|
| Product: | [Fedora] Fedora | Reporter: | Miro Hrončok <mhroncok> |
| Component: | kernel | Assignee: | Kernel Maintainer List <kernel-maint> |
| Status: | NEW --- | QA Contact: | Fedora Extras Quality Assurance <extras-qa> |
| Severity: | unspecified | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | rawhide | CC: | airlied, bskeggs, cstratak, dmalcolm, fweimer, hdegoede, ichavero, itamar, jarodwilson, jeremy, jforbes, jglisse, john.j5live, jonathan, josef, kernel-maint, linville, masami256, mchehab, m.cyprian, mhroncok, mjg59, pviktori, rkuska, shcherbina.iryna, slavek.kabrda, steved, tomspur, torsava, vstinner |
| Target Milestone: | --- | ||
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| URL: | https://koschei.fedoraproject.org/package/python3 | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | Type: | Bug | |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Bug Depends On: | |||
| Bug Blocks: | 1795991 | ||
I managed to reproduce the issue on Fedora 31 on a XFS filesystem:
$ touch testfn
$ strace -o trace python3 -c 'import os; os.utime("testfn", (4386268800, 4386268800))'
$ grep ^utimensat trace
utimensat(AT_FDCWD, "testfn", [{tv_sec=4386268800, tv_nsec=0} /* 2108-12-30T01:00:00+0100 */, {tv_sec=4386268800, tv_nsec=0} /* 2108-12-30T01:00:00+0100 */], 0) = 0
$ stat testfn
(...)
Modify: 2038-01-19 04:14:07.000000000 +0100
(...)
It looks like a Linux kernel issue in the XFS filesystem: I don't think that mtime should be *silently* truncated to 2147483647.
--
Copy of Florian Weimer's message (https://bugs.python.org/issue39460#msg360952):
I believe you might be observing an XFS limitation in combination with a Linux VFS bug.
On disk, XFS only supports 32-bit timestamps:
typedef struct xfs_timestamp {
__be32 t_sec; /* timestamp seconds */
__be32 t_nsec; /* timestamp nanoseconds */
} xfs_timestamp_t;
This is on the roadmap being fixed.
However, the Linux VFS code does not appear to know about this. It caches the full 64-bit value. You only see the truncated value if it is read back from disk:
# touch -t 222201020304 /tmp/t
# ls -l /tmp/t
-rw-r--r--. 1 root root 0 Jan 2 2222 /tmp/t
# echo 3 > /proc/sys/vm/drop_caches
# ls -l /tmp/t
-rw-r--r--. 1 root root 0 Oct 19 1949 /tmp/t
This is a bug in the Linux VFS layer.
Removing the high severity, we can skip this test temporarily if needed. 32-bit timestamps are a missing XFS feature, requiring an on disk-format change (but perhaps without reformatting). In the short term, the Linux VFS layer should check the limits of the underlying file system and reject nonrepresentable timestamps. I can reproduce the VFS bug on Fedora 30 with 5.2.13-200.fc30.x86_64 kernel, so it doesn't seem to be a recent regression:
$ touch testfn
$ python3 -c 'import os; os.utime("testfn", (4386268800, 4386268800))'
$ stat testfn
...
Modify: 2108-12-30 01:00:00.000000000 +0100
...
$ sudo bash -c 'echo 3 > /proc/sys/vm/drop_caches'
$ stat testfn
...
Modify: 1972-11-22 18:31:44.000000000 +0100
...
Some Koji buidelrs were updated: From https://koji.fedoraproject.org/koji/taskinfo?taskID=41127107 (test passed): kernel version == 5.4.8-200.fc31.x86_64 (see root.log) From https://koji.fedoraproject.org/koji/taskinfo?taskID=41133060 (test failed): kernel version == 5.4.12-200.fc31.x86_64 (see root.log) I tested multiple kernel versions on Fedora 31. I noticed that utime/stat behavior changed between 5.3.14-300.fc31.x86_64 and 5.4.8-200.fc31.x86_64: * 5.3.14-300.fc31.x86_64: Modify: 2108-12-30 01:00:00.000000000 +0100 => Modify: 1972-11-22 18:31:44.000000000 +0100 * 5.4.8-200.fc31.x86_64: Modify: 2038-01-19 04:14:07.000000000 +0100 => Modify: 2038-01-19 04:14:07.000000000 +0100 I'm not sure if any of these behavior is correct :-) RHEL issue: https://bugzilla.redhat.com/show_bug.cgi?id=1795991 (sadly private) xfs has had support to fix this since of 5.10 kernels. As it is a new on disk format, you need to either create a new filesystem with bigtime=1 or convert an existing (unmounted) filesystem with 'xfs_admin -O bigtime=1'. This requires a more recent xfsprogs, which exists in rawhide, but not in F34 stable. At some point, I believe it should be the default for creating new filesystems, but it is not with the current xfsprogs. |
Description of problem: Packages python3 and python39 fail to build from source in Fedora rawhide because a test apparently randomly fails on several architectures (in all cases so far at least on one). FAIL: test_add_file_after_2107 (test.test_zipfile.StoredTestsWithSourceFile) ---------------------------------------------------------------------- Traceback (most recent call last): File "/builddir/build/BUILD/Python-3.8.1/Lib/test/test_zipfile.py", line 606, in test_add_file_after_2107 self.assertRaises(struct.error, zipfp.write, TESTFN) AssertionError: error not raised by write https://bugs.python.org/issue39460 Versions: 3.8.1, 3.9.0a2, 3.9.0a3 The packages are tracked by Koschei. See: https://koschei.fedoraproject.org/package/python3 https://koschei.fedoraproject.org/package/python39 But note that the ppc64le build is also blocked by bz1795575. This blocks any changes to Python we need to ship. It started to happen after the update of gcc to 10.