RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1144766 - futimens() is a no-op via guestmount
Summary: futimens() is a no-op via guestmount
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: libguestfs
Version: 7.0
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: rc
: ---
Assignee: Richard W.M. Jones
QA Contact: Virtualization Bugs
URL:
Whiteboard:
Depends On: 1144891
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-09-20 14:34 UTC by Colin Walters
Modified: 2015-03-05 13:45 UTC (History)
5 users (show)

Fixed In Version: libguestfs-1.27.52-1.1.el7
Doc Type: Bug Fix
Doc Text:
Clone Of:
: 1144891 (view as bug list)
Environment:
Last Closed: 2015-03-05 13:45:26 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
utimensat.c (280 bytes, text/plain)
2014-09-23 13:23 UTC, Richard W.M. Jones
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2015:0303 0 normal SHIPPED_LIVE libguestfs bug fix and enhancement update 2015-03-05 17:34:44 UTC

Description Colin Walters 2014-09-20 14:34:08 UTC
Atomic/OSTree uses libguestfs to generate disk images - the images are mounted via guestmount.

# rpm -q libguestfs
libguestfs-1.22.6-22.el7.x86_64
# guestmount -a disk.qcow2 -m ... /mnt/guestmount
# cd /mnt/guestmount
# ostree' '--repo=/mnt/guestmount/ostree/repo' 'pull-local' '--disable-fsync' '--remote=rhel-atomic-host' '/srv/rhel-atomic-host/repo' 'febc5921803258a8c28b0d940a6fbff7613a0875955be9e5d9808b3bc4ef07f2'
...

Now, OSTree calls futimens() to try to set the modification times of files to 0 (see https://bugzilla.gnome.org/show_bug.cgi?id=720363 for background)  The call apparently succeeds:

open("ostree/repo/objects/00/15fd1a8baa527835162e46a7b826b68c2062d446b4b0f337f96736356d7f06.file", O_WRONLY) = 3
utimensat(3, NULL, {UTIME_OMIT, {0, 0}}, 0) = 0
close(3)                                = 0

But:

# stat ostree/repo/objects/00/15fd1a8baa527835162e46a7b826b68c2062d446b4b0f337f96736356d7f06.file
  File: ‘ostree/repo/objects/00/15fd1a8baa527835162e46a7b826b68c2062d446b4b0f337f96736356d7f06.file’
  Size: 2420      	Blocks: 8          IO Block: 4096   regular file
Device: 29h/41d	Inode: 79097       Links: 1
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2014-09-20 10:27:46.000000000 -0400
Modify: 2014-09-20 10:27:46.000000000 -0400
Change: 2014-09-20 10:27:45.000000000 -0400
 Birth: -

However, using utime() via a path *does* succeed, e.g. with this trivial Python program:

#!/usr/bin/env python
import os,sys
os.utime(sys.argv[1], (0,0))

...
utimes("ostree/repo/objects/00/15fd1a8baa527835162e46a7b826b68c2062d446b4b0f337f96736356d7f06.file", {{0, 0}, {0, 0}}) = 0

# stat ostree/repo/objects/00/15fd1a8baa527835162e46a7b826b68c2062d446b4b0f337f96736356d7f06.file
  File: ‘ostree/repo/objects/00/15fd1a8baa527835162e46a7b826b68c2062d446b4b0f337f96736356d7f06.file’
  Size: 2420      	Blocks: 8          IO Block: 4096   regular file
Device: 29h/41d	Inode: 79097       Links: 1
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 1969-12-31 19:00:00.000000000 -0500
Modify: 1969-12-31 19:00:00.000000000 -0500
Change: 2014-09-20 10:33:27.000000000 -0400
 Birth: -

Comment 1 Colin Walters 2014-09-20 14:35:01 UTC
Note: this is not urgent - I can just add a workaround to the disk image generation script to walk the repository and use utimes() instead of utimensat().

Comment 3 Richard W.M. Jones 2014-09-20 16:28:57 UTC
Pino, commit 99d6e2c84ef490feb6ff2e37e4b0684f34bac554 fixes
this doesn't it?

Colin - you might want to try the RHEL 7.1 packages (in general,
not just here - I'd be interested in how well they work for you).
You can get them from brew, or by following the instructions here:
https://www.redhat.com/archives/libguestfs/2014-May/msg00090.html

Comment 4 Colin Walters 2014-09-20 22:07:21 UTC
Downstream workaround: https://github.com/projectatomic/rpm-ostree-toolbox/commit/80fc2dcfa2cd0066ae55adf5afbfd53e59278b78

I tried your updated packages, didn't appear to help.  Here's a quick test program that uses all the different calls:

#include <sys/statvfs.h>
#include <sys/stat.h>
#include <utime.h>
#include <sys/time.h>
#include <errno.h>
#include <time.h>
#include <fcntl.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>

int
main (int argc, char **argv)
{
  const char *path = argv[1];
  int fd;
  struct stat stbuf;
  struct timespec ts[2];

  ts[0].tv_sec = 0;
  ts[0].tv_nsec = UTIME_OMIT;
  ts[1].tv_sec = ts[1].tv_nsec = 0;

  fd = open (path, O_WRONLY);
  if (fd == -1)
    {
      perror ("open: ");
      exit (1);
    }
  
  if (lstat (path, &stbuf) != 0)
    {
      perror ("lstat: ");
      exit (1);
    }

  printf ("orig mtime: %llu\n", (unsigned long long)stbuf.st_mtime);
  
  if (futimens (fd, ts) == -1)
    {
      perror ("futimens: ");
      exit (1);
    }

  if (lstat (path, &stbuf) != 0)
    {
      perror ("fstat: ");
      exit (1);
    }

  printf ("after futimens: %llu\n", (unsigned long long)stbuf.st_mtime);

  if (utimensat (AT_FDCWD, path, ts, AT_SYMLINK_NOFOLLOW) == -1)
    {
      perror ("futimens: ");
      exit (1);
    }

  if (lstat (path, &stbuf) != 0)
    {
      perror ("fstat: ");
      exit (1);
    }

  printf ("after utimensat: %llu\n", (unsigned long long)stbuf.st_mtime);

  {
    struct timeval tvs[2];
    tvs[0].tv_sec = 0;
    tvs[0].tv_usec = 0;
    tvs[1].tv_sec = 0;
    tvs[1].tv_usec = 0;

    if (utimes (path, tvs) == -1)
      {
	perror ("utimes: ");
	exit (1);
      }
  }
  if (lstat (path, &stbuf) != 0)
    {
      perror ("fstat: ");
      exit (1);
    }

  printf ("after utimes: %llu\n", (unsigned long long)stbuf.st_mtime);

  (void) close (fd);

  exit (0);
}

Comment 5 Richard W.M. Jones 2014-09-21 10:37:37 UTC
With Fedora 21 / libguestfs 1.27.50:

orig mtime: 1411295583
after futimens: 1411295583
after utimensat: 1411295583
after utimes: 0

So if I understand the test, only utimes has an effect.

Same with guestmount debugging enabled:

libguestfs: trace: utimens "/testfile" 0 0 0 0
libguestfs: trace: utimens = 0
libguestfs: trace: lstat "/testfile"
libguestfs: trace: lstat = <struct guestfs_stat *>

It looks as if futimens and utimensat don't even get translated
into FUSE calls, so we could be missing those callbacks from the
struct fuse_operations.  I'll have to look into this in more
detail at work.

Comment 6 Richard W.M. Jones 2014-09-21 21:18:44 UTC
In a few hours of after-lunch boredom, I rewrote our FUSE test
shell script in C.  This gives us finer control over how we
test system calls.

https://github.com/libguestfs/libguestfs/commit/aca076e2e2b3e03f4cf5b09cf7766925f97e4b68

This reveals two bugs in utimens: firstly the bug shown here,
but a second bug which is that our stat call throws away the
nanosecond fields (actually, both guestfs_stat AND the FUSE layer).

Comment 7 Richard W.M. Jones 2014-09-22 12:57:51 UTC
I added this patch upstream which was just for fixing bug 1144891:

https://www.redhat.com/archives/libguestfs/2014-September/msg00162.html

Miraculously this has also fixed this (futimens no-op) bug too.

Here is a patch which enables the test, demonstrating that it is working:

https://www.redhat.com/archives/libguestfs/2014-September/msg00165.html

I have no idea at the moment why this has fixed it ...

Comment 9 Wei Shi 2014-09-23 07:57:05 UTC
Reproduce with c4
libguestfs-1.27.52-1.1.el7.x86_64

# guestmount -a /tmp/WinXP-32-hvm.raw -m /dev/sda1 /mnt/
# cd /mnt/
# touch test
# ~/a.out test 
orig mtime: 1411457531
after futimens: 1411457531
after utimensat: 1411457531
after utimes: 0


Rich,
  Is my test steps wrong?

Comment 10 Richard W.M. Jones 2014-09-23 08:03:08 UTC
(In reply to Wei Shi from comment #9)
> Reproduce with c4
> libguestfs-1.27.52-1.1.el7.x86_64
> 
> # guestmount -a /tmp/WinXP-32-hvm.raw -m /dev/sda1 /mnt/
> # cd /mnt/
> # touch test
> # ~/a.out test 
> orig mtime: 1411457531
> after futimens: 1411457531
> after utimensat: 1411457531
> after utimes: 0
> 
> 
> Rich,
>   Is my test steps wrong?

The tests demonstrate the bug.  However:

 - Don't use a Windows guest for testing, because I don't know if it
   supports nanosecond-resolution timestamps.

 - Don't need to run the test as root!

To create an ext4 image which supports nanosecond timestamps, do:

guestfish -N disk mkfs ext4 /dev/sda inode:256 : mount /dev/sda / : touch /test

By the way we also test this as part of the test suite (fuse/test-fuse).

Comment 11 Wei Shi 2014-09-23 09:19:45 UTC
<wshi> $ export LIBGUESTFS_ATTACH_METHOD=appliance
<wshi> $ guestfish -N disk mkfs ext4 /dev/sda inode:256 : mount /dev/sda / : touch /test
<wshi> $ guestmount -a test1.img -m /dev/sda /tmp/tmp/
<wshi> $ touch /tmp/tmp/test
<wshi> $ ~/a.out /tmp/tmp/test 
<wshi> orig mtime: 1411463547
<wshi> after futimens: 1411463547
<wshi> after utimensat: 1411463547
<wshi> after utimes: 0

Comment 12 Richard W.M. Jones 2014-09-23 13:23:42 UTC
Created attachment 940433 [details]
utimensat.c

First of all, we need a simple, reliable test for this problem:

(1) Create a disk image in a format which is known to support nanosecond
timestamps:

guestfish -N disk mkfs ext4 /dev/sda inode:256 : mount /dev/sda / : touch /test

(2) Mount the disk image on /tmp/mnt:

mkdir -p /tmp/mnt
guestmount -a test1.img -m /dev/sda /tmp/mnt

(3) Verify the current timestamp of the test file:

stat /tmp/mnt/test

(4) Run utimensat system call (see attached C program):

./utimensat /tmp/mnt/test

(5) Repeat step (3):

stat /tmp/mnt/test

(6) Clean up:

guestunmount /tmp/mnt
rm test1.img

---------------

* In step (3) you should always see output that looks something like this.  The
exact time will depend on when you run the test of course:

Access: 2014-09-23 14:12:47.785707604 +0100
Modify: 2014-09-23 14:12:47.785707604 +0100

* Incorrect output (test failed).  In step (5) you will EITHER see the same
as step (3), OR you will see:

Access: 1970-01-01 01:02:03.000000000 +0100
Modify: 1970-01-01 01:01:18.000000000 +0100

Note the incorrect thing here is there is no nanosecond part (ie. 000000000).

* Correct output (bug is fixed).  In step (5) you will see:

Access: 1970-01-01 01:02:03.000000456 +0100
Modify: 1970-01-01 01:01:18.000000090 +0100

Note the nanosecond field is not all zeroes.

Comment 13 Richard W.M. Jones 2014-09-23 13:25:28 UTC
Using the steps in comment 12, I am able to verify this bug is fixed
in RHEL 7.

Wei Shi -- can you try the test again with guestmount 1.27.52?

Colin -- would be useful if you could sanity check the program and
steps used in comment 12.

Comment 14 Richard W.M. Jones 2014-09-23 13:27:50 UTC
(In reply to Richard W.M. Jones from comment #12)
> * Correct output (bug is fixed).  In step (5) you will see:
> 
> Access: 1970-01-01 01:02:03.000000456 +0100
> Modify: 1970-01-01 01:01:18.000000090 +0100
> 
> Note the nanosecond field is not all zeroes.

As it's not very clear.  The bug is ONLY fixed when those EXACT
times are shown.  It's not just that the nanosecond part is != 0.

Comment 16 Wei Shi 2014-09-24 05:42:47 UTC
Verified with libguestfs-1.27.52-1.1.el7:

guestfish -N disk mkfs ext4 /dev/sda inode:256 : mount /dev/sda / : touch /test

mkdir -p /tmp/mnt
guestmount -a test1.img -m /dev/sda /tmp/mnt

stat /tmp/mnt/test
  File: ‘/tmp/mnt/test’
  Size: 0               Blocks: 0          IO Block: 1024   regular empty file
Device: 28h/40d Inode: 2           Links: 1
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Context: system_u:object_r:fusefs_t:s0
Access: 2014-09-24 13:40:22.266000000 +0800
Modify: 2014-09-24 13:40:22.266000000 +0800
Change: 2014-09-24 13:40:22.266000000 +0800
 Birth: -

./utimensat /tmp/mnt/test

stat /tmp/mnt/test
  File: ‘/tmp/mnt/test’
  Size: 0               Blocks: 0          IO Block: 1024   regular empty file
Device: 28h/40d Inode: 2           Links: 1
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Context: system_u:object_r:fusefs_t:s0
Access: 1970-01-01 08:02:03.000000456 +0800
Modify: 1970-01-01 08:01:18.000000090 +0800
Change: 2014-09-24 13:40:47.155000000 +0800
 Birth: -

guestunmount /tmp/mnt
rm test1.img

Comment 18 errata-xmlrpc 2015-03-05 13:45:26 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-0303.html


Note You need to log in before you can comment on or make changes to this bug.