Bug 1771257 - virt-sparsify failed when run on the 4.3.0 qcow2 image.
Summary: virt-sparsify failed when run on the 4.3.0 qcow2 image.
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: RHCOS
Version: 4.3.0
Hardware: Unspecified
OS: Linux
high
high
Target Milestone: ---
: ---
Assignee: Micah Abbott
QA Contact: Michael Nguyen
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-11-12 04:49 UTC by Praveen Kumar
Modified: 2020-02-18 06:31 UTC (History)
8 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-11-18 16:35:22 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Comment 1 Christophe Fergeau 2019-11-12 14:01:48 UTC
A shorter reproducer is to pick up the qcow2 image used by the installer from ~/.cache/openshift-install/libvirt/image/ , unzip it, and then run:
$ TMPDIR=$(pwd) virt-sparsify ./rhcos-43.81.201911081536.0-qemu.x86_64.qcow2 sparse.qcow2

Comment 2 Richard W.M. Jones 2019-11-12 15:37:23 UTC
The filesystem on /dev/sda4 is XFS and mounts fine (there are no kernel
messages indicating any problem) but I simply cannot write to it even as root.
All writes return -EPERM but there's no other indication of what is wrong.

Comment 4 Richard W.M. Jones 2019-11-12 16:04:22 UTC
$ virt-rescue --ro -a rhcos-43.81.201911081536.0-qemu.x86_64.qcow2

><rescue> file -bsL /dev/sda4
SGI XFS filesystem data (blksz 4096, inosz 512, v2 dirs)
><rescue> mount /dev/sda4 /sysroot/
><rescue> dmesg | tail
[    0.413665] sd 2:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[    0.413709] sd 2:0:1:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[    0.414962] sd 2:0:1:0: [sdb] Attached SCSI disk
[    0.415296]  sda: sda1 sda2 sda3 sda4
[    0.415732] sd 2:0:0:0: [sda] Attached SCSI disk
[    0.421296] EXT4-fs (sdb): mounting ext2 file system using the ext4 subsystem
[    0.423023] EXT4-fs (sdb): mounted filesystem without journal. Opts: 
[   22.706196] SGI XFS with ACLs, security attributes, scrub, no debug enabled
[   22.709669] XFS (sda4): Mounting V5 Filesystem
[   22.715660] XFS (sda4): Ending clean mount

><rescue> ls -la /sysroot
total 8
drwxr-xr-x.  4 root root   66 Nov  8 15:43 .
drwxr-xr-x  19 root root 4096 Nov 12 16:02 ..
-rw-rw-r--.  1 root root  257 Nov  8 15:43 .coreos-aleph-version.json
drwxrwxr-x.  2 root root    6 Nov  8 15:42 boot
drwxrwxr-x.  5 root root   62 Nov  8 15:43 ostree

><rescue> touch /sysroot/hello
touch: setting times of '/sysroot/hello': No such file or directory
><rescue> strace touch /sysroot/hello
execve("/bin/touch", ["touch", "/sysroot/hello"], 0x7ffe6aacbb68 /* 13 vars */) = 0
[...]
openat(AT_FDCWD, "/sysroot/hello", O_WRONLY|O_CREAT|O_NOCTTY|O_NONBLOCK, 0666) = -1 EPERM (Operation not permitted)
utimensat(AT_FDCWD, "/sysroot/hello", NULL, 0) = -1 ENOENT (No such file or directory)

Comment 5 Richard W.M. Jones 2019-11-12 16:22:23 UTC
It turns out (thanks Jeff Moyer) that the root directory of that
filesystem is marked as immutable:

><rescue> lsattr -a /sysroot
----i--------------- /sysroot/.

which is what causes the problem.  Because of the way that virt-sparsify
works in copying mode it needs to temporarily write a balloon file to
each filesystem, so an immutable filesystem isn't going to work.

As a workaround for this particular disk you can use the --ignore option, eg:

  virt-sparsify --ignore /dev/sda4 [...]

You can also use in-place sparsification instead, which is much faster.  This
ignores the immutable bit and I've checked that it works on this particular
filesystem.

Comment 6 Praveen Kumar 2019-11-12 16:28:43 UTC
@Richard Thank you for quick look and providing a workaround. Before we were using `in-place` option but that doesn't work when we specify `-o lazy_refcounts=on` option and that is something we really required to make this qcow2 image work with mac using hyperkit. We will try to use the --ignore option meanwhile and see if resulted image is work for our case. I still not sure why this change is required for Redhat CoreOS side, may be @Micah or @Steve can answer.

Comment 7 Richard W.M. Jones 2019-11-12 16:33:09 UTC
Another thing we were discussing is temporarily removing the immutable attribute
around the virt-sparsify command, ie:

$ guestfish -a source.qcow2 -m /dev/sda4 set-e2attrs / i clear:true
$ virt-sparsify source.qcow2 dest.qcow2
$ guestfish -a dest.qcow2 -m /dev/sda4 set-e2attrs / i

Although the command is called set-e2attrs it can in fact set XFS attributes
too, but see also bug 1771587.

Comment 8 Praveen Kumar 2019-11-13 09:11:08 UTC
Also when I inspect the Redhat CoreOS image for 4.2 and 4.3 side I see the difference in mount points.

- 4.3 images
```
$ guestfish <<_EOF_
 add crc.qcow2
run
list-filesystems
_EOF_

/dev/sda1: ext4
/dev/sda2: vfat
/dev/sda3: unknown
/dev/sda4: xfs
```

- 4.2 images
```
$  guestfish <<_EOF_
>  add crc.qcow2
> run
> list-filesystems
> _EOF_

/dev/sda1: unknown
/dev/sda2: ext4
/dev/sda3: xfs
```

Comment 9 Colin Walters 2019-11-13 15:15:36 UTC
This came from https://github.com/coreos/coreos-assembler/pull/737/commits/bbc61e3bc0b1666ea951284010602e5c2308b7e2

> Another thing we were discussing is temporarily removing the immutable attribute

Makes sense to me!

I don't think there's any RHCOS bug here, CRC should add those libguestfs commands above to their build scripts, right?

Comment 10 Christophe Fergeau 2019-11-13 15:27:02 UTC
(In reply to Colin Walters from comment #9) 
> CRC should add those libguestfs commands above to their build scripts, right?

Mostly done in https://github.com/code-ready/snc/pull/130

Comment 11 Micah Abbott 2019-11-18 16:18:39 UTC
@praveen Based on the latest comments and PR to `code-ready`, this doesn't appear to be an actual RHCOS bug anymore.  Would you object to closing it?

Comment 12 Christophe Fergeau 2019-11-18 16:35:22 UTC
Yes this can be closed.


Note You need to log in before you can comment on or make changes to this bug.