Bug 1166057

Summary: btrfs filesystem will not work well if you create the filesystem with multiple disks at the same time, such as: mkfs-btrfs "/dev/sda1 /dev/sdb1"
Product: Red Hat Enterprise Linux 7 Reporter: Lingfei Kong <lkong>
Component: libguestfsAssignee: Richard W.M. Jones <rjones>
Status: CLOSED ERRATA QA Contact: Virtualization Bugs <virt-bugs>
Severity: low Docs Contact:
Priority: low    
Version: 7.1CC: leiwang, linl, mbooth, ptoscano, wshi, xchen
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: libguestfs-1.31.7-2.el7.x86_64 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-11-03 17:47:30 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1218766    
Bug Blocks: 1288337, 1301891    

Description Lingfei Kong 2014-11-20 10:54:53 UTC
Description of problem:
btrfs filesystem will not work well if you create the filesystem with multiple disks at the same time, such as: mkfs-btrfs "/dev/sda1 /dev/sdb1"


Version-Release number of selected component (if applicable):
libguestfs-1.28.1-1.9.el7


How reproducible:
100%


Steps to Reproduce:
1. #guestfish
><fs> sparse test-btrfs-devices-1.img 1G
><fs> sparse test-btrfs-devices-2.img 1G
><fs> sparse test-btrfs-devices-3.img 1G
><fs> sparse test-btrfs-devices-4.img 1G
><fs> run
 100% ⟦▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒⟧ --:--
><fs> list-devices
/dev/sda
/dev/sdb
/dev/sdc
/dev/sdd
><fs> part-disk /dev/sda mbr
><fs> part-disk /dev/sdb mbr
><fs> part-disk /dev/sdc mbr
><fs> part-disk /dev/sdd mbr
><fs> trace 1
><fs> mkfs-btrfs "/dev/sda1 /dev/sdb1"
libguestfs: trace: mkfs_btrfs "/dev/sda1 /dev/sdb1"
libguestfs: trace: mkfs_btrfs = 0
><fs> mkfs-btrfs /dev/sdc1
libguestfs: trace: mkfs_btrfs "/dev/sdc1"
libguestfs: trace: mkfs_btrfs = 0
><fs> mkfs-btrfs /dev/sdd1
libguestfs: trace: mkfs_btrfs "/dev/sdd1"
libguestfs: trace: mkfs_btrfs = 0
><fs> list-filesystems
libguestfs: trace: list_filesystems
libguestfs: trace: feature_available "lvm2"
libguestfs: trace: feature_available = 1
libguestfs: trace: feature_available "ldm"
libguestfs: trace: feature_available = 0
libguestfs: trace: list_devices
libguestfs: trace: list_devices = ["/dev/sda", "/dev/sdb", "/dev/sdc", "/dev/sdd"]
libguestfs: trace: list_partitions
libguestfs: trace: list_partitions = ["/dev/sda1", "/dev/sdb1", "/dev/sdc1", "/dev/sdd1"]
libguestfs: trace: list_md_devices
libguestfs: trace: list_md_devices = []
libguestfs: trace: part_to_dev "/dev/sda1"
libguestfs: trace: part_to_dev = "/dev/sda"
libguestfs: trace: part_to_dev "/dev/sdb1"
libguestfs: trace: part_to_dev = "/dev/sdb"
libguestfs: trace: part_to_dev "/dev/sdc1"
libguestfs: trace: part_to_dev = "/dev/sdc"
libguestfs: trace: part_to_dev "/dev/sdd1"
libguestfs: trace: part_to_dev = "/dev/sdd"
libguestfs: trace: vfs_type "/dev/sda1"
libguestfs: trace: vfs_type = "btrfs"
libguestfs: trace: btrfs_subvolume_list "/dev/sda1"
libguestfs: trace: btrfs_subvolume_list = <struct guestfs_btrfssubvolume_list *>
libguestfs: trace: vfs_type "/dev/sdb1"
libguestfs: trace: vfs_type = "btrfs"
libguestfs: trace: btrfs_subvolume_list "/dev/sdb1"
libguestfs: trace: btrfs_subvolume_list = <struct guestfs_btrfssubvolume_list *>
libguestfs: trace: vfs_type "/dev/sdc1"
libguestfs: trace: vfs_type = "btrfs"
libguestfs: trace: btrfs_subvolume_list "/dev/sdc1"
libguestfs: trace: btrfs_subvolume_list = <struct guestfs_btrfssubvolume_list *>
libguestfs: trace: vfs_type "/dev/sdd1"
libguestfs: trace: vfs_type = "btrfs"
libguestfs: trace: btrfs_subvolume_list "/dev/sdd1"
libguestfs: trace: btrfs_subvolume_list = <struct guestfs_btrfssubvolume_list *>
libguestfs: trace: lvs
libguestfs: trace: lvs = []
libguestfs: trace: list_filesystems = ["/dev/sda1", "btrfs", "/dev/sdb1", "btrfs", "/dev/sdc1", "btrfs", "/dev/sdd1", "btrfs"]
/dev/sda1: btrfs
/dev/sdb1: btrfs
/dev/sdc1: btrfs
/dev/sdd1: btrfs
><fs> quit

2. #guestfish -a test-btrfs-devices-1.img
><fs> run
 100% ⟦▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒⟧ --:--
><fs> trace 1
><fs> list-filesystems
libguestfs: trace: list_filesystems
libguestfs: trace: feature_available "lvm2"
libguestfs: trace: feature_available = 1
libguestfs: trace: feature_available "ldm"
libguestfs: trace: feature_available = 0
libguestfs: trace: list_devices
libguestfs: trace: list_devices = ["/dev/sda"]
libguestfs: trace: list_partitions
libguestfs: trace: list_partitions = ["/dev/sda1"]
libguestfs: trace: list_md_devices
libguestfs: trace: list_md_devices = []
libguestfs: trace: part_to_dev "/dev/sda1"
libguestfs: trace: part_to_dev = "/dev/sda"
libguestfs: trace: vfs_type "/dev/sda1"
libguestfs: trace: vfs_type = "btrfs"
libguestfs: trace: btrfs_subvolume_list "/dev/sda1"
libguestfs: trace: btrfs_subvolume_list = NULL (error)
libguestfs: error: btrfs_subvolume_list: /dev/sda1 on <internal> (options: ''): mount: wrong fs type, bad option, bad superblock on /dev/sda1,
       missing codepage or helper program, or other error

       In some cases useful info is found in syslog - try
       dmesg | tail or so.
libguestfs: trace: list_filesystems = NULL (error)
><fs>
><fs> dmesg |tail -20
[    1.421987] input: ImExPS/2 Generic Explorer Mouse as /devices/platform/i8042/serio1/input/input1
[    1.553818] input: PC Speaker as /devices/platform/pcspkr/input/input2
[    1.572177] powernow_k8: Power state transitions not supported
[    2.632306] device-mapper: uevent: version 1.0.3
[    2.634641] device-mapper: ioctl: 4.27.0-ioctl (2013-10-30) initialised: dm-devel
[    8.529190] xor: measuring software checksum speed
[    8.540015]    prefetch64-sse:  5512.000 MB/sec
[    8.550015]    generic_sse:  3840.000 MB/sec
[    8.550834] xor: using function: prefetch64-sse (5512.000 MB/sec)
[    8.575036] raid6: sse2x1    3257 MB/s
[    8.592025] raid6: sse2x2    4367 MB/s
[    8.609017] raid6: sse2x4    4800 MB/s
[    8.609720] raid6: using algorithm sse2x4 (4800 MB/s)
[    8.610725] raid6: using intx1 recovery algorithm
[    8.667487] Btrfs loaded
[    8.695164] BTRFS: device fsid b693c349-4f64-439e-b453-15ce2e1d79f9 devid 1 transid 4 /dev/sda1
[    8.698778] BTRFS info (device sda1): disk space caching is enabled
[    8.700892] BTRFS: failed to read chunk tree on sda1
[    8.708155] BTRFS: open_ctree failed

#guestfish -a test-btrfs-devices-2.img run : list-filesystems
libguestfs: error: btrfs_subvolume_list: /dev/sda1 on <internal> (options: ''): mount: wrong fs type, bad option, bad superblock on /dev/sda1,
       missing codepage or helper program, or other error

       In some cases useful info is found in syslog - try
       dmesg | tail or so.

#guestfish -a test-btrfs-devices-3.img run : list-filesystems
/dev/sda1: btrfs

#guestfish -a test-btrfs-devices-4.img run : list-filesystems
/dev/sda1: btrfs


Actual results:
At step 2, btrfs filesystem in test-btrfs-devices-1.img and test-btrfs-devices-2.img was broken. btrfs filesystem in test-btrfs-devices-3.img and test-btrfs-devices-4.img works well


Expected results:
At step 2, can get the filesystem in test-btrfs-devices-1.img,  test-btrfs-devices-21.img and the filesystem is works well.


Additional info:

Comment 1 Pino Toscano 2014-12-01 09:41:37 UTC
(In reply to Lingfei Kong from comment #0)
> ><fs> mkfs-btrfs "/dev/sda1 /dev/sdb1"

As also mentioned in mkfs.btrfs(8), this will create a filesystem spanning across the specified devices, which in the above case are stored in:

> ><fs> sparse test-btrfs-devices-1.img 1G
> ><fs> sparse test-btrfs-devices-2.img 1G

So later when doing:

> 2. #guestfish -a test-btrfs-devices-1.img
> ><fs> run
>  100%
> ⟦▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒
> ▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒⟧ --:--
> ><fs> trace 1
> ><fs> list-filesystems
> [...]
> libguestfs: error: btrfs_subvolume_list: /dev/sda1 on <internal> (options:
> ''): mount: wrong fs type, bad option, bad superblock on /dev/sda1,
>        missing codepage or helper program, or other error
> 
>        In some cases useful info is found in syslog - try
>        dmesg | tail or so.

... and ...

> #guestfish -a test-btrfs-devices-2.img run : list-filesystems
> libguestfs: error: btrfs_subvolume_list: /dev/sda1 on <internal> (options:
> ''): mount: wrong fs type, bad option, bad superblock on /dev/sda1,
>        missing codepage or helper program, or other error
> 
>        In some cases useful info is found in syslog - try
>        dmesg | tail or so.

... they fails because the filesystem they hold is just a part of the spanned btrfs filesystem. You can also verify with
  debug sh "btrfs filesystem show -d"
the devices used for any btrfs filesystem.

Having both the devices allows using the filesystem correctly:

$ guestfish -a test-btrfs-devices-1.img -a test-btrfs-devices-2.img run : list-filesystems
/dev/sda1: btrfs
/dev/sdb1: btrfs

It seems the "btrfs filesystem show" command could be the only one giving this information, and its output is less than ideal to parse...

> Expected results:
> At step 2, can get the filesystem in test-btrfs-devices-1.img, 
> test-btrfs-devices-21.img and the filesystem is works well.

Definitely you cannot use any of the two without the other as well.

Comment 2 Lingfei Kong 2014-12-02 03:31:47 UTC
(In reply to Pino Toscano from comment #1)
> Having both the devices allows using the filesystem correctly:
> 
> $ guestfish -a test-btrfs-devices-1.img -a test-btrfs-devices-2.img run :
> list-filesystems
> /dev/sda1: btrfs
> /dev/sdb1: btrfs
> 

I tried this way, but also got the same error:

Here are the steps:
Step 1:
#guestfish
><fs> sparse test-btrfs-devices-1.img 1G
><fs> sparse test-btrfs-devices-2.img 1G
><fs> run
><fs> part-disk /dev/sda mbr
><fs> part-disk /dev/sdb mbr
><fs> mkfs-btrfs "/dev/sda1 /dev/sdb1"
><fs> trace 1
><fs> list-filesystem
libguestfs: trace: list_filesystems
libguestfs: trace: feature_available "lvm2"
libguestfs: trace: feature_available = 1
libguestfs: trace: feature_available "ldm"
libguestfs: trace: feature_available = 0
libguestfs: trace: list_devices
libguestfs: trace: list_devices = ["/dev/sda", "/dev/sdb"]
libguestfs: trace: list_partitions
libguestfs: trace: list_partitions = ["/dev/sda1", "/dev/sdb1"]
libguestfs: trace: list_md_devices
libguestfs: trace: list_md_devices = []
libguestfs: trace: part_to_dev "/dev/sda1"
libguestfs: trace: part_to_dev = "/dev/sda"
libguestfs: trace: part_to_dev "/dev/sdb1"
libguestfs: trace: part_to_dev = "/dev/sdb"
libguestfs: trace: vfs_type "/dev/sda1"
libguestfs: trace: vfs_type = "btrfs"
libguestfs: trace: btrfs_subvolume_list "/dev/sda1"
libguestfs: trace: btrfs_subvolume_list = <struct guestfs_btrfssubvolume_list *>
libguestfs: trace: vfs_type "/dev/sdb1"
libguestfs: trace: vfs_type = "btrfs"
libguestfs: trace: btrfs_subvolume_list "/dev/sdb1"
libguestfs: trace: btrfs_subvolume_list = <struct guestfs_btrfssubvolume_list *>
libguestfs: trace: lvs
libguestfs: trace: lvs = []
libguestfs: trace: list_filesystems = ["/dev/sda1", "btrfs", "/dev/sdb1", "btrfs"]
/dev/sda1: btrfs
/dev/sdb1: btrfs
><fs> quit

Step 2:
#guestfish -a test-btrfs-devices-1.img -a test-btrfs-devices-2.img 
><fs> run
><fs> trace 1
><fs> list-filesystems 
libguestfs: trace: list_filesystems
libguestfs: trace: feature_available "lvm2"
libguestfs: trace: feature_available = 1
libguestfs: trace: feature_available "ldm"
libguestfs: trace: feature_available = 0
libguestfs: trace: list_devices
libguestfs: trace: list_devices = ["/dev/sda", "/dev/sdb"]
libguestfs: trace: list_partitions
libguestfs: trace: list_partitions = ["/dev/sda1", "/dev/sdb1"]
libguestfs: trace: list_md_devices
libguestfs: trace: list_md_devices = []
libguestfs: trace: part_to_dev "/dev/sda1"
libguestfs: trace: part_to_dev = "/dev/sda"
libguestfs: trace: part_to_dev "/dev/sdb1"
libguestfs: trace: part_to_dev = "/dev/sdb"
libguestfs: trace: vfs_type "/dev/sda1"
libguestfs: trace: vfs_type = "btrfs"
libguestfs: trace: btrfs_subvolume_list "/dev/sda1"
libguestfs: trace: btrfs_subvolume_list = NULL (error)
libguestfs: error: btrfs_subvolume_list: /dev/sda1 on <internal> (options: ''): mount: wrong fs type, bad option, bad superblock on /dev/sda1,
       missing codepage or helper program, or other error

       In some cases useful info is found in syslog - try
       dmesg | tail or so.
libguestfs: trace: list_filesystems = NULL (error)


When run 'list-filesystems' guestfish will call btrfs_subvolume_list, but i didn't create any subvolume. At Step 1,  btrfs_subvolume_list will run successfully and output:
libguestfs: trace: btrfs_subvolume_list = <struct guestfs_btrfssubvolume_list *>
<no subvolume>

But at Step 2, btrfs_subvolume_list failed to run.

I don't know why list-filesystems will call btrfs_subvolume_list first. 

> It seems the "btrfs filesystem show" command could be the only one giving
> this information, and its output is less than ideal to parse...
> 
> > Expected results:
> > At step 2, can get the filesystem in test-btrfs-devices-1.img, 
> > test-btrfs-devices-21.img and the filesystem is works well.
> 
> Definitely you cannot use any of the two without the other as well.

Is there any way i can use btrfs in test-btrfs-devices-1.img and test-btrfs-devices-2.img again after i quit guestfish? 

Thanks!

Comment 3 Richard W.M. Jones 2014-12-02 10:19:19 UTC
Here's the two line reproducer:

$ guestfish -N part:1G -N part:1G mkfs-btrfs "/dev/sda1 /dev/sdb1" : list-filesystems
$ guestfish -a test1.img -a test2.img run : list-filesystems

On Fedora 22:

$ rpm -q libguestfs
libguestfs-1.29.12-1.fc22.x86_64
$ guestfish -N part:1G -N part:1G mkfs-btrfs "/dev/sda1 /dev/sdb1" : list-filesystems
/dev/sda1: btrfs
/dev/sdb1: btrfs
$ guestfish -a test1.img -a test2.img run : list-filesystems
/dev/sda1: btrfs
/dev/sdb1: btrfs

(No bug)

On RHEL 7.1:

$ rpm -q libguestfs
libguestfs-1.28.1-1.14.el7.x86_64
$ guestfish -N part:1G -N part:1G mkfs-btrfs "/dev/sda1 /dev/sdb1" : list-filesystems
/dev/sda1: btrfs
/dev/sdb1: btrfs
$ guestfish -a test1.img -a test2.img run : list-filesystems
libguestfs: error: btrfs_subvolume_list: /dev/sda1 on <internal> (options: ''): mount: wrong fs type, bad option, bad superblock on /dev/sda1,
       missing codepage or helper program, or other error

       In some cases useful info is found in syslog - try
       dmesg | tail or so.

So yes I agree that there is a bug on RHEL 7.

Comment 6 Xianghua Chen 2016-06-28 07:13:03 UTC
Verified with the packages:
libguestfs-1.32.5-6.el7.x86_64

Verify steps:
1. Create 2 guest image with btrfs:
# guestfish -N part:1G -N part:1G mkfs-btrfs "/dev/sda1 /dev/sdb1" : list-filesystems
/dev/sda1: btrfs
/dev/sdb1: btrfs
# guestfish -a test1.img -a test2.img run : list-filesystems
/dev/sda1: btrfs
/dev/sdb1: btrfs


Command finished successfully.

So verified.

Comment 8 errata-xmlrpc 2016-11-03 17:47:30 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2016-2576.html