RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1462060 - Failed to attach disk with source file in the /dev/ dir
Summary: Failed to attach disk with source file in the /dev/ dir
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: libvirt
Version: 7.4
Hardware: x86_64
OS: Linux
unspecified
unspecified
Target Milestone: rc
: ---
Assignee: Michal Privoznik
QA Contact: Meina Li
URL:
Whiteboard:
Depends On:
Blocks: 1473046
TreeView+ depends on / blocked
 
Reported: 2017-06-16 03:37 UTC by yafu
Modified: 2018-04-10 10:50 UTC (History)
8 users (show)

Fixed In Version: libvirt-3.7.0-1.el7
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-04-10 10:48:37 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHEA-2018:0704 0 None None None 2018-04-10 10:50:02 UTC

Description yafu 2017-06-16 03:37:12 UTC 
Description of problem:
Failed to attach disk with source file in the /dev/ dir

Version-Release number of selected component (if applicable):
libvirt-3.2.0-10.el7.x86_64

How reproducible:
100%

Steps to reproduce:
1.Prepare a img in the /dev:
#qemu-img create -f qcow2 /dev/test.img 10M

2.Prepare the disk xml:
#cat disk.xml
 <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/dev/test.img'/>
      <target dev='vdc' bus='virtio'/>
    </disk>

3.Attach the disk to a running guest:
#virsh attach-device full-73 disk.xml
error: Failed to attach device from disk.xml
error: internal error: child reported: unable to set user and group to '107:107' on '/dev/test.img': No such file or directory

Actual results:
Failed to attach disk with source file in the /dev dir

Expected results:
Should attach the disk successfully.

Additional info:
1.The guest also can not start if disk having source file with qcwo2 format in the /dev dir:
#qemu-img create -f qcow2 /dev/test.img 10M
#virsh edit full-73
...
 <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/dev/test.img'/>
      <target dev='vdc' bus='virtio'/>
    </disk>
...
#virsh start full-73
error: Failed to start domain full-73
error: internal error: qemu unexpectedly closed the monitor: 2017-06-15T15:35:11.690410Z qemu-kvm: -chardev pty,id=charserial0: char device redirected to /dev/pts/18 (label charserial0)
2017-06-15T15:35:11.809120Z qemu-kvm: -drive file=/dev/test.img,format=qcow2,if=none,id=drive-virtio-disk4: Image is not in qcow2 format

2.The issues will not occur when turning off namespace.
Comment 3 Michal Privoznik 2017-06-22 16:20:19 UTC 
Patches proposed upstream:

https://www.redhat.com/archives/libvir-list/2017-June/msg00940.html
Comment 4 Michal Privoznik 2017-07-11 12:57:25 UTC 
I've just pushed the patches upstream:

commit e93d844b90aab2676a1ad910a165c9c83c99de3d
Author:     Michal Privoznik <mprivozn>
AuthorDate: Mon Jun 19 17:05:31 2017 +0200
Commit:     Michal Privoznik <mprivozn>
CommitDate: Tue Jul 11 14:45:15 2017 +0200

    qemu ns: Create chardev backends more frequently

    Currently, the only type of chardev that we create the backend
    for in the namespace is type='dev'. This is not enough, other
    backends might have files under /dev too. For instance channels
    might have a unix socket under /dev (well, bind mounted under
    /dev from a different place).

    Signed-off-by: Michal Privoznik <mprivozn>
    Reviewed-by: John Ferlan <jferlan>

commit 7976d1a5146bc5117e7125f0cbbfe537248dc630
Author:     Michal Privoznik <mprivozn>
AuthorDate: Thu Jun 15 17:05:06 2017 +0200
Commit:     Michal Privoznik <mprivozn>
CommitDate: Tue Jul 11 14:45:15 2017 +0200

    qemuDomainAttachDeviceMknodRecursive: Support file mount points

    https://bugzilla.redhat.com/show_bug.cgi?id=1462060

    Just like in the previous commit, when attaching a file based
    device which has its source living under /dev (that is not a
    device rather than a regular file), calling mknod() is no help.
    We need to:

    1) bind mount device to some temporary location
    2) enter the namespace
    3) move the mount point to desired place
    4) umount it in the parent namespace from the temporary location

    At the same time, the check in qemuDomainNamespaceSetupDisk makes
    no longer sense. Therefore remove it.

    Signed-off-by: Michal Privoznik <mprivozn>
    Reviewed-by: John Ferlan <jferlan>

commit 4f05f188de5de0d5764699ba5231a7d20b42dc9c
Author:     Michal Privoznik <mprivozn>
AuthorDate: Fri Jun 16 13:49:52 2017 +0200
Commit:     Michal Privoznik <mprivozn>
CommitDate: Tue Jul 11 14:45:15 2017 +0200

    qemuDomainCreateDeviceRecursive: Support file mount points

    https://bugzilla.redhat.com/show_bug.cgi?id=1462060

    When building a qemu namespace we might be dealing with bare
    regular files. Files that live under /dev. For instance
    /dev/my_awesome_disk:

      <disk type='file' device='disk'>
        <driver name='qemu' type='qcow2'/>
        <source file='/dev/my_awesome_disk'/>
        <target dev='vdc' bus='virtio'/>
      </disk>

      # qemu-img create -f qcow2 /dev/my_awesome_disk 10M

    So far we were mknod()-ing them which is
    obviously wrong. We need to touch the file and bind mount it to
    the original:

    1) touch /var/run/libvirt/qemu/fedora.dev/my_awesome_disk
    2) mount --bind /dev/my_awesome_disk /var/run/libvirt/qemu/fedora.dev/my_awesome_disk

    Later, when the new /dev is built and replaces original /dev the
    file is going to live at expected location.

    Signed-off-by: Michal Privoznik <mprivozn>
    Reviewed-by: John Ferlan <jferlan>

commit 4fedbac6206ad6c754d936ac39f0b42417b7dbd5
Author:     Michal Privoznik <mprivozn>
AuthorDate: Thu Jun 15 15:50:39 2017 +0200
Commit:     Michal Privoznik <mprivozn>
CommitDate: Tue Jul 11 14:45:15 2017 +0200

    qemuDomainAttachDeviceMknodHelper: Fail on unsupported file type

    Currently, we silently assume that file we are creating in the
    namespace is either a link or a device (character or block one).
    This is not always the case. Therefore instead of doing something
    wrong, claim about unsupported file type.

    Signed-off-by: Michal Privoznik <mprivozn>
    Reviewed-by: John Ferlan <jferlan>

commit 89921f54cd4775909f30cfb56313ed3afd3945ca
Author:     Michal Privoznik <mprivozn>
AuthorDate: Mon Jun 19 10:56:20 2017 +0200
Commit:     Michal Privoznik <mprivozn>
CommitDate: Tue Jul 11 14:45:15 2017 +0200

    qemuDomainCreateDeviceRecursive: Fail on unsupported file type

    Currently, we silently assume that file we are creating in the
    namespace is either a link or a device (character or block one).
    This is not always the case. Therefore instead of doing something
    wrong, claim about unsupported file type.

    Signed-off-by: Michal Privoznik <mprivozn>
    Reviewed-by: John Ferlan <jferlan>

commit 4d8a914be00b5d94b7a5ee8dd3a857d26700592d
Author:     Michal Privoznik <mprivozn>
AuthorDate: Thu Jun 15 15:48:52 2017 +0200
Commit:     Michal Privoznik <mprivozn>
CommitDate: Tue Jul 11 14:45:15 2017 +0200

    qemu: Move preserved mount points path generation into a separate function

    This function is going to be used on other places, so
    instead of copying code we can just call the function.

    Signed-off-by: Michal Privoznik <mprivozn>
    Reviewed-by: John Ferlan <jferlan>

commit 7154917908d9f712a950a2716c4687e57ccb74e7
Author:     Michal Privoznik <mprivozn>
AuthorDate: Tue Jun 20 17:35:54 2017 +0200
Commit:     Michal Privoznik <mprivozn>
CommitDate: Tue Jul 11 14:45:15 2017 +0200

    qemuDomainBuildNamespace: Handle special file mount points

    https://bugzilla.redhat.com/show_bug.cgi?id=1459592

    In 290a00e41d I've tried to fix the process of building a
    qemu namespace when dealing with file mount points. What I
    haven't realized then is that we might be dealing not with just
    regular files but also special files (like sockets). Indeed, try
    the following:

    1) socat unix-listen:/tmp/soket stdio
    2) touch /dev/socket
    3) mount --bind /tmp/socket /dev/socket
    4) virsh start anyDomain

    Problem with my previous approach is that I wasn't creating the
    temporary location (where mount points under /dev are moved) for
    anything but directories and regular files.

    Signed-off-by: Michal Privoznik <mprivozn>
    Reviewed-by: John Ferlan <jferlan>

commit 37ef8763c4ce085d0ea8e7c18b29a9dd27874291
Author:     Michal Privoznik <mprivozn>
AuthorDate: Mon Jun 19 16:43:25 2017 +0200
Commit:     Michal Privoznik <mprivozn>
CommitDate: Tue Jul 11 14:45:15 2017 +0200

    conf: Rename and expose virDomainChrSourceDefPath

    It comes very handy to have source path for chardevs. We already
    have such function: virDomainAuditChardevPath() but it's static
    and has name not suitable for exposing. Moreover, while exposing
    it change its name slightly to virDomainChrSourceDefGetPath.

    Signed-off-by: Michal Privoznik <mprivozn>
    Reviewed-by: John Ferlan <jferlan>


v3.5.0-56-ge93d844b9
Comment 6 Meina Li 2017-11-22 08:03:51 UTC 
Verified it on:
libvirt-3.9.0-2.el7.x86_64
qemu-kvm-rhev-2.10.0-6.el7.x86_64

Verified steps:
Scenario 1 - Attach disk source file in the /dev/ dir directly:
1. Prepare a img in the /dev and prepare the disk xml:
# qemu-img create -f qcow2 /dev/test.img 10M
Formatting '/dev/test.img', fmt=qcow2 size=10485760 cluster_size=65536 lazy_refcounts=off refcount_bits=16
# cat disk.xml
 <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/dev/test.img'/>
      <target dev='vdb' bus='virtio'/>
  </disk>

2. Attach/detach the disk to a running guest:
# virsh attach-device lmn disk.xml 
Device attached successfully
# virsh domblklist lmn
Target     Source
------------------------------------------------
vda        /var/lib/libvirt/images/lmn.qcow2
vdb        /dev/test.img
# virsh detach-device lmn disk.xml 
Device detached successfully

3. Start a guest which the disk has source file with qcow2 format in the /dev dir:
...
 <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/dev/test.img'/>
      <target dev='vdb' bus='virtio'/>
  </disk>
...
# virsh start lmn
Domain lmn started

Scenario 2 -  Attach disk source file in the /dev/shm dir.
1. Prepare a img in the /dev and prepare the disk xml:
# qemu-img create -f qcow2 /dev/shm/test.img 10M
Formatting '/dev/shm/test.img', fmt=qcow2 size=10485760 cluster_size=65536 lazy_refcounts=off refcount_bits=16
# cat disk.xml
 <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/dev/shm/test.img'/>
      <target dev='vdb' bus='virtio'/>
  </disk>

2. Attach/detach the disk to a running guest:
# virsh attach-device lmn disk.xml 
Device attached successfully
# virsh domblklist lmn
Target     Source
------------------------------------------------
vda        /var/lib/libvirt/images/lmn.qcow2
vdb        /dev/shm/test.img
# virsh detach-device lmn disk.xml 
Device detached successfully

3. Start a guest which the disk has source file with qcow2 format in the /dev dir:
...
 <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/dev/shm/test.img'/>
      <target dev='vdb' bus='virtio'/>
  </disk>
...
# virsh start lmn
Domain lmn started


Scenario 3 - Attach disk source file in the /dev/ dir that file is mounted on mount points:
1. Prepare a img in the /dev and prepare the disk xml:
# qemu-img create -f qcow2 /dev/test.img 10M
Formatting '/dev/test.img', fmt=qcow2 size=10485760 cluster_size=65536 lazy_refcounts=off refcount_bits=16
# cat disk.xml
 <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/dev/test.img'/>
      <target dev='vdb' bus='virtio'/>
  </disk>

2. Support file mount points for /dev file:
# touch /tmp/demo.img
# mount --bind /dev/test.img /tmp/demo.img 

3. Attach/detach the disk to a running guest:
# virsh attach-device lmn disk.xml 
Device attached successfully
# virsh domblklist lmn
Target     Source
------------------------------------------------
vda        /var/lib/libvirt/images/lmn.qcow2
vdb        /dev/test.img
# virsh detach-device lmn disk.xml 
Device detached successfully

Scenario 4: Attach disk source file in the /dev/ which has a symlink
1. Prepare a img in the /dev and prepare the disk xml:
# qemu-img create -f qcow2 /dev/test.img 10M
Formatting '/dev/test.img', fmt=qcow2 size=10485760 cluster_size=65536 lazy_refcounts=off refcount_bits=16
# cat disk.xml
 <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/dev/test.img'/>    -->use '/dev/demo.img'is also OK
      <target dev='vdb' bus='virtio'/>
  </disk>

2. Make symlink for /dev file:
# ln -s /dev/test.img /dev/demo.img

3. Attach/detach the disk to a running guest:
# virsh attach-device lmn disk.xml 
Device attached successfully
# virsh domblklist lmn
Target     Source
------------------------------------------------
vda        /var/lib/libvirt/images/lmn.qcow2
vdb        /dev/test.img
# virsh detach-device lmn disk.xml 
Device detached successfully

Scenario 5: Attach disk source file in the /dev/ which has a backing chain image
1. Prepare a img in the /dev and prepare the disk xml:
# qemu-img create -f qcow2 /dev/test.img -b /tmp/demo.img 10M
Formatting '/dev/test.img', fmt=qcow2 size=10485760 backing_file=/tmp/demo.img cluster_size=65536 lazy_refcounts=off refcount_bits=16
 refcount_bits=16
# cat disk.xml
 <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/dev/test.img'/>    
      <target dev='vdb' bus='virtio'/>
  </disk>

2. Attach/detach the disk to a running guest:
# virsh attach-device lmn disk.xml 
Device attached successfully
# virsh domblklist lmn
Target     Source
------------------------------------------------
vda        /var/lib/libvirt/images/lmn.qcow2
vdb        /dev/test.img
# virsh detach-device lmn disk.xml 
Device detached successfully

Move this bug to VERIFIED.
Comment 10 errata-xmlrpc 2018-04-10 10:48:37 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2018:0704
Note You need to log in before you can comment on or make changes to this bug.