Description of problem: "Block format 'nbd' used by node '#block084' does not support reopening files" error when do active commit Version-Release number of selected component (if applicable): qemu-kvm-rhev-2.9.0-14.el7.x86_64 How reproducible: 100% Steps to Reproduce: 1. { "execute": "qmp_capabilities" } {"return": {}} { "execute": "blockdev-snapshot-sync", "arguments": { "device": "drive_image1", "snapshot-file": "/root/sn1", "format": "qcow2" } } {"return": {}} { "execute": "blockdev-snapshot-sync", "arguments": { "device": "drive_image1", "snapshot-file": "/root/sn2", "format": "qcow2" } } {"return": {}} { "execute": "blockdev-snapshot-sync", "arguments": { "device": "drive_image1", "snapshot-file": "/root/sn3", "format": "qcow2" } } {"return": {}} { "execute": "block-commit", "arguments": { "device": "drive_image1", "top": "/root/sn3", "speed": 1000000000 } } {"error": {"class": "GenericError", "desc": "Block format 'nbd' used by node '#block084' does not support reopening files"}} pass if run { "execute": "block-commit", "arguments": { "device": "drive_image1", "top": "/root/sn2", "speed": 1000000000 } } cmd: /usr/libexec/qemu-kvm \ -S \ -name 'avocado-vt-vm1' \ -sandbox off \ -machine pc \ -nodefaults \ -vga std \ -chardev socket,id=qmp_id_qmpmonitor1,path=/tmp/monitor,server,nowait \ -mon chardev=qmp_id_qmpmonitor1,mode=control \ -device pvpanic,ioport=0x505,id=id4FDckd \ -chardev socket,id=serial_id_serial0,path=/tmp/serial,server,nowait \ -device isa-serial,chardev=serial_id_serial0 \ -chardev socket,id=seabioslog_id_20160129-133009-OkYQqhNt,path=/tmp/seabios-20160129-133009-OkYQqhNt,server,nowait \ -device isa-debugcon,chardev=seabioslog_id_20160129-133009-OkYQqhNt,iobase=0x402 \ -device ich9-usb-ehci1,id=usb1,addr=1d.7,multifunction=on,bus=pci.0 \ -device ich9-usb-uhci1,id=usb1.0,multifunction=on,masterbus=usb1.0,addr=1d.0,firstport=0,bus=pci.0 \ -device ich9-usb-uhci2,id=usb1.1,multifunction=on,masterbus=usb1.0,addr=1d.2,firstport=2,bus=pci.0 \ -device ich9-usb-uhci3,id=usb1.2,multifunction=on,masterbus=usb1.0,addr=1d.4,firstport=4,bus=pci.0 \ -drive id=drive_image1,if=none,cache=none,snapshot=off,aio=native,format=qcow2,file=nbd://10.73.196.127:9000 \ -device virtio-blk-pci,id=image1,drive=drive_image1,bootindex=0,bus=pci.0,addr=03 \ -device virtio-net-pci,mac=9a:0d:0e:0f:10:11,id=idKK4Mdq,vectors=4,netdev=idgjGCvt,bus=pci.0,addr=05 \ -netdev tap,id=idgjGCvt,vhost=on,script=/etc/qemu-ifup \ -m 2048 \ -smp 2,maxcpus=2,cores=2,threads=1,sockets=1 \ -cpu Westmere \ -vnc :2 \ -rtc base=localtime,clock=host,driftfix=slew \ -boot order=cdn,once=c,menu=off,strict=off \ -no-kvm \ -monitor stdio 2. 3. Actual results: Expected results: Additional info:
Live commit has always required the ability to convert a backing file from read-only to read-write (if the backing file is not already read-write). Specifying NBD as a backing file when starting qemu defaults to using NBD read-only, and there is no way to convert NBD from read-only to read-write at run-time. But your use case is creating NBD as a read-write file (the initial active image), then adding snapshots on top: > -drive > id=drive_image1,if=none,cache=none,snapshot=off,aio=native,format=qcow2, > file=nbd://10.73.196.127:9000 \ > -device > virtio-blk-pci,id=image1,drive=drive_image1,bootindex=0,bus=pci.0,addr=03 \ Where it gets interesting is that you are showing a difference between live commit (merge the active image to the base NBD file, then make the base NBD file active again): > { "execute": "block-commit", "arguments": { "device": "drive_image1", "top": > "/root/sn3", "speed": 1000000000 } } > {"error": {"class": "GenericError", "desc": "Block format 'nbd' used by node > '#block084' does not support reopening files"}} > from intermediate commit (shrinking the backing chain, but keeping /root/sn3 as the live image): > > pass if run > { "execute": "block-commit", "arguments": { "device": "drive_image1", "top": > "/root/sn2", "speed": 1000000000 } } > So you may have indeed discovered an oddity in the qemu code for live-commit (both versions of commit SHOULD be able to succeed, if the NBD image is indeed read-write). I'll have to investigate further.
reproduce with qemu-kvm-rhev-2.9.0-2.el7.x86_64, not regression
For my own notes, pared down reproducer: window 1: qemu-img create -f qcow2 bar 1m qemu-nbd -t -f raw bar window 2: ./x86_64-softmmu/qemu-system-x86_64 -nographic -nodefaults -qmp stdio \ -drive id=drive-image1,if=none,format=qcow2,file=nbd://localhost:10809 \ -device virtio-blk-pci,id=image1,drive=drive-image1 {'execute':'qmp_capabilities'} {'execute':'blockdev-snapshot-sync','arguments':{'device':'drive-image1', 'snapshot-file':'bar1','format':'qcow2'}} {'execute':'blockdev-snapshot-sync','arguments':{'device':'drive-image1', 'snapshot-file':'bar2','format':'qcow2'}} {'execute':'blockdev-snapshot-sync','arguments':{'device':'drive-image1', 'snapshot-file':'bar3','format':'qcow2'}} intermediate works: {'execute':'block-commit','arguments':{'device':'drive-image1','top':'bar2'}} active fails: {'execute':'block-commit','arguments':{'device':'drive-image1','top':'bar3'}}
This may need a resolution similar to bug 1529666 where LUKS reports the same error message; it may also need to wait for upstream to revamp block job support.
The following case also hit this error: 1. Prepare a NBD backend image # qemu-img info nbd:10.0.1.8:5000 image: nbd://10.0.1.8:5000 file format: qcow2 virtual size: 2.0G (2147483648 bytes) disk size: unavailable cluster_size: 65536 Format specific information: compat: 1.1 lazy refcounts: false refcount bits: 16 corrupt: false 2. Create snapshot sn1 # qemu-img create -f qcow2 -F qcow2 -b nbd:10.0.1.8:5000 sn1.qcow2 Formatting 'sn1.qcow2', fmt=qcow2 size=2147483648 backing_file=nbd:10.0.1.8:5000 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 3. Create snapshot sn2 # qemu-img create -f qcow2 -F qcow2 -b sn1.qcow2 sn2.qcow2 Formatting 'sn2.qcow2', fmt=qcow2 size=2147483648 backing_file=sn1.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 4. Commit sn2 to base # qemu-img commit -f qcow2 -b nbd://10.0.1.8:5000 sn2.qcow2 qemu-img: Block format 'nbd' used by node '#block474' does not support reopening files
Test on qemu-kvm-4.2.0-4.module+el8.2.0+5220+e82621dc, don't hit this issue any more. Set bug's status to "Closed"->"CURRENTRELEASE" Test Steps: 1. Expose data disk qemu-nbd -t -p 9000 /home/data.qcow2 2. Start guest with qemu cmds: /usr/libexec/qemu-kvm \ -name 'avocado-vt-vm1' \ -sandbox on \ -machine q35 \ -nodefaults \ -device VGA,bus=pcie.0,addr=0x1 \ -m 14336 \ -smp 16,maxcpus=16,cores=8,threads=1,dies=1,sockets=2 \ -cpu 'EPYC',+kvm_pv_unhalt \ -chardev socket,id=qmp_id_qmpmonitor1,path=/var/tmp/monitor-qmpmonitor1-20191219-050834-hFzNWwvu,server,nowait \ -mon chardev=qmp_id_qmpmonitor1,mode=control \ -chardev socket,id=qmp_id_catch_monitor,path=/var/tmp/monitor-catch_monitor-20191219-050834-hFzNWwvu,server,nowait \ -mon chardev=qmp_id_catch_monitor,mode=control \ -device pvpanic,ioport=0x505,id=idPQPIm8 \ -chardev socket,id=chardev_serial0,path=/var/tmp/serial-serial0-20191219-050834-hFzNWwvu,server,nowait \ -device isa-serial,id=serial0,chardev=chardev_serial0 \ -chardev socket,id=seabioslog_id_20191219-050834-hFzNWwvu,path=/var/tmp/seabios-20191219-050834-hFzNWwvu,server,nowait \ -device isa-debugcon,chardev=seabioslog_id_20191219-050834-hFzNWwvu,iobase=0x402 \ -object iothread,id=iothread0 \ -object iothread,id=iothread1 \ -device pcie-root-port,id=pcie.0-root-port-2,slot=2,chassis=2,addr=0x2,bus=pcie.0,multifunction=on \ -device qemu-xhci,id=usb1,bus=pcie.0-root-port-2,addr=0x0 \ -device pcie-root-port,id=pcie.0-root-port-3,slot=3,chassis=3,addr=0x2.0x1 \ -device virtio-scsi-pci,id=virtio_scsi_pci0,bus=pcie.0-root-port-3,addr=0x0 \ -blockdev node-name=file_image1,driver=file,aio=threads,filename=/home/kvm_autotest_root/images/rhel820-64-virtio-scsi.qcow2,cache.direct=on,cache.no-flush=off \ -blockdev node-name=drive_image1,driver=qcow2,cache.direct=on,cache.no-flush=off,file=file_image1 \ -device scsi-hd,id=image1,drive=drive_image1,write-cache=on \ -device pcie-root-port,id=pcie.0-root-port-4,slot=4,chassis=4,addr=0x2.0x2 \ -blockdev node-name=file_data1,driver=nbd,server.host=10.73.196.71,server.port=9000,server.type=inet,cache.direct=on,cache.no-flush=off \ -blockdev node-name=drive_data1,driver=raw,cache.direct=on,cache.no-flush=off,file=file_data1 \ -device virtio-blk-pci,id=data1,drive=drive_data1,write-cache=on,bus=pcie.0-root-port-4,iothread=iothread1 \ -device pcie-root-port,id=pcie.0-root-port-5,slot=5,chassis=5,addr=0x2.0x3 \ -device virtio-net-pci,mac=9a:aa:82:61:c1:00,id=idmt8SOs,netdev=idsyxRmM,bus=pcie.0-root-port-5,addr=0x0 \ -netdev tap,id=idsyxRmM,vhost=on \ -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 \ -vnc :1 \ -rtc base=utc,clock=host,driftfix=slew \ -boot menu=off,order=cdn,once=c,strict=off \ -enable-kvm \ -device pcie-root-port,id=pcie_extra_root_port_0,slot=6,chassis=6,addr=0x2.0x4 \ -monitor stdio \ -qmp tcp:0:3000,server,nowait \ 3. Create snapshot target in advance. {'execute':'blockdev-create','arguments':{'options': {'driver':'file','filename':'/root/sn1','size':2147483648},'job-id':'job1'}} {'execute':'blockdev-add','arguments':{'driver':'file','node-name':'drive_sn1','filename':'/root/sn1'}} {'execute':'blockdev-create','arguments':{'options': {'driver': 'qcow2','file':'drive_sn1','size':2147483648,'backing-file':'json:{"driver":"raw","file":{"driver":"nbd","server.host":"10.73.196.71","server.port":"9000","server.type":"inet"}}','backing-fmt':'raw'},'job-id':'job2'}} {'execute':'blockdev-add','arguments':{'driver':'qcow2','node-name':'sn1','file':'drive_sn1','backing':null}} 4. Do snapshot {'execute':'blockdev-snapshot','arguments':{'node':'drive_data1','overlay':'sn1'}} 5. In guest, dd a new file (guest)# dd if=/dev/urandom of=test bs=1M count=1000 6. Do block commit {'execute': 'block-commit', 'arguments': { 'device': 'sn1','job-id':'j1'}} {"timestamp": {"seconds": 1578389387, "microseconds": 486817}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "j1"}} {"timestamp": {"seconds": 1578389387, "microseconds": 486891}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "j1"}} {"return": {}} {"timestamp": {"seconds": 1578389391, "microseconds": 447252}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "j1"}} {"timestamp": {"seconds": 1578389391, "microseconds": 447372}, "event": "BLOCK_JOB_READY", "data": {"device": "j1", "len": 1150943232, "offset": 1150943232, "speed": 0, "type": "commit"}} 7. Complete the block job { "execute": "block-job-complete", "arguments": { "device": "j1" } } {"return": {}} {"timestamp": {"seconds": 1578389405, "microseconds": 678954}, "event": "JOB_STATUS_CHANGE", "data": {"status": "waiting", "id": "j1"}} {"timestamp": {"seconds": 1578389405, "microseconds": 679022}, "event": "JOB_STATUS_CHANGE", "data": {"status": "pending", "id": "j1"}} {"timestamp": {"seconds": 1578389405, "microseconds": 679154}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "j1", "len": 1150943232, "offset": 1150943232, "speed": 0, "type": "commit"}} {"timestamp": {"seconds": 1578389405, "microseconds": 679230}, "event": "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "j1"}} {"timestamp": {"seconds": 1578389405, "microseconds": 679282}, "event": "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "j1"}}