Bug 1735609
| Summary: | Incremental backup: handle snapshots with bitmaps | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux Advanced Virtualization | Reporter: | aihua liang <aliang> |
| Component: | qemu-kvm | Assignee: | John Snow <jsnow> |
| Status: | CLOSED CURRENTRELEASE | QA Contact: | aihua liang <aliang> |
| Severity: | high | Docs Contact: | |
| Priority: | high | ||
| Version: | 8.1 | CC: | coli, ddepaula, eblake, jferlan, jinzhao, jsnow, juzhang, mtessun, ngu, qzhang, rbalakri, virt-maint, xutian |
| Target Milestone: | rc | ||
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | qemu-kvm-4.2.0-1.module+el8.2.0+4793+b09dd2fb | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2019-12-16 03:19:15 UTC | Type: | Enhancement |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Bug Depends On: | |||
| Bug Blocks: | 1758964, 1771318 | ||
Re-assign it to Kevin to check what happened on the patch? (In reply to aihua liang from comment #0) > > 5.Merge base.bitmap0 to sn1.bitmap0 > { "execute": > "block-dirty-bitmap-merge","arguments":{"node":"sn1","bitmaps":[{"node": > "drive_image1","name":"bitmap0"}],"target":"bitmap0"}} > > Actual results: > After step5, bitmap merge failed with info: > {"error": {"class": "GenericError", "desc": "Bitmaps are incompatible and > can't be merged"}} > > Expected results: > Bitmap merge with external bitmap can success. > > Additional info: > Commit info: > commit eff0829b0710d60c1846eb99baeb47b2478a414a > Author: Vladimir Sementsov-Ogievskiy <vsementsov> > Date: Tue May 28 19:33:31 2019 -0400 > > qapi: support external bitmaps in block-dirty-bitmap-merge > > Add new optional parameter making possible to merge bitmaps from > different nodes. It is needed to maintain external snapshots during > incremental backup chain history. > > Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov> > Reviewed-by: John Snow <jsnow> > Message-id: 20190517152111.206494-2-vsementsov > Signed-off-by: John Snow <jsnow> > > Have confirmed with John by mail, this function may not work normally until > 4.2, file this bug to track its progress according to his suggestion. > > > Issue1. Bitmap merge with external bitmap > > I'm checking bitmap-merge with external bitmap with upstream qemu, > > but it failed with info: > > bitmaps and external snapshots aren't 100% gauranteed to work together > yet, we're working on this for 4.2 > > I'll have more info soon. If you want to file a tracker BZ that would be > useful. Hi, - eff0829b0710d60c1846eb99baeb47b2478a414a should be in 4.1 and present in our 4.2-based build. We should have the ability to merge bitmaps *across nodes* now. I expect that to work. - c5b40c1f9cd should allow any two bitmaps of the same size to be merged together. I expect that to work, too. - Several commits in the 4.2 time span enabled commits, and I expected those to work. ... Is it still breaking at step #5? With the exact same error message? (In reply to John Snow from comment #12) > (In reply to aihua liang from comment #0) > > > > > 5.Merge base.bitmap0 to sn1.bitmap0 > > { "execute": > > "block-dirty-bitmap-merge","arguments":{"node":"sn1","bitmaps":[{"node": > > "drive_image1","name":"bitmap0"}],"target":"bitmap0"}} > > > > Actual results: > > After step5, bitmap merge failed with info: > > {"error": {"class": "GenericError", "desc": "Bitmaps are incompatible and > > can't be merged"}} > > > > Expected results: > > Bitmap merge with external bitmap can success. > > > > Additional info: > > Commit info: > > commit eff0829b0710d60c1846eb99baeb47b2478a414a > > Author: Vladimir Sementsov-Ogievskiy <vsementsov> > > Date: Tue May 28 19:33:31 2019 -0400 > > > > qapi: support external bitmaps in block-dirty-bitmap-merge > > > > Add new optional parameter making possible to merge bitmaps from > > different nodes. It is needed to maintain external snapshots during > > incremental backup chain history. > > > > Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov> > > Reviewed-by: John Snow <jsnow> > > Message-id: 20190517152111.206494-2-vsementsov > > Signed-off-by: John Snow <jsnow> > > > > Have confirmed with John by mail, this function may not work normally until > > 4.2, file this bug to track its progress according to his suggestion. > > > > > Issue1. Bitmap merge with external bitmap > > > I'm checking bitmap-merge with external bitmap with upstream qemu, > > > but it failed with info: > > > > bitmaps and external snapshots aren't 100% gauranteed to work together > > yet, we're working on this for 4.2 > > > > I'll have more info soon. If you want to file a tracker BZ that would be > > useful. > > > Hi, > > - eff0829b0710d60c1846eb99baeb47b2478a414a should be in 4.1 and present in > our 4.2-based build. We should have the ability to merge bitmaps *across > nodes* now. I expect that to work. > - c5b40c1f9cd should allow any two bitmaps of the same size to be merged > together. I expect that to work, too. > - Several commits in the 4.2 time span enabled commits, and I expected those > to work. > > > ... Is it still breaking at step #5? With the exact same error message? John, External bitmap works ok on qemu-kvm-4.2.0-2.module+el8.2.0+5135+ed3b2489.x86_64, will set it closed as current release, for other new features, I'm still working on it. As it works on with -blockdev + qemu-kvm-4.2.0-2.module+el8.2.0+5135+ed3b2489.x86_64, close it as currentrelease. Clearing NEEDINFO, thank you aliang. Please reach out to me again as-needed if I missed something during the November-December timeframe. --js |
Description of problem: Failed to merge external bitmap by block-dirty-bitmap-merge Version-Release number of selected component (if applicable): qemu-kvm version:v4.1.0-rc2 kernel version: 4.18.0-122.el8.x86_64 How reproducible: 100% Steps to Reproduce: 1.Start guest with qemu cmds: /usr/local/bin/qemu-system-x86_64 \ -name 'avocado-vt-vm1' \ -machine q35 \ -nodefaults \ -device VGA,bus=pcie.0,addr=0x1 \ -chardev socket,id=qmp_id_qmpmonitor1,path=/var/tmp/monitor-qmpmonitor1-20190602-221944-MrlxVzia,server,nowait \ -mon chardev=qmp_id_qmpmonitor1,mode=control \ -chardev socket,id=qmp_id_catch_monitor,path=/var/tmp/monitor-catch_monitor-20190602-221944-MrlxVzia,server,nowait \ -mon chardev=qmp_id_catch_monitor,mode=control \ -device pvpanic,ioport=0x505,id=idn20piu \ -chardev socket,id=serial_id_serial0,path=/var/tmp/serial-serial0-20190602-221944-MrlxVzia,server,nowait \ -device isa-serial,chardev=serial_id_serial0 \ -chardev socket,id=seabioslog_id_20190602-221944-MrlxVzia,path=/var/tmp/seabios-20190602-221944-MrlxVzia,server,nowait \ -device isa-debugcon,chardev=seabioslog_id_20190602-221944-MrlxVzia,iobase=0x402 \ -device pcie-root-port,id=pcie.0-root-port-2,slot=2,chassis=2,addr=0x2,bus=pcie.0 \ -device qemu-xhci,id=usb1,bus=pcie.0-root-port-2,addr=0x0 \ -device pcie-root-port,id=pcie.0-root-port-3,slot=3,chassis=3,addr=0x3,bus=pcie.0 \ -device virtio-scsi-pci,id=virtio_scsi_pci0,bus=pcie.0-root-port-3,addr=0x0 \ -blockdev driver=file,node-name=file_node,filename=/home/kvm_autotest_root/images/rhel77-64-virtio-scsi.qcow2.backup \ -blockdev driver=qcow2,node-name=drive_image1,file=file_node \ -device scsi-hd,id=image1,drive=drive_image1 \ -device pcie-root-port,id=pcie.0-root-port-5,slot=5,chassis=5,addr=0x5,bus=pcie.0 \ -device virtio-scsi-pci,id=virtio_scsi_pci1,bus=pcie.0-root-port-5,addr=0x0 \ -blockdev driver=file,filename=/home/data.qcow2,node-name=file_data \ -blockdev driver=qcow2,node-name=drive_data1,file=file_data \ -device scsi-hd,id=data1,drive=drive_data1 \ -drive id=drive_cd1,if=none,snapshot=off,aio=threads,cache=none,media=cdrom,file=/home/kvm_autotest_root/iso/linux/RHEL8.1.0-BaseOS-x86_64.iso \ -device ide-cd,id=cd1,drive=drive_cd1,bus=ide.0,unit=0 \ -device pcie-root-port,id=pcie.0-root-port-4,slot=4,chassis=4,addr=0x4,bus=pcie.0 \ -device virtio-net-pci,mac=9a:33:34:35:36:37,id=idj01pFr,vectors=4,netdev=idMgbx8B,bus=pcie.0-root-port-4,addr=0x0 \ -netdev tap,id=idMgbx8B,vhost=on \ -m 4096 \ -smp 4,maxcpus=4,cores=2,threads=1,sockets=2 \ -cpu 'Skylake-Client',hv_stimer,hv_synic,hv_vpindex,hv_reset,hv_relaxed,hv_spinlocks=0x1fff,hv_vapic,hv_time,hv-tlbflush,+kvm_pv_unhalt \ -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 \ -vnc :0 \ -rtc base=utc,clock=host,driftfix=slew \ -boot order=cdn,once=c,menu=off,strict=off \ -enable-kvm \ -monitor stdio \ -qmp tcp:0:3000,server,nowait \ 2.Add a bitmap to drive_image1 and check bitmap info {"execute":"block-dirty-bitmap-add","arguments":{"node":"drive_image1","name":"bitmap0"}} {"execute":"query-block"} "qdev": "image1", "dirty-bitmaps": [{"name": "bitmap0", "recording": true, "persistent": false, "busy": false, "status": "active", "granularity": 65536, "count": 131072}] 3.Create a snapshot target and do live snapshot. #create snapshot target node {'execute':'blockdev-create','arguments':{'options': {'driver':'file','filename':'/root/sn$i','size':21474836480},'job-id':'job1'}} {'execute':'blockdev-add','arguments':{'driver':'file','node-name':'drive_sn$i','filename':'/root/sn$i'}} {'execute':'blockdev-create','arguments':{'options': {'cluster-size':4096, 'driver': 'qcow2','file':'drive_sn$i','size':21474836480},'job-id':'job2'}} {'execute':'blockdev-add','arguments':{'driver':'qcow2','node-name':'sn$i','file':'drive_sn$i'}} {'execute':'job-dismiss','arguments':{'id':'job1'}} {'execute':'job-dismiss','arguments':{'id':'job2'}} #do live snapshot {"execute":"blockdev-snapshot","arguments":{"node":"drive_image1","overlay":"sn1"}} 4.Add a bitmap to sn1 {"execute":"block-dirty-bitmap-add","arguments":{"node":"sn1","name":"bitmap0"}} {"execute":"query-block"} "qdev": "image1", "dirty-bitmaps": [{"name": "bitmap0", "recording": true, "persistent": false, "busy": false, "status": "active", "granularity": 4096, "count": 16384}] 5.Merge base.bitmap0 to sn1.bitmap0 { "execute": "block-dirty-bitmap-merge","arguments":{"node":"sn1","bitmaps":[{"node":"drive_image1","name":"bitmap0"}],"target":"bitmap0"}} Actual results: After step5, bitmap merge failed with info: {"error": {"class": "GenericError", "desc": "Bitmaps are incompatible and can't be merged"}} Expected results: Bitmap merge with external bitmap can success. Additional info: Commit info: commit eff0829b0710d60c1846eb99baeb47b2478a414a Author: Vladimir Sementsov-Ogievskiy <vsementsov> Date: Tue May 28 19:33:31 2019 -0400 qapi: support external bitmaps in block-dirty-bitmap-merge Add new optional parameter making possible to merge bitmaps from different nodes. It is needed to maintain external snapshots during incremental backup chain history. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov> Reviewed-by: John Snow <jsnow> Message-id: 20190517152111.206494-2-vsementsov Signed-off-by: John Snow <jsnow> Have confirmed with John by mail, this function may not work normally until 4.2, file this bug to track its progress according to his suggestion. > Issue1. Bitmap merge with external bitmap > I'm checking bitmap-merge with external bitmap with upstream qemu, > but it failed with info: bitmaps and external snapshots aren't 100% gauranteed to work together yet, we're working on this for 4.2 I'll have more info soon. If you want to file a tracker BZ that would be useful.