Note: This bug is displayed in read-only format because
the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
(In reply to xiywang from comment #0)
[...]
> Actual results:
> Error message displayed after step 4 is executed:
> migrate: Migration disabled: vhost-user backend lacks
> VHOST_USER_PROTOCOL_F_LOG_SHMFD feature.
>
> Expected results:
> migration works without error
>
> Additional info:
> I've tested migration with upstream dpdk and ovs, it works fine.
> I'm wondering if downstream gonna support migration or not. If not, please
> feel free to close this bug.
I assume by "upstream dpdk and ovs" you mean "built from upstream git repo recently". In that case yes, vhost-user live migration support was just added in January and will be part of the next DPDK version (16.04) which should be released in the next few weeks.
So the answer basically is: "when openvswitch-dpdk is updated to use dpdk >= 16.04", but as to when that is going to happen I dont have an answer at the moment.
(In reply to Panu Matilainen from comment #2)
> (In reply to xiywang from comment #0)
> [...]
> > Actual results:
> > Error message displayed after step 4 is executed:
> > migrate: Migration disabled: vhost-user backend lacks
> > VHOST_USER_PROTOCOL_F_LOG_SHMFD feature.
> >
> > Expected results:
> > migration works without error
> >
> > Additional info:
> > I've tested migration with upstream dpdk and ovs, it works fine.
> > I'm wondering if downstream gonna support migration or not. If not, please
> > feel free to close this bug.
>
> I assume by "upstream dpdk and ovs" you mean "built from upstream git repo
> recently". In that case yes, vhost-user live migration support was just
> added in January and will be part of the next DPDK version (16.04) which
> should be released in the next few weeks.
>
> So the answer basically is: "when openvswitch-dpdk is updated to use dpdk >=
> 16.04", but as to when that is going to happen I dont have an answer at the
> moment.
OK, we can skip the "when" part of this question. But I get the information (correct me if I'm wrong), that we're going to use dpdk >= 16.04 in the future eventually and support migration, am I right? I just need to get the message that whether we will support live migration, no matter when.
(In reply to xiywang from comment #3)
> OK, we can skip the "when" part of this question. But I get the information
> (correct me if I'm wrong), that we're going to use dpdk >= 16.04 in the
> future eventually and support migration, am I right? I just need to get the
> message that whether we will support live migration, no matter when.
Some day in some product version, sure.
As for the current RHEL-7 based offerings, I dont think anybody knows what the precise plan is. OVS 2.5 is a LTS release so we'll probably be sticking with that for a good while, and DPDK 2.2 is the officially blessed DPDK version with it. DPDK 16.04 seems to work fine with OVS 2.5 but its not officially supported upstream, at least yet. Its of course not even released yet so...
Flavio (cc'd), thoughts/comments?
Well, it's WIP in upstream yet as already said and it is too soon to say anything else.
I don't know why this bug was opened since vhost-user migration is not supported in any official product. There is no customer case attached as well and it doesn't look like a RFE either.
Am I missing something?
Thanks,
fbl
Description of problem: If use vhostuser to boot a guest, then migration fails with error message displayed. Version-Release number of selected component (if applicable): guest and host kernel: 3.10.0-364.rt56.241.el7.x86_64 host qemu: qemu-kvm-rhev-2.5.0-3.el7.x86_64 ovs-dpdk: openvswitch-dpdk-2.5.0-3.el7.x86_64 How reproducible: 100% Steps to Reproduce: 1. setup vhostuser environment with script below: #!/bin/sh HUGETLB_PATH="/mnt/hugetlbfs/" eth_model="X540-AT2" # mount hugetlbfs if [ ! -d "$HUGETLB_PATH" ]; then mkdir $HUGETLB_PATH fi PAGESIZE_1G=`cat /proc/cmdline | grep "default_hugepagesz=1G"` if [ ! -z "$PAGESIZE_1G" ]; then mount -t hugetlbfs hugetlbfs $HUGETLB_PATH -o size=16G else PAGESIZE_2M=`cat /proc/cmdline | grep "default_hugepagesz=2M"` if [ ! -z "$PAGESIZE_2M" ]; then mount -t hugetlbfs -o pagesize=2048K none $HUGETLB_PATH else echo "please add hugepage setting in /boot/grub2/grub.cfg first." exit fi fi # load/unload modules for kmod in openvswitch vfio vfio-pci; do if lsmod | grep -q $kmod; then echo "not loading $kmod (already loaded)" else if modprobe -v $kmod; then echo "loaded $kmod module" else echo "Failed to load $kmmod module, exiting" exit 1 fi fi done # bind the devices to dpdk module dpdk_nics=`lspci -D | grep $eth_model | awk '{print $1}'` echo DPDK adapters: $dpdk_nics dpdk_nic_kmod=vfio-pci for nic in $dpdk_nics; do dpdk_nic_bind.py -b $dpdk_nic_kmod $nic done dpdk_nic_bind.py --status # completely remove old ovs configuration echo "remove old ovs configuration" killall ovs-vswitchd killall ovsdb-server sleep 3 rm -rf /var/run/openvswitch/* rm -rf /etc/openvswitch/*db* rm -rf /var/log/openvswitch/* # start database server echo "start new ovs" export DB_SOCK="/var/run/openvswitch/db.sock" mkdir -p /var/run/openvswitch mkdir -p /etc/openvswitch /bin/ovsdb-tool create /etc/openvswitch/conf.db /usr/share/openvswitch/vswitch.ovsschema rm -rf /dev/usvhost-1 /sbin/ovsdb-server -v --remote=punix:$DB_SOCK \ --remote=db:Open_vSwitch,Open_vSwitch,manager_options \ --pidfile --detach || exit 1 # start ovs echo "openvswitch init" /bin/ovs-vsctl --no-wait init screen -dmS ovs sudo su -g qemu -c "umask 002; /sbin/ovs-vswitchd --dpdk -c 0x1 -n 3 --socket-mem 1024,1024 -- unix:$DB_SOCK --pidfile --mlockall --log-file=/var/log/openvswitch/ovs-vswitchd.log" # configure the bridge echo "configure the bridge0" /bin/ovs-vsctl --if-exists del-br ovsbr0 /bin/ovs-vsctl add-br ovsbr0 -- set bridge ovsbr0 datapath_type=netdev /bin/ovs-vsctl add-port ovsbr0 dpdk0 -- set Interface dpdk0 type=dpdk /bin/ovs-vsctl add-port ovsbr0 vhost-user1 -- set Interface vhost-user1 type=dpdkvhostuser echo "configure the bridge1" /bin/ovs-vsctl --if-exists del-br ovsbr1 /bin/ovs-vsctl add-br ovsbr1 -- set bridge ovsbr1 datapath_type=netdev /bin/ovs-vsctl add-port ovsbr1 dpdk1 -- set Interface dpdk1 type=dpdk /bin/ovs-vsctl add-port ovsbr1 vhost-user2 -- set Interface vhost-user2 type=dpdkvhostuser /bin/ovs-vsctl show echo "configure flowtable for bridge0" /bin/ovs-ofctl del-flows ovsbr0 /bin/ovs-ofctl add-flow ovsbr0 "in_port=1,idle_timeout=0 actions=output:2" /bin/ovs-ofctl add-flow ovsbr0 "in_port=2,idle_timeout=0 actions=output:1" echo "configure flowtable for bridge1" /bin/ovs-ofctl del-flows ovsbr1 /bin/ovs-ofctl add-flow ovsbr1 "in_port=1,idle_timeout=0 actions=output:2" /bin/ovs-ofctl add-flow ovsbr1 "in_port=2,idle_timeout=0 actions=output:1" 2. boot a guest in src host /usr/libexec/qemu-kvm -name rhel7.2-rt-355 -machine pc-i440fx-rhel7.2.0 -cpu host,+tsc-deadline,pmu=off -m 4096 -realtime mlock=off -smp 4,sockets=4,cores=1,threads=1 \ -object memory-backend-file,id=ram-node0,prealloc=yes,mem-path=/mnt/hugetlbfs,share=on,size=4096M -numa node,nodeid=0,cpus=0-3,memdev=ram-node0 \ -drive file=/home/rhel7.2-rt-355.qcow2,if=none,id=drive-virtio-disk0,format=qcow2,snapshot=off -device virtio-blk-pci,drive=drive-virtio-disk0,id=virtio-disk0 \ -netdev tap,id=hostnet0,vhost=on -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:a1:d0:5f \ -chardev socket,id=charnet1,path=/var/run/openvswitch/vhost-user1 -netdev type=vhost-user,id=hostnet1,chardev=charnet1 -device virtio-net-pci,netdev=hostnet1,id=net1,mac=52:54:00:b2:02:01 -chardev socket,id=charnet2,path=/var/run/openvswitch/vhost-user2 -netdev type=vhost-user,id=hostnet2,chardev=charnet2 -device virtio-net-pci,netdev=hostnet2,id=net2,mac=52:54:00:b2:02:02 \ -monitor stdio -device qxl-vga,id=video0 -serial unix:/tmp/console,server,nowait -vnc :1 -spice port=5900,disable-ticketing 3. boot a guest in the same host (as dst host), with 'incoming' 4. do local migration, failed (qemu) migrate -d tcp:10.73.64.240:4444 Actual results: Error message displayed after step 4 is executed: migrate: Migration disabled: vhost-user backend lacks VHOST_USER_PROTOCOL_F_LOG_SHMFD feature. Expected results: migration works without error Additional info: I've tested migration with upstream dpdk and ovs, it works fine. I'm wondering if downstream gonna support migration or not. If not, please feel free to close this bug.