Created attachment 1657830 [details] rhel8-tcp.png Description of problem: Can't import guests from rhel8 kvm host by qemu+tcp connection on rhv4.3 Version-Release number of selected component (if applicable): RHEL8 KVM host: kernel-4.18.0-147.el8.x86_64 libvirt-5.6.0-10.module+el8.1.1+5309+6d656f05.x86_64 qemu-kvm-4.1.0-23.module+el8.1.1+5467+ba2d821b.x86_64 rhv:4.3.7.0-0.1.el7 vdsm-4.30.30-1.el7ev.x86_64 How reproducible: 100% Steps to Reproduce: 1. Edit libvirtd configuration files to enable tcp connection and restart libvirtd service on rhel8 host #vi /etc/sysconfig/libvirtd and remove # for LIBVIRTD_ARGS="--listen" #vi /etc/libvirt/libvirtd.conf and remove # for listen_tcp = 1 and listen_tls = 0 #vi /etc/libvirt/libvirtd.conf and remove # for auth_tcp = "sasl" and change "sasl" to "none" #service libvirtd restart 2. Disable firewalld service and selinux on rhel8 host # service firewalld stop # setenforce 0 3.Log in rhv and import guest from rhel8 as below steps: Open virtual machine option at rhv4.3 web -> click import button -> choose source as KVM(via libvirt)->input URL:qemu+tcp://ip/system, username/password->click 'Load' button 4.Failed to load guest from rhel8 host with info'Failed to communicate with the external provider, see log for additional details.', event log shows 'VDSM NFS command GetVmsNamesFromExternalProviderVDS failed: unable to connect to server at '10.73.224.193:16509': Connection refused'. details pls refer to vdsm.log and sreenshot"rhel8-tcp" Actual results: As above description Expected results: Can import guests from rhel8 host by qemu+tcp connection on rhv4.3 Additional info： 1.Can import guests from rhel7 host by qemu+tcp connection on rhv4.3 2. Still cannot import guests from rhel8 host by qemu+tcp connection on rhv4.3 even if clear all network rules by "iptables -F" 3. Can import guests from rhel8 kvm host by qemu+ssh connection on rhv4.3 3.1 Generate ssh key for vdsm user on rhv-node # sudo -u vdsm ssh-keygen 3.2 Copy vdsms public key to rhel8 host # sudo -u vdsm ssh-copy-id root.224.193 3.3 Login rhv4.3 env, then can load vms from rhel8 host by qemu+ssh on rhv4.3 successfully, pls refer to screenshot"rhel8-ssh"
Created attachment 1657832 [details] rhel8-ssh.png
Created attachment 1657833 [details] vdsm.log
Can you run the following from your VDSM host? sudo -u vdsm virsh -c 'qemu+tcp://10.73.224.193/system' list
Error happened at: 2020-02-05 16:40:34,365+0800 ERROR (jsonrpc/4) [root] error connecting to hypervisor (v2v:194) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/v2v.py", line 192, in get_external_vm_names passwd=password) File "/usr/lib/python2.7/site-packages/vdsm/common/libvirtconnection.py", line 107, in open_connection return function.retry(libvirtOpen, timeout=10, sleep=0.2) File "/usr/lib/python2.7/site-packages/vdsm/common/function.py", line 58, in retry return func() File "/usr/lib64/python2.7/site-packages/libvirt.py", line 104, in openAuth if ret is None:raise libvirtError('virConnectOpenAuth() failed') Could you please try as comment3 and check if your libvirtd is started. I followed your settings in libvirtd sysconf and libvirtd.conf but find libvirtd failed to start: ➜ ~ virsh list error: failed to connect to the hypervisor error: Cannot recv data: Connection reset by peer ➜ ~ virsh -c 'qemu+tcp://localhost/system' list error: failed to connect to the hypervisor error: unable to connect to server at 'localhost:16509': Connection refused ➜ ~ pidof libvirtd ➜ ~
(In reply to Tomáš Golembiovský from comment #3) > Can you run the following from your VDSM host? > > sudo -u vdsm virsh -c 'qemu+tcp://10.73.224.193/system' list Run above command on rhv-node, will show below info # sudo -u vdsm virsh -c 'qemu+tcp://10.73.224.193/system' list error: failed to connect to the hypervisor error: unable to connect to server at '10.73.224.193:16509': Connection refused Hi Hanhan service libvirtd is running normally in my rhel8 host after doing comment3, I don't understand your question, is service libvirtd running is your host after editing libvirtd configuration files as comment0? # systemctl status libvirtd ● libvirtd.service - Virtualization daemon Loaded: loaded (/usr/lib/systemd/system/libvirtd.service; enabled; vendor preset: enabled) Active: active (running) since Wed 2020-02-05 03:02:41 EST; 3h 28min ago Docs: man:libvirtd(8) https://libvirt.org Main PID: 12026 (libvirtd) Tasks: 19 (limit: 32768) Memory: 65.5M CGroup: /system.slice/libvirtd.service ├─ 2360 /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/libexec/libvirt_leases> ├─ 2361 /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/libexec/libvirt_leases> └─12026 /usr/sbin/libvirtd --listen
(In reply to mxie from comment #5) > (In reply to Tomáš Golembiovský from comment #3) > > Can you run the following from your VDSM host? > > > > sudo -u vdsm virsh -c 'qemu+tcp://10.73.224.193/system' list > > Run above command on rhv-node, will show below info > # sudo -u vdsm virsh -c 'qemu+tcp://10.73.224.193/system' list > error: failed to connect to the hypervisor > error: unable to connect to server at '10.73.224.193:16509': Connection > refused > > Hi Hanhan > service libvirtd is running normally in my rhel8 host after doing > comment3, I don't understand your question, is service libvirtd running is > your host after editing libvirtd configuration files as comment0? Yes. I followed the configrations in comment0. Since virsh cannot establish the qemu+tcp connection, I suspect the problem is from libvirt part. > > # systemctl status libvirtd > ● libvirtd.service - Virtualization daemon > Loaded: loaded (/usr/lib/systemd/system/libvirtd.service; enabled; vendor > preset: enabled) > Active: active (running) since Wed 2020-02-05 03:02:41 EST; 3h 28min ago > Docs: man:libvirtd(8) > https://libvirt.org > Main PID: 12026 (libvirtd) > Tasks: 19 (limit: 32768) > Memory: 65.5M > CGroup: /system.slice/libvirtd.service > ├─ 2360 /usr/sbin/dnsmasq > --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro > --dhcp-script=/usr/libexec/libvirt_leases> > ├─ 2361 /usr/sbin/dnsmasq > --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro > --dhcp-script=/usr/libexec/libvirt_leases> > └─12026 /usr/sbin/libvirtd --listen
(In reply to mxie from comment #5) > (In reply to Tomáš Golembiovský from comment #3) > > Can you run the following from your VDSM host? > > > > sudo -u vdsm virsh -c 'qemu+tcp://10.73.224.193/system' list > > Run above command on rhv-node, will show below info > # sudo -u vdsm virsh -c 'qemu+tcp://10.73.224.193/system' list > error: failed to connect to the hypervisor > error: unable to connect to server at '10.73.224.193:16509': Connection > refused > I am still thinking this is some miss-configuration on your RHEL8 hypervisor or your network rather than RHV problem. Can you please make sure the above command (virsh -c 'qemu+tcp://10.73.224.193/system' list) works from the RHEL8 itself and from some other machine on your network?
> Can you please make sure the above command (virsh -c > 'qemu+tcp://10.73.224.193/system' list) works from the RHEL8 itself and from > some other machine on your network? Hi Tomas, Sorry for late reply, I can't reproduce the problem when import guest from rhel8.3 host on rhv4.3 and rhv4.4 now, I just knew the way of configuration of tcp has changed since libvirt-5.6.0, I think the bug can be closed now. Packages: libvirt-6.4.0-1.module+el8.3.0+6881+88468c00.x86_64 qemu-kvm-5.0.0-0.module+el8.3.0+6620+5d5e1420.x86_64 rhv:4.4.1.2-0.10.el8ev vdsm-4.40.19-1.el8ev.x86_64 Steps: 1.Modify /etc/libvirt/libvirtd.conf to change authentication for tcp # grep -v '#' /etc/libvirt/libvirtd.conf auth_tcp = "none" 2.Stop libvirtd service to start libvirtd-tcp.socket, then start libvirtd service again # systemctl stop libvirtd Warning: Stopping libvirtd.service, but it can still be activated by: libvirtd.socket libvirtd-admin.socket # systemctl start libvirtd-tcp.socket # systemctl start libvirtd 3.Use virsh to connect qemu:///system by tcp on local host, can see qemu+tcp works from the RHEL8.3 itself # virsh -c qemu+tcp://10.73.72.77/system Welcome to virsh, the virtualization interactive terminal. Type: 'help' for help with commands 'quit' to quit virsh # 4.Log into rhv4.4 to import guest from above rhel8.3 host with below steps, then can load guests from rhel8 host successfully Open virtual machine option at rhv4.4 web -> click import button -> choose source as KVM(via libvirt)->input URL:qemu+tcp://10.73.72.77/system, correct username and password->click 'Load' button
Ok, thanks. I am closing the bug.