Bug 1481624

Summary: After ugprade from 3.6 ngn to 4.1 via yum update migration fails / firewalld has no 16514 enabled
Product: [oVirt] ovirt-node Reporter: Jiri Belka <jbelka>
Component: Installation & UpdateAssignee: Ryan Barry <rbarry>
Status: CLOSED DUPLICATE QA Contact: Huijuan Zhao <huzhao>
Severity: urgent Docs Contact:
Priority: unspecified    
Version: 4.1CC: bugs, cshao, dguo, huzhao, jiawu, pbrilla, qiyuan, rbarry, sbonazzo, weiwang, yaniwang, ycui, yzhao
Target Milestone: ---Flags: cshao: testing_ack?
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-08-15 12:33:05 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Node RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Jiri Belka 2017-08-15 08:37:35 UTC
Description of problem:

I was migrating a VM from 3.6ngn to 4.1ngn on SHE 3.6 env:

...
Thread-166763::ERROR::2017-08-15 10:25:19,274::migration::313::virt.vm::(run) vmId=`4377393e-65b7-42a0-badd-4d650327937f`::Failed to migrate
Traceback (most recent call last):
  File "/usr/share/vdsm/virt/migration.py", line 301, in run
    self._startUnderlyingMigration(time.time())
  File "/usr/share/vdsm/virt/migration.py", line 365, in _startUnderlyingMigration
    self._perform_migration(duri, muri)
  File "/usr/share/vdsm/virt/migration.py", line 406, in _perform_migration
    self._vm._dom.migrateToURI3(duri, params, flags)
  File "/usr/share/vdsm/virt/virdomain.py", line 68, in f
    ret = attr(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/vdsm/libvirtconnection.py", line 124, in wrapper
    ret = f(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/vdsm/utils.py", line 1313, in wrapper
    return func(inst, *args, **kwargs)
  File "/usr/lib64/python2.7/site-packages/libvirt.py", line 1939, in migrateToURI3
    if ret == -1: raise libvirtError ('virDomainMigrateToURI3() failed', dom=self)
libvirtError: operation failed: Failed to connect to remote libvirt URI qemu+tls://dell-r210ii-03.example.com/system: unable to connect to server at 'dell-r210ii-03.example.com:16514': Network is unreachable
...


# telnet dell-r210ii-03.example.com 16514
Trying 10.34.63.222...
telnet: connect to address 10.34.63.222: No route to host
Trying 2620:52:0:223c:d267:e5ff:feee:63ec...
telnet: connect to address 2620:52:0:223c:d267:e5ff:feee:63ec: Network is unreachable

~~ destination host

# iptables -nL | grep '^ACCEPT.*16514'
# firewall-cmd --list-ports ; firewall-cmd --list-services ; firewall-cmd --info-service=libvirt

ssh dhcpv6-client cockpit vdsm libvirt glusterfs
libvirt
  ports: 16509/tcp
  protocols: 
  source-ports: 
  modules: 
  destination: 

^^ see, there is no 16514/tcp for libvirtd/ssl.

~~~ comparison with a clean 4.1ngn

# iptables -nL | grep '^ACCEPT.*16514'
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0            tcp dpt:16514
# firewall-cmd --list-ports ; firewall-cmd --list-services ; firewall-cmd --info-service=libvirt
FirewallD is not running
FirewallD is not running
FirewallD is not running

^^ so why is firewalld not running here but 16514/tcp is enabled in netfilter?

Version-Release number of selected component (if applicable):
rhvh-4.1-0.20170808.0+1

How reproducible:
just happened

Steps to Reproduce:
1. 3.6ngn & 3.6ngn -> 4.1ngn
2. 3.6ngn & 4.1ngn
3. migrate VM running on 3.6ngn to 4.1ngn

Actual results:
migration fails, 16514/tcp not enabled on 4.1ngn

Expected results:
should work

Additional info:

Comment 1 Jiri Belka 2017-08-15 08:41:14 UTC
workaround for now which makes migration working...

iptables -I INPUT -m tcp -p tcp --dport 16514 -j ACCEPT

Comment 2 Ryan Barry 2017-08-15 12:33:05 UTC
This strongly appears to be a duplicate of https://bugzilla.redhat.com/show_bug.cgi?id=1473307

Closing

*** This bug has been marked as a duplicate of bug 1473307 ***