RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1019237 - qemu live migration port conflicts with other users of ephemeral port(s)
Summary: qemu live migration port conflicts with other users of ephemeral port(s)
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: libvirt
Version: 7.0
Hardware: Unspecified
OS: Linux
urgent
high
Target Milestone: rc
: ---
Assignee: Jiri Denemark
QA Contact: Virtualization Bugs
URL:
Whiteboard:
Depends On:
Blocks: 1018178 1018383
TreeView+ depends on / blocked
 
Reported: 2013-10-15 10:54 UTC by Daniel Berrangé
Modified: 2016-06-20 17:18 UTC (History)
24 users (show)

Fixed In Version: libvirt-1.1.1-12.el7
Doc Type: Bug Fix
Doc Text:
Prior to this update, migrating a virtual machine failed when the libvirtd service used a transmission control protocol (TCP) port that was already in use. Now, it is possible to predefine a custom migration TCP port range in case the default port is in use. In addition, libvirtd now ensures that the port it chooses from the custom range is not used by another process.
Clone Of: 1018695
Environment:
Last Closed: 2014-06-13 09:51:13 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)

Comment 2 Jiri Denemark 2013-10-18 14:44:29 UTC 
This is now fixed upstream by v1.1.3-188-g0196845 and v1.1.3-189-ge3ef20d:

commit 0196845d3abd0d914cf11f7ad6c19df8b47c32ed
Author: Wang Yufei <james.wangyufei>
Date:   Fri Oct 11 11:27:13 2013 +0800

    qemu: Avoid assigning unavailable migration ports
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1019053
    
    When we migrate vms concurrently, there's a chance that libvirtd on
    destination assigns the same port for different migrations, which will
    lead to migration failure during prepare phase on destination. So we use
    virPortAllocator here to solve the problem.
    
    Signed-off-by: Wang Yufei <james.wangyufei>
    Signed-off-by: Jiri Denemark <jdenemar>

commit e3ef20d7f7fee595ac4fc6094e04b7d65ee0583a
Author: Jiri Denemark <jdenemar>
Date:   Tue Oct 15 15:26:52 2013 +0200

    qemu: Make migration port range configurable
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1019053
Comment 3 Jiri Denemark 2013-10-18 21:29:10 UTC 
One more patch is needed to fully support configurable migration ports:

commit d9be5a7157515eeae99379e9544c34b34c5e5198
Author: Michal Privoznik <mprivozn>
Date:   Fri Oct 18 18:28:14 2013 +0200

    qemu: Fix augeas support for migration ports
    
    Commit e3ef20d7 allows user to configure migration ports range via
    qemu.conf. However, it forgot to update augeas definition file and
    even the test data was malicious.
    
    Signed-off-by: Michal Privoznik <mprivozn>
Comment 6 zhe peng 2013-10-25 10:55:58 UTC 
I can reproduce this with libvirt-1.1.1-9.el7
will get error:
2013-10-25 10:38:24.397+0000: 16830: debug : virCommandHandshakeChild:388 : Handshake with parent is done
char device redirected to /dev/pts/4 (label charserial0)
qemu-kvm: -incoming tcp:[::]:49179: Failed to bind socket: Address already in use

verify with build :  
libvirt-1.1.1-10.el7
glusterfs-3.4.0.36rhs-1.el7.x86_64

step:
1: prepare gluster env. and migration env.
2: create a guest on source with xml:
....
 <disk type='network' device='disk'>
      <driver name='qemu' type='raw' cache='none'/>
      <source protocol='gluster' name='gluster-vol1/rh6.img'>
        <host name='10.66.82.251' transport='rdma'/>
      </source>
      <target dev='hda' bus='ide'/>
      <alias name='ide0-0-0'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
....
3: on the destination host run cmd:
#nc -l -p 49152
4: do migration on source:
# virsh migrate --live rhel6 qemu+ssh://10.66.106.25/system --verbose --unsafe
root.106.25's password: 
Migration: [100 %]

no error occured. move to verified.
Comment 7 Jiri Denemark 2013-11-06 14:16:13 UTC 
Cancelled or failed migrations leave migration ports reserved on destination which can easily lead in migration range to be exhausted. The following upstream commit fixes this issue:

commit c92ca769af2bacefdd451802d7eb1adac5e6597c
Author: Zeng Junliang <zengjunliang>
Date:   Wed Nov 6 11:36:57 2013 +0800

    qemu: clean up migration ports when migration cancelled
    
    If there's a migration cancelled, the bitmap of migration port should be
    cleaned up too.
    
    Signed-off-by: Zeng Junliang <zengjunliang>
    Signed-off-by: Jiri Denemark <jdenemar>
Comment 10 zhe peng 2013-11-13 07:23:06 UTC 
verify with build:
libvirt-1.1.1-12.el7.x86_64

step:
1: prepare gluster env. and migration env.
2: on source do live migrate then cancelled
# virsh migrate rhel --live qemu+ssh://10.66.100.118/system --verbose --unsafe 
root.100.118's password: 
Migration: [ 72 %]^Cerror: operation aborted: migration job: canceled by client
3: on target , check ports:
# netstat -nlap | grep 491
tcp        0      0 10.66.100.118:57155     10.66.6.69:49152        TIME_WAIT   -                   
tcp        0      0 10.66.100.118:43629     10.66.7.236:49152       TIME_WAIT   -    
4: do migrate again and cancelled.
5: check ports:
tcp        0      0 10.66.100.118:43634     10.66.7.236:49152       TIME_WAIT   -                   
tcp        0      0 10.66.100.118:57165     10.66.6.69:49152        TIME_WAIT   -                   
tcp        0      0 10.66.100.118:43639     10.66.7.236:49152       TIME_WAIT   -                   
tcp        0      0 10.66.100.118:57160     10.66.6.69:49152        TIME_WAIT   

if cancelled the job, the port cleaned up, can reused in next migration, so move to verified.
Comment 11 Ludek Smid 2014-06-13 09:51:13 UTC 
This request was resolved in Red Hat Enterprise Linux 7.0.

Contact your manager or support representative in case you have further questions about the request.
Note You need to log in before you can comment on or make changes to this bug.