| Summary: | qemu live migration port conflicts with other users of ephemeral port(s) | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux 7 | Reporter: | Daniel Berrangé <berrange> |
| Component: | libvirt | Assignee: | Jiri Denemark <jdenemar> |
| Status: | CLOSED CURRENTRELEASE | QA Contact: | Virtualization Bugs <virt-bugs> |
| Severity: | high | Docs Contact: | |
| Priority: | urgent | ||
| Version: | 7.0 | CC: | acathrow, barumuga, berrange, bili, cdhouch, chhu, clalancette, dallan, dani-rh, dyuan, gianluca.cecchi, gluster-bugs, herrold, itamar, jforbes, jherrman, kkeithle, laine, libvirt-maint, stefanha, veillard, virt-maint, ydu, zpeng |
| Target Milestone: | rc | ||
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | libvirt-1.1.1-12.el7 | Doc Type: | Bug Fix |
| Doc Text: |
Prior to this update, migrating a virtual machine failed when the libvirtd service used a transmission control protocol (TCP) port that was already in use. Now, it is possible to predefine a custom migration TCP port range in case the default port is in use. In addition, libvirtd now ensures that the port it chooses from the custom range is not used by another process.
|
Story Points: | --- |
| Clone Of: | 1018695 | Environment: | |
| Last Closed: | 2014-06-13 09:51:13 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Bug Depends On: | |||
| Bug Blocks: | 1018178, 1018383 | ||
One more patch is needed to fully support configurable migration ports:
commit d9be5a7157515eeae99379e9544c34b34c5e5198
Author: Michal Privoznik <mprivozn>
Date: Fri Oct 18 18:28:14 2013 +0200
qemu: Fix augeas support for migration ports
Commit e3ef20d7 allows user to configure migration ports range via
qemu.conf. However, it forgot to update augeas definition file and
even the test data was malicious.
Signed-off-by: Michal Privoznik <mprivozn>
I can reproduce this with libvirt-1.1.1-9.el7
will get error:
2013-10-25 10:38:24.397+0000: 16830: debug : virCommandHandshakeChild:388 : Handshake with parent is done
char device redirected to /dev/pts/4 (label charserial0)
qemu-kvm: -incoming tcp:[::]:49179: Failed to bind socket: Address already in use
verify with build :
libvirt-1.1.1-10.el7
glusterfs-3.4.0.36rhs-1.el7.x86_64
step:
1: prepare gluster env. and migration env.
2: create a guest on source with xml:
....
<disk type='network' device='disk'>
<driver name='qemu' type='raw' cache='none'/>
<source protocol='gluster' name='gluster-vol1/rh6.img'>
<host name='10.66.82.251' transport='rdma'/>
</source>
<target dev='hda' bus='ide'/>
<alias name='ide0-0-0'/>
<address type='drive' controller='0' bus='0' target='0' unit='0'/>
</disk>
....
3: on the destination host run cmd:
#nc -l -p 49152
4: do migration on source:
# virsh migrate --live rhel6 qemu+ssh://10.66.106.25/system --verbose --unsafe
root.106.25's password:
Migration: [100 %]
no error occured. move to verified.
Cancelled or failed migrations leave migration ports reserved on destination which can easily lead in migration range to be exhausted. The following upstream commit fixes this issue:
commit c92ca769af2bacefdd451802d7eb1adac5e6597c
Author: Zeng Junliang <zengjunliang>
Date: Wed Nov 6 11:36:57 2013 +0800
qemu: clean up migration ports when migration cancelled
If there's a migration cancelled, the bitmap of migration port should be
cleaned up too.
Signed-off-by: Zeng Junliang <zengjunliang>
Signed-off-by: Jiri Denemark <jdenemar>
verify with build: libvirt-1.1.1-12.el7.x86_64 step: 1: prepare gluster env. and migration env. 2: on source do live migrate then cancelled # virsh migrate rhel --live qemu+ssh://10.66.100.118/system --verbose --unsafe root.100.118's password: Migration: [ 72 %]^Cerror: operation aborted: migration job: canceled by client 3: on target , check ports: # netstat -nlap | grep 491 tcp 0 0 10.66.100.118:57155 10.66.6.69:49152 TIME_WAIT - tcp 0 0 10.66.100.118:43629 10.66.7.236:49152 TIME_WAIT - 4: do migrate again and cancelled. 5: check ports: tcp 0 0 10.66.100.118:43634 10.66.7.236:49152 TIME_WAIT - tcp 0 0 10.66.100.118:57165 10.66.6.69:49152 TIME_WAIT - tcp 0 0 10.66.100.118:43639 10.66.7.236:49152 TIME_WAIT - tcp 0 0 10.66.100.118:57160 10.66.6.69:49152 TIME_WAIT if cancelled the job, the port cleaned up, can reused in next migration, so move to verified. This request was resolved in Red Hat Enterprise Linux 7.0. Contact your manager or support representative in case you have further questions about the request. |
This is now fixed upstream by v1.1.3-188-g0196845 and v1.1.3-189-ge3ef20d: commit 0196845d3abd0d914cf11f7ad6c19df8b47c32ed Author: Wang Yufei <james.wangyufei> Date: Fri Oct 11 11:27:13 2013 +0800 qemu: Avoid assigning unavailable migration ports https://bugzilla.redhat.com/show_bug.cgi?id=1019053 When we migrate vms concurrently, there's a chance that libvirtd on destination assigns the same port for different migrations, which will lead to migration failure during prepare phase on destination. So we use virPortAllocator here to solve the problem. Signed-off-by: Wang Yufei <james.wangyufei> Signed-off-by: Jiri Denemark <jdenemar> commit e3ef20d7f7fee595ac4fc6094e04b7d65ee0583a Author: Jiri Denemark <jdenemar> Date: Tue Oct 15 15:26:52 2013 +0200 qemu: Make migration port range configurable https://bugzilla.redhat.com/show_bug.cgi?id=1019053