Bug 1018178
| Summary: | Glusterfs ports conflict with qemu live migration | ||
|---|---|---|---|
| Product: | [Community] GlusterFS | Reporter: | Kaleb KEITHLEY <kkeithle> |
| Component: | transport | Assignee: | Kaleb KEITHLEY <kkeithle> |
| Status: | CLOSED UPSTREAM | QA Contact: | |
| Severity: | medium | Docs Contact: | |
| Priority: | urgent | ||
| Version: | mainline | CC: | BSipos, cdhouch, dani-rh, danken, gianluca.cecchi, gluster-bugs, herrold, jherrman, kkeithle, ndevos, rgowdapp |
| Target Milestone: | --- | Keywords: | Reopened |
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | Bug Fix | |
| Doc Text: |
Prior to this update, migrating a virtual machine failed when the libvirtd service used a transmission control protocol (TCP) port that was already in use. Now, it is possible to predefine a custom migration TCP port range in case the default port is in use. In addition, libvirtd now ensures that the port it chooses from the custom range is not used by another process.
|
Story Points: | --- |
| Clone Of: | 987555 | Environment: | |
| Last Closed: | 2013-11-06 12:06:38 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Bug Depends On: | 987555, 1018695, 1019058, 1019237, 1340368 | ||
| Bug Blocks: | 1018383 | ||
|
Description
Kaleb KEITHLEY
2013-10-11 11:32:52 UTC
Out of curiosity, why isn't this a bug in qemu-kvm? Shouldn't qemu-kvm be trying another port if 49152 (or any other port) is in use? And using portmapper to register the port it does end up using? REVIEW: http://review.gluster.org/6075 (xlators/mgmt/glusterd: ports conflict with qemu live migration) posted (#1) for review on master by Kaleb KEITHLEY (kkeithle) I confirm that taking source rpm provided by fedora 19 (glusterfs-3.4.1-1.fc19.src.rpm), changing sources as in comment#2 , rebuilding them and updating my gluster rpms on a Fedora 19 system in oVirt now I have [g.cecchi@f18ovn01 ~]$ ps -ef|grep glusterfs root 2131 1 0 18:44 ? 00:00:00 /usr/sbin/glusterfsd -s f18ovn01.mydomain --volfile-id gvdata.f18ovn01.mydomain.gluster-DATA_GLUSTER-brick1 -p /var/lib/glusterd/vols/gvdata/run/f18ovn01.mydomain-gluster-DATA_GLUSTER-brick1.pid -S /var/run/3f792df94642bdbf308ce4c368dc05bf.socket --brick-name /gluster/DATA_GLUSTER/brick1 -l /var/log/glusterfs/bricks/gluster-DATA_GLUSTER-brick1.log --xlator-option *-posix.glusterd-uuid=ebaf2f1a-65a8-409a-b911-6e631a5f182f --brick-port 50152 --xlator-option gvdata-server.listen-port=50152 [g.cecchi@f18ovn01 ~]$ sudo lsof -Pp 2131 | egrep "491|501" glusterfs 2131 root 9u IPv4 28933 0t0 TCP f18ovn01.mydomain:50152->f18ovn01.mydomain:1011 (ESTABLISHED) glusterfs 2131 root 10u IPv4 25260 0t0 TCP *:50152 (LISTEN) glusterfs 2131 root 13u IPv4 101260 0t0 TCP f18ovn01.mydomain:50152->f18ovn01.mydomain:1001 (ESTABLISHED) And I'm able to successfully live migrate my CentOS 6.4 VM from one host to the other one. I have one brick that is replicated ditributed between two hosts [g.cecchi@f18ovn01 ~]$ sudo gluster volume info gvdata Volume Name: gvdata Type: Replicate Volume ID: ed71a4c2-6205-4aad-9aab-85da086d5ba3 Status: Started Number of Bricks: 1 x 2 = 2 Transport-type: tcp Bricks: Brick1: f18ovn01.mydomain:/gluster/DATA_GLUSTER/brick1 Brick2: f18ovn03.mydomain:/gluster/DATA_GLUSTER/brick1 Options Reconfigured: server.allow-insecure: on storage.owner-uid: 36 storage.owner-gid: 36 Hope to see something like a configurable range via configuration file too. Thanks, Gianluca RFC 6335 is very clear about this:
8.1.2. Variances for Specific Port Number Ranges
...
o Ports in the Dynamic Ports range (49152-65535) have been
specifically set aside for local and dynamic use and cannot be
assigned through IANA. Application software may simply use any
dynamic port that is available on the local host, without any sort
of assignment. On the other hand, application software MUST NOT
assume that a specific port number in the Dynamic Ports range will
always be available for communication at all times...
This is clearly a libvirt bug. Libvirt needs to handle the case when a port is not available. It cannot assume that it can exclusively use ports starting at 49152.
See Red Hat bugs 1018530, 1018696, and 1019237.
REVIEW: http://review.gluster.org/6210 (mgmt/glusterd: add option to specify a different base-port) posted (#1) for review on master by Kaleb KEITHLEY (kkeithle) COMMIT: http://review.gluster.org/6210 committed in master by Anand Avati (avati) ------ commit c47408e896c9bcaf21e7f8956bdae85633f873e0 Author: Kaleb S. KEITHLEY <kkeithle> Date: Thu Oct 31 08:18:56 2013 -0400 mgmt/glusterd: add option to specify a different base-port This is (arguably) a hack to work around a bug in libvirt which is not well behaved wrt to using TCP ports in the unreserved space between 49152-65535. (See RFC 6335) Normally glusterd starts and binds to the first available port in range, usually 49152. libvirt's live migration also tries to use ports in this range, but has no fallback to use (an)other port(s) when the one it wants is already in use. Change-Id: Id8fe35c08b6ce4f268d46804bbb6dddab7a6b7bb BUG: 1018178 Signed-off-by: Kaleb S. KEITHLEY <kkeithle> Reviewed-on: http://review.gluster.org/6210 Tested-by: Gluster Build System <jenkins.com> Reviewed-by: Anand Avati <avati> In fact, closing with UPSTREAM. Bug 987555 is tracked for an update in release-3.4. *** Bug 1196678 has been marked as a duplicate of this bug. *** |