Bug 1446003 - vnc cannot find a free port to use
Summary: vnc cannot find a free port to use
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: qemu-kvm-rhev
Version: 7.4
Hardware: x86_64
OS: All
urgent
medium
Target Milestone: rc
: ---
Assignee: Daniel Berrange
QA Contact: Guo, Zhiyi
URL:
Whiteboard:
Keywords: Regression
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-04-27 00:16 UTC by Guo, Zhiyi
Modified: 2017-08-02 04:35 UTC (History)
8 users (show)

(edit)
Clone Of:
(edit)
Last Closed: 2017-08-02 04:35:59 UTC


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2017:2392 normal SHIPPED_LIVE Important: qemu-kvm-rhev security, bug fix, and enhancement update 2017-08-01 20:04:36 UTC

Description Guo, Zhiyi 2017-04-27 00:16:01 UTC
Description of problem:
vnc cannot find a free port to use

Version-Release number of selected component (if applicable):
qemu-kvm-rhev-2.9.0-1.el7.x86_64

How reproducible:
100%

Steps to Reproduce:
1.boot qemu as /usr/libexec/qemu-kvm -vnc :1,to=2
2.boot another qemu as /usr/libexec/qemu-kvm -vnc :1,to=2
3.

Actual results:
Second qemu fail to start with:
qemu-kvm: -vnc :1,to=2: Failed to start VNC server: Failed to bind socket: Address already in use

Expected results:
Second qemu can start and use free vnc port 2

Additional info:
This is a regression of qemu-kvm 2.9. Compile qemu-kvm from 2.8.0 upstream/downstream source, these can start:
$ ./qemu-system-x86_64 -vnc :0,to=2 -monitor stdio
QEMU 2.8.1.1 monitor - type 'help' for more information
(qemu) info vnc
Server:
     address: 0.0.0.0:5900
        auth: none

./qemu-system-x86_64 -vnc :0,to=2 -monitor stdio
QEMU 2.8.1.1 monitor - type 'help' for more information
(qemu) info vnc
Server:
     address: 0.0.0.0:5901

./qemu-system-x86_64 -vnc :0,to=2 -monitor stdio
QEMU 2.8.1.1 monitor - type 'help' for more information
(qemu) info vnc
Server:
     address: 0.0.0.0:5902

Comment 2 Gerd Hoffmann 2017-04-27 06:54:55 UTC
kraxel@nilsson ~/Downloads/patches# netstat -tnap | grep qemu
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
tcp        0      0 0.0.0.0:5910            0.0.0.0:*               LISTEN      8774/qemu-system-x8 
tcp6       0      0 :::5911                 :::*                    LISTEN      8774/qemu-system-x8 

qemu appears to use two ports now, one for ipv4, one for ipv6.

Comment 3 Guo, Zhiyi 2017-04-27 08:00:02 UTC
(In reply to Gerd Hoffmann from comment #2)
> kraxel@nilsson ~/Downloads/patches# netstat -tnap | grep qemu
> (Not all processes could be identified, non-owned process info
>  will not be shown, you would have to be root to see it all.)
> tcp        0      0 0.0.0.0:5910            0.0.0.0:*               LISTEN  
> 8774/qemu-system-x8 
> tcp6       0      0 :::5911                 :::*                    LISTEN  
> 8774/qemu-system-x8 
> 
> qemu appears to use two ports now, one for ipv4, one for ipv6.

Hi Gerd,

   I want to verify Bug 1038963 - [RFE] qemu can't listen on both IPv6 and IPv4 localhost for VNC  but qemu only bind to ipv4 address with my cli:
# /usr/libexec/qemu-kvm -vnc :0 -monitor stdio
QEMU 2.9.0 monitor - type 'help' for more information
(qemu) info vnc
Server:
     address: 0.0.0.0:5900
        auth: none
Client: none
(qemu) 

# netstat -tnap | grep qemu
tcp        0      0 0.0.0.0:5900            0.0.0.0:*               LISTEN      5840/qemu-kvm       

Could you give us a reference how to verify that bug?

BR/
Guo, Zhiyi

Comment 4 Daniel Berrange 2017-04-27 08:41:59 UTC
(In reply to Gerd Hoffmann from comment #2)
> kraxel@nilsson ~/Downloads/patches# netstat -tnap | grep qemu
> (Not all processes could be identified, non-owned process info
>  will not be shown, you would have to be root to see it all.)
> tcp        0      0 0.0.0.0:5910            0.0.0.0:*               LISTEN  
> 8774/qemu-system-x8 
> tcp6       0      0 :::5911                 :::*                    LISTEN  
> 8774/qemu-system-x8 
> 
> qemu appears to use two ports now, one for ipv4, one for ipv6.

That would be a bug - we should only ever use a single port, even if using 'to='. If we do end up having multiple listening sockets, then we must ensure they all use the same port as the first socket. I guess this is a regression from the listener rewrite I did

Comment 5 Daniel Berrange 2017-04-27 09:30:08 UTC
So when you use ':0', the host name is "" and getaddrinfo resolves that to "0.0.0.0" and "::", in that order.

We listen on "0.0.0.0" first and succeed.

The old code stopped there and didn't even bother trying to listen on "::".

The new code tries "::" and because we have IPV6_V6ONLY=0, it fails to listen on "::" since it clashes with 0.0.0.0. Because to=2 is set, we try "::" again on the next port.

What we really need here is for IPV6_V6ONLY to be set to 1, so that we when try "::", we don't clash with "0.0.0.0".

The problem with changing IPV6_V6ONLY to 1 though, is that then breaks people who have explicitly set '::' as their listen address.

Comment 6 Ademar Reis 2017-04-28 16:05:29 UTC
(In reply to Daniel Berrange from comment #4)
> (In reply to Gerd Hoffmann from comment #2)
> > kraxel@nilsson ~/Downloads/patches# netstat -tnap | grep qemu
> > (Not all processes could be identified, non-owned process info
> >  will not be shown, you would have to be root to see it all.)
> > tcp        0      0 0.0.0.0:5910            0.0.0.0:*               LISTEN  
> > 8774/qemu-system-x8 
> > tcp6       0      0 :::5911                 :::*                    LISTEN  
> > 8774/qemu-system-x8 
> > 
> > qemu appears to use two ports now, one for ipv4, one for ipv6.
> 
> That would be a bug - we should only ever use a single port, even if using
> 'to='. If we do end up having multiple listening sockets, then we must
> ensure they all use the same port as the first socket. I guess this is a
> regression from the listener rewrite I did

Would you be able to fix it and submit the backport?

Comment 7 Daniel Berrange 2017-05-02 08:10:09 UTC
Possible patch provided upstream

https://lists.gnu.org/archive/html/qemu-devel/2017-04/msg05659.html

Comment 8 Ademar Reis 2017-05-16 14:42:27 UTC
(In reply to Daniel Berrange from comment #7)
> Possible patch provided upstream
> 
> https://lists.gnu.org/archive/html/qemu-devel/2017-04/msg05659.html

I see there was some discussion upstream, but looks like it should be merged. v2?

Comment 10 Daniel Berrange 2017-05-16 14:46:19 UTC
(In reply to Ademar Reis from comment #8)
> (In reply to Daniel Berrange from comment #7)
> > Possible patch provided upstream
> > 
> > https://lists.gnu.org/archive/html/qemu-devel/2017-04/msg05659.html
> 
> I see there was some discussion upstream, but looks like it should be
> merged. v2?

Yeah, i'll be sending a v2

Comment 11 Daniel Berrange 2017-05-22 12:43:37 UTC
The fix for this has triggered a broader cleanup. The specific fix for this bug is in the 1st patch here

https://lists.gnu.org/archive/html/qemu-devel/2017-05/msg04706.html

I'll file a separate bugs to track the other issues described in that series.

Comment 16 Miroslav Rezanina 2017-06-23 06:12:51 UTC
Fix included in qemu-kvm-rhev-2.9.0-14.el7

Comment 18 Guo, Zhiyi 2017-06-26 02:45:51 UTC
Test against qemu-kvm-rhev-2.9.0-14.el7.x86_64

# /usr/libexec/qemu-kvm -vnc :0

# netstat -pant | grep 5900
tcp        0      0 0.0.0.0:5900            0.0.0.0:*               LISTEN      30179/qemu-kvm      
tcp6       0      0 :::5900                 :::*                    LISTEN      30179/qemu-kvm      
   
vnc can listen on both ipv4 and ipv6

Launch 1st qemu process:
# /usr/libexec/qemu-kvm -vnc :0,to=1 -monitor stdio
QEMU 2.9.0 monitor - type 'help' for more information
(qemu) VNC server running on 0.0.0.0:5900

Keep 1st qemu process alive and luanch 2nd qemu process:
# /usr/libexec/qemu-kvm -vnc :0,to=1 -monitor stdio
QEMU 2.9.0 monitor - type 'help' for more information
(qemu) VNC server running on 0.0.0.0:5901

Keep 1st and 2nd qemu process alive and launch 3rd qemu process:
# /usr/libexec/qemu-kvm -vnc :0,to=1
qemu-kvm: -vnc :0,to=1: Failed to start VNC server: Failed to bind socket: Address already in use

Both 5900 and 5901 can be connected with remote-viewer
So the bug is fixed

Comment 19 Guo, Zhiyi 2017-06-26 02:46:31 UTC
Verified per comment 18

Comment 21 errata-xmlrpc 2017-08-02 04:35:59 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:2392


Note You need to log in before you can comment on or make changes to this bug.