Bug 1698373

Summary: Some communication between pcsd daemons does not work over IPv6 addresses
Product: Red Hat Enterprise Linux 8 Reporter: Miroslav Lisik <mlisik>
Component: pcsAssignee: Tomas Jelinek <tojeline>
Status: CLOSED ERRATA QA Contact: cluster-qe <cluster-qe>
Severity: urgent Docs Contact:
Priority: urgent    
Version: 8.0CC: aherr, cfeist, cluster-maint, idevat, nhostako, omular, tojeline, toneata
Target Milestone: rcKeywords: EasyFix, ZStream
Target Release: 8.1Flags: pm-rhel: mirror+
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: pcs-0.10.1-6.el8 Doc Type: Bug Fix
Doc Text:
Cause: IPv6 addresses specified in 'pcs host auth' command. Consequence: Some pcs commands are unable to connect to cluster nodes. Fix: Wrap IPv6 addresses in [] when constructing URLs to connect to. Result: Pcs is able to communicate with nodes over IPv6.
Story Points: ---
Clone Of:
: 1710067 (view as bug list) Environment:
Last Closed: 2019-11-05 20:40:02 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1710067    
Attachments:
Description Flags
proposed fix none

Description Miroslav Lisik 2019-04-10 09:10:38 UTC
Description of problem:
Some communication between pcsd daemons does not work over IPv6 addresses.

Version-Release number of selected component (if applicable):
pcs-0.10.1-4.el8

How reproducible:
always

Steps to Reproduce:

1. authenticate pcsd using IPv6 addresses
[root@virt-157 ~]# pcs host auth -u hacluster -p password 2620:52:0:25a4:1800:ff:fe00:9d 2620:52:0:25a4:1800:ff:fe00:9e
2620:52:0:25a4:1800:ff:fe00:9d: Authorized
2620:52:0:25a4:1800:ff:fe00:9e: Authorized

2. check authentication

[root@virt-157 ~]# pcs status pcsd 2620:52:0:25a4:1800:ff:fe00:9d 2620:52:0:25a4:1800:ff:fe00:9e
  2620:52:0:25a4:1800:ff:fe00:9d: Offline
  2620:52:0:25a4:1800:ff:fe00:9e: Offline


3. create cluster
[root@virt-157 ~]# pcs cluster setup TestCluster 2620:52:0:25a4:1800:ff:fe00:9d 2620:52:0:25a4:1800:ff:fe00:9e
No addresses specified for host '2620:52:0:25a4:1800:ff:fe00:9d', using '2620:52:0:25a4:1800:ff:fe00:9d'
No addresses specified for host '2620:52:0:25a4:1800:ff:fe00:9e', using '2620:52:0:25a4:1800:ff:fe00:9e'
Destroying cluster on hosts: '2620:52:0:25a4:1800:ff:fe00:9d', '2620:52:0:25a4:1800:ff:fe00:9e'...
2620:52:0:25a4:1800:ff:fe00:9d: Successfully destroyed cluster
2620:52:0:25a4:1800:ff:fe00:9e: Successfully destroyed cluster
Requesting remove 'pcsd settings' from '2620:52:0:25a4:1800:ff:fe00:9d', '2620:52:0:25a4:1800:ff:fe00:9e'
2620:52:0:25a4:1800:ff:fe00:9e: successful removal of the file 'pcsd settings'
2620:52:0:25a4:1800:ff:fe00:9d: successful removal of the file 'pcsd settings'
Sending 'corosync authkey', 'pacemaker authkey' to '2620:52:0:25a4:1800:ff:fe00:9d', '2620:52:0:25a4:1800:ff:fe00:9e'
2620:52:0:25a4:1800:ff:fe00:9d: successful distribution of the file 'corosync authkey'
2620:52:0:25a4:1800:ff:fe00:9d: successful distribution of the file 'pacemaker authkey'
2620:52:0:25a4:1800:ff:fe00:9e: successful distribution of the file 'corosync authkey'
2620:52:0:25a4:1800:ff:fe00:9e: successful distribution of the file 'pacemaker authkey'
Synchronizing pcsd SSL certificates on nodes '2620:52:0:25a4:1800:ff:fe00:9d', '2620:52:0:25a4:1800:ff:fe00:9e'...
2620:52:0:25a4:1800:ff:fe00:9e: Success
2620:52:0:25a4:1800:ff:fe00:9d: Success
Sending 'corosync.conf' to '2620:52:0:25a4:1800:ff:fe00:9d', '2620:52:0:25a4:1800:ff:fe00:9e'
2620:52:0:25a4:1800:ff:fe00:9d: successful distribution of the file 'corosync.conf'
2620:52:0:25a4:1800:ff:fe00:9e: successful distribution of the file 'corosync.conf'
Cluster has been successfully set up.

4. Try to start all nodes.
[root@virt-157 ~]# pcs cluster start --all
2620:52:0:25a4:1800:ff:fe00:9d: Unable to connect to 2620:52:0:25a4:1800:ff:fe00:9d, try setting higher timeout in --request-timeout option (Port number ended with ':')
2620:52:0:25a4:1800:ff:fe00:9e: Unable to connect to 2620:52:0:25a4:1800:ff:fe00:9e, try setting higher timeout in --request-timeout option (Port number ended with ':')
Error: unable to start all nodes
2620:52:0:25a4:1800:ff:fe00:9d: Unable to connect to 2620:52:0:25a4:1800:ff:fe00:9d, try setting higher timeout in --request-timeout option (Port number ended with ':')
2620:52:0:25a4:1800:ff:fe00:9e: Unable to connect to 2620:52:0:25a4:1800:ff:fe00:9e, try setting higher timeout in --request-timeout option (Port number ended with ':')

Actual results:

Communication between pcsd daemons does not work over IPv6 addresses.

Expected results:

Pcsd communication works over IPv6 addresses.


Additional info:

It looks like there are missing square brackets around ipv6 address:
> Sending HTTP Request to: https://2620:52:0:25a4:1800:ff:fe00:9d:2224/remote/check_auth


[root@virt-157 ~]# pcs status pcsd 2620:52:0:25a4:1800:ff:fe00:9d --debug | tail

--Debug Stdout End--
--Debug Stderr Start--

--Debug Stderr End--

Sending HTTP Request to: https://2620:52:0:25a4:1800:ff:fe00:9d:2224/remote/check_auth
Data: None
Response Reason: Port number ended with ':'
  2620:52:0:25a4:1800:ff:fe00:9d: Offline

Comment 2 Tomas Jelinek 2019-04-10 16:07:52 UTC
Created attachment 1554300 [details]
proposed fix

Comment 3 Miroslav Lisik 2019-04-11 09:31:54 UTC
Also parsing of port when ipv6 address is used should be improved:

[root@virt-157 ~]# pcs host auth node-01 -u hacluster -p password addr=2620:52:0:25a4:1800:ff:fe00:9d:3224
Error: Unable to communicate with node-01
[root@virt-157 ~]# echo $?
1
[root@virt-157 ~]# pcs host auth node-01 -u hacluster -p password addr=2620:52:0:25a4:1800:ff:fe00:9d:3224 --debug | tail -n 20
Error: Unable to communicate with node-01
    "sync_successful": true,
    "sync_nodes_err": [

    ],
    "sync_responses": {
    }
  },
  "log": [
    "I, [2019-04-11T11:29:43.957895 #5344]  INFO -- : PCSD Debugging enabled\n",
    "D, [2019-04-11T11:29:43.957955 #5344] DEBUG -- : Detected systemd is in use\n",
    "I, [2019-04-11T11:29:44.159604 #5344]  INFO -- : Connecting to: https://[2620:52:0:25a4:1800:ff:fe00:9d:3224]:2224/remote/auth\n",
    "I, [2019-04-11T11:29:44.168959 #5344]  INFO -- : No response from: 2620:52:0:25a4:1800:ff:fe00:9d:3224 request: auth, error: couldnt_resolve_host\n"
  ]
}

--Debug Stdout End--
--Debug Stderr Start--

--Debug Stderr End--

Comment 4 Miroslav Lisik 2019-04-11 09:35:00 UTC
Scratch comment 3. Brackets should be used on command line when port is given:

[root@virt-157 ~]# pcs host auth node-01 -u hacluster -p password addr=[2620:52:0:25a4:1800:ff:fe00:9d]:3224
node-01: Authorized

Comment 5 Ondrej Mular 2019-05-02 12:46:21 UTC
After fix:
[root@rhel81-node1 ~]# rpm -q pcs
pcs-0.10.1-6.el8.x86_64

[root@rhel81-node1 ~]# pcs host auth -uhacluster ::1
Password:
::1: Authorized
[root@rhel81-node1 ~]# pcs status pcsd ::1
  ::1: Online

Comment 10 errata-xmlrpc 2019-11-05 20:40:02 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2019:3311