Bug 851312 - pNFS client fails to select correct DS from multipath
pNFS client fails to select correct DS from multipath
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: kernel (Show other bugs)
All Linux
unspecified Severity medium
: rc
: ---
Assigned To: Steve Dickson
Jian Li
Depends On:
Blocks: 910083
  Show dependency treegraph
Reported: 2012-08-23 14:29 EDT by Tigran Mkrtchyan
Modified: 2014-03-03 19:09 EST (History)
5 users (show)

See Also:
Fixed In Version: kernel-2.6.32-335.el6
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2013-02-21 01:48:13 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)
possible fix (845 bytes, patch)
2012-08-23 14:29 EDT, Tigran Mkrtchyan
no flags Details | Diff
Patch the fixes the problem (1.51 KB, patch)
2012-10-03 15:12 EDT, Steve Dickson
no flags Details | Diff

  None (edit)
Description Tigran Mkrtchyan 2012-08-23 14:29:54 EDT
Created attachment 606683 [details]
possible fix

Description of problem:

The pNFS client implementation in RHEL 6.2 does not supports multipath for DS and simply picks the first one from the list. At the same time client does not supports IPv6 addresses as well. As a result, if a server reply with a multipath address where first one is IPv6 and second IPv4, then client will pick the first one and fail. It would be nice if client will try to take first supported one. 

Version-Release number of selected component (if applicable):
RHEL 6.2, kernel-2.6.32-220.23.1.el6.x86_64

How reproducible:
You will need a pNFS server which supports IPv6 and IPv4 as well as multipath.

Steps to Reproduce:
1. server have to return IPv6 and IPv4, where IPv6 entry is the first one
Actual results:
IO error with following error messages:
decode_device: Multipath count 2 not supported, skipping all greater than 1
decode_and_add_device: Could not decode or add device

Expected results:
Client picks the first supported address

Additional info:

A possible fix is attached. Please notice, that this is just a hint and not tested.
Comment 2 Ric Wheeler 2012-08-24 07:48:16 EDT
pNFS code in RHEL lags the upstream (but will get significant updates).

Have you seen this same issue with upstream (try Fedora 17 for example)?
Comment 3 Tigran Mkrtchyan 2012-08-24 08:27:50 EDT
No. Fedora 17 support IPv6 as well as multipath data servers.
Comment 4 RHEL Product and Program Management 2012-10-02 14:11:29 EDT
This request was evaluated by Red Hat Product Management for
inclusion in a Red Hat Enterprise Linux release.  Product
Management has requested further review of this request by
Red Hat Engineering, for potential inclusion in a Red Hat
Enterprise Linux release for currently deployed products.
This request is not yet committed for inclusion in a release.
Comment 5 Ric Wheeler 2012-10-03 01:42:33 EDT
Hi Tigran, it would be good to have you try out the updated pNFS code that is coming with RHEL6.4. We have a significant update in that release.
Comment 6 Tigran Mkrtchyan 2012-10-03 09:29:37 EDT
Hi Ric.

Sure, just give me a pointer to the rpm or srpm.
Comment 7 Steve Dickson 2012-10-03 15:12:28 EDT
Created attachment 621082 [details]
Patch the fixes the problem

This patch was tested at this fall's BAT...
Comment 8 Tigran Mkrtchyan 2012-10-03 16:08:31 EDT
I can confirm, that the patch fixes the problem I see.
Comment 10 Jian Li 2012-10-07 21:34:53 EDT
Try to test this bug with newpynfs(4.1), set qa_ack+ .
Comment 11 Jarod Wilson 2012-10-19 16:51:27 EDT
Patch(es) available on kernel-2.6.32-335.el6
Comment 14 Jian Li 2013-01-30 02:29:15 EST
In 2.6.32-356, this patch still would not be tested, kernel still only support ipv4.

274     /* Currently only support ipv4 address */
275     if (in4_pton(buf, rlen, (u8 *)&ip_addr, '-', &ipend) == 0) {
276         dprintk("%s: Only ipv4 addresses supported\n", __func__);
277         goto out_free;
278     }

SanityOnly test. Please check that Steve, should we adapt kernel?
Comment 15 Ric Wheeler 2013-02-01 07:26:50 EST
Do we need a separate BZ to track the IPV4 only issue?
Comment 16 Jian Li 2013-02-03 22:14:53 EST
(In reply to comment #14)
> In 2.6.32-356, this patch still would not be tested, kernel still only
> support ipv4.

This patch could be tested with newpynfs, server offer a multipath deviceinfo, the first is ipv6 style ([::1]), the second is ipv4 (

INFO   :nfs.server:****************************************
INFO   :nfs.server:Handling COMPOUND
INFO   :nfs.server:COMPOUND4args(tag='', minorversion=1, argarray=[nfs_argop4(argop=OP_SEQUENCE, opsequence=SEQUENCE4args(sa_sessionid='0000000000000001', sa_sequenceid=24, sa_slotid=0, sa_highest_slotid=0, sa_cachethis=False)), nfs_argop4(argop=OP_GETDEVICEINFO, opgetdeviceinfo=GETDEVICEINFO4args(gdia_device_id='\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', gdia_layout_type=LAYOUT4_NFSV4_1_FILES, gdia_maxcount=16384, gdia_notify_types=0L))])
INFO   :nfs.server:*** OP_SEQUENCE (53) ***
INFO   :nfs.server:*** OP_GETDEVICEINFO (47) ***
INFO   :nfs.server:Replying.  Status NFS4_OK (0)
INFO   :nfs.server:COMPOUND4res(status=NFS4_OK, tag='', resarray=[nfs_resop4(resop=OP_SEQUENCE, opsequence=SEQUENCE4res(sr_status=NFS4_OK, sr_resok4=SEQUENCE4resok(sr_sessionid='0000000000000001', sr_sequenceid=24, sr_slotid=0, sr_highest_slotid=0, sr_target_highest_slotid=8, sr_status_flags=0))), nfs_resop4(resop=OP_GETDEVICEINFO, opgetdeviceinfo=GETDEVICEINFO4res(gdir_status=NFS4_OK, gdir_resok4=GETDEVICEINFO4resok(gdir_device_addr=device_addr4(da_layout_type=LAYOUT4_NFSV4_1_FILES, da_addr_body='\x00\x00\x00\x01\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x02\x00\x00\x00\x04tcp6\x00\x00\x00\t::1.48.58\x00\x00\x00\x00\x00\x00\x03tcp\x00\x00\x00\x00\x0f127.\x00'), gdir_notification=0)))])

client mount with 
[root@hp-xw4600-01 ~]# mount [::1]:/files /mnt/test -o vers=4,minorversion=1
[root@hp-xw4600-01 ~]# grep "/mnt/test" /proc/mounts 
[::1]:/files/ /mnt/test nfs4 rw,relatime,vers=4,rsize=4096,wsize=4096,namlen=255,hard,proto=tcp6,port=0,timeo=600,retrans=2,sec=sys,clientaddr=::1,minorversion=1,local_lock=none,addr=::1 0 0
[root@hp-xw4600-01 ~]# cat /proc/fs/nfsfs/servers 
v4 7f000001 3039   1 (null)
v4 00000000000000000000000000000001  801   1 ::1
Comment 17 Jian Li 2013-02-03 22:24:12 EST
(In reply to comment #15)
> Do we need a separate BZ to track the IPV4 only issue?

This patch aim to deal with IPV4 only issue, so this bug is finished.
Comment 19 errata-xmlrpc 2013-02-21 01:48:13 EST
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.