Description of problem: I have a replica 3 volume homes with numerous clients, I need to change the auth.allow to enable mounting by clients on a new subnet. any attempt to change the setting using: gluster volume set homes auth.allow 129.174.126.*,129.174.120.*,129.174.121.*,129.174.122.*,129.174.123.* generates: volume set: failed: Staging failed on 129.174.126.56. Error: One or more connected clients cannot support the feature being set. These clients need to be upgraded or disconnected before running this command again To my knowledge this should never be possible as auth.allow has existed in every version of Gluster. Version-Release number of selected component (if applicable): 3.6.4 How reproducible: always Steps to Reproduce: 1.run gluster volume set volname aut.allow <list> 2. 3. Actual results: auth.allow parameters unchanged Expected results: auth.allow parameters set to new value Additional info: This cluster was originally installed with 3.3, and hase been progressively upgraded.
Please attach glusterd logs, with out the log files its difficult to analyze the problem.
let me know what if any other logs you need. Node Gluster0: etc-glusterfs-glusterd.vol.log: [2015-07-28 13:23:55.157052] I [glusterd-handler.c:1296:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2015-07-28 13:23:55.157832] I [glusterd-handler.c:1296:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2015-07-28 13:23:55.159092] I [glusterd-handler.c:1296:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2015-07-28 13:24:00.453804] E [glusterd-syncop.c:105:gd_collate_errors] 0-: Staging failed on 129.174.126.56. Error: One or more connected clients cannot support the feature being set. These clients need to be upgraded or disconnected before running this command again [2015-07-28 13:24:00.681844] I [glusterd-handler.c:1241:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2015-07-28 13:24:00.740495] I [glusterd-handler.c:1296:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2015-07-28 13:24:00.741785] I [glusterd-handler.c:1296:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2015-07-28 13:24:00.743125] I [glusterd-handler.c:1296:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2015-07-28 13:24:00.744396] I [glusterd-handler.c:1296:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2015-07-28 13:24:00.745617] I [glusterd-handler.c:1296:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2015-07-28 13:24:00.746575] I [glusterd-handler.c:1296:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2015-07-28 13:24:00.747446] I [glusterd-handler.c:1296:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2015-07-28 13:24:00.748670] I [glusterd-handler.c:1296:__glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req Node Gluster1: etc-glusterfs-glusterd.vol.log: [2015-07-28 13:25:10.295707] E [glusterd-utils.c:7410:glusterd_add_inode_size_to_dict] 0-management: tune2fs exited with non-zero exit status [2015-07-28 13:25:10.295758] E [glusterd-utils.c:7436:glusterd_add_inode_size_to_dict] 0-management: failed to get inode size [2015-07-28 13:25:10.719401] E [glusterd-utils.c:7410:glusterd_add_inode_size_to_dict] 0-management: xfs_info exited with non-zero exit status [2015-07-28 13:25:10.719445] E [glusterd-utils.c:7436:glusterd_add_inode_size_to_dict] 0-management: failed to get inode size [2015-07-28 13:25:11.571048] E [glusterd-utils.c:7410:glusterd_add_inode_size_to_dict] 0-management: xfs_info exited with non-zero exit status [2015-07-28 13:25:11.571097] E [glusterd-utils.c:7436:glusterd_add_inode_size_to_dict] 0-management: failed to get inode size [2015-07-28 13:25:11.957819] E [glusterd-utils.c:7410:glusterd_add_inode_size_to_dict] 0-management: xfs_info exited with non-zero exit status [2015-07-28 13:25:11.957861] E [glusterd-utils.c:7436:glusterd_add_inode_size_to_dict] 0-management: failed to get inode size [2015-07-28 13:25:12.345546] E [glusterd-utils.c:7410:glusterd_add_inode_size_to_dict] 0-management: xfs_info exited with non-zero exit status [2015-07-28 13:25:12.345587] E [glusterd-utils.c:7436:glusterd_add_inode_size_to_dict] 0-management: failed to get inode size [2015-07-28 13:25:12.835625] E [glusterd-utils.c:7410:glusterd_add_inode_size_to_dict] 0-management: xfs_info exited with non-zero exit status [2015-07-28 13:25:12.835689] E [glusterd-utils.c:7436:glusterd_add_inode_size_to_dict] 0-management: failed to get inode size [2015-07-28 13:25:13.344794] E [glusterd-utils.c:7410:glusterd_add_inode_size_to_dict] 0-management: xfs_info exited with non-zero exit status [2015-07-28 13:25:13.344836] E [glusterd-utils.c:7436:glusterd_add_inode_size_to_dict] 0-management: failed to get inode size [2015-07-28 13:25:36.304764] E [glusterd-op-sm.c:207:glusterd_get_txn_opinfo] 0-: Unable to get transaction opinfo for transaction ID : bbb60160-66e9-44b9-a778-aa3d45755075 [2015-07-28 13:25:47.242369] E [glusterd-op-sm.c:671:glusterd_check_client_op_version_support] 0-management: One or more clients don't support the required op-version [2015-07-28 13:25:47.242421] E [glusterd-op-sm.c:4585:glusterd_op_ac_stage_op] 0-management: Stage failed on operation 'Volume Set', Status : -1 Node Gluster-2: etc-glusterfs-glusterd.vol.log: [2015-07-28 13:25:10.295758] E [glusterd-utils.c:7436:glusterd_add_inode_size_to_dict] 0-management: failed to get inode size [2015-07-28 13:25:10.719401] E [glusterd-utils.c:7410:glusterd_add_inode_size_to_dict] 0-management: xfs_info exited with non-zero exit status [2015-07-28 13:25:10.719445] E [glusterd-utils.c:7436:glusterd_add_inode_size_to_dict] 0-management: failed to get inode size [2015-07-28 13:25:11.571048] E [glusterd-utils.c:7410:glusterd_add_inode_size_to_dict] 0-management: xfs_info exited with non-zero exit status [2015-07-28 13:25:11.571097] E [glusterd-utils.c:7436:glusterd_add_inode_size_to_dict] 0-management: failed to get inode size [2015-07-28 13:25:11.957819] E [glusterd-utils.c:7410:glusterd_add_inode_size_to_dict] 0-management: xfs_info exited with non-zero exit status [2015-07-28 13:25:11.957861] E [glusterd-utils.c:7436:glusterd_add_inode_size_to_dict] 0-management: failed to get inode size [2015-07-28 13:25:12.345546] E [glusterd-utils.c:7410:glusterd_add_inode_size_to_dict] 0-management: xfs_info exited with non-zero exit status [2015-07-28 13:25:12.345587] E [glusterd-utils.c:7436:glusterd_add_inode_size_to_dict] 0-management: failed to get inode size [2015-07-28 13:25:12.835625] E [glusterd-utils.c:7410:glusterd_add_inode_size_to_dict] 0-management: xfs_info exited with non-zero exit status [2015-07-28 13:25:12.835689] E [glusterd-utils.c:7436:glusterd_add_inode_size_to_dict] 0-management: failed to get inode size [2015-07-28 13:25:13.344794] E [glusterd-utils.c:7410:glusterd_add_inode_size_to_dict] 0-management: xfs_info exited with non-zero exit status [2015-07-28 13:25:13.344836] E [glusterd-utils.c:7436:glusterd_add_inode_size_to_dict] 0-management: failed to get inode size [2015-07-28 13:25:36.304764] E [glusterd-op-sm.c:207:glusterd_get_txn_opinfo] 0-: Unable to get transaction opinfo for transaction ID : bbb60160-66e9-44b9-a778-aa3d45755075 [2015-07-28 13:25:47.242369] E [glusterd-op-sm.c:671:glusterd_check_client_op_version_support] 0-management: One or more clients don't support the required op-version [2015-07-28 13:25:47.242421] E [glusterd-op-sm.c:4585:glusterd_op_ac_stage_op] 0-management: Stage failed on operation 'Volume Set', Status : -1 [2015-07-28 13:26:33.168980] E [glusterd-op-sm.c:671:glusterd_check_client_op_version_support] 0-management: One or more clients don't support the required op-version [2015-07-28 13:26:33.169032] E [glusterd-op-sm.c:4585:glusterd_op_ac_stage_op] 0-management: Stage failed on operation 'Volume Set', Status : -1 [2015-07-28 13:26:36.502079] E [glusterd-op-sm.c:207:glusterd_get_txn_opinfo] 0-: Unable to get transaction opinfo for transaction ID : 7c4b3693-77f0-4087-a236-76f4f41edb2f [2015-07-28 13:27:36.685167] E [glusterd-op-sm.c:207:glusterd_get_txn_opinfo] 0-: Unable to get transaction opinfo for transaction ID : 2588f9c9-2e33-47fc-9566-c95ab381dfd7 [2015-07-28 13:28:36.834453] E [glusterd-op-sm.c:207:glusterd_get_txn_opinfo] 0-: Unable to get transaction opinfo for transaction ID : d90dc97f-f6bc-40d7-bef9-85f5f73fcb14
From the log: One or more clients don't support the required op-version This suggests that there is a client that does not support the options for the volume that needs this auth.allow change. Could you check if all the clients that connect to the volume have the same version?
Not an easy thing to do if you have hundreds of clients. I have been able to eventually track down the client that was causing the issue. It was an ubuntu 14.04 client, I'm not sure what gluster version it was using as I don't have access to the system. This would have been much easier to track down if the ip addresses of the problem clients were logged with the error. "One or more clients don't support the required op-version" useless! Alternatively, have "gluster volume status volname clients" report some information about the clients other than just ip address, client version springs to mind.
(In reply to Alastair Neil from comment #4) > Not an easy thing to do if you have hundreds of clients. I have been able to > eventually track down the client that was causing the issue. It was an > ubuntu 14.04 client, I'm not sure what gluster version it was using as I > don't have access to the system. > > This would have been much easier to track down if the ip addresses of the > problem clients were logged with the error. > > "One or more clients don't support the required op-version" I agree that there is a scope of improvement in logging. I've sent a patch http://review.gluster.org/11831 to address it. Hopefully I will try to get it pushed for next 3.7 release. > > useless! > > Alternatively, have "gluster volume status volname clients" report some > information about the clients other than just ip address, client version > springs to mind.