Bug 1378371

Summary:	"ganesha.so cannot open" warning message in glusterd log in non ganesha setup.
Product:	[Red Hat Storage] Red Hat Gluster Storage	Reporter:	Byreddy <bsrirama>
Component:	glusterd	Assignee:	Jiffin <jthottan>
Status:	CLOSED ERRATA	QA Contact:	Manisha Saini <msaini>
Severity:	high	Docs Contact:
Priority:	low
Version:	rhgs-3.2	CC:	alsilva, amukherj, jthottan, kramdoss, msaini, nchilaka, ndevos, psony, rhinduja, rhs-bugs, sarora, sasundar, sheggodu, skoduri, storage-qa-internal, vbellur
Target Milestone:	---	Keywords:	ZStream
Target Release:	RHGS 3.4.0
Hardware:	x86_64
OS:	Linux
Whiteboard:
Fixed In Version:	glusterfs-3.12.2-1	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:
Clones:	1486542 (view as bug list)		Environment:
Last Closed:	2018-09-04 06:29:44 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1472361, 1503135

Description Byreddy 2016-09-22 09:44:55 UTC

Description of problem:
=======================
I am seeing below warning message in glusterd log after node reboot.


[2016-09-22 09:38:33.987943] I [MSGID: 106502] [glusterd-handler.c:2831:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2016-09-22 09:39:37.453790] W [MSGID: 101095] [xlator.c:147:xlator_volopt_dynload] 0-xlator: /usr/lib64/glusterfs/3.8.4/xlator/features/ganesha.so: cannot open shared object file: No such file or directory

Version-Release number of selected component (if applicable):
=============================================================
glusterfs-3.8.4-1


How reproducible:
=================
Always


Steps to Reproduce:
===================
1. Have 3.1.3 setup with simple distribute/replica volume and update it to 3.2 
2. reboot the node
3. Check the glusterd log for the ganesha.so warning message 

Actual results:
===============
Getting "ganesha.so  cannot open shared object file: No such file or directory" warning message in glusterd log after node reboot in non ganesha setup.


Expected results:
==================
This warning should not throw after node reboot.

Additional info:

Comment 2 Niels de Vos 2016-09-29 13:58:43 UTC

Possibly glusterd did not update the .vol files? The upstream glusterfs packaging does this with the command mentioned here:

https://github.com/gluster/glusterfs/blob/master/glusterfs.spec.in#L852

Comment 3 Jiffin 2016-09-30 12:27:08 UTC

I was not able to reproduce this issue.
Steps I followed :
Create a vol in 3.1.3
stop all the gluster services
upgrade to gluster 3.2
Restart gluster services.Still I can't find the warning message.

Am I missing anything?

In your set up whether nodes in cluster have different rhgs versions. I mean one node with rhgs3.1.3 and another with rhgs3.2 ?

I request qa to reproduce issue and give a set up to debug.

@Niels 
Similar change is present in downstream spec file as well.

Comment 4 Jiffin 2016-10-14 06:23:34 UTC

IMHO with current bandwidth it might be difficult fix atm. Since it does not have any huge impact on functionality issue, can we move this bug to 3.2.0-beyond.?

Comment 5 Atin Mukherjee 2016-10-17 09:30:37 UTC

I am moving this BZ to 3.2.0-beyond given there is no functionality impact.

Comment 8 Byreddy 2016-11-28 06:03:08 UTC

I am checking the update from 3.1.3 to 3.2 and found this issue 

[2016-11-28 05:53:23.343467] I [MSGID: 106143] [glusterd-pmap.c:250:pmap_registry_bind] 0-pmap: adding brick /bricks/brick1/br2 on port 49153
[2016-11-28 05:53:43.458663] W [MSGID: 101095] [xlator.c:147:xlator_volopt_dynload] 0-xlator: /usr/lib64/glusterfs/3.8.4/xlator/features/ganesha.so: cannot open shared object file: No such file or directory
[2016-11-28 05:56:56.896

It's very simple to reproduce with the steps mentioned, every time i am seeing this one while testing the update path.

Comment 10 SATHEESARAN 2017-01-19 08:56:35 UTC

(In reply to Byreddy from comment #8)
> I am checking the update from 3.1.3 to 3.2 and found this issue 
> 
> [2016-11-28 05:53:23.343467] I [MSGID: 106143]
> [glusterd-pmap.c:250:pmap_registry_bind] 0-pmap: adding brick
> /bricks/brick1/br2 on port 49153
> [2016-11-28 05:53:43.458663] W [MSGID: 101095]
> [xlator.c:147:xlator_volopt_dynload] 0-xlator:
> /usr/lib64/glusterfs/3.8.4/xlator/features/ganesha.so: cannot open shared
> object file: No such file or directory
> [2016-11-28 05:56:56.896
> 
> It's very simple to reproduce with the steps mentioned, every time i am
> seeing this one while testing the update path.

I am seeing such an error on fresh setup, where I installed RHGS 3.2.0 interim build ( glusterfs-3.8.4-12.el7rhgs ) with RHEL 7.3 platform

Comment 21 Manisha Saini 2018-02-16 12:53:23 UTC

Observing the similar warning message on upgraded setup from 3.3.1 to RHGS 3.4
(Only package upgrade)

#glusterfs-3.12.2-4.el7rhgs.x86_64

[2018-02-15 06:20:43.467752] W [MSGID: 101095] [xlator.c:162:xlator_volopt_dynload] 0-xlator: /usr/lib64/glusterfs/3.12.2/xlator/features/ganesha.so: cannot open shared object file: No such file or directory
[2018-02-16 04:41:02.694805] W [MSGID: 101095] [xlator.c:162:xlator_volopt_dynload] 0-xlator: /usr/lib64/glusterfs/3.12.2/xlator/features/ganesha.so: cannot open shared object file: No such file or directory
[2018-02-16 07:50:04.599384] W [MSGID: 101095] [xlator.c:162:xlator_volopt_dynload] 0-xlator: /usr/lib64/glusterfs/3.12.2/xlator/features/ganesha.so: cannot open shared object file: No such file or directory


Tried reproducing the same on freshly installed RHGS 3.4 setup.Unable to repro. Will try once again on freshly installed setup if I could reproduce and will update the BZ.

As the issue reported in BZ is been observed in upgraded setup,hence moving this BZ to assigned to state

Comment 22 Atin Mukherjee 2018-02-16 15:55:47 UTC

(In reply to Manisha Saini from comment #21)
> Observing the similar warning message on upgraded setup from 3.3.1 to RHGS
> 3.4
> (Only package upgrade)
> 
> #glusterfs-3.12.2-4.el7rhgs.x86_64
> 
> [2018-02-15 06:20:43.467752] W [MSGID: 101095]
> [xlator.c:162:xlator_volopt_dynload] 0-xlator:
> /usr/lib64/glusterfs/3.12.2/xlator/features/ganesha.so: cannot open shared
> object file: No such file or directory
> [2018-02-16 04:41:02.694805] W [MSGID: 101095]
> [xlator.c:162:xlator_volopt_dynload] 0-xlator:
> /usr/lib64/glusterfs/3.12.2/xlator/features/ganesha.so: cannot open shared
> object file: No such file or directory
> [2018-02-16 07:50:04.599384] W [MSGID: 101095]
> [xlator.c:162:xlator_volopt_dynload] 0-xlator:
> /usr/lib64/glusterfs/3.12.2/xlator/features/ganesha.so: cannot open shared
> object file: No such file or directory
> 
> 
> Tried reproducing the same on freshly installed RHGS 3.4 setup.Unable to
> repro. Will try once again on freshly installed setup if I could reproduce
> and will update the BZ.

When the above logs are seen? Is it post upgrade?

> 
> As the issue reported in BZ is been observed in upgraded setup,hence moving
> this BZ to assigned to state

Comment 31 errata-xmlrpc 2018-09-04 06:29:44 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2018:2607