Bug 1018619

Summary: All Glupy translators nonfunctional in upstream GlusterFS git master
Product: [Community] GlusterFS Reporter: Justin Clift <jclift>
Component: coreAssignee: Justin Clift <jclift>
Status: CLOSED CURRENTRELEASE QA Contact:
Severity: urgent Docs Contact:
Priority: urgent    
Version: pre-releaseCC: gluster-bugs, kwade, ppai, rraja
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: glusterfs-3.5.0 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-04-17 11:49:42 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1049981    

Description Justin Clift 2013-10-13 21:38:24 UTC
Description of problem:

  All Glupy based translators in GlusterFS are nonfunctional, failing with
  error "NameError: name 'Translator' is not defined".

  For example, using the "helloworld" translator (shipped with Gluster)
  in the NFS server fails with:

    [2013-10-13 21:16:34.639044] E [glupy.c:2372:init] 0-firstvol-helloworld: Python import failed
    Traceback (most recent call last):
      File "/usr/lib64/glusterfs/3git/xlator/features/glupy/helloworld.py", line 4, in <module>
        class xlator (Translator):
    NameError: name 'Translator' is not defined


Version-Release number of selected component (if applicable):

  Upstream GlusterFS git master head, as of Sun 13th Oct 2013


How reproducible:

  100%


Steps to Reproduce:
1. Create a GlusterFS volume of any type, named "firstvol"
2. Add the "helloworld" Glupy translator to the nfs-server.vol file (example attached to this issue)
3. Kill any existing Gluster NFS server process running, then start a new one manually using the new translator file (enable the --debug option if desired):

     # /usr/sbin/glusterfs -f ~/nfs-server-helloworld.vol --debug
     [2013-10-13 21:32:36.174254] I [glusterfsd.c:1954:main] 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3git (/usr/sbin/glusterfs -f /root/nfs-server-helloworld.vol --debug)
     [2013-10-13 21:32:36.175328] D [glusterfsd.c:593:get_volfp] 0-glusterfsd: loading volume file /root/nfs-server-helloworld.vol
     [2013-10-13 21:32:36.185233] D [rpcsvc.c:1949:rpcsvc_init] 0-rpc-service: RPC service inited.
     [2013-10-13 21:32:36.185263] D [rpcsvc.c:1715:rpcsvc_program_register] 0-rpc-service: New program registered: GF-DUMP, Num: 123451501, Ver: 1, Port: 0
     [2013-10-13 21:32:36.185274] D [nfs.c:398:nfs_init_subvolumes] 0-nfs: Initing subvolume: firstvol
     [2013-10-13 21:32:36.202679] D [nfs.c:121:nfs_init_versions] 0-nfs: Initing protocol versions
     [2013-10-13 21:32:36.202707] D [mount3.c:2573:mnt3svc_init] 0-nfs-mount: Initing Mount v3 state
     [2013-10-13 21:32:36.202751] D [rpc-transport.c:249:rpc_transport_load] 0-rpc-transport: attempt to load file /usr/lib64/glusterfs/3git/rpc-transport/socket.so
     [2013-10-13 21:32:36.207834] I [socket.c:3513:socket_init] 0-socket.nfs-server: SSL support is NOT enabled
     [2013-10-13 21:32:36.207861] I [socket.c:3528:socket_init] 0-socket.nfs-server: using system polling thread
     [2013-10-13 21:32:36.207872] D [name.c:557:server_fill_address_family] 0-socket.nfs-server: option address-family not specified, defaulting to inet
     [2013-10-13 21:32:36.214253] D [nfs.c:139:nfs_init_versions] 0-nfs: Starting program: MOUNT3
     [2013-10-13 21:32:36.214277] D [rpcsvc.c:1715:rpcsvc_program_register] 0-rpc-service: New program registered: MOUNT3, Num: 100005, Ver: 3, Port: 38465
     [2013-10-13 21:32:36.227870] D [mount3.c:2660:mnt1svc_init] 0-nfs-mount: Initing Mount v1 state
     [2013-10-13 21:32:36.227914] D [rpc-transport.c:249:rpc_transport_load] 0-rpc-transport: attempt to load file /usr/lib64/glusterfs/3git/rpc-transport/socket.so
     [2013-10-13 21:32:36.227958] I [socket.c:3513:socket_init] 0-socket.nfs-server: SSL support is NOT enabled
     [2013-10-13 21:32:36.227966] I [socket.c:3528:socket_init] 0-socket.nfs-server: using system polling thread
     [2013-10-13 21:32:36.227973] D [name.c:557:server_fill_address_family] 0-socket.nfs-server: option address-family not specified, defaulting to inet
     [2013-10-13 21:32:36.228102] D [nfs.c:139:nfs_init_versions] 0-nfs: Starting program: MOUNT1
     [2013-10-13 21:32:36.228114] D [rpcsvc.c:1715:rpcsvc_program_register] 0-rpc-service: New program registered: MOUNT1, Num: 100005, Ver: 1, Port: 38466
     [2013-10-13 21:32:36.239486] D [rpc-transport.c:249:rpc_transport_load] 0-rpc-transport: attempt to load file /usr/lib64/glusterfs/3git/rpc-transport/socket.so
     [2013-10-13 21:32:36.239563] I [socket.c:3513:socket_init] 0-socket.nfs-server: SSL support is NOT enabled
     [2013-10-13 21:32:36.239572] I [socket.c:3528:socket_init] 0-socket.nfs-server: using system polling thread
     [2013-10-13 21:32:36.239580] D [name.c:557:server_fill_address_family] 0-socket.nfs-server: option address-family not specified, defaulting to inet
     [2013-10-13 21:32:36.240204] D [nfs.c:139:nfs_init_versions] 0-nfs: Starting program: NFS3
     [2013-10-13 21:32:36.240217] D [rpcsvc.c:1715:rpcsvc_program_register] 0-rpc-service: New program registered: NFS3, Num: 100003, Ver: 3, Port: 2049
     [2013-10-13 21:32:36.240531] D [rpc-transport.c:249:rpc_transport_load] 0-rpc-transport: attempt to load file /usr/lib64/glusterfs/3git/rpc-transport/socket.so
     [2013-10-13 21:32:36.240565] I [socket.c:3513:socket_init] 0-socket.NLM: SSL support is NOT enabled
     [2013-10-13 21:32:36.240573] I [socket.c:3528:socket_init] 0-socket.NLM: using system polling thread
     [2013-10-13 21:32:36.312589] D [nfs.c:139:nfs_init_versions] 0-nfs: Starting program: NLM4
     [2013-10-13 21:32:36.312653] D [rpcsvc.c:1715:rpcsvc_program_register] 0-rpc-service: New program registered: NLM4, Num: 100021, Ver: 4, Port: 38468
     [2013-10-13 21:32:36.313071] D [rpc-transport.c:249:rpc_transport_load] 0-rpc-transport: attempt to load file /usr/lib64/glusterfs/3git/rpc-transport/socket.so
     [2013-10-13 21:32:36.313120] I [socket.c:3513:socket_init] 0-socket.ACL: SSL support is NOT enabled
     [2013-10-13 21:32:36.313129] I [socket.c:3528:socket_init] 0-socket.ACL: using system polling thread
     [2013-10-13 21:32:36.313175] D [nfs.c:139:nfs_init_versions] 0-nfs: Starting program: ACL3
     [2013-10-13 21:32:36.313184] D [rpcsvc.c:1715:rpcsvc_program_register] 0-rpc-service: New program registered: ACL3, Num: 100227, Ver: 3, Port: 2049
     [2013-10-13 21:32:36.313373] D [rpc-drc.c:752:rpcsvc_drc_init] 0-rpc-service: drc type not set. Continuing with default
     [2013-10-13 21:32:36.313388] D [rpc-drc.c:762:rpcsvc_drc_init] 0-rpc-service: drc size not set. Continuing with default size
     [2013-10-13 21:32:36.332143] D [rpc-drc.c:781:rpcsvc_drc_init] 0-rpc-service: drc lru factor not set. Continuing with policy default
     [2013-10-13 21:32:36.332190] D [rpc-drc.c:797:rpcsvc_drc_init] 0-rpc-service: drc init successful
     [2013-10-13 21:32:36.332200] I [nfs.c:931:init] 0-nfs: NFS service started
     [2013-10-13 21:32:36.332239] D [options.c:1109:xlator_option_init_bool] 0-firstvol: option count-fop-hits using set value off
     [2013-10-13 21:32:36.332250] D [options.c:1109:xlator_option_init_bool] 0-firstvol: option latency-measurement using set value off
     [2013-10-13 21:32:36.343974] E [glupy.c:2368:init] 0-firstvol-helloworld: py_mod_name = helloworld
     [2013-10-13 21:32:36.344361] E [glupy.c:2372:init] 0-firstvol-helloworld: Python import failed
     Traceback (most recent call last):
       File "/usr/lib64/glusterfs/3git/xlator/features/glupy/helloworld.py", line 4, in <module>
         class xlator (Translator):
     NameError: name 'Translator' is not defined
     [2013-10-13 21:32:36.344430] E [xlator.c:425:xlator_init] 0-firstvol-helloworld: Initialization of volume 'firstvol-helloworld' failed, review your volfile again
     [2013-10-13 21:32:36.344440] E [graph.c:307:glusterfs_graph_init] 0-firstvol-helloworld: initializing translator failed
     [2013-10-13 21:32:36.344446] E [graph.c:502:glusterfs_graph_activate] 0-graph: init failed
     [2013-10-13 21:32:36.345172] W [glusterfsd.c:1086:cleanup_and_exit] (-->/usr/sbin/glusterfs(main+0x6c7) [0x406a27] (-->/usr/sbin/glusterfs(glusterfs_volumes_init+0xb7) [0x405217] (-->/usr/sbin/glusterfs(glusterfs_process_volfp+0x106) [0x405126]))) 0-: received signum (0), shutting down
     [2013-10-13 21:32:36.345289] D [glusterfsd-mgmt.c:2087:glusterfs_mgmt_pmap_signout] 0-fsd-mgmt: portmapper signout arguments not given

   This fails and exits, whereas it should instead start the NFS server and keep in the foreground.


Additional info:

  * GlusterFS is otherwise installed and operating correctly.  FUSE clients are
    mounting and working fine (without Glupy translator being used).

  * The /usr/lib64/glusterfs/3git/xlator/features/glupy/helloworld.py file is as
    distributed in GlusterFS git.  No modifications have been done.

  * No other Glupy translator seems to work either (tried a few).

Comment 1 Justin Clift 2013-10-13 21:40:04 UTC
Note, this failure is occurring on CentOS 6.4.  Haven't tried on Fedora or RHEL 6.x yet.

Comment 2 Anand Avati 2014-02-12 04:00:42 UTC
REVIEW: http://review.gluster.org/6979 (Rename Glupy module to avoid namespace conflict) posted (#1) for review on master by Justin Clift (jclift)

Comment 3 Justin Clift 2014-02-12 04:02:48 UTC
It's not the greatest approach, since it's still using "import *" on the import line, but it works and isn't regressing anything.

Comment 4 Prashanth Pai 2014-02-12 06:27:16 UTC
Hi Justin, how about we have everything under 'gluster' python package ?

Which is everything would be installed under - "/usr/lib/python2.7/site-packages/gluster"

gluster.swift (gluster-swift project already uses this)
gluster.glupy 
gluster.gfapi (libfgapi-python project)

Comment 5 Justin Clift 2014-02-12 06:41:06 UTC
No real objection to that.  I'll look into updating my patch for this later on today. :)

Comment 6 Anand Avati 2014-02-13 08:58:25 UTC
REVIEW: http://review.gluster.org/6979 (Rename Glupy module to avoid namespace conflict) posted (#2) for review on master by Justin Clift (jclift)

Comment 7 Anand Avati 2014-02-13 10:19:09 UTC
REVIEW: http://review.gluster.org/6979 (Rename Glupy module to avoid namespace conflict) posted (#3) for review on master by Justin Clift (jclift)

Comment 8 Anand Avati 2014-02-16 03:12:37 UTC
REVIEW: http://review.gluster.org/6979 (Rename Glupy module to avoid namespace conflict) posted (#4) for review on master by Justin Clift (jclift)

Comment 9 Anand Avati 2014-02-17 22:59:54 UTC
REVIEW: http://review.gluster.org/6979 (features/glupy: Rename Glupy module to avoid namespace conflict) posted (#5) for review on master by Justin Clift (jclift)

Comment 10 Anand Avati 2014-02-19 01:52:56 UTC
REVIEW: http://review.gluster.org/6979 (features/glupy: Rename Glupy module to avoid namespace conflict) posted (#6) for review on master by Justin Clift (jclift)

Comment 11 Anand Avati 2014-02-19 03:12:16 UTC
REVIEW: http://review.gluster.org/6979 (features/glupy: Rename Glupy module to avoid namespace conflict) posted (#7) for review on master by Justin Clift (jclift)

Comment 12 Anand Avati 2014-02-21 15:33:43 UTC
REVIEW: http://review.gluster.org/6979 (features/glupy: Rename Glupy module to avoid namespace conflict) posted (#8) for review on master by Justin Clift (jclift)

Comment 13 Anand Avati 2014-03-02 17:23:07 UTC
REVIEW: http://review.gluster.org/6979 (features/glupy: Rename Glupy module to avoid namespace conflict) posted (#9) for review on master by Justin Clift (jclift)

Comment 14 Anand Avati 2014-03-02 17:29:19 UTC
REVIEW: http://review.gluster.org/6979 (features/glupy: Rename Glupy module to avoid namespace conflict) posted (#10) for review on master by Justin Clift (jclift)

Comment 15 Anand Avati 2014-03-21 20:23:32 UTC
REVIEW: http://review.gluster.org/6979 (features/glupy: Rename Glupy python module to avoid namespace conflict) posted (#11) for review on master by Justin Clift (justin)

Comment 16 Anand Avati 2014-03-22 03:47:14 UTC
REVIEW: http://review.gluster.org/7316 (features/glupy: Rename Glupy python module to avoid namespace conflict) posted (#1) for review on release-3.5 by Justin Clift (justin)

Comment 17 Anand Avati 2014-03-24 17:31:34 UTC
COMMIT: http://review.gluster.org/6979 committed in master by Vijay Bellur (vbellur) 
------
commit f7a815a2d0e7e9d7ed1ec2da587790bd3ddda9e5
Author: Justin Clift <justin>
Date:   Fri Mar 21 20:13:16 2014 +0000

    features/glupy: Rename Glupy python module to avoid namespace conflict
    
    * Rename gluster.py to glupy.py to avoid namespace conflict (#1018619)
    * Move the main Glupy files into glusterfs-extra-xlators rpm
    * Move the Glupy Translator examples into glusterfs-devel rpm
    * Add Glupy entry to the MAINTAINERS file
    
    BUG: 1018619
    Change-Id: I48de598ba5ae8eec0e7e276bbcca1abb0e549cef
    Signed-off-by: Justin Clift <justin>
    Reviewed-on: http://review.gluster.org/6979
    Reviewed-by: Kaleb KEITHLEY <kkeithle>
    Reviewed-by: Jeff Darcy <jdarcy>

Comment 18 Anand Avati 2014-03-24 17:32:11 UTC
COMMIT: http://review.gluster.org/7316 committed in release-3.5 by Vijay Bellur (vbellur) 
------
commit 446547559952e3b8af943f87c7475b80dd090153
Author: Justin Clift <justin>
Date:   Fri Mar 21 13:18:29 2014 +0000

    features/glupy: Rename Glupy python module to avoid namespace conflict
    
    * Rename gluster.py to glupy.py to avoid namespace conflict (#1018619)
    * Move the main Glupy files into glusterfs-extra-xlators rpm
    * Move the Glupy Translator examples into glusterfs-devel rpm
    
    Backport of: http://review.gluster.org/#/c/6979/
    
    Change-Id: Ie9b71b56502f8e98c527ade381c918446bc7f022
    BUG: 1018619
    Signed-off-by: Justin Clift <justin>
    Reviewed-on: http://review.gluster.org/7316
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Kaleb KEITHLEY <kkeithle>
    Reviewed-by: Jeff Darcy <jdarcy>

Comment 19 Niels de Vos 2014-04-17 11:49:42 UTC
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.5.0, please reopen this bug report.

glusterfs-3.5.0 has been announced on the Gluster Developers mailinglist [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://thread.gmane.org/gmane.comp.file-systems.gluster.devel/6137
[2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user