Description of problem: ----------------------- While performing 'In-service Software Upgrade', glusterfs package update should be prevented, when there are glusterfsd, glusterfs, glusterd processes are running. The same is the case when there are distributed volume is running. But while upgrading from RHS 3.0.3-async to latest RHS 3.0.4 build, glusterfs-geo-replication and glusterfs-cli packages are getting updated, even when there is a distributed volume or glusterfsd( brick process ) of replicate/distributed-replicate volumes up / running. Version-Release number of selected component (if applicable): ------------------------------------------------------------- glusterfs-3.6.0.48-1.el6rhs glusterfs-3.6.0.42-1.1.el6rhs How reproducible: ----------------- Always Steps to Reproduce: ------------------- 1. Install RHS 3.0.3 async update ( glusterfs-3.6.0.42-1.1.el6rhs ) 2. Create a distributed volume and start it 3. Upgrade to latest RHS-3.0.4 candidate ( glusterfs-3.6.0.48-1.el6rhs ) [or] 1. Install RHS 3.0.3 async update ( glusterfs-3.6.0.42-1.1.el6rhs ) 2. Create a distributed-replicate or replicate volume and start it 3. Upgrade to latest RHS-3.0.4 candidate ( glusterfs-3.6.0.48-1.el6rhs ) Actual results: --------------- glusterfs-geo-replication and glusterfs-cli packages are updated Expected results: ----------------- Updating all glusterfs packages should be prevented, as there are distribute volume is not stopped. The same holds true for any existing replicate or distributed-replicate volume Additional info: ---------------- Refer - https://bugzilla.redhat.com/show_bug.cgi?id=1016385#c14
Providinf the console log, when upgrading the setup with replicate volume. You can observe that the glusterfs-geo-replication and glusterfs-cli packages are updated. [root@ ~]# yum update glusterfs* --disablerepo=* --enablerepo=rhs Loaded plugins: aliases, changelog, downloadonly, product-id, security, : subscription-manager, tmprepo, verify, versionlock This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register. Setting up Update Process rhs | 2.9 kB 00:00 rhs/primary_db | 9.8 kB 00:00 Resolving Dependencies --> Running transaction check ---> Package glusterfs.x86_64 0:3.6.0.42-1.el6rhs will be updated ---> Package glusterfs.x86_64 0:3.6.0.48-1.el6rhs will be an update ---> Package glusterfs-api.x86_64 0:3.6.0.42-1.el6rhs will be updated ---> Package glusterfs-api.x86_64 0:3.6.0.48-1.el6rhs will be an update ---> Package glusterfs-cli.x86_64 0:3.6.0.42-1.el6rhs will be updated ---> Package glusterfs-cli.x86_64 0:3.6.0.48-1.el6rhs will be an update ---> Package glusterfs-fuse.x86_64 0:3.6.0.42-1.el6rhs will be updated ---> Package glusterfs-fuse.x86_64 0:3.6.0.48-1.el6rhs will be an update ---> Package glusterfs-geo-replication.x86_64 0:3.6.0.42-1.el6rhs will be updated ---> Package glusterfs-geo-replication.x86_64 0:3.6.0.48-1.el6rhs will be an update ---> Package glusterfs-libs.x86_64 0:3.6.0.42-1.el6rhs will be updated ---> Package glusterfs-libs.x86_64 0:3.6.0.48-1.el6rhs will be an update ---> Package glusterfs-rdma.x86_64 0:3.6.0.42-1.el6rhs will be updated ---> Package glusterfs-rdma.x86_64 0:3.6.0.48-1.el6rhs will be an update ---> Package glusterfs-server.x86_64 0:3.6.0.42-1.el6rhs will be updated ---> Package glusterfs-server.x86_64 0:3.6.0.48-1.el6rhs will be an update --> Finished Dependency Resolution Dependencies Resolved ================================================================================== Package Arch Version Repository Size ================================================================================== Updating: glusterfs x86_64 3.6.0.48-1.el6rhs rhs 1.3 M glusterfs-api x86_64 3.6.0.48-1.el6rhs rhs 59 k glusterfs-cli x86_64 3.6.0.48-1.el6rhs rhs 118 k glusterfs-fuse x86_64 3.6.0.48-1.el6rhs rhs 80 k glusterfs-geo-replication x86_64 3.6.0.48-1.el6rhs rhs 154 k glusterfs-libs x86_64 3.6.0.48-1.el6rhs rhs 269 k glusterfs-rdma x86_64 3.6.0.48-1.el6rhs rhs 38 k glusterfs-server x86_64 3.6.0.48-1.el6rhs rhs 704 k Transaction Summary ================================================================================== Upgrade 8 Package(s) Total download size: 2.7 M Is this ok [y/N]: y Downloading Packages: (1/8): glusterfs-3.6.0.48-1.el6rhs.x86_64.rpm | 1.3 MB 00:00 (2/8): glusterfs-api-3.6.0.48-1.el6rhs.x86_64.rpm | 59 kB 00:00 (3/8): glusterfs-cli-3.6.0.48-1.el6rhs.x86_64.rpm | 118 kB 00:00 (4/8): glusterfs-fuse-3.6.0.48-1.el6rhs.x86_64.rpm | 80 kB 00:00 (5/8): glusterfs-geo-replication-3.6.0.48-1.el6rhs.x86_64. | 154 kB 00:00 (6/8): glusterfs-libs-3.6.0.48-1.el6rhs.x86_64.rpm | 269 kB 00:00 (7/8): glusterfs-rdma-3.6.0.48-1.el6rhs.x86_64.rpm | 38 kB 00:00 (8/8): glusterfs-server-3.6.0.48-1.el6rhs.x86_64.rpm | 704 kB 00:00 ---------------------------------------------------------------------------------- Total 42 MB/s | 2.7 MB 00:00 Running rpm_check_debug Running Transaction Test Transaction Test Succeeded Running Transaction Error in PRETRANS scriptlet in rpm package glusterfs-libs-3.6.0.48-1.el6rhs.x86_64 error: lua script failed: Detected running glusterfs processes WARNING: Updating glusterfs requires its processes to be killed. This action does NOT incur downtime. WARNING: Ensure to wait for the upgraded server to finish healing before proceeding. WARNING: Refer upgrade section of install guide for more details Please run # service glusterd stop; pkill glusterfs; pkill glusterfsd; pkill gsyncd.py; Error in PRETRANS scriptlet in rpm package glusterfs-3.6.0.48-1.el6rhs.x86_64 error: lua script failed: Detected running glusterfs processes Error in PRETRANS scriptlet in rpm package glusterfs-api-3.6.0.48-1.el6rhs.x86_64 error: lua script failed: Detected running glusterfs processes Error in PRETRANS scriptlet in rpm package glusterfs-fuse-3.6.0.48-1.el6rhs.x86_64 error: lua script failed: Detected running glusterfs processes Error in PRETRANS scriptlet in rpm package glusterfs-server-3.6.0.48-1.el6rhs.x86_64 error: lua script failed: Detected running glusterfs processes Error in PRETRANS scriptlet in rpm package glusterfs-rdma-3.6.0.48-1.el6rhs.x86_64 error: lua script failed: Detected running glusterfs processes Updating : glusterfs-cli-3.6.0.48-1.el6rhs.x86_64 1/16 Updating : glusterfs-geo-replication-3.6.0.48-1.el6rhs.x86_64 2/16 Cleanup : glusterfs-geo-replication-3.6.0.42-1.el6rhs.x86_64 3/16 Cleanup : glusterfs-cli-3.6.0.42-1.el6rhs.x86_64 4/16 Verifying : glusterfs-cli-3.6.0.48-1.el6rhs.x86_64 1/16 Verifying : glusterfs-geo-replication-3.6.0.48-1.el6rhs.x86_64 2/16 Verifying : glusterfs-geo-replication-3.6.0.42-1.el6rhs.x86_64 3/16 glusterfs-server-3.6.0.42-1.el6rhs.x86_64 was supposed to be removed but is not! Verifying : glusterfs-server-3.6.0.42-1.el6rhs.x86_64 4/16 glusterfs-libs-3.6.0.42-1.el6rhs.x86_64 was supposed to be removed but is not! Verifying : glusterfs-libs-3.6.0.42-1.el6rhs.x86_64 5/16 glusterfs-api-3.6.0.42-1.el6rhs.x86_64 was supposed to be removed but is not! Verifying : glusterfs-api-3.6.0.42-1.el6rhs.x86_64 6/16 Verifying : glusterfs-libs-3.6.0.48-1.el6rhs.x86_64 7/16 Verifying : glusterfs-fuse-3.6.0.48-1.el6rhs.x86_64 8/16 glusterfs-3.6.0.42-1.el6rhs.x86_64 was supposed to be removed but is not! Verifying : glusterfs-3.6.0.42-1.el6rhs.x86_64 9/16 glusterfs-fuse-3.6.0.42-1.el6rhs.x86_64 was supposed to be removed but is not! Verifying : glusterfs-fuse-3.6.0.42-1.el6rhs.x86_64 10/16 Verifying : glusterfs-3.6.0.48-1.el6rhs.x86_64 11/16 Verifying : glusterfs-cli-3.6.0.42-1.el6rhs.x86_64 12/16 Verifying : glusterfs-server-3.6.0.48-1.el6rhs.x86_64 13/16 Verifying : glusterfs-api-3.6.0.48-1.el6rhs.x86_64 14/16 glusterfs-rdma-3.6.0.42-1.el6rhs.x86_64 was supposed to be removed but is not! Verifying : glusterfs-rdma-3.6.0.42-1.el6rhs.x86_64 15/16 Verifying : glusterfs-rdma-3.6.0.48-1.el6rhs.x86_64 16/16 Updated: glusterfs-cli.x86_64 0:3.6.0.48-1.el6rhs glusterfs-geo-replication.x86_64 0:3.6.0.48-1.el6rhs Failed: glusterfs.x86_64 0:3.6.0.42-1.el6rhs glusterfs.x86_64 0:3.6.0.48-1.el6rhs glusterfs-api.x86_64 0:3.6.0.42-1.el6rhs glusterfs-api.x86_64 0:3.6.0.48-1.el6rhs glusterfs-fuse.x86_64 0:3.6.0.42-1.el6rhs glusterfs-fuse.x86_64 0:3.6.0.48-1.el6rhs glusterfs-libs.x86_64 0:3.6.0.42-1.el6rhs glusterfs-libs.x86_64 0:3.6.0.48-1.el6rhs glusterfs-rdma.x86_64 0:3.6.0.42-1.el6rhs glusterfs-rdma.x86_64 0:3.6.0.48-1.el6rhs glusterfs-server.x86_64 0:3.6.0.42-1.el6rhs glusterfs-server.x86_64 0:3.6.0.48-1.el6rhs
Providing the console log, when upgrading the setup with distribute volume. You can observe that the glusterfs-geo-replication and glusterfs-cli packages are updated. [root@ ~]# yum update glusterfs* --disablerepo=* --enablerepo=rhs-async Loaded plugins: aliases, changelog, downloadonly, product-id, security, : subscription-manager, tmprepo, verify, versionlock This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register. Setting up Update Process rhs-async | 2.9 kB 00:00 rhs-async/primary_db | 9.5 kB 00:00 Resolving Dependencies --> Running transaction check ---> Package glusterfs.x86_64 0:3.6.0.42-1.el6rhs will be updated ---> Package glusterfs.x86_64 0:3.6.0.42.1-1.el6rhs will be an update ---> Package glusterfs-api.x86_64 0:3.6.0.42-1.el6rhs will be updated ---> Package glusterfs-api.x86_64 0:3.6.0.42.1-1.el6rhs will be an update ---> Package glusterfs-cli.x86_64 0:3.6.0.42-1.el6rhs will be updated ---> Package glusterfs-cli.x86_64 0:3.6.0.42.1-1.el6rhs will be an update ---> Package glusterfs-fuse.x86_64 0:3.6.0.42-1.el6rhs will be updated ---> Package glusterfs-fuse.x86_64 0:3.6.0.42.1-1.el6rhs will be an update ---> Package glusterfs-geo-replication.x86_64 0:3.6.0.42-1.el6rhs will be updated ---> Package glusterfs-geo-replication.x86_64 0:3.6.0.42.1-1.el6rhs will be an update ---> Package glusterfs-libs.x86_64 0:3.6.0.42-1.el6rhs will be updated ---> Package glusterfs-libs.x86_64 0:3.6.0.42.1-1.el6rhs will be an update ---> Package glusterfs-rdma.x86_64 0:3.6.0.42-1.el6rhs will be updated ---> Package glusterfs-rdma.x86_64 0:3.6.0.42.1-1.el6rhs will be an update ---> Package glusterfs-server.x86_64 0:3.6.0.42-1.el6rhs will be updated ---> Package glusterfs-server.x86_64 0:3.6.0.42.1-1.el6rhs will be an update --> Finished Dependency Resolution Dependencies Resolved ================================================================================== Package Arch Version Repository Size ================================================================================== Updating: glusterfs x86_64 3.6.0.42.1-1.el6rhs rhs-async 1.3 M glusterfs-api x86_64 3.6.0.42.1-1.el6rhs rhs-async 59 k glusterfs-cli x86_64 3.6.0.42.1-1.el6rhs rhs-async 116 k glusterfs-fuse x86_64 3.6.0.42.1-1.el6rhs rhs-async 80 k glusterfs-geo-replication x86_64 3.6.0.42.1-1.el6rhs rhs-async 152 k glusterfs-libs x86_64 3.6.0.42.1-1.el6rhs rhs-async 266 k glusterfs-rdma x86_64 3.6.0.42.1-1.el6rhs rhs-async 37 k glusterfs-server x86_64 3.6.0.42.1-1.el6rhs rhs-async 699 k Transaction Summary ================================================================================== Upgrade 8 Package(s) Total download size: 2.7 M Is this ok [y/N]: y Downloading Packages: (1/8): glusterfs-3.6.0.42.1-1.el6rhs.x86_64.rpm | 1.3 MB 00:00 (2/8): glusterfs-api-3.6.0.42.1-1.el6rhs.x86_64.rpm | 59 kB 00:00 (3/8): glusterfs-cli-3.6.0.42.1-1.el6rhs.x86_64.rpm | 116 kB 00:00 (4/8): glusterfs-fuse-3.6.0.42.1-1.el6rhs.x86_64.rpm | 80 kB 00:00 (5/8): glusterfs-geo-replication-3.6.0.42.1-1.el6rhs.x86_6 | 152 kB 00:00 (6/8): glusterfs-libs-3.6.0.42.1-1.el6rhs.x86_64.rpm | 266 kB 00:00 (7/8): glusterfs-rdma-3.6.0.42.1-1.el6rhs.x86_64.rpm | 37 kB 00:00 (8/8): glusterfs-server-3.6.0.42.1-1.el6rhs.x86_64.rpm | 699 kB 00:00 ---------------------------------------------------------------------------------- Total 44 MB/s | 2.7 MB 00:00 Running rpm_check_debug Running Transaction Test Transaction Test Succeeded Running Transaction Error in PRETRANS scriptlet in rpm package glusterfs-libs-3.6.0.42.1-1.el6rhs.x86_64 error: lua script failed: Detected running glusterfs processes ERROR: Distribute volumes detected. In-service rolling upgrade requires distribute volume(s) to be stopped. ERROR: Please stop distribute volume(s) before proceeding... exiting! Error in PRETRANS scriptlet in rpm package glusterfs-3.6.0.42.1-1.el6rhs.x86_64 error: lua script failed: Detected running glusterfs processes Error in PRETRANS scriptlet in rpm package glusterfs-api-3.6.0.42.1-1.el6rhs.x86_64 error: lua script failed: Detected running glusterfs processes Error in PRETRANS scriptlet in rpm package glusterfs-fuse-3.6.0.42.1-1.el6rhs.x86_64 error: lua script failed: Detected running glusterfs processes Error in PRETRANS scriptlet in rpm package glusterfs-server-3.6.0.42.1-1.el6rhs.x86_64 error: lua script failed: Detected running glusterfs processes Error in PRETRANS scriptlet in rpm package glusterfs-rdma-3.6.0.42.1-1.el6rhs.x86_64 error: lua script failed: Detected running glusterfs processes Updating : glusterfs-cli-3.6.0.42.1-1.el6rhs.x86_64 1/16 Updating : glusterfs-geo-replication-3.6.0.42.1-1.el6rhs.x86_64 2/16 Cleanup : glusterfs-geo-replication-3.6.0.42-1.el6rhs.x86_64 3/16 Cleanup : glusterfs-cli-3.6.0.42-1.el6rhs.x86_64 4/16 Verifying : glusterfs-geo-replication-3.6.0.42.1-1.el6rhs.x86_64 1/16 Verifying : glusterfs-cli-3.6.0.42.1-1.el6rhs.x86_64 2/16 Verifying : glusterfs-cli-3.6.0.42-1.el6rhs.x86_64 3/16 glusterfs-server-3.6.0.42-1.el6rhs.x86_64 was supposed to be removed but is not! Verifying : glusterfs-server-3.6.0.42-1.el6rhs.x86_64 4/16 glusterfs-libs-3.6.0.42-1.el6rhs.x86_64 was supposed to be removed but is not! Verifying : glusterfs-libs-3.6.0.42-1.el6rhs.x86_64 5/16 Verifying : glusterfs-api-3.6.0.42.1-1.el6rhs.x86_64 6/16 Verifying : glusterfs-rdma-3.6.0.42.1-1.el6rhs.x86_64 7/16 glusterfs-3.6.0.42-1.el6rhs.x86_64 was supposed to be removed but is not! Verifying : glusterfs-3.6.0.42-1.el6rhs.x86_64 8/16 glusterfs-fuse-3.6.0.42-1.el6rhs.x86_64 was supposed to be removed but is not! Verifying : glusterfs-fuse-3.6.0.42-1.el6rhs.x86_64 9/16 Verifying : glusterfs-libs-3.6.0.42.1-1.el6rhs.x86_64 10/16 glusterfs-api-3.6.0.42-1.el6rhs.x86_64 was supposed to be removed but is not! Verifying : glusterfs-api-3.6.0.42-1.el6rhs.x86_64 11/16 Verifying : glusterfs-server-3.6.0.42.1-1.el6rhs.x86_64 12/16 Verifying : glusterfs-geo-replication-3.6.0.42-1.el6rhs.x86_64 13/16 Verifying : glusterfs-3.6.0.42.1-1.el6rhs.x86_64 14/16 glusterfs-rdma-3.6.0.42-1.el6rhs.x86_64 was supposed to be removed but is not! Verifying : glusterfs-rdma-3.6.0.42-1.el6rhs.x86_64 15/16 Verifying : glusterfs-fuse-3.6.0.42.1-1.el6rhs.x86_64 16/16 Updated: glusterfs-cli.x86_64 0:3.6.0.42.1-1.el6rhs glusterfs-geo-replication.x86_64 0:3.6.0.42.1-1.el6rhs Failed: glusterfs.x86_64 0:3.6.0.42-1.el6rhs glusterfs.x86_64 0:3.6.0.42.1-1.el6rhs glusterfs-api.x86_64 0:3.6.0.42-1.el6rhs glusterfs-api.x86_64 0:3.6.0.42.1-1.el6rhs glusterfs-fuse.x86_64 0:3.6.0.42-1.el6rhs glusterfs-fuse.x86_64 0:3.6.0.42.1-1.el6rhs glusterfs-libs.x86_64 0:3.6.0.42-1.el6rhs glusterfs-libs.x86_64 0:3.6.0.42.1-1.el6rhs glusterfs-rdma.x86_64 0:3.6.0.42-1.el6rhs glusterfs-rdma.x86_64 0:3.6.0.42.1-1.el6rhs glusterfs-server.x86_64 0:3.6.0.42-1.el6rhs glusterfs-server.x86_64 0:3.6.0.42.1-1.el6rhs Complete!
RHSS Cli stopped respnding post comment1 and comment2, due to the mix of packages now. [root@d~]# gluster volume status [root@~]# gluster volume list [root@~]# gluster pe s [root@~]# ps aux | grep glusterfsd root 5050 0.0 2.5 811436 26412 ? Ssl Mar03 0:04 /usr/sbin/glusterfsd -s dhcp37-220.lab.eng.blr.redhat.com --volfile-id rep.dhcp37-220.lab.eng.blr.redhat.com.tmp-6 -p /var/lib/glusterd/vols/rep/run/dhcp37-220.lab.eng.blr.redhat.com-tmp-6.pid -S /var/run/7145c16dc42f2275cbddd2178c14b44b.socket --brick-name /tmp/6 -l /var/log/glusterfs/bricks/tmp-6.log --xlator-option *-posix.glusterd-uuid=1e12a8b3-32a8-4cf6-aab1-7e551a12be72 --brick-port 49152 --xlator-option rep-server.listen-port=49152 root 6572 0.0 0.0 103252 840 pts/0 S+ 02:30 0:00 grep glusterfsd [root@ ~]# pidof glusterfsd 5050 I have killed this brick process and proceeded to upgrade once again. All worked well. [root@~]# kill 5050 [root@~]# pkill glusterfsd [root@~]# pkill glusterfs [root@~]# service glusterd stop [ OK ] [root@~]# yum update glusterfs* --disablerepo=* --enablerepo=rhs Loaded plugins: aliases, changelog, downloadonly, product-id, security, : subscription-manager, tmprepo, verify, versionlock This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register. Repository 'rhs' is missing name in configuration, using id Setting up Update Process Resolving Dependencies --> Running transaction check ---> Package glusterfs.x86_64 0:3.6.0.42-1.el6rhs will be updated ---> Package glusterfs.x86_64 0:3.6.0.48-1.el6rhs will be an update ---> Package glusterfs-api.x86_64 0:3.6.0.42-1.el6rhs will be updated ---> Package glusterfs-api.x86_64 0:3.6.0.48-1.el6rhs will be an update ---> Package glusterfs-fuse.x86_64 0:3.6.0.42-1.el6rhs will be updated ---> Package glusterfs-fuse.x86_64 0:3.6.0.48-1.el6rhs will be an update ---> Package glusterfs-libs.x86_64 0:3.6.0.42-1.el6rhs will be updated ---> Package glusterfs-libs.x86_64 0:3.6.0.48-1.el6rhs will be an update ---> Package glusterfs-rdma.x86_64 0:3.6.0.42-1.el6rhs will be updated ---> Package glusterfs-rdma.x86_64 0:3.6.0.48-1.el6rhs will be an update ---> Package glusterfs-server.x86_64 0:3.6.0.42-1.el6rhs will be updated ---> Package glusterfs-server.x86_64 0:3.6.0.48-1.el6rhs will be an update --> Finished Dependency Resolution Dependencies Resolved ================================================================================== Package Arch Version Repository Size ================================================================================== Updating: glusterfs x86_64 3.6.0.48-1.el6rhs rhs 1.3 M glusterfs-api x86_64 3.6.0.48-1.el6rhs rhs 59 k glusterfs-fuse x86_64 3.6.0.48-1.el6rhs rhs 80 k glusterfs-libs x86_64 3.6.0.48-1.el6rhs rhs 269 k glusterfs-rdma x86_64 3.6.0.48-1.el6rhs rhs 38 k glusterfs-server x86_64 3.6.0.48-1.el6rhs rhs 704 k Transaction Summary ================================================================================== Upgrade 6 Package(s) Total download size: 2.4 M Is this ok [y/N]: y Downloading Packages: (1/6): glusterfs-3.6.0.48-1.el6rhs.x86_64.rpm | 1.3 MB 00:00 (2/6): glusterfs-api-3.6.0.48-1.el6rhs.x86_64.rpm | 59 kB 00:00 (3/6): glusterfs-fuse-3.6.0.48-1.el6rhs.x86_64.rpm | 80 kB 00:00 (4/6): glusterfs-libs-3.6.0.48-1.el6rhs.x86_64.rpm | 269 kB 00:00 (5/6): glusterfs-rdma-3.6.0.48-1.el6rhs.x86_64.rpm | 38 kB 00:00 (6/6): glusterfs-server-3.6.0.48-1.el6rhs.x86_64.rpm | 704 kB 00:00 ---------------------------------------------------------------------------------- Total 46 MB/s | 2.4 MB 00:00 Running rpm_check_debug Running Transaction Test Transaction Test Succeeded Running Transaction Updating : glusterfs-libs-3.6.0.48-1.el6rhs.x86_64 1/12 Updating : glusterfs-3.6.0.48-1.el6rhs.x86_64 2/12 Updating : glusterfs-api-3.6.0.48-1.el6rhs.x86_64 3/12 Updating : glusterfs-fuse-3.6.0.48-1.el6rhs.x86_64 4/12 Updating : glusterfs-server-3.6.0.48-1.el6rhs.x86_64 5/12 error reading information on service glusterfsd: No such file or directory warning: /var/lib/glusterd/vols/rep/rep.tcp-fuse.vol saved as /var/lib/glusterd/vols/rep/rep.tcp-fuse.vol.rpmsave warning: /var/lib/glusterd/vols/rep/rep.dhcp37-170.lab.eng.blr.redhat.com.tmp-6.vol saved as /var/lib/glusterd/vols/rep/rep.dhcp37-170.lab.eng.blr.redhat.com.tmp-6.vol.rpmsave warning: /var/lib/glusterd/vols/rep/trusted-rep.tcp-fuse.vol saved as /var/lib/glusterd/vols/rep/trusted-rep.tcp-fuse.vol.rpmsave warning: /var/lib/glusterd/vols/rep/rep-rebalance.vol saved as /var/lib/glusterd/vols/rep/rep-rebalance.vol.rpmsave warning: /var/lib/glusterd/vols/rep/rep.dhcp37-220.lab.eng.blr.redhat.com.tmp-6.vol saved as /var/lib/glusterd/vols/rep/rep.dhcp37-220.lab.eng.blr.redhat.com.tmp-6.vol.rpmsave Updating : glusterfs-rdma-3.6.0.48-1.el6rhs.x86_64 6/12 Cleanup : glusterfs-server-3.6.0.42-1.el6rhs.x86_64 7/12 Cleanup : glusterfs-rdma-3.6.0.42-1.el6rhs.x86_64 8/12 Cleanup : glusterfs-fuse-3.6.0.42-1.el6rhs.x86_64 9/12 Cleanup : glusterfs-api-3.6.0.42-1.el6rhs.x86_64 10/12 Cleanup : glusterfs-3.6.0.42-1.el6rhs.x86_64 11/12 Cleanup : glusterfs-libs-3.6.0.42-1.el6rhs.x86_64 12/12 Verifying : glusterfs-libs-3.6.0.48-1.el6rhs.x86_64 1/12 Verifying : glusterfs-fuse-3.6.0.48-1.el6rhs.x86_64 2/12 Verifying : glusterfs-3.6.0.48-1.el6rhs.x86_64 3/12 Verifying : glusterfs-api-3.6.0.48-1.el6rhs.x86_64 4/12 Verifying : glusterfs-server-3.6.0.48-1.el6rhs.x86_64 5/12 Verifying : glusterfs-rdma-3.6.0.48-1.el6rhs.x86_64 6/12 Verifying : glusterfs-server-3.6.0.42-1.el6rhs.x86_64 7/12 Verifying : glusterfs-libs-3.6.0.42-1.el6rhs.x86_64 8/12 Verifying : glusterfs-3.6.0.42-1.el6rhs.x86_64 9/12 Verifying : glusterfs-fuse-3.6.0.42-1.el6rhs.x86_64 10/12 Verifying : glusterfs-api-3.6.0.42-1.el6rhs.x86_64 11/12 Verifying : glusterfs-rdma-3.6.0.42-1.el6rhs.x86_64 12/12 Updated: glusterfs.x86_64 0:3.6.0.48-1.el6rhs glusterfs-api.x86_64 0:3.6.0.48-1.el6rhs glusterfs-fuse.x86_64 0:3.6.0.48-1.el6rhs glusterfs-libs.x86_64 0:3.6.0.48-1.el6rhs glusterfs-rdma.x86_64 0:3.6.0.48-1.el6rhs glusterfs-server.x86_64 0:3.6.0.48-1.el6rhs Complete!
I have seen the same issue while upgrading from RHS 3.0.3 ( glusterfs-3.6.0.42-1.el6rhs ) to RHS 3.0.4 ( glusterfs-3.6.0.53-1.el6rhs ) gluster cli package and glusterfs-geo-replication package got updated and gluster-cli is no longer responding cli.log had the following: -------------------------- [2015-04-08 13:41:40.973056] T [cli.c:264:cli_rpc_notify] 0-glusterfs: got RPC_CLNT_CONNECT [2015-04-08 13:41:40.973080] T [cli-quotad-client.c:94:cli_quotad_notify] 0-glusterfs: got RPC_CLNT_CONNECT [2015-04-08 13:41:40.973090] I [socket.c:2263:socket_event_handler] 0-transport: disconnecting now [2015-04-08 13:41:40.973111] T [cli-quotad-client.c:100:cli_quotad_notify] 0-glusterfs: got RPC_CLNT_DISCONNECT [2015-04-08 13:41:40.973358] T [rpc-clnt.c:1381:rpc_clnt_record] 0-glusterfs: Auth Info: pid: 0, uid: 0, gid: 0, owner: [2015-04-08 13:41:40.973427] T [rpc-clnt.c:1238:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 120, payload: 56, rpc hdr: 64 [2015-04-08 13:41:40.973807] T [socket.c:2732:socket_connect] (-->gluster(cli_cmd_submit+0x8e) [0x40b38e] (-->gluster(cli_submit_request+0xdb) [0x40a57b] (-->/usr/lib64/libgfrpc.so.0(rpc_clnt_submit+0x468) [0x35c880ece8]))) 0-glusterfs: connect () called on transport already connected [2015-04-08 13:41:40.973929] T [rpc-clnt.c:1573:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x1 Program: Gluster CLI, ProgVers: 2, Proc: 27) to rpc-transport (glusterfs) [2015-04-08 13:41:40.973965] D [rpc-clnt-ping.c:231:rpc_clnt_start_ping] 0-glusterfs: ping timeout is 0, returning [2015-04-08 13:41:40.980219] T [rpc-clnt.c:660:rpc_clnt_reply_init] 0-glusterfs: received rpc message (RPC XID: 0x1 Program: Gluster CLI, ProgVers: 2, Proc: 27) from rpc-transport (glusterfs) [2015-04-08 13:41:40.980279] D [cli-rpc-ops.c:6498:gf_cli_status_cbk] 0-cli: Received response to status cmd [2015-04-08 13:41:40.980343] D [cli-cmd.c:388:cli_cmd_submit] 0-cli: Returning 0 [2015-04-08 13:41:40.980365] D [cli-rpc-ops.c:6761:gf_cli_status_volume] 0-cli: Returning: 0 [2015-04-08 13:41:40.980405] T [rpc-clnt.c:1381:rpc_clnt_record] 0-glusterfs: Auth Info: pid: 0, uid: 0, gid: 0, owner: [2015-04-08 13:41:40.980427] T [rpc-clnt.c:1238:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 144, payload: 80, rpc hdr: 64 [2015-04-08 13:41:40.980499] T [socket.c:2732:socket_connect] (-->gluster(cli_cmd_submit+0x8e) [0x40b38e] (-->gluster(cli_submit_request+0xdb) [0x40a57b] (-->/usr/lib64/libgfrpc.so.0(rpc_clnt_submit+0x468) [0x35c880ece8]))) 0-glusterfs: connect () called on transport already connected [2015-04-08 13:41:40.980534] T [rpc-clnt.c:1573:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x2 Program: Gluster CLI, ProgVers: 2, Proc: 27) to rpc-transport (glusterfs) [2015-04-08 13:41:40.980555] D [rpc-clnt-ping.c:231:rpc_clnt_start_ping] 0-glusterfs: ping timeout is 0, returning [2015-04-08 13:41:40.991180] T [rpc-clnt.c:660:rpc_clnt_reply_init] 0-glusterfs: received rpc message (RPC XID: 0x2 Program: Gluster CLI, ProgVers: 2, Proc: 27) from rpc-transport (glusterfs) [2015-04-08 13:41:40.991256] D [cli-rpc-ops.c:6498:gf_cli_status_cbk] 0-cli: Received response to status cmd [2015-04-08 13:41:40.991579] D [cli-cmd.c:388:cli_cmd_submit] 0-cli: Returning 0 [2015-04-08 13:41:40.991604] D [cli-rpc-ops.c:6761:gf_cli_status_volume] 0-cli: Returning: 0 [2015-04-08 13:41:40.991630] I [input.c:36:cli_batch] 0-: Exiting with: 0 [2015-04-08 13:46:14.532670] E [mem-pool.c:353:mem_get0] (-->gluster(cli_rpc_init+0x2c) [0x40987c] (-->/usr/lib64/libglusterfs.so.0(dict_new+0xb) [0x35c801bd6b] (-->/usr/lib64/libglusterfs.so.0(get_new_dict_full+0x25) [0x35c801bc25]))) 0-mem-pool: invalid argument [2015-04-08 13:46:21.163614] E [mem-pool.c:353:mem_get0] (-->gluster(cli_rpc_init+0x2c) [0x40987c] (-->/usr/lib64/libglusterfs.so.0(dict_new+0xb) [0x35c801bd6b] (-->/usr/lib64/libglusterfs.so.0(get_new_dict_full+0x25) [0x35c801bc25]))) 0-mem-pool: invalid argument [2015-04-08 13:46:25.613171] E [mem-pool.c:353:mem_get0] (-->gluster(cli_rpc_init+0x2c) [0x40987c] (-->/usr/lib64/libglusterfs.so.0(dict_new+0xb) [0x35c801bd6b] (-->/usr/lib64/libglusterfs.so.0(get_new_dict_full+0x25) [0x35c801bc25]))) 0-mem-pool: invalid argument [2015-04-08 13:46:29.868359] E [mem-pool.c:353:mem_get0] (-->gluster(cli_rpc_init+0x2c) [0x40987c] (-->/usr/lib64/libglusterfs.so.0(dict_new+0xb) [0x35c801bd6b] (-->/usr/lib64/libglusterfs.so.0(get_new_dict_full+0x25) [0x35c801bc25]))) 0-mem-pool: invalid argument
Re-assigning the component as this problem has nothing to do with build.
There is a customer, who hit this issue, while performing the update. Refer - https://bugzilla.redhat.com/show_bug.cgi?id=1201221 Raising the severity to Urgent
*** Bug 1201221 has been marked as a duplicate of this bug. ***