Bug 1257468
| Summary: | gdeploy: creation of thin pool stuck after brick cleanup | ||
|---|---|---|---|
| Product: | [Red Hat Storage] Red Hat Gluster Storage | Reporter: | Anush Shetty <ashetty> |
| Component: | gdeploy | Assignee: | Nandaja Varma <nvarma> |
| Status: | CLOSED ERRATA | QA Contact: | Anush Shetty <ashetty> |
| Severity: | unspecified | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | rhgs-3.1 | CC: | musoni, nvarma, smohan, surs, vagarwal |
| Target Milestone: | --- | Keywords: | ZStream |
| Target Release: | RHGS 3.1.1 | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | gdeploy-1.0-8 | Doc Type: | Bug Fix |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2015-10-05 07:24:50 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Bug Depends On: | |||
| Bug Blocks: | 1251815 | ||
Anush, Can you provide the configuration file used? (Just to clarify some doubts) The root cause of the issue was, lvcreate was prompting if it should wipe the xfs signature. This prompt was not shown by ansible. The prompt was as follows: Rounding up size to full physical extent 99.50 GiB WARNING: xfs signature detected on /dev/GLUSTER_vg1/GLUSTER_pool1 at offset 0. Wipe it? [y/n]: The lvcreate command has been modified to not wipe any signatures left behind. It will not prompt for this anymore. Verified with gdeploy-1.0-9.el7rhgs.noarch Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHSA-2015-1845.html |
Description of problem: After a successful run of gdeploy backend setup, the backend bricks were clean up by umounting the brick mount and deleting the logical volumes, volume groups and physical volumes on the devices. Running gdeploy to configure bricks again resulted in the process being stuck while creating logical volumes on the hosts. Version-Release number of selected component (if applicable): gdeploy-1.0-6.el7rhgs.noarch.rpm How reproducible: Always Steps to Reproduce: 1. Run gdeploy to configure backend bricks 2. After a successful run, clean up by the brick mounts: umount, remove logical and physical volumes. 3. Run gdeploy again with the same configuration file. Actual results: gdeploy stuck while creating logical volumes. gdeploy in verbose mode: changed: [rhshdp05.lab.eng.blr.redhat.com] => (item=GLUSTER_vg2) => {"changed": 1, "item": "GLUSTER_vg2", "msg": " Rounding up size to full physical extent 1.75 GiB\n Logical volume \"metadata\" created.\n"} changed: [rhshdp06.lab.eng.blr.redhat.com] => (item=GLUSTER_vg2) => {"changed": 1, "item": "GLUSTER_vg2", "msg": " Rounding up size to full physical extent 1.75 GiB\n Logical volume \"metadata\" created.\n"} changed: [rhshdp03.lab.eng.blr.redhat.com] => (item=GLUSTER_vg2) => {"changed": 1, "item": "GLUSTER_vg2", "msg": " Rounding up size to full physical extent 512.00 MiB\n Logical volume \"metadata\" created.\n"} TASK: [Create a thin pool] **************************************************** <rhshdp03.lab.eng.blr.redhat.com> ESTABLISH CONNECTION FOR USER: root <rhshdp03.lab.eng.blr.redhat.com> REMOTE_MODULE lv action=create lvname=GLUSTER_pool1 lvtype='thin' compute=rhs disktype="jbod" vgname=GLUSTER_vg1 <rhshdp03.lab.eng.blr.redhat.com> EXEC ssh -C -tt -v -o ControlMaster=auto -o ControlPersist=60s -o ControlPath="/root/.ansible/cp/ansible-ssh-%h-%p-%r" -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ConnectTimeout=10 rhshdp03.lab.eng.blr.redhat.com /bin/sh -c 'mkdir -p $HOME/.ansible/tmp/ansible-tmp-1440658478.89-25346136994452 && echo $HOME/.ansible/tmp/ansible-tmp-1440658478.89-25346136994452' <rhshdp05.lab.eng.blr.redhat.com> ESTABLISH CONNECTION FOR USER: root <rhshdp05.lab.eng.blr.redhat.com> REMOTE_MODULE lv action=create lvname=GLUSTER_pool1 lvtype='thin' compute=rhs disktype="jbod" vgname=GLUSTER_vg1 <rhshdp06.lab.eng.blr.redhat.com> ESTABLISH CONNECTION FOR USER: root <rhshdp05.lab.eng.blr.redhat.com> EXEC ssh -C -tt -v -o ControlMaster=auto -o ControlPersist=60s -o ControlPath="/root/.ansible/cp/ansible-ssh-%h-%p-%r" -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ConnectTimeout=10 rhshdp05.lab.eng.blr.redhat.com /bin/sh -c 'mkdir -p $HOME/.ansible/tmp/ansible-tmp-1440658478.91-59915647038493 && echo $HOME/.ansible/tmp/ansible-tmp-1440658478.91-59915647038493' <rhshdp06.lab.eng.blr.redhat.com> REMOTE_MODULE lv action=create lvname=GLUSTER_pool1 lvtype='thin' compute=rhs disktype="jbod" vgname=GLUSTER_vg1 <rhshdp06.lab.eng.blr.redhat.com> EXEC ssh -C -tt -v -o ControlMaster=auto -o ControlPersist=60s -o ControlPath="/root/.ansible/cp/ansible-ssh-%h-%p-%r" -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ConnectTimeout=10 rhshdp06.lab.eng.blr.redhat.com /bin/sh -c 'mkdir -p $HOME/.ansible/tmp/ansible-tmp-1440658478.93-259397830188625 && echo $HOME/.ansible/tmp/ansible-tmp-1440658478.93-259397830188625' <rhshdp03.lab.eng.blr.redhat.com> PUT /tmp/tmp_rmobG TO /root/.ansible/tmp/ansible-tmp-1440658478.89-25346136994452/lv <rhshdp05.lab.eng.blr.redhat.com> PUT /tmp/tmpNfYBjg TO /root/.ansible/tmp/ansible-tmp-1440658478.91-59915647038493/lv <rhshdp06.lab.eng.blr.redhat.com> PUT /tmp/tmpfnOCHC TO /root/.ansible/tmp/ansible-tmp-1440658478.93-259397830188625/lv <rhshdp03.lab.eng.blr.redhat.com> EXEC ssh -C -tt -v -o ControlMaster=auto -o ControlPersist=60s -o ControlPath="/root/.ansible/cp/ansible-ssh-%h-%p-%r" -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ConnectTimeout=10 rhshdp03.lab.eng.blr.redhat.com /bin/sh -c 'LANG=C LC_CTYPE=C /usr/bin/python /root/.ansible/tmp/ansible-tmp-1440658478.89-25346136994452/lv; rm -rf /root/.ansible/tmp/ansible-tmp-1440658478.89-25346136994452/ >/dev/null 2>&1' <rhshdp05.lab.eng.blr.redhat.com> EXEC ssh -C -tt -v -o ControlMaster=auto -o ControlPersist=60s -o ControlPath="/root/.ansible/cp/ansible-ssh-%h-%p-%r" -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ConnectTimeout=10 rhshdp05.lab.eng.blr.redhat.com /bin/sh -c 'LANG=C LC_CTYPE=C /usr/bin/python /root/.ansible/tmp/ansible-tmp-1440658478.91-59915647038493/lv; rm -rf /root/.ansible/tmp/ansible-tmp-1440658478.91-59915647038493/ >/dev/null 2>&1' <rhshdp06.lab.eng.blr.redhat.com> EXEC ssh -C -tt -v -o ControlMaster=auto -o ControlPersist=60s -o ControlPath="/root/.ansible/cp/ansible-ssh-%h-%p-%r" -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ConnectTimeout=10 rhshdp06.lab.eng.blr.redhat.com /bin/sh -c On the hosts: # lvs ^C Interrupted... Giving up waiting for lock. /run/lock/lvm/V_GLUSTER_vg1: flock failed: Interrupted system call Can't get lock for GLUSTER_vg1 Cannot process volume group GLUSTER_vg1 LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert metadata GLUSTER_vg2 -wi-a----- 1.75g # ps ax | grep lv 386 ? Ssl 0:00 /usr/sbin/lvmetad -f 32031 pts/1 Ss+ 0:00 /bin/sh -c LANG=C LC_CTYPE=C /usr/bin/python /root/.ansible/tmp/ansible-tmp-1440658478.91-59915647038493/lv; rm -rf /root/.ansible/tmp/ansible-tmp-1440658478.91-59915647038493/ >/dev/null 2>&1 32037 pts/1 S+ 0:00 /usr/bin/python /root/.ansible/tmp/ansible-tmp-1440658478.91-59915647038493/lv 32046 pts/1 S+ 0:00 /usr/sbin/lvcreate -L 104326144.0K -n GLUSTER_pool1 GLUSTER_vg1 32257 pts/0 S+ 0:00 grep --color=auto lv Additional info: The lvs, vgs and pvs had to cleaned up again to get the gdepoy run successfully.