Bug 1021824

Summary: Failed clean up RHEVM DB - rhevm-cleanup
Product: Red Hat Enterprise Virtualization Manager Reporter: vvyazmin <vvyazmin>
Component: ovirt-engine-setupAssignee: Sandro Bonazzola <sbonazzo>
Status: CLOSED NOTABUG QA Contact: vvyazmin <vvyazmin>
Severity: high Docs Contact:
Priority: medium    
Version: 3.3.0CC: acanan, acathrow, alonbl, alourie, bazulay, didi, emesika, hateya, iheim, oschreib, pstehlik, Rhev-m-bugs, sbonazzo, vvyazmin, yeylon
Target Milestone: ---Keywords: Regression, Triaged
Target Release: 3.3.0   
Hardware: x86_64   
OS: Linux   
Whiteboard: integration
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-10-29 15:04:25 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
# rhevm-cleanup log none

Description vvyazmin@redhat.com 2013-10-22 07:41:44 UTC
Created attachment 814874 [details]
# rhevm-cleanup log

Description of problem:
Failed clean up RHEVM DB - rhevm-cleanup

Version-Release number of selected component (if applicable):
RHEVM 3.3 - IS19 environment:

Host OS: RHEL 6.5

RHEVM:  rhevm-3.3.0-0.27.beta1.el6ev.noarch
PythonSDK:  rhevm-sdk-python-3.3.0.16-1.el6ev.noarch
VDSM:  vdsm-4.13.0-0.3.beta1.el6ev.x86_64
LIBVIRT:  libvirt-0.10.2-29.el6.x86_64
QEMU & KVM:  qemu-kvm-rhev-0.12.1.2-2.414.el6.x86_64
SANLOCK:  sanlock-2.8-1.el6.x86_64

How reproducible:
100 %

Steps to Reproduce:
1. Run rhevm-cleanup 

Actual results:
Failed clean up RHEVM DB

Expected results:
Succeed clean up RHEVM DB

Impact on user:
Failed clean up RHEVM DB

Workaround:
Reinstall RHEVM

Additional info:

[root@kipi-rhevm ~]# rhevm-cleanup 
[ INFO  ] Stage: Initializing
[ INFO  ] Stage: Environment setup
          Configuration files: ['/etc/ovirt-engine-setup.conf.d/10-packaging.conf', '/etc/ovirt-engine-setup.conf.d/20-setup-ovirt-post.conf']
          Log file: /var/log/ovirt-engine/setup/ovirt-engine-remove-20131022095057.log
          Version: otopi-1.1.2 (otopi-1.1.2-1.el6ev)
[WARNING] Cannot connect to database using existing credentials: engine@localhost:5432
[ ERROR ] Failed to execute stage 'Environment setup': Command '/usr/share/ovirt-engine/bin/ovirt-engine-log-setup-event.sh' failed to execute
[ INFO  ] Stage: Clean up
          Log file is located at /var/log/ovirt-engine/setup/ovirt-engine-remove-20131022095057.log
[ INFO  ] Stage: Pre-termination
[ INFO  ] Stage: Termination
[ ERROR ] Execution of cleanup failed
[root@kipi-rhevm ~]# less /var/log/ovirt-engine/setup/ovirt-engine-remove-20131022095057.log

Comment 1 vvyazmin@redhat.com 2013-10-22 07:56:20 UTC
Workaround:
Run "engine-cleanup" command

Comment 3 Sandro Bonazzola 2013-10-28 08:43:25 UTC
Looking at the logs:

> sort: write failed: standard output: No space left on device

Can you check if your disk is full and that caused postgres to die before cleanup?

Comment 4 Alon Bar-Lev 2013-10-28 08:46:24 UTC
Just for my regular note...

ovirt-engine-log-setup-event.sh should either die or go either to:

1. redhat support plugin

2. rhevm-setup-plugins

But this has nothing to do with the problem...

no disk space:
---
sort: write failed: standard output: No space left on device
sort: write error
/usr/share/ovirt-engine/bin/ovirt
---

database is down:
---
    error=e,
RuntimeError: Cannot connect to database: could not connect to server: Connection refused
	Is the server running on host "localhost" and accepting
	TCP/IP connections on port 5432?
could not connect to server: Connection refused
	Is the server running on host "localhost" and accepting
	TCP/IP connections on port 5432?
---

Comment 5 Sandro Bonazzola 2013-10-29 15:04:25 UTC
since it seems that this is not a real bug but something depending on the host without enough disk space, I'm closing as not a bug.
Please reopen if you can provide more info or reproduce without running out of disk space.