Bug 1038808 - SRPM: ovirt-engine -- database initial setup fails
Summary: SRPM: ovirt-engine -- database initial setup fails
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: oVirt
Classification: Retired
Component: ovirt-engine-core
Version: unspecified
Hardware: Unspecified
OS: Linux
urgent
medium
Target Milestone: ---
: ---
Assignee: Eli Mesika
QA Contact:
URL:
Whiteboard: integration
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2013-12-05 20:53 UTC by R P Herrold
Modified: 2014-01-07 17:05 UTC (History)
9 users (show)

Fixed In Version: ovirt-engine-3.3.2-1
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2014-01-01 08:52:07 UTC
oVirt Team: ---


Attachments (Terms of Use)
requested log file (657.04 KB, text/x-log)
2014-01-07 16:25 UTC, R P Herrold
no flags Details

Description R P Herrold 2013-12-05 20:53:00 UTC
The component I would file in is not listed (the SRPM name)

Description of problem:

engine-setup fails in an initial database setup


Version-Release number of selected component (if applicable):

ovirt-engine-setup-3.3.1-2.el6.noarch


How reproducible:

fresh minimal install of CentOS 6, not expressly updated yet

Relevant history is as follows
                                             
[root@engine1 ~]# history | tail -20                                                     
   29  yum install -y ovirt-engine                                                       
   30  rpm -e ovirt-engine-jbossas711                                                    
   31  yum install -y ovirt-engine                                                       
   32  engine-cleanup                                                                    
   33  engine-setup                                                                                          
   38  grep -v DEBUG /var/log/ovirt-engine/setup/ovirt-engine-setup-20131205153005.log | grep -v INFO | grep ERROR
   39  rpm -qa ovirt\*
   40  rpm -qa ovirt\* | grep -v el6
   41  wc /usr/share/ovirt-engine/dbscripts/create_schema.sh
   42  less  /usr/share/ovirt-engine/dbscripts/create_schema.sh
   43  < /usr/share/ovirt-engine/dbscripts/create_schema.sh | grep encrypt_chap_password.sh
   44  grep encrypt_chap_password.sh /usr/share/ovirt-engine/dbscripts/create_schema.sh
   45  cd
   46  rpm -qf ` which engine-setup `
   47  rpm -qi ovirt-engine-setup
   48  history | tail -20
[root@engine1 ~]# ls /etc/yum.repos.d/
attic             CentOS-Debuginfo.repo  el6-ovirt.repo  epel-testing.repo
CentOS-Base.repo  CentOS-Media.repo      epel.repo       glusterfs-epel.repo
[root@engine1 ~]#


Steps to Reproduce:

as above


Actual results:

[root@engine1 yum.repos.d]# grep -v DEBUG /var/log/ovirt-engine/setup/ovirt-engine-setup-20131205153005.log | grep -v INFO | grep ERROR
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 18188
ALTER DATABASE "engine_20131205153450" SET client_min_messages=ERROR;
insert into event_map(event_up_name, event_down_name) values('IRS_DISK_SPACE_LOW_ERROR', 'UNASSIGNED');
insert into event_map(event_up_name, event_down_name) values('DWH_ERROR', 'UNASSIGNED');
psql:upgrade/03_01_1440_encrypt_chap_password.sh:6: ERROR:  syntax error at or near "#!/"
2013-12-05 15:36:14 ERROR otopi.context context._executeMethod:146 Failed to execute stage 'Misc configuration': Command '/usr/share/ovirt-engine/dbscripts/create_schema.sh' failed to execute
2013-12-05 15:36:26 ERROR otopi.plugins.ovirt_engine_common.core.misc misc._terminate:146 Execution of setup failed
[root@engine1 yum.repos.d]#


Expected results:

a clean initial DB setup


Additional info:
         
          --== CONFIGURATION PREVIEW ==--                                                
                                                                                         
          Database name                      : engine                                    
          Database secured connection        : False                                     
          Database host                      : localhost
          Database user name                 : engine
          Database host name validation      : False
          Datbase port                       : 5432
          NFS setup                          : False
          PKI organization                   : catigator.net
          Application mode                   : both
          Firewall manager                   : iptables
          Configure WebSocket Proxy          : True
          Host FQDN                          : engine1.catigator.net
          Datacenter storage type            : nfs
          Configure local database           : True
          Set application as default page    : True
          Configure Apache SSL               : True

          Please confirm installation settings (OK, Cancel) [OK]


[root@engine1 ~]# ls /etc/yum.repos.d/
attic             CentOS-Debuginfo.repo  el6-ovirt.repo  epel-testing.repo
CentOS-Base.repo  CentOS-Media.repo      epel.repo       glusterfs-epel.repo
[root@engine1 ~]#


[root@engine1 ~]# rpm -qf ` which engine-setup `
ovirt-engine-setup-3.3.1-2.el6.noarch

[root@engine1 ~]# rpm -qi ovirt-engine-setup
Name        : ovirt-engine-setup           Relocations: (not relocatable)
Version     : 3.3.1                             Vendor: (none)
Release     : 2.el6                         Build Date: Thu 14 Nov 2013 06:38:21 AM EST
Install Date: Thu 05 Dec 2013 03:24:28 PM EST      Build Host: jenkins-slave-vm11.ovirt.org
Group       : Virtualization/Management     Source RPM: ovirt-engine-3.3.1-2.el6.src.rpm
Size        : 1016029                          License: ASL 2.0
Signature   : (none)
URL         : http://www.ovirt.org
Summary     : Setup and upgrade scripts for oVirt Engine
Description :
Setup and upgrade scripts for oVirt Engine


[root@engine1 yum.repos.d]# rpm -qa ovirt\*                                              
ovirt-engine-sdk-python-3.3.0.8-1.el6.noarch                                             
ovirt-host-deploy-1.1.2-1.el6.noarch                                                     
ovirt-iso-uploader-3.3.1-1.el6.noarch                                                    
ovirt-engine-lib-3.3.1-2.el6.noarch                                                      
ovirt-engine-restapi-3.3.1-2.el6.noarch
ovirt-engine-dbscripts-3.3.1-2.el6.noarch
ovirt-engine-setup-3.3.1-2.el6.noarch
ovirt-engine-3.3.1-2.el6.noarch
ovirt-engine-cli-3.2.0.6-1.el6.centos.alt.noarch
ovirt-release-el6-8-1.noarch
ovirt-log-collector-3.3.1-1.el6.noarch
ovirt-image-uploader-3.3.1-1.el6.noarch
ovirt-engine-websocket-proxy-3.3.1-2.el6.noarch
ovirt-host-deploy-java-1.1.2-1.el6.noarch
ovirt-engine-userportal-3.3.1-2.el6.noarch
ovirt-engine-webadmin-portal-3.3.1-2.el6.noarch
ovirt-engine-tools-3.3.1-2.el6.noarch
ovirt-engine-backend-3.3.1-2.el6.noarch
[root@engine1 yum.repos.d]#

Comment 1 R P Herrold 2013-12-05 21:06:41 UTC
The failing script is intact and unchanged

[root@engine1 ~]# rpm -V `rpm -qf   /usr/share/ovirt-engine/dbscripts/upgrade/03_01_1440_encrypt_chap_password.sh `
[root@engine1 ~]#

Comment 2 R P Herrold 2013-12-09 16:26:45 UTC
do you need any additional information? -- I can provide as wished ... so I can find it, a self-reminder
my test host:  engine1.c(elided).net  
VM Name: vm_67842

-- Russ herrold

Comment 3 Eli Mesika 2013-12-17 08:35:35 UTC
(In reply to R P Herrold from comment #2)
> do you need any additional information? -- I can provide as wished ... so I
> can find it, a self-reminder
> my test host:  engine1.c(elided).net  
> VM Name: vm_67842
> 
> -- Russ herrold

The script even did not start to execute, it fails on the shell declaration :

#!/bin/bash

Please check if you can run this script manually on your setup 

change dir to dbscript/upgrade and run from there :

>./03_01_1440_encrypt_chap_password.sh

Is this working or generating the same error ???

Comment 4 R P Herrold 2013-12-17 22:54:55 UTC
[root@engine1 ~]# locate 03_01_1440_encrypt_chap_password.sh
/usr/share/ovirt-engine/dbscripts/upgrade/03_01_1440_encrypt_chap_password.sh
[root@engine1 ~]# cd /usr/share/ovirt-engine/dbscripts/upgrade/
[root@engine1 upgrade]# ./03_01_1440_encrypt_chap_password.sh -x
./03_01_1440_encrypt_chap_password.sh: line 4: ./dbfunctions.sh: No such file or directory
Engine certificate was not set, skipping
[root@engine1 upgrade]#

and there is no such script in that directory

[root@engine1 upgrade]# grep dbfunctions.sh *
03_01_1440_encrypt_chap_password.sh:source ./dbfunctions.sh
[root@engine1 upgrade]# pwd
/usr/share/ovirt-engine/dbscripts/upgrade
[root@engine1 upgrade]#

Comment 5 Eli Mesika 2013-12-18 09:52:15 UTC
(In reply to R P Herrold from comment #4)
> [root@engine1 ~]# locate 03_01_1440_encrypt_chap_password.sh
> /usr/share/ovirt-engine/dbscripts/upgrade/03_01_1440_encrypt_chap_password.sh
> [root@engine1 ~]# cd /usr/share/ovirt-engine/dbscripts/upgrade/
> [root@engine1 upgrade]# ./03_01_1440_encrypt_chap_password.sh -x
> ./03_01_1440_encrypt_chap_password.sh: line 4: ./dbfunctions.sh: No such
> file or directory

Sorry, please retry from dbscripts directory :
> upgrade/03_01_1440_encrypt_chap_password.sh

Comment 6 Sandro Bonazzola 2013-12-18 10:20:21 UTC
Hi herrold, are you able to reproduce this? also on 3.3.2 RC?
Ohad tested 3.3.2 RC on a clean CentOS without hitting this.

Comment 7 R P Herrold 2013-12-18 22:29:19 UTC
checking

Comment 8 R P Herrold 2013-12-31 15:35:53 UTC
backstop L0 backup taken so we can bisect if needed

Comment 9 R P Herrold 2013-12-31 15:45:49 UTC
trying: 
Updating:
 ovirt-engine                       noarch       3.3.2-1.el6          ovirt-3.3.2       1.2 M
Updating for dependencies:
 ovirt-engine-backend               noarch       3.3.2-1.el6          ovirt-3.3.2       9.6 M
 ovirt-engine-dbscripts             noarch       3.3.2-1.el6          ovirt-3.3.2       197 k
 ovirt-engine-lib                   noarch       3.3.2-1.el6          ovirt-3.3.2        18 k
 ovirt-engine-restapi               noarch       3.3.2-1.el6          ovirt-3.3.2       1.0 M
 ovirt-engine-setup                 noarch       3.3.2-1.el6          ovirt-3.3.2       230 k
 ovirt-engine-tools                 noarch       3.3.2-1.el6          ovirt-3.3.2       146 k
 ovirt-engine-userportal            noarch       3.3.2-1.el6          ovirt-3.3.2        77 M
 ovirt-engine-webadmin-portal       noarch       3.3.2-1.el6          ovirt-3.3.2        97 M
 ovirt-engine-websocket-proxy       noarch       3.3.2-1.el6          ovirt-3.3.2        12 k

Comment 10 R P Herrold 2013-12-31 16:08:44 UTC
Initial install concluded without error;  I will try the code path:

   32  engine-cleanup 
   33  engine-setup

in a moment


[root@engine1 ~]# updatedb
[root@engine1 ~]# engine-setup
[ INFO  ] Stage: Initializing 
[ INFO  ] Stage: Environment setup
          Configuration files: ['/etc/ovirt-engine-setup.conf.d/10-packaging.conf']
          Log file: /var/log/ovirt-engine/setup/ovirt-engine-setup-20131231105518.log
          Version: otopi-1.1.2 (otopi-1.1.2-1.el6)                                   
[ INFO  ] Stage: Environment packages setup                                          
[ INFO  ] Stage: Programs detection                                                  
[ INFO  ] Stage: Environment setup                                                   
[ INFO  ] Stage: Environment customization                                           
                                                                                     
          --== PACKAGES ==--                                                         
                                                                                     
[ INFO  ] Checking for product updates...                                            
[ INFO  ] No product updates found                                                   
                                                                                     
          --== NETWORK CONFIGURATION ==--                                            
                                                                                     
          Host fully qualified DNS name of this server [engine1.c(elided).net]: 
          iptables was detected on your computer. Do you wish Setup to configure it? (yes, no) [yes]:                                                                                       
                                                                                              
          --== DATABASE CONFIGURATION ==--                                                    
                                                                                              
          Where is the database located? (Local, Remote) [Local]:                             
          Setup can configure the local postgresql server automatically for the engine to run. This may conflict with existing applications.                                                
          Would you like Setup to automatically configure postgresql, or prefer to perform that manually? (Automatic, Manual) [Automatic]:                                                  
                                                                                              
          --== OVIRT ENGINE CONFIGURATION ==--                                                
                                                                                              
          Engine admin password:                                                              
          Confirm engine admin password:                                                      
          Application mode (Both, Virt, Gluster) [Both]:                                      
          Default storage type: (NFS, FC, ISCSI, POSIXFS) [NFS]:                              
                                                                                              
          --== PKI CONFIGURATION ==--                                                         
                                                                                              
          Organization name for certificate [c(elided).net]:                                  
                                                                                              
          --== APACHE CONFIGURATION ==--                                                      
                                                                                              
          Setup can configure the default page of the web server to present the application home page. This may conflict with existing applications.                                        
          Do you wish to set the application as the default page of the web server? (Yes, No) [Yes]:                                                                                        
          Setup can configure apache to use SSL using a certificate issued from the internal CA.                                                                                            
          Do you wish Setup to configure that, or prefer to perform that manually? (Automatic, Manual) [Automatic]:                                                                         
                                                                                              
          --== SYSTEM CONFIGURATION ==--                                                      
                                                                                              
          Configure an NFS share on this server to be used as an ISO Domain? (Yes, No) [Yes]: 
          Local ISO domain path [/var/lib/exports/iso]:                                       
          Local ISO domain name [ISO_DOMAIN]:                                                 
          Configure WebSocket Proxy on this machine? (Yes, No) [Yes]:                         
                                                                                              
          --== END OF CONFIGURATION ==--                                                      
                                                                                              
[ INFO  ] Stage: Setup validation                                                             
[WARNING] Less than 16384MB of memory is available                                            
                                                                                              
          --== CONFIGURATION PREVIEW ==--                                                     
                                                                                              
          Database name                      : engine                                         
          Database secured connection        : False                                          
          Database host                      : localhost                                      
          Database user name                 : engine                                         
          Database host name validation      : False                                          
          Datbase port                       : 5432                                           
          NFS setup                          : True                                           
          PKI organization                   : c(elided).net                                  
          NFS mount point                    : /var/lib/exports/iso                           
          Application mode                   : both                                           
          Firewall manager                   : iptables                                       
          Configure WebSocket Proxy          : True                                           
          Host FQDN                          : engine1.c(elided).net                          
          Datacenter storage type            : nfs                                            
          Configure local database           : True                                           
          Set application as default page    : True                                           
          Configure Apache SSL               : True                                           
                                                                                              
          Please confirm installation settings (OK, Cancel) [OK]:                             
[ INFO  ] Stage: Transaction setup                                                            
[ INFO  ] Stopping engine service                                                             
[ INFO  ] Stopping websocket-proxy service                                                    
[ INFO  ] Stage: Misc configuration                                                           
[ INFO  ] Stage: Package installation                                                         
[ INFO  ] Stage: Misc configuration                                                           
[ INFO  ] Creating PostgreSQL database                                                        
[ INFO  ] Configuring PostgreSQL                                                              
[ INFO  ] Creating database schema                                                            
[ INFO  ] Creating CA                                                                         
[ INFO  ] Configuring WebSocket Proxy                                                         
[ INFO  ] Generating post install configuration file '/etc/ovirt-engine-setup.conf.d/20-setup-ovirt-post.conf'                                                                              
[ INFO  ] Stage: Transaction commit                                                           
[ INFO  ] Stage: Closing up                                                                   
                                                                                              
          --== SUMMARY ==--                                                                   
                                                                                              
[WARNING] Less than 16384MB of memory is available                                            
          An ISO NFS share has been created on this host.                                     
              If IP based access restrictions are required, edit:                             
              entry /var/lib/exports/iso in /etc/exports                                      
          Database resources:                                                                 
              Database name:      engine_20131231105828                                       
              Database user name: engine_20131231105828                                       
          SSH fingerprint: 97:C7:41:61:4F:48:18:D4:B9:CA:5A:61:5D:6C:69:2E                    
          Internal CA 6F:25:F6:82:6B:78:10:6B:C1:71:F0:06:A2:C3:A6:69:D2:4A:05:B3             
          Web access is enabled at:                                                           
              http://engine1.c(elided).net:80/ovirt-engine                                    
              https://engine1.c(elided).net:443/ovirt-engine                                  
          Please use the user "admin" and password specified in order to login into oVirt Engine                                                                                            

          --== END OF SUMMARY ==--

[ INFO  ] Starting engine service
[ INFO  ] Restarting httpd
[ INFO  ] Restarting nfs services
[ INFO  ] Generating answer file '/var/lib/ovirt-engine/setup/answers/20131231110322-setup.conf'
[ INFO  ] Stage: Clean up
          Log file is located at /var/log/ovirt-engine/setup/ovirt-engine-setup-20131231105518.log
[ INFO  ] Stage: Pre-termination
[ INFO  ] Stage: Termination
[ INFO  ] Execution of setup completed successfully
[root@engine1 ~]# grep -v DEBUG /var/log/ovirt-engine/setup/ovirt-engine-setup-20131231105518.log | grep -v INFO | grep ERROR
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 28738
ALTER DATABASE "engine_20131231105828" SET client_min_messages=ERROR;
insert into event_map(event_up_name, event_down_name) values('IRS_DISK_SPACE_LOW_ERROR', 'UNASSIGNED');
insert into event_map(event_up_name, event_down_name) values('DWH_ERROR', 'UNASSIGNED');
[root@engine1 ~]# date
Tue Dec 31 11:04:18 EST 2013
[root@engine1 ~]#

Comment 11 R P Herrold 2013-12-31 16:12:07 UTC
while running: engine-cleanup

I get one error, which may be an artifact of me typing 'yes' rather than 'Yes' on one question

[root@engine1 bin]# grep -v DEBUG /var/log/ovirt-engine/setup/ovirt-engine-remove-20131231110920.log | grep -v INFO | grep ERROR
2013-12-31 11:09:29 ERROR otopi.plugins.otopi.dialog.human human.queryString:176 Invalid value
[root@engine1 bin]#

Comment 12 R P Herrold 2013-12-31 16:14:20 UTC
in the preceding stage, I had assumed answers were case insensitive, as I say an entry thus:
          iptables was detected on your computer. Do you wish Setup to configure it? (yes, no) [yes]:


I will file a separate bug,. to seek this minor change

Comment 13 R P Herrold 2013-12-31 16:21:42 UTC
and a fresh:
   engine-setup 
completes without error, so I consider this bug fixed at ovirt-engine-3.3.2-1.el6.noarch

[root@engine1 bin]# grep -v DEBUG /var/log/ovirt-engine/setup/ovirt-engine-setup-20131231111213.log | grep -v INFO | grep ERROR
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 24991
ALTER DATABASE "engine_20131231111400" SET client_min_messages=ERROR;
insert into event_map(event_up_name, event_down_name) values('IRS_DISK_SPACE_LOW_ERROR', 'UNASSIGNED');
insert into event_map(event_up_name, event_down_name) values('DWH_ERROR', 'UNASSIGNED');
[root@engine1 bin]#

Comment 14 Yedidyah Bar David 2013-12-31 18:13:32 UTC
Hi Herrold,

(In reply to R P Herrold from comment #11)
> while running: engine-cleanup
> 
> I get one error, which may be an artifact of me typing 'yes' rather than
> 'Yes' on one question
> 
> [root@engine1 bin]# grep -v DEBUG
> /var/log/ovirt-engine/setup/ovirt-engine-remove-20131231110920.log | grep -v
> INFO | grep ERROR
> 2013-12-31 11:09:29 ERROR otopi.plugins.otopi.dialog.human
> human.queryString:176 Invalid value
> [root@engine1 bin]#

Thanks for the updates.

I think it should have worked with 'Yes'.

Can you please attach the log? It might be some other issue.

Best regards,
-- 
Didi

Comment 15 Eli Mesika 2014-01-01 08:52:07 UTC
Closing due to BZ reporter comment #13

Comment 16 R P Herrold 2014-01-07 16:25:04 UTC
Didi

here is the requested log, elided a bit

Comment 17 R P Herrold 2014-01-07 16:25:57 UTC
Created attachment 846773 [details]
requested log file

Comment 18 Yedidyah Bar David 2014-01-07 17:05:33 UTC
(In reply to R P Herrold from comment #16)
> Didi
> 
> here is the requested log, elided a bit

Seems like it did accept your 'Yes' - the "Invalid value" was a response to you inputting 'y'.


Note You need to log in before you can comment on or make changes to this bug.