Description of problem: The problem is how rhevm-dwh-setup (and its friends) get DB password. I modified the code to print env and content of PGPASSFILE. As you can see closing '"' is dropped from password! Discovered as part of BZ922854. [root@bz ~]# diff -uNp /usr/share/ovirt-engine-dwh/common_utils.py.orig /usr/share/ovirt-engine-dwh/common_utils.py --- /usr/share/ovirt-engine-dwh/common_utils.py.orig 2014-01-13 11:35:23.384086498 +0100 +++ /usr/share/ovirt-engine-dwh/common_utils.py 2014-01-13 11:31:31.633114947 +0100 @@ -936,6 +936,10 @@ def execCmd( else: env["PGPASSFILE"] = FILE_PG_PASS + ##kuku + print env + subprocess.call(["cat",env["PGPASSFILE"]]) + # We use close_fds to close any file descriptors we have so it won't be copied to forked childs proc = subprocess.Popen( cmd, [root@bz ~]# rhevm-dwh-setup Welcome to ovirt-engine-dwh setup utility {'HISTTIMEFORMAT': '%F %T ', 'LESSOPEN': '|/usr/bin/lesspipe.sh %s', 'SSH_CLIENT': '10.36.7.48 37502 22', 'CVS_RSH': 'ssh', 'LOGNAME': 'root', 'USER': 'root', 'HOME': '/root', 'PATH': '/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin', 'LANG': 'en_US.utf8', 'TERM': 'screen', 'SHELL': '/bin/bash', 'SHLVL': '1', 'G_BROKEN_FILENAMES': '1', 'HISTSIZE': '1000', 'ENGINE_PGPASS': '/tmp/pgpassHIEOqx.tmp', 'XMODIFIERS': '@im=none', 'SSH_AUTH_SOCK': '/tmp/ssh-uryjL27870/agent.27870', 'PGPASSFILE': '/tmp/pgpassHIEOqx.tmp', 'SELINUX_ROLE_REQUESTED': '', '_': '/usr/bin/rhevm-dwh-setup', 'LS_COLORS': 'rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=01;05;37;41:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arj=01;31:*.taz=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lz=01;31:*.xz=01;31:*.bz2=01;31:*.tbz=01;31:*.tbz2=01;31:*.bz=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.rar=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.axv=01;35:*.anx=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=01;36:*.au=01;36:*.flac=01;36:*.mid=01;36:*.midi=01;36:*.mka=01;36:*.mp3=01;36:*.mpc=01;36:*.ogg=01;36:*.ra=01;36:*.wav=01;36:*.axa=01;36:*.oga=01;36:*.spx=01;36:*.xspf=01;36:', 'SSH_TTY': '/dev/pts/0', 'HOSTNAME': 'bz.rhev.lab.eng.brq.redhat.com', 'SELINUX_LEVEL_REQUESTED': '', 'HISTCONTROL': 'ignoredups', 'PWD': '/root', 'SELINUX_USE_CURRENT_RANGE': '', 'MAIL': '/var/spool/mail/root', 'SSH_CONNECTION': '10.36.7.48 37502 10.34.60.121 22'} # DB USER credentials. testovic.rhev.lab.eng.brq.redhat.com:5432:*:engine_history:0080MSJr testovic.rhev.lab.eng.brq.redhat.com:5432:*:remoteengine:Z6AA"4txi\ testovic.rhev.lab.eng.brq.redhat.com:5432:remoteengine:remoteengine:Z6AA"4txi\ Error encountered while installing rhevm-dwh, please consult the log file: /var/log/ovirt-engine/rhevm-dwh-setup-2014_01_13_11_31_33.log [root@bz ~]# grep -i pass /etc/ovirt-engine/engine.conf.d/10-setup-database.conf ENGINE_DB_PASSWORD="Z6AA"4txi\"" [root@bz ~]# cat /var/log/ovirt-engine/rhevm-dwh-setup-2014_01_13_11_31_33.log 2014-01-13 11:31:33::DEBUG::rhevm-dwh-setup::408::root:: starting main() 2014-01-13 11:31:33::DEBUG::common_utils::446::root:: running sql query on host: testovic.rhev.lab.eng.brq.redhat.com, port: 5432, db: remoteengine, user: remoteengine, query: 'copy ( select option_value from vdc_options where option_name like 'MinimalETLVersion' ) to stdout with csv header;'. 2014-01-13 11:31:33::DEBUG::common_utils::907::root:: Executing command --> '/usr/bin/psql --pset=tuples_only=on --set ON_ERROR_STOP=1 --dbname remoteengine --host testovic.rhev.lab.eng.brq.redhat.com --port 5432 --username remoteengine -w -c copy ( select option_value from vdc_options where option_name like 'MinimalETLVersion' ) to stdout with csv header;' in working directory '/root' 2014-01-13 11:31:33::DEBUG::common_utils::966::root:: output = 2014-01-13 11:31:33::DEBUG::common_utils::967::root:: stderr = psql: FATAL: password authentication failed for user "remoteengine" 2014-01-13 11:31:33::DEBUG::common_utils::968::root:: retcode = 2 2014-01-13 11:31:33::ERROR::rhevm-dwh-setup::685::root:: Exception caught! 2014-01-13 11:31:33::ERROR::rhevm-dwh-setup::686::root:: Traceback (most recent call last): File "/usr/bin/rhevm-dwh-setup", line 431, in main temp_pgpass=PGPASS_TEMP, File "/usr/share/ovirt-engine-dwh/common_utils.py", line 151, in getVDCOption envDict={'ENGINE_PGPASS': temp_pgpass} File "/usr/share/ovirt-engine-dwh/common_utils.py", line 432, in parseRemoteSqlCommand envDict, File "/usr/share/ovirt-engine-dwh/common_utils.py", line 470, in execSqlCmd output, rc = execCmd(cmdList=cmd, failOnError=fail_on_error, msg=err_msg, envDict=envDict) File "/usr/share/ovirt-engine-dwh/common_utils.py", line 971, in execCmd raise Exception(msg) Exception: Failed running sql query Version-Release number of selected component (if applicable): is31 rhevm-dwh-3.3.0-27.el6ev.noarch How reproducible: 100% Steps to Reproduce: 1. have a remote db install environment working (base rhevm) with password engine with '"' (see above for password) 2. yum install rhevm-dwh 3. rhevm-dwh-setup Actual results: failure because of authentication (password not read correctly) Expected results: read password with all funny chars in it correctly Additional info:
*** Bug 1052848 has been marked as a duplicate of this bug. ***
This happens due to us removing all '"' from all credentials. In ovirt-engine-dwh-setup.py:getDbDictFromOptions: db_dict[k] = s.strip('"') To fix this properly, we should not do that, and instead of parsing ourselves, use the module configfile from ovirt-engine-lib (rhevm-lib). This module does not support writing, just reading, so a partial solution will be to copy the parsing from it to the current parser (common_utils.py:TextConfigFileHandler). For the meantime, we might want to add a note to the release notes that a remote db user's password should not contain '"'.
Well I think the password should be saved in its real form. Right now the code escapes and saves escaped specific chars in password. See: [root@bz ~]# grep -i pass /etc/ovirt-engine/engine.conf.d/10-setup-database.conf ENGINE_DB_PASSWORD="Z6AA"4txi\"" Real password's form is: Z6AA"4txi" I have never seen any application saving plain-text password in files escaped.
(In reply to Jiri Belka from comment #3) > Well I think the password should be saved in its real form. Right now the > code escapes and saves escaped specific chars in password. See: > > [root@bz ~]# grep -i pass > /etc/ovirt-engine/engine.conf.d/10-setup-database.conf > ENGINE_DB_PASSWORD="Z6AA"4txi\"" > > Real password's form is: Z6AA"4txi" > > I have never seen any application saving plain-text password in files > escaped. Any application whose configuration is intended to be parsed by a shell does that. E.g. most of the files in /etc/sysconfig. It's not specific to the password, btw. These files are read by at least 3 different parsers: 1. They are sourced by sh - in engine-prolog.sh 2. They are read by Java code, in LocalConfig.java 3. They are read by the above-mentioned configfile python code dwh and reports have their own simple parser (two unsynced copies of it) and as I said we better get rid of it in favor of configfile. Anyway, accepting your suggestion of keeping unescaped strings in these files means rewriting quite a lot of code. So it won't happen.
Barak, do we want this fixed for z stream? Yaniv
simplest solution for now is just to forbid '"', if you can please check the new setup and see if problem exists there.
Do we want this fixed in 3.3.z? See comment #4 for the (somewhat) complex fix this will require. In 3.4 the setup is rewritten and so porting a fix from there to 3.3 is not practical. As Alon said, we can simply forbid '"' in passwords for 3.3.
Arthur, We intend to ban the use of '"' in the setup entirely (this is consistent with ethe engin's behaviour (see comment #7). Please ack
Moving to QA as 24464 is irrelevant for 3.4 - the code there was rewritten and should behave well.
Verified on av2.1 rhevm-dwh-3.4.0-0.4.master.20140224152332.el6ev.noarch rhevm-dwh-setup-3.4.0-0.4.master.20140224152332.el6ev.noarch rhevm-reports-setup-3.4.0-0.4.master.20140226133324.el6ev.noarch rhevm-reports-3.4.0-0.4.master.20140226133324.el6ev.noarch jasperreports-server-pro-5.5.0-8.el6ev.noarch # grep -i pass /etc/ovirt-engine/engine.conf.d/10-setup-database.conf ENGINE_DB_PASSWORD="Z6AA"4txi\"" Reports installation passed. Is that enough Jiri ?
OK.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. http://rhn.redhat.com/errata/RHEA-2014-0601.html