Description of problem: ----------------------- During Database role upgrade 'TASK [Upgrade Mysql database from a temporary container]' failed openstack overcloud upgrade run \ --nodes Database --playbook all 2>&1 ... u'TASK [Upgrade Mysql database from a temporary container] ***********************', u'fatal: [192.168.24.14]: FAILED! => {"changed": true, "cmd": "/usr/bin/docker run --rm --log-driver=syslog -u root --net=host -e \\"KOLLA_CONFIG_STRATEGY=COPY_ALWAYS\\" -v /etc/hosts:/etc/hosts:ro -v /etc/localtime:/etc/localtime:ro -v /etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro -v /etc/pki/tls/certs/ca-bundle.crt:/etc/pki/tls/certs/ca-bundle.crt:ro -v /etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro -v /etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro -v /dev/log:/dev/log -v /etc/ssh/ssh_known_hosts:/etc/ssh/ssh_known_hosts:ro -v /etc/puppet:/etc/puppet:ro -v /var/lib/kolla/config_files/mysql.json:/var/lib/kolla/config_files/config.json -v /var/lib/config-data/puppet-generated/mysql/:/var/lib/kolla/config_files/src:ro -v /var/lib/mysql:/var/lib/mysql -v /tmp/mariadb-upgrade:/var/log/mariadb:rw \\"192.168.24.1:8787/rhosp13/openstack-mariadb:pcmklatest\\" /bin/bash -ecx \\"sudo -E kolla_set_configs; mysqld_safe --user=mysql --wsrep-provider=none --skip-networking --wsrep-on=off & timeout 60 sh -c \'while ! mysqladmin ping --silent; do sleep 1; done\'; mysql_upgrade; mysqladmin shutdown\\"", "delta": "0:01:00.349754", "end": "2018-04-06 13:58:00.615572", "msg": "non-zero return code", "rc": 124, "start": "2018-04-06 13:57:00.265818", "stderr": "+ sudo -E kolla_set_configs\\nINFO:__main__:Loading config file at /var/lib/kolla/config_files/config.json\\nINFO:__main__:Validating config file\\nINFO:__main__:Kolla config strategy set to: COPY_ALWAYS\\nINFO:__main__:Copying service configuration files\\nINFO:__main__:Copying /dev/null to /etc/libqb/force-filesystem-sockets\\nINFO:__main__:Setting permission for /etc/libqb/force-filesystem-sockets\\nINFO:__main__:Deleting /etc/my.cnf.d/galera.cnf\\nINFO:__main__:Copying /var/lib/kolla/config_files/src/etc/my.cnf.d/galera.cnf to /etc/my.cnf.d/galera.cnf\\nINFO:__main__:Copying /var/lib/kolla/config_files/src/etc/sysconfig/clustercheck to /etc/sysconfig/clustercheck\\nINFO:__main__:Copying /var/lib/kolla/config_files/src/root/.my.cnf to /root/.my.cnf\\nINFO:__main__:Writing out command to execute\\n+ timeout 60 sh -c \'while ! mysqladmin ping --silent; do sleep 1; done\'\\n+ mysqld_safe --user=mysql --wsrep-provider=none --skip-networking --wsrep-on=off\\n/usr/bin/mysqld_safe: line 755: ulimit: -1: invalid option\\nulimit: usage: ulimit [-SHacdefilmnpqrstuvx] [limit]", "stderr_lines": ["+ sudo -E kolla_set_configs", "INFO:__main__:Loading config file at /var/lib/kolla/config_files/config.json", "INFO:__main__:Validating config file", "INFO:__main__:Kolla config strategy set to: COPY_ALWAYS", "INFO:__main__:Copying service configuration files", "INFO:__main__:Copying /dev/null to /etc/libqb/force-filesystem-sockets", "INFO:__main__:Setting permission for /etc/libqb/force-filesystem-sockets", "INFO:__main__:Deleting /etc/my.cnf.d/galera.cnf", "INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/my.cnf.d/galera.cnf to /etc/my.cnf.d/galera.cnf", "INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/sysconfig/clustercheck to /etc/sysconfig/clustercheck", "INFO:__main__:Copying /var/lib/kolla/config_files/src/root/.my.cnf to /root/.my.cnf", "INFO:__main__:Writing out command to execute", "+ timeout 60 sh -c \'while ! mysqladmin ping --silent; do sleep 1; done\'", "+ mysqld_safe --user=mysql --wsrep-provider=none --skip-networking --wsrep-on=off", "/usr/bin/mysqld_safe: line 755: ulimi'headers' t: -1: invalid option", "ulimit: usage: ulimit [-SHacdefilmnpqrstuvx] [limit]"], "stdout": "180406 13:57:00 mysqld_safe Logging to \'/var/log/mariadb/mariadb.log\'.\\n180406 13:57:00 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql\\n180406 13:57:01 mysqld_safe mysqld from pid file /var/lib/mysql/mariadb.pid ended", "stdout_lines": ["180406 13:57:00 mysqld_safe Logging to \'/var/log/mariadb/mariadb.log\'.", "180406 13:57:00 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql", "180406 13:57:01 mysqld_safe mysqld from pid file /var/lib/mysql/mariadb.pid ended"]}'] Version-Release number of selected component (if applicable): ------------------------------------------------------------- openstack-tripleo-heat-templates-8.0.2-0.20180327213843.f25e2d8.el7ost.noarch 192.168.24.1:8787/rhosp13/openstack-mariadb:2018-03-29.1 How reproducible: ----------------- So far 100% Steps to Reproduce: ------------------- 1. Upgrade UC 2. Setup latest repos on OC 3. Setup latest containers 4. Run 'overcloud upgrade prepare...' 5. Upgrade Controller role 6. Start upgrade of Database role Additional info: ---------------- Virtual environment: 3controller + 2compute + 3database + 3messaging + 3ceph + 2networker
This one is a warning due to the way we generate the galera config in OSP. It is not fatal. The failure was probably due to other is a previous revision of https://review.openstack.org/#/c/560408, and should be fixed in the latest revisions.
Updated patch to point to queens branch. merged end of last week
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2018:2086