Bug 1605189

Summary: [UPGRADES][14] ERROR 1 (HY000) at line 1: Can't create/write to file './glance/db.op t' (Errcode: 13 \"Permission denied\")
Product: Red Hat OpenStack Reporter: Yurii Prokulevych <yprokule>
Component: openstack-tripleo-heat-templatesAssignee: Jose Luis Franco <jfrancoa>
Status: CLOSED ERRATA QA Contact: Yurii Prokulevych <yprokule>
Severity: high Docs Contact:
Priority: high    
Version: 14.0 (Rocky)CC: augol, ccamacho, jfrancoa, jjoyce, josorior, mburns, mcornea, racedoro, slinaber, yprokule
Target Milestone: rcKeywords: Triaged
Target Release: 14.0 (Rocky)   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: openstack-tripleo-heat-templates-9.0.0-0.20180919080946.0rc1.0rc1.el7ost Doc Type: No Doc Update
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-01-11 11:50:43 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Yurii Prokulevych 2018-07-20 12:18:33 UTC
Description of problem:
-----------------------
Failed to upgrade undercloud from RHOS-13 to RHOS-14
undercloud upgrade -y --use-heat 2>&1
...
        "Warning: ModuleLoader: module 'openstacklib' has unresolved dependencies - it will only see those that are resolved. Use 'puppet module list --tree' to see information about modules",
        "                    with Pattern[]. There is further documentation for validate_legacy function in the README. at [\"/etc/puppet/modules/openstacklib/manifests/db/mysql/host_access.pp\", 43]:",
        "Error: Execution of '/usr/bin/mysql --defaults-extra-file=/root/.my.cnf -NBe alter database `glance` CHARACTER SET utf8' returned 1: ERROR 1 (HY000) at line 1: Can't create/write to file './glance/db.op
t' (Errcode: 13 \"Permission denied\")",
        "Error: /Stage[main]/Glance::Db::Mysql/Openstacklib::Db::Mysql[glance]/Mysql_database[glance]/charset: change from latin1 to utf8 failed: Execution of '/usr/bin/mysql --defaults-extra-file=/root/.my.cnf 
-NBe alter database `glance` CHARACTER SET utf8' returned 1: ERROR 1 (HY000) at line 1: Can't create/write to file './glance/db.opt' (Errcode: 13 \"Permission denied\")",
        "Error: Execution of '/usr/bin/mysql --defaults-extra-file=/root/.my.cnf -NBe alter database `glance` COLLATE utf8_general_ci' returned 1: ERROR 1 (HY000) at line 1: Can't create/write to file './glance/
db.opt' (Errcode: 13 \"Permission denied\")",
        "Error: /Stage[main]/Glance::Db::Mysql/Openstacklib::Db::Mysql[glance]/Mysql_database[glance]/collate: change from latin1_swedish_ci to utf8_general_ci failed: Execution of '/usr/bin/mysql --defaults-ext
ra-file=/root/.my.cnf -NBe alter database `glance` COLLATE utf8_general_ci' returned 1: ERROR 1 (HY000) at line 1: Can't create/write to file './glance/db.opt' (Errcode: 13 \"Permission denied\")",
        "Error: Execution of '/usr/bin/mysql --defaults-extra-file=/root/.my.cnf -NBe alter database `heat` CHARACTER SET utf8' returned 1: ERROR 1 (HY000) at line 1: Can't create/write to file './heat/db.opt' (
Errcode: 13 \"Permission denied\")",
        "Error: /Stage[main]/Heat::Db::Mysql/Openstacklib::Db::Mysql[heat]/Mysql_database[heat]/charset: change from latin1 to utf8 failed: Execution of '/usr/bin/mysql --defaults-extra-file=/root/.my.cnf -NBe a
lter database `heat` CHARACTER SET utf8' returned 1: ERROR 1 (HY000) at line 1: Can't create/write to file './heat/db.opt' (Errcode: 13 \"Permission denied\")"



Version-Release number of selected component (if applicable):
-------------------------------------------------------------
openstack-tripleo-heat-templates-9.0.0-0.20180710202746.d2994ca.el7ost.noarch
python-tripleoclient-heat-installer-10.2.1-0.20180709114814.e5ce9a8.el7ost.noarch
python-tripleoclient-10.2.1-0.20180709114814.e5ce9a8.el7ost.noarch


How reproducible:
-----------------
100%

Steps to Reproduce:
-------------------
1. Upgrade UC to 13(or install 13)
2. Prepare 14 repos and images
3. Set undercloud_ntp_servers variable in undercloud.conf
4. Set container_images_file in undercloud.conf
5. Run `openstack undercloud upgrade -y --use-heat 2>&1`


Actual results:
---------------
Upgrade fails


Expected results:
-----------------
Upgrade succeeds

Additional info:
----------------
Attempt to alter table fails when next command executed manually:
(undercloud) [stack@undercloud-0 ~ (undercloud-13-US)]$ sudo /usr/bin/mysql --defaults-extra-file=/root/.my.cnf -NBe "alter database zaqar CHARACTER SET utf8" -vv                                                 
--------------
alter database zaqar CHARACTER SET utf8
--------------

ERROR 1 (HY000) at line 1: Can't create/write to file './zaqar/db.opt' (Errcode: 13 "Permission denied")
Bye

Comment 1 Juan Antonio Osorio 2018-07-23 14:47:37 UTC
The environment that yprokule showed me had the following issue:

The running mysql instance was the host's, not the container's. The database files at that point are owned by the container's mysql user, not the host's mysql user; this is what causes the permission issues. The mysql in the host should be stopped, and the one from the container should be configured correctly and it's what should be running.

Comment 3 Jose Luis Franco 2018-07-27 12:16:40 UTC
Shouldn't this patch fix this upgrade issue? https://github.com/openstack/tripleo-heat-templates/commit/cc05e522335bc9f1f66478a744d74b8c565e7845

Comment 4 Juan Antonio Osorio 2018-07-30 06:21:02 UTC
That looks like a possible fix. It seems that that patch wasn't included in the undercloud we tested, or it didn't run.
Yurii, could you check out if that patch was included?

Comment 14 Yurii Prokulevych 2018-12-12 07:30:54 UTC
Verified with openstack-tripleo-heat-templates-9.0.1-0.20181013060901.el7ost.noarch

Comment 17 errata-xmlrpc 2019-01-11 11:50:43 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2019:0045