Bug 1279404
Summary: | The clustered mysql pod keeps CrashLoopBackOff for cannot create directory '/var/lib/mysql/data/mysql': Permission denied | |||
---|---|---|---|---|
Product: | OpenShift Container Platform | Reporter: | zhou ying <yinzhou> | |
Component: | ImageStreams | Assignee: | Scott Dodson <sdodson> | |
Status: | CLOSED ERRATA | QA Contact: | DeShuai Ma <dma> | |
Severity: | medium | Docs Contact: | ||
Priority: | unspecified | |||
Version: | 3.0.0 | CC: | aos-bugs, bleanhar, bparees, haowang, hhorak, jokerman, mmccomas, mnagy, pruan, wzheng, xiuwang | |
Target Milestone: | --- | Keywords: | Regression, UpcomingRelease | |
Target Release: | --- | |||
Hardware: | Unspecified | |||
OS: | Unspecified | |||
Whiteboard: | ||||
Fixed In Version: | Doc Type: | Bug Fix | ||
Doc Text: | Story Points: | --- | ||
Clone Of: | ||||
: | 1280466 1373099 (view as bug list) | Environment: | ||
Last Closed: | 2016-01-26 19:17:00 UTC | Type: | Bug | |
Regression: | --- | Mount Type: | --- | |
Documentation: | --- | CRM: | ||
Verified Versions: | Category: | --- | ||
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
Cloudforms Team: | --- | Target Upstream Version: | ||
Embargoed: | ||||
Bug Depends On: | ||||
Bug Blocks: | 1280466, 1373099 |
Description
zhou ying
2015-11-09 11:45:06 UTC
This is almost certainly because the image is 7 weeks old: registry.access.redhat.com/openshift3/mysql-55-rhel7 latest c3d990247510 7 weeks ago 306 MB Assigning to Scott to move to ON_QA after a new version is built. mysql-56-rhel7 has same issue with log as below: -bash-4.2# docker logs a3472a940df7 The 'master' server-id is 1 Running mysql_install_db ... 2015-11-11 02:06:07 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details). 2015-11-11 02:06:07 0 [Note] /opt/rh/rh-mysql56/root/usr/libexec/mysqld (mysqld 5.6.26-log) starting as process 15 ... 2015-11-11 02:06:07 15 [Warning] Can't create test file /var/lib/mysql/data/mysql-master-1-f5903.lower-test 2015-11-11 02:06:07 15 [Warning] Can't create test file /var/lib/mysql/data/mysql-master-1-f5903.lower-test /opt/rh/rh-mysql56/root/usr/libexec/mysqld: File '/var/lib/mysql/data/mysql-bin.index' not found (Errcode: 13 - Permission denied) 2015-11-11 02:06:07 15 [ERROR] Aborting 2015-11-11 02:06:07 15 [Note] Binlog end 2015-11-11 02:06:07 15 [Note] /opt/rh/rh-mysql56/root/usr/libexec/mysqld: Shutdown complete Wenjing, can you please provide steps to reproduce? I might be off-topic here, so scratch that if that is not valid -- "Permission denied" errors are often caused by wrong selinux context being used. I don't see the code, but just be sure you use :Z modificator in -v docker option and use correct owner of the directory for data, that is volume-mounted into the container. Brenton explained to me that the steps are the same as in the description. Wenjing, we test the MySQL image with a hostPath storage plugin pretty regularly and no such errors are there. Maybe there is something wrong with your PV setup? Honza, your suggestion might not be off-base, though remember that they are running it in OS not Docker. Ben, maybe somebody from the storage team can take a look at this and help QE out? It looks to me like the 55 and 56 rhel images need to be republished to pick up the script renaming that happened. the centos images work fine(I just tested them), and as Wenjing noted, the jenkins-built rhel images work fine. This is not a blocker for 3.1 since 1) the images don't ship w/ the release 2) replication is a tech preview Assigning to Scott to move to ON_QA when the mysql55 image is formally published (the image on RCM looks good: rcm-img-docker01.build.eng.bos.redhat.com:5001/openshift3/mysql-55-rhel7), and will create a clone for Honza to cover the mysql56 image. Note that none of this relates to the permission denied error that's being reported in comment 5 and 6, i did not see those issues when testing with an NFS volume and the Centos 55 and 56 images, so I suspect it is a configuration issue. Sorry, the sebool virt_use_nfs is not turned on by default in AEP container installation which causes this issue. I verified again with this option turned on, both mysql-55 and mysql-56 works well now. Thanks for your patience. @sdodson, @mnagy please feel free to change this bug status to ON_QA, so that I can verify it. Thanks for the quick verification Wenjing :) Verified with below version: mysql-56-rhel7 8101d9386118 mysql-55-rhel7 4aa22c02fb8a Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2016:0070 |