Bug 1895093

Summary: Dockerfile builds cannot change /etc/pki/ca-trust
Product: OpenShift Container Platform Reporter: OpenShift BugZilla Robot <openshift-bugzilla-robot>
Component: BuildAssignee: Adam Kaplan <adam.kaplan>
Status: CLOSED ERRATA QA Contact: wewang <wewang>
Severity: urgent Docs Contact: Rolfe Dlugy-Hegwer <rdlugyhe>
Priority: unspecified    
Version: 4.6CC: aaleman, adam.kaplan, aos-bugs, bparees, gmontero, rdlugyhe, wking, wzheng
Target Milestone: ---Keywords: Regression
Target Release: 4.6.z   
Hardware: Unspecified   
OS: Unspecified   
Fixed In Version: Doc Type: Bug Fix
Doc Text:
* Previously, Dockerfile builds could not change permissions of `/etc/pki/ca-trust` or create files inside it. This issue was caused by fixing link:https://bugzilla.redhat.com/show_bug.cgi?id=1826183[] in version 4.6, which added support for HTTPS proxies with CAs for builds and ALWAYS mounted `/etc/pki/ca-trust`. This prevented builds that included their own CAs or modified the system trust store from working correctly at runtime. The current release fixes this issue by reverting Bug 1826183. Now, builder images that include their own CAs work again. (link:https://bugzilla.redhat.com/show_bug.cgi?id=1895093[*BZ#1895093*])
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-11-16 14:37:43 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Bug Depends On: 1891759    
Bug Blocks:    

Description OpenShift BugZilla Robot 2020-11-05 18:12:08 UTC
+++ This bug was initially created as a clone of Bug #1891759 +++

Description of problem:

Dockerfile builds cannot change permissions of /etc/pki/ca-trust or create files inside it.

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1. create a Dockerfile with RUN chmod a+w -R /etc/pki/ca-trust/extracted
2. build an image
3. inspect /etc/pki/ca-trust/extracted permissions

Actual results:


Expected results:


Additional info:

It affects our ability to build release images:

  * https://github.com/openshift/image-registry/blob/edd1c864a4cd87e496fdce38aa166f22683eb234/Dockerfile.rhel7#L10
  * https://github.com/openshift/cloud-credential-operator/blob/1e5a13b5a796630ef6fa9de5ec974b330b77fd5f/Dockerfile#L11

--- Additional comment from adam.kaplan on 2020-10-27 12:51:43 UTC ---

This was a consequence of fixing https://bugzilla.redhat.com/show_bug.cgi?id=1826183. /etc/pki/ca-trust is a mount point for buildah, the contents of which are managed by the build container. When the build container starts, it adds the cluster-wide CA to the set of anchors, and then runs `update-ca-trust extract` as root.

In the case of the image registry, it wants /etc/pki/ca-trust to be writeable so that `update-ca-trust` can be run as non-root.

--- Additional comment from bparees on 2020-11-04 04:45:12 UTC ---

I think https://bugzilla.redhat.com/show_bug.cgi?id=1826183 needs to be considered for reversion, given the impact on existing build behavior.

It means that no one w/ a builder image that includes their own CAs can continue to function as expected, right?  Because their CAs will be stomped by the always present mount that that fix introduced.

--- Additional comment from bparees on 2020-11-04 14:36:16 UTC ---

more specifically, here is my suggested course of action:

1) put in a change to 4.7 and 4.6 that does not mount the content (you can leave the rest of the logic in place) 
2) get CI to update to the 4.6.z patch asap 
3) reintroduce the mount based on a buildconfig api opt-in field. in 4.6+4.7.

Comment 2 wewang 2020-11-09 05:46:10 UTC
Verified in version:

STEP 1: FROM registry.svc.ci.openshift.org/ocp/4.7:base
STEP 2: RUN chmod a+w -R /etc/pki/ca-trust/extracted
--> e416ed50a7b
STEP 3: RUN ls -ld /etc/pki/ca-trust/extracted
drwxrwxrwx. 1 root root 70 Sep  1 19:39 /etc/pki/ca-trust/extracted

Comment 5 errata-xmlrpc 2020-11-16 14:37:43 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.6.4 bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.


Comment 10 Rolfe Dlugy-Hegwer 2021-02-24 12:05:24 UTC
Replaced the doc text with the one from the "older sibling" clone.

Comment 11 Rolfe Dlugy-Hegwer 2021-03-04 20:03:25 UTC
Published in the enterprise-4.7 OpenShift release notes.