Bug 1810069 - buildah is not expanding env vars in file paths
Summary: buildah is not expanding env vars in file paths
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Containers
Version: 4.4
Hardware: Unspecified
OS: Unspecified
Target Milestone: ---
: 4.5.0
Assignee: Jindrich Novy
QA Contact: weiwei jiang
Depends On: 1817175 1822031 1822032 1822033 1822034 1822035 1822036 1822037 1822038
Blocks: 1822207
TreeView+ depends on / blocked
Reported: 2020-03-04 14:07 UTC by Jeff Cantrill
Modified: 2020-07-13 17:18 UTC (History)
10 users (show)

Fixed In Version: buildah 1.13.2 and newer, buildah-1.11.6-6.rhaos4.4.el8, buildah-1.11.6-8.rhaos4.5.el8
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1810174 1822207 (view as bug list)
Last Closed: 2020-07-13 17:18:09 UTC
Target Upstream Version:

Attachments (Terms of Use)

System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2020:2409 0 None None None 2020-07-13 17:18:34 UTC

Description Jeff Cantrill 2020-03-04 14:07:00 UTC
Description of problem:
buildah fails when a file path uses a variable in its name:

STEP 1: FROM centos
STEP 2: ENV foo=oo
STEP 3: ADD ./f${foo} /tmp/foo
error dry-running "ADD ./f${foo} /tmp/foo": no files found matching "/tmp/t/f": no such file or directory
ERRO exit status 1   

Version-Release number of selected component (if applicable):
$ buildah version
Version:         1.12.0
Go Version:      go1.12.13
Image Spec:      1.0.1-dev
Runtime Spec:    1.0.1-dev
CNI Spec:        0.4.0
libcni Version:  
image Version:   5.0.0
Git Commit:      
Built:           Wed Dec 31 19:00:00 1969
OS/Arch:         linux/amd64

How reproducible: Always

Steps to Reproduce:

Actual results:
STEP 15: ADD kibana_ui_objects/ ${ES_HOME}/kibana_ui_objects/
STEP 16: ADD probe/ ${ES_HOME}/probe/
STEP 17: ADD init.sh run.sh prep-install.${RELEASE_STREAM} install.sh ${HOME}/
error dry-running "ADD init.sh run.sh prep-install.${RELEASE_STREAM} install.sh ${HOME}/": no files found matching "/home/jeff.cantrill/git/origin-aggregated-logging/elasticsearch/prep-install.": no such file or directory
ERRO exit status 1    

Expected results:

Additional info:
 * Actual Build failure: https://prow.svc.ci.openshift.org/view/gcs/origin-ci-test/pr-logs/pull/openshift_origin-aggregated-logging/1841/pull-ci-openshift-origin-aggregated-logging-master-images/1210
* https://github.com/openshift/origin-aggregated-logging/blob/master/elasticsearch/Dockerfile.centos7#L48

Comment 1 Ben Parees 2020-03-04 14:13:38 UTC
this is breaking logging builds on CI now that CI moved to 4.x.

note that this works ok:

FROM centos
ENV foo=foo
ADD ./${foo} /tmp/foo

it's only substitution within a path segment that seems to be a problem:
FROM centos
ENV foo=oo
ADD ./f${foo} /tmp/foo

Comment 2 Tom Sweeney 2020-03-04 16:25:39 UTC
I believe this was fixed with https://github.com/containers/buildah/pull/2095 which landed in Buildah v1.13.2, but I'll need to verify.

Comment 3 Tom Sweeney 2020-03-04 19:02:06 UTC
Confirmed the issue is in Buildah v1.12.0 and fixed in Buildah v1.13.2 (and for that matter Buildah v1.14.0) via https://github.com/containers/buildah/pull/2095 

Jindrich, Nalin, Ben, next step?

Comment 4 Ben Parees 2020-03-04 19:23:13 UTC
Please work with Adam Kaplan on the devex team to get the fix vendored into OCP.

can start w/ this bug for 4.5:

but should probably be backported at least to 4.4.0 (if we can get it in before march 6th) or 4.4.z(if we can't).

Comment 8 Daniel Walsh 2020-03-06 16:28:10 UTC
At this point is this a buildah bug or imagebuilder?

Comment 9 Tom Sweeney 2020-03-06 20:21:59 UTC

It's a Buildah bug that was fixed with https://github.com/containers/buildah/pull/2095  which landed in Buildah 1.13.2.  I need to figure out how to get that fix to where it needs to go for OCP 4.5

Comment 11 Tom Sweeney 2020-05-21 21:50:30 UTC
Assigning to Jindrich as I think this is just down to packaging now.  Jindrich let's talk if not.

Comment 17 weiwei jiang 2020-05-25 06:01:18 UTC
Checked with buildah-1.11.6-8.rhaos4.5.el8.x86_64, moved to verified.

# buildah --version 
buildah version 1.11.6 (image-spec 1.0.1-dev, runtime-spec 1.0.1-dev)
# rpm -q buildah

# cat Dockerfile 
FROM centos
ENV foo=oo
ADD ./f${foo} /tmp/foo

# echo "bar" > foo
# buildah bud -t foo:bar .
STEP 1: FROM centos
Getting image source signatures
Copying blob 294dced27509 done
Copying config c460095972 done
Writing manifest to image destination
Storing signatures
STEP 2: ENV foo=oo
STEP 3: ADD ./f${foo} /tmp/foo
STEP 4: COMMIT foo:bar
Getting image source signatures
Copying blob ca4be3487ff9 skipped: already exists
Copying blob 128588b1e263 done
Copying config 518518f0e2 done
Writing manifest to image destination
Storing signatures

# podman run -it foo:bar /bin/bash -c "cat /tmp/foo"

Comment 19 errata-xmlrpc 2020-07-13 17:18:09 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.