Bug 1694859 - image source extraction w/ symlink fails
Summary: image source extraction w/ symlink fails
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Build
Version: 4.1.0
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
: 4.1.0
Assignee: Adam Kaplan
QA Contact: wewang
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-04-01 21:18 UTC by Ben Parees
Modified: 2019-06-04 10:46 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: No Doc Update
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-06-04 10:46:44 UTC
Target Upstream Version:


Attachments (Terms of Use)
relative symlink bc with source (568 bytes, text/plain)
2019-04-09 19:32 UTC, Adam Kaplan
no flags Details
relative symlink bc no source (485 bytes, text/plain)
2019-04-09 19:32 UTC, Adam Kaplan
no flags Details
abs symlink bc source (568 bytes, text/plain)
2019-04-09 19:34 UTC, Adam Kaplan
no flags Details
abs symlink bc link only (485 bytes, text/plain)
2019-04-09 19:35 UTC, Adam Kaplan
no flags Details
symlink bc test case (472 bytes, text/plain)
2019-04-09 20:00 UTC, Adam Kaplan
no flags Details
source Dockerfile (206 bytes, text/plain)
2019-04-10 18:15 UTC, Adam Kaplan
no flags Details


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2019:0758 None None None 2019-06-04 10:46:53 UTC
Red Hat Bugzilla 1698152 None CLOSED image extraction path behavior has changed 2019-07-18 07:29:37 UTC

Internal Links: 1698152

Description Ben Parees 2019-04-01 21:18:28 UTC
Description of problem:
Performing a build where the image source input path specified is a symlink.  The extraction process appears to copy the symlink, not the content referenced by the symlink, meaning the build ultimately fails when it can't find the expected content.

This appears to be a regression from the behavior we had in 3.x when extracting image content to use for build inputs.



How reproducible:
Always


Steps to Reproduce:
1. Create an input image with a symlink like "/symlink" that points to "/some/file"
2. Create a build that uses the input image:
    images:
    - as: someimage
      from:
        kind: DockerImage
        name: docker.io/my/symylink:image
      paths:
      - destinationDir: .
        sourcePath: /symlink

3. Make the build try to use the referenced input content as "./some/file"


Actual results:
build fails, "./some" is not a valid path in the input content

Expected results:
build succeeds, ./some/file is a valid file because the symlink was followed during content extraction/copying.

Additional Info:
Should first confirm the behavior on 3.11 but we are seeing this as a difference in behavior when moving CI from 3.x to 4.x.

Comment 1 Ben Parees 2019-04-03 18:37:19 UTC
fyi we hit this because i was using an old ci-operator binary which referenced content via the symlink location.  The new ci-operator binary uses the paths to the real files and thus works on 4.x clusters.

But this is still a valid bug/regression in behavior (well actually maybe we need to confirm what the 3.x behavior was...I had assumed ci-operator was always doing this and it worked on 3.x but perhaps that assumption is not valid).  

Regardless it does not appear to block api.ci from moving to 4.x. (unless there are specific build configurations that do use symlinks in a way that breaks here)

Comment 2 Ben Parees 2019-04-03 18:39:13 UTC
specifically this change in ci-operator moved from symlink to real file:
https://github.com/openshift/ci-operator/commit/35523cc58ae1674fec3791656759eb1d6c039f60#diff-dd3690bad12236edb7359fb0e2df4ccd

Comment 3 Adam Kaplan 2019-04-04 20:51:00 UTC
Testing this on a local 3.11 cluster - looks like COPYing symlinks only works if:
a) The symlink and target are extracted.
b) The relative paths between the symlink and target are preserved.

I suspect this is because the source is TARed and sent as input to the docker daemon.

In 4.0 this doesn't work - buildah mounts in the local filesystem as the context for its build. So if our local file system contains a symlink with an absolute path, it will likely break.

Comment 4 Adam Kaplan 2019-04-09 19:32:11 UTC
Created attachment 1553922 [details]
relative symlink bc with source

BuildConfig with symlink image source that works on 3.11. Uses relative paths.

Comment 5 Adam Kaplan 2019-04-09 19:32:58 UTC
Created attachment 1553923 [details]
relative symlink bc no source

Build config with symlink, but only copies the link. Fails on 3.11.

Comment 6 Adam Kaplan 2019-04-09 19:34:25 UTC
Created attachment 1553925 [details]
abs symlink bc source

BuildConfig with source using absolute symlinks. Fails on 3.11, though there may be some edge cases where this succeeds.

Comment 7 Adam Kaplan 2019-04-09 19:35:14 UTC
Created attachment 1553927 [details]
abs symlink bc link only

BuildConfig with absolute symlink, no source copied. Always fails on 3.11.

Comment 8 Adam Kaplan 2019-04-09 20:00:07 UTC
Created attachment 1553931 [details]
symlink bc test case

Attached bc succeeds on 3.11, breaks on 4.1.

This is a blocker IMO - developers will extract a parent directory and not care about the content within.

Comment 9 Adam Kaplan 2019-04-10 18:15:30 UTC
Created attachment 1554339 [details]
source Dockerfile

Dockerfile for source, symlink uses a relative path

Comment 10 wewang 2019-04-15 07:37:28 UTC
Add a test case:OCP-23174 for the bug

Comment 11 Adam Kaplan 2019-04-16 13:50:25 UTC
PR to ensure we're extracting symlinks in images correctly: https://github.com/openshift/origin/pull/22456

Comment 12 Adam Kaplan 2019-04-17 12:43:10 UTC
Moving this to MODIFIED.

Origin PR [1] verifies that the copy behavior is correct if the source image contains a relative symlink.
Related bug 1698152 [2] contains the actual code fix.

[1] https://github.com/openshift/origin/pull/22456
[2] https://bugzilla.redhat.com/show_bug.cgi?id=1698152

Comment 14 wewang 2019-04-23 08:33:13 UTC
Verified in version:
4.1.0-0.nightly-2019-04-22-005054
payload: registry.svc.ci.openshift.org/ocp/release@sha256:3f3628cd9b694705cb0627ce72e61932df5d9938a291fabba1ed691230f7b548

Comment 16 errata-xmlrpc 2019-06-04 10:46:44 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:0758


Note You need to log in before you can comment on or make changes to this bug.