Bug 1771273 - The `opm alpha bundle build` command should support to build image for the manifest, which has multi sub-folders
Summary: The `opm alpha bundle build` command should support to build image for the ma...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: OLM
Version: 4.3.0
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ---
: 4.3.0
Assignee: Vu Dinh
QA Contact: Jian Zhang
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-11-12 05:33 UTC by Jian Zhang
Modified: 2020-01-23 11:12 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-01-23 11:12:18 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github operator-framework operator-registry pull 124 0 'None' closed Bug 1771273: Fix bundle build command with Docker and update documentation 2020-10-13 14:56:34 UTC
Github operator-framework operator-registry pull 127 0 'None' closed Bug 1771273: Ensure provided directory has at least one yaml manifest file 2020-10-13 14:56:35 UTC
Red Hat Product Errata RHBA-2020:0062 0 None None None 2020-01-23 11:12:36 UTC

Description Jian Zhang 2019-11-12 05:33:07 UTC
Description of problem:
Got below error when building the image for the manifest, which multi sub-folders
Step 8/9 : ADD /Users/jianzhang/goproject/src/github.com/example-inc/learn-operator/manifests/learn/*.yaml /manifests/
ADD failed: no source files were specified
Error: Failed to exec []string{"docker", "build", "-f", "/Users/jianzhang/goproject/src/github.com/example-inc/learn-operator/manifests/learn/Dockerfile", "-t", "quay.io/jiazha/learn-operator-image:v0.0.1", "."}: exit status 1

The directories of the manifest:
mac:learn jianzhang$ tree
.
├── 0.0.1
│   ├── learn-operator.v0.0.1.clusterserviceversion.yaml
│   └── learn.crd.yaml
├── 0.0.2
│   ├── learn-operator.v0.0.2.clusterserviceversion.yaml
│   └── learn.crd.yaml
├── Dockerfile
├── learn.package.yaml
└── metadata
    └── annotations.yaml


Version-Release number of selected component (if applicable):
mac:bin jianzhang$ git log
commit 237e0d77dd18fee42be455f29ed6b9191f9a6160 (HEAD -> master, origin/master, origin/HEAD)
Merge: 5d342b2 ab7034f
Author: OpenShift Merge Robot <openshift-merge-robot.github.com>
Date:   Fri Nov 8 19:39:32 2019 +0100

    Merge pull request #121 from kevinrizza/fix-docker-image-unpack

How reproducible:
always

Steps to Reproduce:
1. $ git clone git:operator-framework/operator-registry.git
2. $ cd operator-registry && go build ./cmd/opm
3. Create a manifest, which multi sub-folders, like below:
mac:learn jianzhang$ tree
.
├── 0.0.1
│   ├── learn-operator.v0.0.1.clusterserviceversion.yaml
│   └── learn.crd.yaml
├── 0.0.2
│   ├── learn-operator.v0.0.2.clusterserviceversion.yaml
│   └── learn.crd.yaml
├── learn.package.yaml

4, Run " ./opm alpha bundle build --directory /Users/jianzhang/goproject/src/github.com/example-inc/learn-operator/manifests/learn --tag quay.io/jiazha/learn-operator-image:v0.0.1 --package learn-operator --channels alpha --default alpha --overwrite"

Actual results:
Failed to build the docker image. This 
`ADD /Users/jianzhang/goproject/src/github.com/example-inc/learn-operator/manifests/learn/*.yaml /manifests/` command in the generated Docker file is not available for the folder, which has multi sub-folders.

mac:bin jianzhang$ ./opm alpha bundle build --directory /Users/jianzhang/goproject/src/github.com/example-inc/learn-operator/manifests/learn --tag quay.io/jiazha/learn-operator-image:v0.0.1 --package learn-operator --channels alpha --default alpha --overwrite
INFO[0000] Building annotations.yaml                    
INFO[0000] Building Dockerfile                          
INFO[0000] Building bundle image                        
Sending build context to Docker daemon  259.3MB
Step 1/9 : FROM scratch
 ---> 
Step 2/9 : LABEL operators.operatorframework.io.bundle.mediatype.v1=plain
 ---> Running in aa73843411ab
Removing intermediate container aa73843411ab
 ---> 8c12a4b9976f
Step 3/9 : LABEL operators.operatorframework.io.bundle.manifests.v1=manifests/
 ---> Running in 6e4bbd2f0fd0
Removing intermediate container 6e4bbd2f0fd0
 ---> 9f6f1fd92d8a
Step 4/9 : LABEL operators.operatorframework.io.bundle.metadata.v1=metadata/
 ---> Running in 111e764fc000
Removing intermediate container 111e764fc000
 ---> a4dc4377a311
Step 5/9 : LABEL operators.operatorframework.io.bundle.package.v1=learn-operator
 ---> Running in 38e30e5528f7
Removing intermediate container 38e30e5528f7
 ---> 4e6fc6da41c9
Step 6/9 : LABEL operators.operatorframework.io.bundle.channels.v1=alpha
 ---> Running in a6e31b84ef2e
Removing intermediate container a6e31b84ef2e
 ---> 5821ccc470fd
Step 7/9 : LABEL operators.operatorframework.io.bundle.channel.default.v1=alpha
 ---> Running in ad6d60c02ee9
Removing intermediate container ad6d60c02ee9
 ---> f940412c22d6
Step 8/9 : ADD /Users/jianzhang/goproject/src/github.com/example-inc/learn-operator/manifests/learn/*.yaml /manifests/
ADD failed: no source files were specified
Error: Failed to exec []string{"docker", "build", "-f", "/Users/jianzhang/goproject/src/github.com/example-inc/learn-operator/manifests/learn/Dockerfile", "-t", "quay.io/jiazha/learn-operator-image:v0.0.1", "."}: exit status 1
Usage:


Expected results:
The `opm alpha bundle build` command should support to build image for the manifest, which has multi sub-folders

Additional info:
mac:learn jianzhang$ cat Dockerfile 
FROM scratch

LABEL operators.operatorframework.io.bundle.mediatype.v1=plain
LABEL operators.operatorframework.io.bundle.manifests.v1=manifests/
LABEL operators.operatorframework.io.bundle.metadata.v1=metadata/
LABEL operators.operatorframework.io.bundle.package.v1=learn-operator
LABEL operators.operatorframework.io.bundle.channels.v1=alpha
LABEL operators.operatorframework.io.bundle.channel.default.v1=alpha

ADD /Users/jianzhang/goproject/src/github.com/example-inc/learn-operator/manifests/learn/*.yaml /manifests/
ADD /Users/jianzhang/goproject/src/github.com/example-inc/learn-operator/manifests/learn/metadata/annotations.yaml /metadata/annotations.yaml

Comment 1 Vu Dinh 2019-11-15 01:59:47 UTC
Hey Jian,

The PR has been merged to fix the `ADD failed: no source files were specified` error. The `bundle build` command is meant to package a specific version of an operator. As a result, you will need to specify the directory to manifest yaml of a specific version. So the correct command would be:

$ ./opm alpha bundle build --directory /Users/jianzhang/goproject/src/github.com/example-inc/learn-operator/manifests/learn/0.0.1/ --tag quay.io/jiazha/learn-operator-image:v0.0.1 --package learn-operator --channels alpha --default alpha --overwrite

Thanks,
Vu

Comment 3 Jian Zhang 2019-11-18 08:30:26 UTC
Hi, Vu

> So the correct command would be:
Yes, I know, it works well if only one version in my manifest folder.
But, for the users, they have many sub-folders in their manifest.

> The `bundle build` command is meant to package a specific version of an operator. As a result, you will need to specify the directory to manifest yaml of a specific version. 

Make sense. I think we should highlight it in the help info so that the users can know how to use it clearly. What do you think? 

mac:bin jianzhang$ ./opm alpha bundle build --help
The "opm alpha bundle build" command will generate operator
        bundle metadata if needed and build bundle image with operator manifest
        and metadata.

        For example: The command will generate annotations.yaml metadata plus
        Dockerfile for bundle image and then build a container image from
        provided operator bundle manifests generated metadata
        e.g. "quay.io/example/operator:v0.0.1".

        After the build process is completed, a container image would be built
        locally in docker and available to push to a container registry.

        $ opm alpha bundle build --directory /test/ --tag quay.io/example/operator:v0.1.0 \
		--package test-operator --channels stable,beta --default stable --overwrite

        Note: Bundle image is not runnable.

Comment 4 Vu Dinh 2019-11-18 16:06:21 UTC
Hey Jian,

I have opened another PR to clarify the help command + docs to highlight the `specific version` part. This phrase is already included in the bundle docs from the start [1] but I would repeat it in more place to make sure the idea is as visual as possible in the doc.

I also add some information to make sure users are aware of that requirement that all manifests yaml must be in the same directory. There is no reason for users to add yaml in some sub folders given we make it very clear what the `registry manifest format` looks like. We design the `opm` to align with that registry format and that's what we use for community operators [2].

Thanks,
Vu

[1]: https://github.com/operator-framework/operator-registry/blob/master/docs/design/operator-bundle.md
[2]: https://github.com/operator-framework/community-operators/tree/master/community-operators

Comment 6 Jian Zhang 2019-11-19 10:02:26 UTC
Vu, 

Great! Thanks very much! LGTM, verify it.

mac:bin jianzhang$ ./opm alpha bundle build --help
The "opm alpha bundle build" command will generate operator
        bundle metadata if needed and build bundle image with operator manifest
        and metadata for a specific version.

        For example: The command will generate annotations.yaml metadata plus
        Dockerfile for bundle image and then build a container image from
        provided operator bundle manifests generated metadata
        e.g. "quay.io/example/operator:v0.0.1".

        After the build process is completed, a container image would be built
        locally in docker and available to push to a container registry.

        $ opm alpha bundle build --directory /test/0.1.0/ --tag quay.io/example/operator:v0.1.0 \
		--package test-operator --channels stable,beta --default stable --overwrite
...

Error return, LGTM.
mac:bin jianzhang$ ./opm alpha bundle build --directory /Users/jianzhang/goproject/src/github.com/example-inc/learn-operator/manifests/learn --tag quay.io/jiazha/learn-operator-image:v0.0.2 --package learn-operator --channels alpha --default alpha --overwrite
Error: The directory /Users/jianzhang/goproject/src/github.com/example-inc/learn-operator/manifests/learn contains no yaml files

It works well when special a version:
mac:bin jianzhang$ ./opm alpha bundle build --directory /Users/jianzhang/goproject/src/github.com/example-inc/learn-operator/manifests/learn/0.0.2 --tag quay.io/jiazha/learn-operator-image:v0.0.2 --package learn-operator --channels alpha --default alpha --overwrite
INFO[0000] Building annotations.yaml                    
INFO[0000] Building Dockerfile                          
INFO[0000] Building bundle image                        
Sending build context to Docker daemon  176.1kB
Step 1/9 : FROM scratch
 ---> 
Step 2/9 : LABEL operators.operatorframework.io.bundle.mediatype.v1=registry+v1
 ---> Using cache
 ---> 909a04cdbe1a
Step 3/9 : LABEL operators.operatorframework.io.bundle.manifests.v1=manifests/
 ---> Using cache
 ---> 3bc7040cbc40
Step 4/9 : LABEL operators.operatorframework.io.bundle.metadata.v1=metadata/
 ---> Using cache
 ---> db089c4543c3
Step 5/9 : LABEL operators.operatorframework.io.bundle.package.v1=learn-operator
 ---> Using cache
 ---> b2d24af7248b
Step 6/9 : LABEL operators.operatorframework.io.bundle.channels.v1=alpha
 ---> Using cache
 ---> 0d29cd04d586
Step 7/9 : LABEL operators.operatorframework.io.bundle.channel.default.v1=alpha
 ---> Using cache
 ---> 43c3a9a60352
Step 8/9 : COPY /*.yaml /manifests/
 ---> bc67458e200b
Step 9/9 : COPY /metadata/annotations.yaml /metadata/annotations.yaml
 ---> 12c178269571
Successfully built 12c178269571
Successfully tagged quay.io/jiazha/learn-operator-image:v0.0.2

mac:operator-registry jianzhang$ git log
commit 768dcf1c08cab0fbd784767344be036498325039 (HEAD -> master, origin/master, origin/HEAD)
Merge: 3f13ca1 02e2a97
Author: OpenShift Merge Robot <openshift-merge-robot.github.com>
Date:   Tue Nov 19 00:34:16 2019 +0100

    Merge pull request #125 from anik120/opm-bundle-extract-bug
    
    Bug 1771850: Error logging for opm alpha bundle extract command

commit 02e2a97ae811ac94cacf47fde0048624821ffff0

Comment 8 errata-xmlrpc 2020-01-23 11:12:18 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:0062


Note You need to log in before you can comment on or make changes to this bug.