Bug 1758305 - Source To Image fails to add custom image labels with "image_metadata.json" file
Summary: Source To Image fails to add custom image labels with "image_metadata.json" file
Keywords:
Status: CLOSED EOL
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Documentation
Version: 4.1.z
Hardware: All
OS: Linux
unspecified
high
Target Milestone: ---
: 4.3.0
Assignee: Vikram Goyal
QA Contact: wewang
Vikram Goyal
URL:
Whiteboard:
: 1780622 1823362 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-10-03 19:41 UTC by Ricardo Zanini
Modified: 2023-10-06 18:38 UTC (History)
9 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-05-18 06:56:02 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
build config on OCP 4 (104.11 KB, application/octet-stream)
2019-10-03 19:41 UTC, Ricardo Zanini
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker KOGITO-287 0 Major Resolved Add generated protobuf files from Kogito Runtime Services during build into Kogito Data Index configMap 2020-06-04 19:52:44 UTC
Red Hat Issue Tracker KOGITO-290 0 Major Resolved Operator discover and automatically register metrics endpoints in OpenShift provisioned Prometheus 2020-06-04 19:52:44 UTC

Description Ricardo Zanini 2019-10-03 19:41:19 UTC
Created attachment 1622400 [details]
build config on OCP 4

Description of problem:

As stated in the docs, to have custom labels in the s2i builds, it's required to have the file "image_metadata.json" in this path "/tmp/.s2i/image_metadata.json" 

This works perfectly on OCP 3.x, but fails on OCP 4.x (no custom label added). It seems that the s2i is ignoring this file.

Also works locally with the s2i command line.

Version-Release number of selected component (if applicable):

4.1.16

How reproducible:

Always

Steps to Reproduce:

1. Deploy the reproducer into a OpenShift 4.x cluster:

~~~
oc new-app https://github.com/ricardozanini/tenkichannel --build-env MAVEN_ARGS_APPEND="-pl rain-forecast-process -am" --build-env ARTIFACT_DIR="rain-forecast-process/target" --build-env JAVA_APP_JAR="rain-forecast-process-1.0.0-SNAPSHOT-runner.jar" --strategy=source --docker-image=openshift/openjdk-11-rhel8:latest
~~~

(make sure to use this image: https://access.redhat.com/containers/?tab=overview#/registry.access.redhat.com/openjdk/openjdk-11-rhel8)

2. Wait for the build to finish

3.Inspect the imagestreamtag, in the dockerImageMetadata.Config.Labels field, you won't see the custom label described in the "image_metadata.json" file:

~~~
      Labels:
        org.jboss.container.deployments-dir: /deployments
        vcs-type: git
        usage: >-
          https://access.redhat.com/documentation/en-us/red_hat_jboss_middleware_for_openshift/3/html/red_hat_java_s2i_for_openshift/
        io.openshift.build.commit.author: Ricardo Zanini <zanini>
        io.k8s.display-name: Java Applications
        com.redhat.component: openjdk-11-rhel8-container
        summary: Source To Image (S2I) image for Red Hat OpenShift providing OpenJDK 11
        release: '13'
        io.openshift.build.commit.date: 'Thu Oct 3 15:53:10 2019 -0300'
        io.openshift.s2i.scripts-url: 'image:///usr/local/s2i'
        vendor: 'Red Hat, Inc.'
        io.openshift.expose-services: ''
        io.openshift.build.source-location: 'https://github.com/ricardozanini/tenkichannel'
        distribution-scope: public
        name: openjdk/openjdk-11-rhel8
        vcs-ref: e742320ec83194f7e012063f664054884c33c3cf
        io.openshift.s2i.destination: /tmp
        io.k8s.description: >-
          Platform for building and running plain Java applications (fat-jar and
          flat classpath)
        build-date: '2019-09-19T14:57:20.411924'
        io.fabric8.s2i.version.maven: '3.5'
        org.jboss.product: openjdk
        io.openshift.build.commit.message: minor changes
        url: >-
          https://access.redhat.com/containers/#/registry.access.redhat.com/openjdk/openjdk-11-rhel8/images/1.0-13
        org.jboss.product.openjdk.version: '11.0'
        com.redhat.license_terms: 'https://www.redhat.com/en/about/red-hat-end-user-license-agreements'
        architecture: x86_64
        io.openshift.build.commit.id: 7e4ea89b9413003dc294beba56e828b0913247ab
        io.cekit.version: 3.4.0
        io.openshift.tags: 'builder,java'
        version: '1.0'
        org.jboss.product.version: '11.0'
        maintainer: 'Red Hat, Inc.'
        io.openshift.build.commit.ref: master
        authoritative-source-url: registry.access.redhat.com
        description: Source To Image (S2I) image for Red Hat OpenShift providing OpenJDK 11
        io.openshift.build.image: >-
          image-registry.openshift-image-registry.svc:5000/openshift/openjdk-11-rhel8
        io.fabric8.s2i.version.jolokia: 1.5.0-redhat-1
        com.redhat.build-host: cpt-1002.osbs.prod.upshift.rdu2.redhat.com
~~~

This same steps can be reproduced in a OCP 3.x version to see the label being added.

Actual results:

No custom labels are added to the final image on OCP 4.1 builds.

Expected results:

Labels being added

Additional info:

One can use this repo for testing:
https://github.com/ricardozanini/tenkichannel

This is the command line for running locally:

(clone the repo locally first)

~~~
 s2i build .  openjdk/openjdk-11-rhel8 quay.io/ricardozanini/rain-forecast-process:1.0.0  -e MAVEN_ARGS_APPEND="-pl rain-forecast-process -am" -e ARTIFACT_DIR="rain-forecast-process/target" -e JAVA_APP_JAR="rain-forecast-process-1.0.0-SNAPSHOT-runner.jar" 
~~~

Then use "docker inspect", you should see a custom label named "org.tenkichannel/service":"rain-forecast-process"

Attached the build config logs in the OCP 4.x

Comment 1 Ricardo Zanini 2019-10-03 19:46:11 UTC
Forgot to mention the command to run locally with s2i command line tool:

~~~
s2i build . openjdk/openjdk-11-rhel8 rain-forecast-process  -e MAVEN_ARGS_APPEND="-pl rain-forecast-process -am" -e ARTIFACT_DIR="rain-forecast-process/target" -e JAVA_APP_JAR="rain-forecast-process-1.0.0-SNAPSHOT-runner.jar"
~~~

Comment 4 Ricardo Zanini 2019-10-08 21:52:31 UTC
Updating the reproducer command:

~~~
oc new-app https://github.com/ricardozanini/tenkichannel  --context-dir="rain-forecast-process" --strategy=source --docker-image=openshift/openjdk-11-rhel8:latest
~~~

Locally:

~~~
s2i build rain-forecast-process/ openjdk/openjdk-11-rhel8 rain-forecast-process
~~~

Comment 5 Gabe Montero 2019-10-24 19:35:36 UTC
OK confirmed that the "image_metadata.json" file is only used during s2i's commit image post executor step for labelling the image,
so it would have been lost in the 3.x to 4.x move off of docker and use of s2i's as-dockerfile support.

I was able to reproduce it with an s2i invocation with some tweaks based on Ricardo's oc new-app invocation:

gmontero ~/go/src/github.com/openshift/source-to-image  (master)$ s2i build https://github.com/ricardozanini/tenkichannel  gmontero/openjdk-11-rhel8 --context-dir="rain-forecast-process" --as-dockerfile=./ggm
Application dockerfile generated in ./ggm
gmontero ~/go/src/github.com/openshift/source-to-image  (master)$ cat ./ggm
FROM gmontero/openjdk-11-rhel8
LABEL "io.openshift.s2i.build.image"="gmontero/openjdk-11-rhel8" \
      "io.openshift.s2i.build.commit.author"="Ricardo Zanini <zanini>" \
      "io.openshift.s2i.build.commit.date"="Mon Oct 21 21:06:13 2019 -0300" \
      "io.openshift.s2i.build.commit.id"="7be7d8e060f1d8ff6606063283a7ec5c52ac3716" \
      "io.openshift.s2i.build.commit.ref"="master" \
      "io.openshift.s2i.build.commit.message"="Fix #10" \
      "io.openshift.s2i.build.source-location"="https://github.com/ricardozanini/tenkichannel" \
      "io.openshift.s2i.build.source-context-dir"="rain-forecast-process"

USER root
# Copying in source code
COPY upload/src /tmp/src
# Change file ownership to the assemble user. Builder image must support chown command.
RUN chown -R 1001:0 /tmp/src
USER 1001
# Assemble script sourced from builder image based on user input or image metadata.
# If this file does not exist in the image, the build will fail.
RUN /usr/libexec/s2i/assemble
# Run script sourced from builder image based on user input or image metadata.
# If this file does not exist in the image, the build will fail.
CMD /usr/libexec/s2i/run
gmontero ~/go/src/github.com/openshift/source-to-image  (master)$ 

I'm hopeful we can add use of it in https://github.com/openshift/source-to-image/blob/master/pkg/util/labels.go for --as-dockerfile

Comment 6 Gabe Montero 2019-10-24 22:26:25 UTC
I now get this dockerfile:

gmontero ~/go/src/github.com/openshift/source-to-image  (image_metadata_labels)$ cat ggm
FROM gmontero/openjdk-11-rhel8
LABEL "io.openshift.s2i.build.commit.author"="Ricardo Zanini <zanini>" \
      "io.openshift.s2i.build.commit.ref"="master" \
      "io.openshift.s2i.build.source-location"="https://github.com/ricardozanini/tenkichannel" \
      "org.tenkichannel/service"="rain-forecast-process" \
      "io.openshift.s2i.build.image"="gmontero/openjdk-11-rhel8" \
      "io.openshift.s2i.build.commit.date"="Mon Oct 21 21:06:13 2019 -0300" \
      "io.openshift.s2i.build.commit.id"="7be7d8e060f1d8ff6606063283a7ec5c52ac3716" \
      "io.openshift.s2i.build.commit.message"="Fix #10" \
      "io.openshift.s2i.build.source-context-dir"="rain-forecast-process"

USER root
# Copying in source code
COPY upload/src /tmp/src
# Change file ownership to the assemble user. Builder image must support chown command.
RUN chown -R 1001:0 /tmp/src
USER 1001
# Assemble script sourced from builder image based on user input or image metadata.
# If this file does not exist in the image, the build will fail.
RUN /usr/libexec/s2i/assemble
# Run script sourced from builder image based on user input or image metadata.
# If this file does not exist in the image, the build will fail.
CMD /usr/libexec/s2i/run

  "org.tenkichannel/service"="rain-forecast-process"  is from Ricardo's image_metadata.json

Note, after the source-to-image PR merges, we'll need a bump of s2i in https://github.com/openshift/builder for the oc new-app induced build to work as expected.

Comment 7 Ricardo Zanini 2019-10-24 23:38:09 UTC
Many thanks for this, Gabe!

Please also note that BuildConfigs with SourceStrategy also have to work with this new patch. I assume it will since the container used for sti has the s2i binary installed.

Also, in the docs [1] the JSON format file follows this pattern:


```
{
  "labels": [
    {"labelkey1":"value1"},
    {"labelkey2":"value2"},
    .........
  ]
}
```

But I see some image_metadata.json in the following format as well (working on 3.11):

```
{
  "labels": [ {
    "labelkey1" : "value1",
    "labelkey2" : "value2"
    }
  ]
}

```

Could you please confirm if both formats will work (maybe adding a new test case to your PR)?


[1] https://github.com/openshift/source-to-image/blob/master/docs/new_labels.md#example

Comment 8 Gabe Montero 2019-10-28 15:15:17 UTC
Both formats work @Ricardo.  I've added both cases to the new unit tests in the PR noted above.

Also, my note above in reference to bumping openshift/builder is what will facilitate this working with Source Strategy build configs.

Comment 9 Ricardo Zanini 2019-10-28 15:41:26 UTC
Thanks for the heads up, @Gabe.

Comment 12 wewang 2019-11-06 06:12:29 UTC
Verified in version:
4.3.0-0.nightly-2019-11-02-092336
s2i v1.2.0-12-g1877e115


$s2i build https://github.com/ricardozanini/tenkichannel  gmontero/openjdk-11-rhel8 --context-dir="rain-forecast-process" --as-dockerfile=./ggm
FROM gmontero/openjdk-11-rhel8
LABEL "io.openshift.s2i.build.commit.message"="Fix #10" \
      "org.tenkichannel/service"="rain-forecast-process" \        

      "io.openshift.s2i.build.commit.id"="7be7d8e060f1d8ff6606063283a7ec5c52ac3716" \
      "io.openshift.s2i.build.commit.ref"="master" \
      "io.openshift.s2i.build.source-location"="https://github.com/ricardozanini/tenkichannel" \
      "io.openshift.s2i.build.source-context-dir"="rain-forecast-process" \        #  "org.tenkichannel/service"="rain-forecast-process"  is from Ricardo's image_metadata.json                                  
      "io.openshift.s2i.build.image"="gmontero/openjdk-11-rhel8" \
      "io.openshift.s2i.build.commit.author"="Ricardo Zanini <zanini>" \
      "io.openshift.s2i.build.commit.date"="Mon Oct 21 21:06:13 2019 -0300"

USER root
# Copying in source code
COPY upload/src /tmp/src
# Change file ownership to the assemble user. Builder image must support chown command.
RUN chown -R 1001:0 /tmp/src
USER 1001
# Assemble script sourced from builder image based on user input or image metadata.
# If this file does not exist in the image, the build will fail.
RUN /usr/libexec/s2i/assemble
# Run script sourced from builder image based on user input or image metadata.
# If this file does not exist in the image, the build will fail.
CMD /usr/libexec/s2i/run

Comment 13 Karel Suta 2019-12-05 15:12:52 UTC
I have hit an issue while checking this fix.
The reproducer provided by Ricardo works in fixed version, however this reproducer is placing "image_metadata.json" file into wrong folder - "/tmp/src/.s2i/image_metadata.json"

According to [1] the correct place for "image_metadata.json" is "/tmp/.s2i/image_metadata.json". If image_metadata.json is placed into this folder then the labels are not processed by S2I build and are missing in built image.

To reproduce this issue you can use this command:
~~~
oc new-app https://github.com/kiegroup/kogito-examples  --context-dir="onboarding-example/hr" --strategy=source --docker-image=quay.io/kiegroup/kogito-quarkus-ubi8-s2i:0.5.1
~~~

Expected image label and value is for example "prometheus.io/path" : "/metrics".

[1] https://github.com/openshift/source-to-image/blob/master/docs/new_labels.md

Comment 14 Gabe Montero 2019-12-09 17:34:10 UTC
I just tried 

oc new-app https://github.com/ricardozanini/tenkichannel  --context-dir="rain-forecast-process" --strategy=source --docker-image=gmontero/openjdk-11-rhel8:latest

and see the label from the image_metadata.json file ("org.tenkichannel/service":"rain-forecast-process") added to the generated dockerfile:

STEP 2: LABEL "io.openshift.build.image"="gmontero/openjdk-11-rhel8@sha256:8d2177bdfc307bb023ce8b9438f2c1d4bd024fb15049a50e2672aa8c7fa69745" "io.openshift.build.commit.author"="Ricardo Zanini <1538000+ricardozanini.github.com>" "io.openshift.build.commit.id"="712bb849ceb6df820c2f6d88015f4dd529a8b6a4" "io.openshift.build.source-location"="https://github.com/ricardozanini/tenkichannel" "org.tenkichannel/service"="rain-forecast-process" "io.openshift.build.commit.date"="Wed Nov 27 12:01:31 2019 -0300" "io.openshift.build.commit.ref"="master" "io.openshift.build.commit.message"="bump quarkus, kogito and camel / fixing yahoo weather bug (#13)" "io.openshift.build.source-context-dir"="rain-forecast-process"

where the image_metadata.json file is located directly off of the context root (i.e. no "src" dir):  https://github.com/ricardozanini/tenkichannel/blob/master/rain-forecast-process/.s2i/image_metadata.json

I git cloned https://github.com/kiegroup/kogito-examples and see no image_metadata.json file

We are continuing to work this in the bug I had Karel open:  https://bugzilla.redhat.com/show_bug.cgi?id=1780622

Comment 15 Gabe Montero 2019-12-10 14:54:43 UTC
Per https://bugzilla.redhat.com/show_bug.cgi?id=1780622#c5 Ben wants us to revert the changes done with this bug.

Starting that process.

Subsequently, doc updates in the interim noting the lack of 4.x support will be tracked with this bug.  

Open Jira feature requests for further debate on implementing the feature.

Note:  output labels can also be set via https://docs.openshift.com/container-platform/4.2/builds/managing-build-output.html#builds-output-image-labels_managing-build-output

Comment 16 Gabe Montero 2019-12-10 14:56:10 UTC
*** Bug 1780622 has been marked as a duplicate of this bug. ***

Comment 17 Gabe Montero 2019-12-10 15:26:40 UTC
*** Bug 1780622 has been marked as a duplicate of this bug. ***

Comment 18 Gabe Montero 2019-12-10 15:32:50 UTC
So we need to update https://docs.openshift.com/container-platform/4.2/builds/managing-build-output.html#builds-output-image-labels_managing-build-output
to note that the s2i command line feature noted at https://github.com/openshift/source-to-image/blob/master/docs/new_labels.md is currently not supported
in 4.x

See Ben's comments in https://bugzilla.redhat.com/show_bug.cgi?id=1780622 if you want details.

Comment 19 Spolti 2019-12-17 20:21:34 UTC
Gabe, Ben

Could you guys please clarify what would be the official way to set custom labels at build time in a similar way we have for ocp 3.11?
If my understanding is correct, by using buildconfig.spec.output.imageLabels we can't update it during build time (assemble) automatically, it needs to be updated by a operator or manually, is that correct?

Or will be the buildconfig.spec.output.imageLabels label the only supported option?

Comment 20 Ben Parees 2019-12-17 20:25:05 UTC
> Could you guys please clarify what would be the official way to set custom labels at build time in a similar way we have for ocp 3.11?

unfortunately it's currently a gap in what is possible with v4.

> Or will be the buildconfig.spec.output.imageLabels label the only supported option?

for now it's the only way, but i think we'd strongly consider an RFE to bring the "dynamic" behavior back if you have a use case.  Just be aware that it's not a trivial rfe unfortunately.

Comment 21 Wenjing Zheng 2020-01-08 07:17:25 UTC
So, Gabe, the fix of this bug will be add a doc to say buildconfig.spec.output.imageLabels label is the only supported option?

Comment 22 Gabe Montero 2020-01-08 15:09:20 UTC
Yes Wenjing, to reiterate what Ben noted #Comment 20

Comment 24 Adam Kaplan 2020-04-13 15:04:39 UTC
*** Bug 1823362 has been marked as a duplicate of this bug. ***


Note You need to log in before you can comment on or make changes to this bug.