Bug 2075135

Summary: Latest ose-jenkins-agent-base:v4.9.0 image fails to start on OpenShift due to FIPS error
Product: OpenShift Container Platform Reporter: OpenShift BugZilla Robot <openshift-bugzilla-robot>
Component: JenkinsAssignee: Gabe Montero <gmontero>
Status: CLOSED ERRATA QA Contact: Jitendar Singh <jitsingh>
Severity: high Docs Contact:
Priority: high    
Version: 4.8CC: aos-bugs, cdaley, dkarde, gmontero, jitsingh, pbhattac, spandura
Target Milestone: ---   
Target Release: 4.10.z   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 2077289 (view as bug list) Environment:
Last Closed: 2022-05-02 18:38:50 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 2066019    
Bug Blocks: 2077289    

Comment 3 Jitendar Singh 2022-04-22 07:01:50 UTC
verified
==============
used private-templates/functionality-testing/aos-4_10/ipi-on-aws/versioned-installer-fips flexy template with installer_payload_image: quay.io/openshift-release-dev/ocp-release:4.10.11-x86_64 to provision fips enabled cluster

jitsingh@fedora  ~/go/src/github.com/openshift  oc new-app jenkins-ephemeral -p NAMESPACE=$(oc project -q) -p JENKINS_IMAGE_STREAM_TAG=jenkins-jitsingh:latest
--> Deploying template "openshift/jenkins-ephemeral" to project jenkins-test

     Jenkins (Ephemeral)
     ---------
     Jenkins service, without persistent storage.
     
     WARNING: Any data stored will be lost upon pod destruction. Only use this template for testing.

     A Jenkins service has been created in your project.  Log into Jenkins with your OpenShift account.  The tutorial at https://github.com/openshift/origin/blob/master/examples/jenkins/README.md contains more information about using this template.

     * With parameters:
        * Jenkins Service Name=jenkins
        * Jenkins JNLP Service Name=jenkins-jnlp
        * Enable OAuth in Jenkins=true
        * Memory Limit=1Gi
        * Jenkins ImageStream Namespace=jenkins-test
        * Disable memory intensive administrative monitors=false
        * Jenkins ImageStreamTag=jenkins-jitsingh:latest
        * Allows use of Jenkins Update Center repository with invalid SSL certificate=false
        * Image used for the 'jnlp' container of the sample 'java-sidecar' and 'nodejs-sidecar' PodTemplates=image-registry.openshift-image-registry.svc:5000/openshift/jenkins-agent-base:latest
        * Image used for the 'java' container of the sample 'java-builder' PodTemplate=image-registry.openshift-image-registry.svc:5000/openshift/java:latest
        * Image used for the 'nodejs' container of the sample 'nodejs-builder' PodTemplate=image-registry.openshift-image-registry.svc:5000/openshift/nodejs:latest

--> Creating resources ...
    route.route.openshift.io "jenkins" created
    configmap "jenkins-trusted-ca-bundle" created
    deploymentconfig.apps.openshift.io "jenkins" created
    serviceaccount "jenkins" created
    rolebinding.authorization.openshift.io "jenkins_edit" created
    service "jenkins-jnlp" created
    service "jenkins" created
--> Success
    Access your application via route 'jenkins-jenkins-test.apps.fps.qe.devcluster.openshift.com' 
    Run 'oc status' to view your app.
 jitsingh@fedora  ~/go/src/github.com/openshift  cd jenkins 
 jitsingh@fedora  ~/go/src/github.com/openshift/jenkins   master  cd smoke/samples 
 jitsingh@fedora  ~/go/src/github.com/openshift/jenkins/smoke/samples   master  ls
java-builder-cm.yaml  kubeconfig  maven_pipeline.yaml  nodejs-builder-cm.yaml  nodejs_pipeline.yaml
 jitsingh@fedora  ~/go/src/github.com/openshift/jenkins/smoke/samples   master  oc get pods -w
NAME               READY   STATUS    RESTARTS   AGE
jenkins-1-deploy   1/1     Running   0          36s
jenkins-1-w674p    0/1     Running   0          31s
jenkins-1-w674p    1/1     Running   0          50s
jenkins-1-deploy   0/1     Completed   0          56s
jenkins-1-deploy   0/1     Completed   0          61s
^C%                                                                                                                                                                                            ✘ jitsingh@fedora  ~/go/src/github.com/openshift/jenkins/smoke/samples   master  oc create -f java-builder-cm.yaml
configmap/jenkins-agent-java-builder created
 jitsingh@fedora  ~/go/src/github.com/openshift/jenkins/smoke/samples   master  ls
java-builder-cm.yaml  kubeconfig  maven_pipeline.yaml  nodejs-builder-cm.yaml  nodejs_pipeline.yaml
 jitsingh@fedora  ~/go/src/github.com/openshift/jenkins/smoke/samples   master  oc create -f nodejs-builder-cm.yaml 
configmap/jenkins-agent-nodejs created
 jitsingh@fedora  ~/go/src/github.com/openshift/jenkins/smoke/samples   master  oc get routes                                                   
NAME      HOST/PORT                                                   PATH   SERVICES   PORT    TERMINATION     WILDCARD
jenkins   jenkins-jenkins-test.apps.fps.qe.devcluster.openshift.com          jenkins    <all>   edge/Redirect   None
 jitsingh@fedora  ~/go/src/github.com/openshift/jenkins/smoke/samples   master  oc new-build https://github.com/akram/pipes.git\#container-nodes
    * A pipeline build using source code from https://github.com/akram/pipes.git#container-nodes will be created
      * Use 'oc start-build' to trigger a new build

--> Creating resources with label build=pipes ...
    buildconfig.build.openshift.io "pipes" created
--> Success
 jitsingh@fedora  ~/go/src/github.com/openshift/jenkins/smoke/samples   master  oc get builds
NAME      TYPE              FROM                  STATUS   STARTED         DURATION
pipes-1   JenkinsPipeline   Git@container-nodes   New      6 seconds ago   
 jitsingh@fedora  ~/go/src/github.com/openshift/jenkins/smoke/samples   master  oc start-build pipes
build.build.openshift.io/pipes-2 started
 jitsingh@fedora  ~/go/src/github.com/openshift/jenkins/smoke/samples   master  oc start-build pipes
build.build.openshift.io/pipes-3 started
 jitsingh@fedora  ~/go/src/github.com/openshift/jenkins/smoke/samples   master  oc start-build pipes
build.build.openshift.io/pipes-4 started
 jitsingh@fedora  ~/go/src/github.com/openshift/jenkins/smoke/samples   master  oc start-build pipes
build.build.openshift.io/pipes-5 started
 jitsingh@fedora  ~/go/src/github.com/openshift/jenkins/smoke/samples   master  oc start-build pipes
build.build.openshift.io/pipes-6 started
 jitsingh@fedora  ~/go/src/github.com/openshift/jenkins/smoke/samples   master  oc start-build pipes
build.build.openshift.io/pipes-7 started
 jitsingh@fedora  ~/go/src/github.com/openshift/jenkins/smoke/samples   master  oc get builds -w
NAME      TYPE              FROM                  STATUS    STARTED          DURATION
pipes-1   JenkinsPipeline   Git@container-nodes   Running   54 seconds ago   
pipes-2   JenkinsPipeline   Git@container-nodes   New                        
pipes-3   JenkinsPipeline   Git@container-nodes   New                        
pipes-4   JenkinsPipeline   Git@container-nodes   New                        
pipes-5   JenkinsPipeline   Git@container-nodes   New                        
pipes-6   JenkinsPipeline   Git@container-nodes   New                        
pipes-7   JenkinsPipeline   Git@container-nodes   New                        
pipes-1   JenkinsPipeline   Git@container-nodes   Running   About a minute ago   
pipes-1   JenkinsPipeline   Git@container-nodes   Running   About a minute ago   
pipes-1   JenkinsPipeline   Git@container-nodes   Running   About a minute ago   
pipes-1   JenkinsPipeline   Git@container-nodes   Complete   About a minute ago   
pipes-2   JenkinsPipeline   Git@container-nodes   Pending                         
pipes-2   JenkinsPipeline   Git@container-nodes   Running    1 second ago         
pipes-2   JenkinsPipeline   Git@container-nodes   Running    11 seconds ago       
pipes-2   JenkinsPipeline   Git@container-nodes   Running    21 seconds ago       
pipes-2   JenkinsPipeline   Git@container-nodes   Running    31 seconds ago       
pipes-2   JenkinsPipeline   Git@container-nodes   Running    41 seconds ago       
pipes-2   JenkinsPipeline   Git@container-nodes   Complete   48 seconds ago       
pipes-3   JenkinsPipeline   Git@container-nodes   Pending                         
pipes-3   JenkinsPipeline   Git@container-nodes   Running    3 seconds ago        
pipes-3   JenkinsPipeline   Git@container-nodes   Running    18 seconds ago       
pipes-3   JenkinsPipeline   Git@container-nodes   Running    23 seconds ago       
pipes-3   JenkinsPipeline   Git@container-nodes   Running    38 seconds ago       
pipes-3   JenkinsPipeline   Git@container-nodes   Running    48 seconds ago       
pipes-3   JenkinsPipeline   Git@container-nodes   Complete   52 seconds ago       
pipes-4   JenkinsPipeline   Git@container-nodes   Pending                         
pipes-4   JenkinsPipeline   Git@container-nodes   Running    1 second ago         
pipes-4   JenkinsPipeline   Git@container-nodes   Running    11 seconds ago       
pipes-4   JenkinsPipeline   Git@container-nodes   Running    21 seconds ago       
pipes-4   JenkinsPipeline   Git@container-nodes   Running    31 seconds ago       
pipes-4   JenkinsPipeline   Git@container-nodes   Running    51 seconds ago       
pipes-4   JenkinsPipeline   Git@container-nodes   Complete   55 seconds ago       
pipes-5   JenkinsPipeline   Git@container-nodes   Pending                         
pipes-5   JenkinsPipeline   Git@container-nodes   Running    1 second ago         
pipes-5   JenkinsPipeline   Git@container-nodes   Running    6 seconds ago        
pipes-5   JenkinsPipeline   Git@container-nodes   Running    11 seconds ago       
pipes-5   JenkinsPipeline   Git@container-nodes   Running    21 seconds ago       
pipes-5   JenkinsPipeline   Git@container-nodes   Running    31 seconds ago       
pipes-5   JenkinsPipeline   Git@container-nodes   Running    51 seconds ago       
pipes-5   JenkinsPipeline   Git@container-nodes   Complete   About a minute ago   
pipes-6   JenkinsPipeline   Git@container-nodes   Pending                         
pipes-6   JenkinsPipeline   Git@container-nodes   Running    1 second ago         
pipes-6   JenkinsPipeline   Git@container-nodes   Running    11 seconds ago       
pipes-6   JenkinsPipeline   Git@container-nodes   Running    21 seconds ago       
pipes-6   JenkinsPipeline   Git@container-nodes   Running    31 seconds ago       
pipes-6   JenkinsPipeline   Git@container-nodes   Running    46 seconds ago       
pipes-6   JenkinsPipeline   Git@container-nodes   Complete   50 seconds ago       
pipes-1   JenkinsPipeline   Git@container-nodes   Complete   6 minutes ago        
pipes-7   JenkinsPipeline   Git@container-nodes   Pending                         
pipes-7   JenkinsPipeline   Git@container-nodes   New        Less than a second ago   
pipes-7   JenkinsPipeline   Git@container-nodes   Running    5 seconds ago            
pipes-7   JenkinsPipeline   Git@container-nodes   Running    10 seconds ago           
pipes-7   JenkinsPipeline   Git@container-nodes   Running    20 seconds ago           
pipes-7   JenkinsPipeline   Git@container-nodes   Running    30 seconds ago           
^C%                                                                                                                                                                                            ✘ jitsingh@fedora  ~/go/src/github.com/openshift/jenkins/smoke/samples   master  oc get builds
NAME      TYPE              FROM                  STATUS     STARTED              DURATION
pipes-2   JenkinsPipeline   Git@container-nodes   Complete   5 minutes ago        
pipes-3   JenkinsPipeline   Git@container-nodes   Complete   4 minutes ago        
pipes-4   JenkinsPipeline   Git@container-nodes   Complete   3 minutes ago        
pipes-5   JenkinsPipeline   Git@container-nodes   Complete   2 minutes ago        
pipes-6   JenkinsPipeline   Git@container-nodes   Complete   About a minute ago   
pipes-7   JenkinsPipeline   Git@container-nodes   Running    44 seconds ago       
 jitsingh@fedora  ~/go/src/github.com/openshift/jenkins/smoke/samples   master  oc get pods
NAME                            READY   STATUS        RESTARTS   AGE
java-builder-template-5p67n     2/2     Terminating   0          55s
jenkins-1-deploy                0/1     Completed     0          10m
jenkins-1-w674p                 1/1     Running       0          10m
nodejs-builder-template-cq4fm   2/2     Terminating   0          28s
==========================================================================
oc logs -f jenkins-1-w674p
2022/04/22 06:42:44 [go-init] No pre-start command defined, skip
2022/04/22 06:42:44 [go-init] Main command launched : /usr/libexec/s2i/run
CONTAINER_MEMORY_IN_MB='1024', using /usr/lib/jvm/java-11-openjdk-11.0.15.0.9-2.el8_4.x86_64/bin/java and /usr/lib/jvm/java-11-openjdk-11.0.15.0.9-2.el8_4.x86_64/bin/javac
Linking /usr/lib/jenkins/ace-editor.hpi RPM installed Jenkins plugins to /var/lib/jenkins ...
Linking /usr/lib/jenkins/ant.hpi RPM installed Jenkins plugins to /var/lib/jenkins ...
Linking /usr/lib/jenkins/apache-httpcomponents-client-4-api.hpi RPM installed Jenkins plugins to /var/lib/jenkins ...
Linking /usr/lib/jenkins/authentication-tokens.hpi RPM installed Jenkins plugins to /var/lib/jenkins ...
Linking /usr/lib/jenkins/blueocean-autofavorite.hpi RPM installed Jenkins plugins to /var/lib/jenkins ...
Linking /usr/lib/jenkins/blueocean-bitbucket-pipeline.hpi RPM installed Jenkins plugins to /var/lib/jenkins ...
Linking /usr/lib/jenkins/blueocean-commons.hpi RPM installed Jenkins plugins to /var/lib/jenkins ...
Linking /usr/lib/jenkins/blueocean-config.hpi RPM installed Jenkins plugins to /var/lib/jenkins ...
Linking /usr/lib/jenkins/blueocean-core-js.hpi RPM installed Jenkins plugins to /var/lib/jenkins ...
Linking /usr/lib/jenkins/blueocean-dashboard.hpi RPM installed Jenkins plugins to /var/lib/jenkins ...
Linking /usr/lib/jenkins/blueocean-display-url.hpi RPM installed Jenkins plugins to /var/lib/jenkins ...
Linking /usr/lib/jenkins/blueocean-events.hpi RPM installed Jenkins plugins to /var/lib/jenkins ...
Linking /usr/lib/jenkins/blueocean-github-pipeline.hpi RPM installed Jenkins plugins to /var/lib/jenkins ...
Linking /usr/lib/jenkins/blueocean-git-pipeline.hpi RPM installed Jenkins plugins to /var/lib/jenkins ...
Linking /usr/lib/jenkins/blueocean.hpi RPM installed Jenkins plugins to /var/lib/jenkins ...
Linking /usr/lib/jenkins/blueocean-i18n.hpi RPM installed Jenkins plugins to /var/lib/jenkins ...
Linking /usr/lib/jenkins/blueocean-jwt.hpi RPM installed Jenkins plugins to /var/lib/jenkins ...
Linking /usr/lib/jenkins/blueocean-personalization.hpi RPM installed Jenkins plugins to /var/lib/jenkins ...
.
.
.
.+ exec java -XX:+UseParallelGC -XX:MinHeapFreeRatio=5 -XX:MaxHeapFreeRatio=10 -XX:GCTimeRatio=4 -XX:AdaptiveSizePolicyWeight=90 -Xmx512m -Dfile.encoding=UTF8 -Djavamelody.displayed-counters=log,error -Djava.util.logging.config.file=/var/lib/jenkins/logging.properties -Djavax.net.ssl.trustStore=/var/lib/jenkins/ca-anchors-keystore -Dcom.redhat.fips=false -Djdk.http.auth.tunneling.disabledSchemes= -Djdk.http.auth.proxying.disabledSchemes= -Duser.home=/var/lib/jenkins -Djavamelody.application-name=jenkins -Dhudson.security.csrf.GlobalCrumbIssuerConfiguration.DISABLE_CSRF_PROTECTION=true -Djenkins.install.runSetupWizard=false -jar /usr/lib/jenkins/jenkins.war
Picked up JAVA_TOOL_OPTIONS: -XX:+UnlockExperimentalVMOptions -Dsun.zip.disableMemoryMapping=true

========================
checked jenkins master pod log and found "-Dcom.redhat.fips=false"  in the below section which is desired behavior as per the PR

Comment 6 errata-xmlrpc 2022-05-02 18:38:50 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.10.12 bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2022:1601