Bug 1808312 - template-service-broker can not be installed with FIPS on
Summary: template-service-broker can not be installed with FIPS on
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Service Broker
Version: 4.4
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ---
: 4.4.0
Assignee: Fabian von Feilitzsch
QA Contact: Cuiping HUO
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-02-28 09:09 UTC by Cuiping HUO
Modified: 2020-05-04 11:43 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-05-04 11:43:15 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2020:0581 0 None None None 2020-05-04 11:43:33 UTC

Description Cuiping HUO 2020-02-28 09:09:40 UTC
Description of problem:
template-service-broker can not be installed with FIPS on

Version-Release number of selected component (if applicable):
4.4.0-0.nightly-2020-02-27-020932
tsb csv:openshifttemplateservicebrokeroperator.4.4.0-202002272346


How reproducible:
Always

Steps to Reproduce:
1.spin up a cluster with FIPs on 
2.install tsb operator
3.install tsb

Actual results:
template-service-broker operator install sucessfully, but template-service-broker failed with error 
message: 'An unhandled exception occurred while running the lookup plugin ''k8s''.
      Error was a <type ''exceptions.ValueError''>, original message: error:060800A3:digital
      envelope routines:EVP_DigestInit_ex:disabled for fips'

Expected results:
template-service-broker operator and template-service-broker should all be installed sucessfully


Additional info:
This issue is much the same like: https://bugzilla.redhat.com/show_bug.cgi?id=1779101 in 4.3.z

$ oc get csv
NAME                                                        DISPLAY                                      VERSION              REPLACES                                                    PHASE
openshifttemplateservicebrokeroperator.4.4.0-202002272346   OpenShift Template Service Broker Operator   4.4.0-202002272346   openshifttemplateservicebrokeroperator.4.4.0-202002270901   Succeeded

$ oc get templateservicebroker template-service-broker -o yaml
apiVersion: osb.openshift.io/v1
kind: TemplateServiceBroker
metadata:
  creationTimestamp: "2020-02-28T07:36:14Z"
  finalizers:
  - finalizer.osb.openshift.io
  generation: 1
  name: template-service-broker
  namespace: openshift-template-service-broker
  resourceVersion: "571470"
  selfLink: /apis/osb.openshift.io/v1/namespaces/openshift-template-service-broker/templateservicebrokers/template-service-broker
  uid: b003f593-7cca-44b6-b50d-1f62cd0197e6
spec: {}
status:
  conditions:
  - lastTransitionTime: "2020-02-28T07:36:15Z"
    message: Running reconciliation
    reason: Running
    status: "False"
    type: Running
  - ansibleResult:
      changed: 0
      completion: 2020-02-28T07:36:17.953242
      failures: 1
      ok: 2
      skipped: 0
    lastTransitionTime: "2020-02-28T07:36:18Z"
    message: 'An unhandled exception occurred while running the lookup plugin ''k8s''.
      Error was a <type ''exceptions.ValueError''>, original message: error:060800A3:digital
      envelope routines:EVP_DigestInit_ex:disabled for fips'
    reason: Failed
    status: "True"

$ oc logs -f openshift-template-service-broker-operator-7d5c5bd89f-nfvlw
{"level":"info","ts":1582874856.5567164,"logger":"cmd","msg":"Go Version: go1.13.4"}
{"level":"info","ts":1582874856.5567417,"logger":"cmd","msg":"Go OS/Arch: linux/amd64"}
{"level":"info","ts":1582874856.556747,"logger":"cmd","msg":"Version of operator-sdk: v0.12.0+git"}
{"level":"info","ts":1582874856.556765,"logger":"cmd","msg":"Watching namespace.","Namespace":"openshift-template-service-broker"}
{"level":"info","ts":1582874858.8183043,"logger":"controller-runtime.metrics","msg":"metrics server is starting to listen","addr":"0.0.0.0:8383"}
{"level":"info","ts":1582874858.8190265,"logger":"watches","msg":"Failed to parse %v from environment. Using default %v","WORKER_TEMPLATESERVICEBROKER_OSB_OPENSHIFT_IO":1}
{"level":"info","ts":1582874858.8190532,"logger":"watches","msg":"Failed to parse %v from environment. Using default %v","ANSIBLE_VERBOSITY_TEMPLATESERVICEBROKER_OSB_OPENSHIFT_IO":2}
{"level":"info","ts":1582874858.819094,"logger":"ansible-controller","msg":"Watching resource","Options.Group":"osb.openshift.io","Options.Version":"v1","Options.Kind":"TemplateServiceBroker"}
{"level":"info","ts":1582874858.8196156,"logger":"controller-runtime.controller","msg":"Starting EventSource","controller":"templateservicebroker-controller","source":"kind source: osb.openshift.io/v1, Kind=TemplateServiceBroker"}
{"level":"info","ts":1582874858.8198798,"logger":"leader","msg":"Trying to become the leader."}
{"level":"info","ts":1582874861.0997918,"logger":"leader","msg":"No pre-existing lock was found."}
{"level":"info","ts":1582874861.1245904,"logger":"leader","msg":"Became the leader."}
{"level":"info","ts":1582874865.7081823,"logger":"metrics","msg":"Metrics Service object created","Service.Name":"openshift-template-service-broker-operator-metrics","Service.Namespace":"openshift-template-service-broker"}
{"level":"info","ts":1582874865.7091684,"logger":"proxy","msg":"Starting to serve","Address":"127.0.0.1:8888"}
{"level":"info","ts":1582874865.7092674,"logger":"controller-runtime.manager","msg":"starting metrics server","path":"/metrics"}
{"level":"info","ts":1582874865.809408,"logger":"controller-runtime.controller","msg":"Starting Controller","controller":"templateservicebroker-controller"}
{"level":"info","ts":1582874865.909596,"logger":"controller-runtime.controller","msg":"Starting workers","controller":"templateservicebroker-controller","worker count":1}
{"level":"info","ts":1582875377.4792526,"logger":"logging_event_handler","msg":"[playbook task]","name":"template-service-broker","namespace":"openshift-template-service-broker","gvk":"osb.openshift.io/v1, Kind=TemplateServiceBroker","event_type":"playbook_on_task_start","job":"4037200794235010051","EventData.Name":"Validation"}
{"level":"info","ts":1582875377.665456,"logger":"logging_event_handler","msg":"[playbook task]","name":"template-service-broker","namespace":"openshift-template-service-broker","gvk":"osb.openshift.io/v1, Kind=TemplateServiceBroker","event_type":"playbook_on_task_start","job":"4037200794235010051","EventData.Name":"Run template-service-broker role"}
{"level":"error","ts":1582875377.9539332,"logger":"logging_event_handler","msg":"","name":"template-service-broker","namespace":"openshift-template-service-broker","gvk":"osb.openshift.io/v1, Kind=TemplateServiceBroker","event_type":"runner_on_failed","job":"4037200794235010051","EventData.Task":"Get available cluster APIs","EventData.TaskArgs":"","EventData.FailedTaskPath":"/opt/ansible/roles/template-service-broker/tasks/main.yml:7","error":"[playbook task failed]","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\tsrc/github.com/operator-framework/operator-sdk/vendor/github.com/go-logr/zapr/zapr.go:128\ngithub.com/operator-framework/operator-sdk/pkg/ansible/events.loggingEventHandler.Handle\n\tsrc/github.com/operator-framework/operator-sdk/pkg/ansible/events/log_events.go:84"}
{"level":"error","ts":1582875378.1451657,"logger":"runner","msg":"\u001b[0;34mansible-playbook 2.8.8\u001b[0m\r\n\u001b[0;34m  config file = /etc/ansible/ansible.cfg\u001b[0m\r\n\u001b[0;34m  configured module search path = [u'/usr/share/ansible/openshift']\u001b[0m\r\n\u001b[0;34m  ansible python module location = /usr/lib/python2.7/site-packages/ansible\u001b[0m\r\n\u001b[0;34m  executable location = /usr/bin/ansible-playbook\u001b[0m\r\n\u001b[0;34m  python version = 2.7.5 (default, Jun 11 2019, 14:33:56) [GCC 4.8.5 20150623 (Red Hat 4.8.5-39)]\u001b[0m\r\n\u001b[0;34mUsing /etc/ansible/ansible.cfg as config file\u001b[0m\r\n\r\nPLAYBOOK: playbook.yaml ********************************************************\n\u001b[0;34m1 plays in /opt/ansible/playbook.yaml\u001b[0m\n\r\nPLAY [template-service-broker-operator] ****************************************\n\u001b[0;34mMETA: ran handlers\u001b[0m\n\r\nTASK [Validation] **************************************************************\r\n\u001b[1;30mtask path: /opt/ansible/playbook.yaml:13\u001b[0m\n\u001b[0;32mok: [localhost] => {\u001b[0m\r\n\u001b[0;32m    \"changed\": false, \u001b[0m\r\n\u001b[0;32m    \"msg\": \"All assertions passed\"\u001b[0m\r\n\u001b[0;32m}\u001b[0m\n\r\nTASK [Run template-service-broker role] ****************************************\r\n\u001b[1;30mtask path: /opt/ansible/playbook.yaml:20\u001b[0m\n\r\nTASK [template-service-broker : Set apiserver_config if not already defined] ***\r\n\u001b[1;30mtask path: /opt/ansible/roles/template-service-broker/tasks/main.yml:2\u001b[0m\n\u001b[0;32mok: [localhost] => {\"ansible_facts\": {\"apiserver_config\": \"\\\"kind: TemplateServiceBrokerConfig\\\\napiVersion: config.templateservicebroker.openshift.io/v1\\\\ntemplateNamespaces:\\\\n  - openshift\\\\n\\\"\"}, \"changed\": false}\u001b[0m\n\r\nTASK [template-service-broker : Get available cluster APIs] ********************\r\n\u001b[1;30mtask path: /opt/ansible/roles/template-service-broker/tasks/main.yml:7\u001b[0m\n\u001b[0;31mfatal: [localhost]: FAILED! => {\"msg\": \"An unhandled exception occurred while running the lookup plugin 'k8s'. Error was a <type 'exceptions.ValueError'>, original message: error:060800A3:digital envelope routines:EVP_DigestInit_ex:disabled for fips\"}\u001b[0m\n\r\nPLAY RECAP *********************************************************************\r\n\u001b[0;31mlocalhost\u001b[0m                  : \u001b[0;32mok=2   \u001b[0m changed=0    unreachable=0    \u001b[0;31mfailed=1   \u001b[0m skipped=0    rescued=0    ignored=0   \r\n\n","job":"4037200794235010051","name":"template-service-broker","namespace":"openshift-template-service-broker","error":"exit status 2","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\tsrc/github.com/operator-framework/operator-sdk/vendor/github.com/go-logr/zapr/zapr.go:128\ngithub.com/operator-framework/operator-sdk/pkg/ansible/runner.(*runner).Run.func1\n\tsrc/github.com/operator-framework/operator-sdk/pkg/ansible/runner/runner.go:199"}

Comment 2 Jesus M. Rodriguez 2020-02-28 21:02:00 UTC
I suspect we need the fix from this BUG https://bugzilla.redhat.com/show_bug.cgi?id=1779101

Comment 3 Jesus M. Rodriguez 2020-02-28 21:39:16 UTC
Tagged python-openshift-0.8.11-1.el7 to 4.4 repo.

---
$ brew tag rhaos-4.4-rhel-7-candidate python-openshift-0.8.11-1.el7
Created task 26874030
Watching tasks (this may be safely interrupted)...
26874030 tagBuild (noarch): open (x86-034.build.eng.bos.redhat.com)
26874030 tagBuild (noarch): open (x86-034.build.eng.bos.redhat.com) -> closed
  0 free  0 open  1 done  0 failed

26874030 tagBuild (noarch) completed successfully
---

Comment 6 Jesus M. Rodriguez 2020-03-02 14:39:06 UTC
Image has the correct package now:

$ docker run -it --entrypoint=/bin/bash registry-proxy.engineering.redhat.com/rh-osbs/openshift-ose-ansible-operator:v4.4.0-202003020732
Unable to find image 'registry-proxy.engineering.redhat.com/rh-osbs/openshift-ose-ansible-operator:v4.4.0-202003020732' locally
v4.4.0-202003020732: Pulling from rh-osbs/openshift-ose-ansible-operator
bb13d92caffa: Already exists 
455ea8ab0621: Already exists 
935ce2f796a9: Already exists 
e656c348f409: Pull complete 
7a6bafee777b: Pull complete 
Digest: sha256:973d152c2d76dba69fdbac42e5031b685d5e56d5bd7679e3ee1ffafd05e46d62
Status: Downloaded newer image for registry-proxy.engineering.redhat.com/rh-osbs/openshift-ose-ansible-operator:v4.4.0-202003020732
bash-4.2$ rpm -q python2-openshift
python2-openshift-0.8.11-1.el7.noarch

Comment 9 Cuiping HUO 2020-03-06 11:04:09 UTC
Verified.

Cluster version: 4.4.0-0.nightly-2020-03-05-205718
tsb version:4.4.0-202003021217


$ oc get mc | grep fip
99-master-fips                                                                                         2.2.0             7h57m
99-worker-fips                                                                                         2.2.0             7h57m

$ oc get csv
NAME                                                        DISPLAY                                      VERSION              REPLACES   PHASE
openshifttemplateservicebrokeroperator.4.4.0-202003021217   OpenShift Template Service Broker Operator   4.4.0-202003021217              Succeeded

$ oc get po
NAME                                                         READY   STATUS      RESTARTS   AGE
apiserver-1-deploy                                           0/1     Completed   0          62s
apiserver-1-rc6l8                                            1/1     Running     0          58s
openshift-template-service-broker-operator-fb56f46cd-vrrrs   1/1     Running     0          2m28s
[chuo@localhost .kube]$ oc get po
NAME                                                         READY   STATUS      RESTARTS   AGE
apiserver-1-deploy                                           0/1     Completed   0          105s
apiserver-1-rc6l8                                            1/1     Running     0          101s
openshift-template-service-broker-operator-fb56f46cd-vrrrs   1/1     Running     0          3m11s

$ oc get templateservicebroker template-service-broker -o yaml
apiVersion: osb.openshift.io/v1
kind: TemplateServiceBroker
metadata:
  creationTimestamp: "2020-03-06T10:50:08Z"
  finalizers:
  - finalizer.osb.openshift.io
  generation: 1
  name: template-service-broker
  namespace: openshift-template-service-broker
  resourceVersion: "165687"
  selfLink: /apis/osb.openshift.io/v1/namespaces/openshift-template-service-broker/templateservicebrokers/template-service-broker
  uid: 28276e3f-785d-4d05-bc8d-3f6a76b2deb1
spec: {}
status:
  conditions:
  - ansibleResult:
      changed: 0
      completion: 2020-03-06T10:59:02.687137
      failures: 0
      ok: 6
      skipped: 0
    lastTransitionTime: "2020-03-06T10:50:08Z"
    message: Awaiting next reconciliation
    reason: Successful
    status: "True"
    type: Running


$ oc logs -f openshift-template-service-broker-operator-fb56f46cd-vrrrs
{"level":"info","ts":1583491746.595241,"logger":"cmd","msg":"Go Version: go1.13.4"}
{"level":"info","ts":1583491746.5952742,"logger":"cmd","msg":"Go OS/Arch: linux/amd64"}
{"level":"info","ts":1583491746.5952828,"logger":"cmd","msg":"Version of operator-sdk: v0.12.0+git"}
{"level":"info","ts":1583491746.5953023,"logger":"cmd","msg":"Watching namespace.","Namespace":"openshift-template-service-broker"}
{"level":"info","ts":1583491748.8039806,"logger":"controller-runtime.metrics","msg":"metrics server is starting to listen","addr":"0.0.0.0:8383"}
{"level":"info","ts":1583491748.8049648,"logger":"watches","msg":"Failed to parse %v from environment. Using default %v","WORKER_TEMPLATESERVICEBROKER_OSB_OPENSHIFT_IO":1}
{"level":"info","ts":1583491748.8049927,"logger":"watches","msg":"Failed to parse %v from environment. Using default %v","ANSIBLE_VERBOSITY_TEMPLATESERVICEBROKER_OSB_OPENSHIFT_IO":2}
{"level":"info","ts":1583491748.805142,"logger":"ansible-controller","msg":"Watching resource","Options.Group":"osb.openshift.io","Options.Version":"v1","Options.Kind":"TemplateServiceBroker"}
{"level":"info","ts":1583491748.8063536,"logger":"controller-runtime.controller","msg":"Starting EventSource","controller":"templateservicebroker-controller","source":"kind source: osb.openshift.io/v1, Kind=TemplateServiceBroker"}
{"level":"info","ts":1583491748.8065119,"logger":"leader","msg":"Trying to become the leader."}
{"level":"info","ts":1583491751.0248592,"logger":"leader","msg":"No pre-existing lock was found."}
{"level":"info","ts":1583491751.030805,"logger":"leader","msg":"Became the leader."}
{"level":"info","ts":1583491755.4771461,"logger":"metrics","msg":"Metrics Service object created","Service.Name":"openshift-template-service-broker-operator-metrics","Service.Namespace":"openshift-template-service-broker"}
{"level":"info","ts":1583491755.4788537,"logger":"proxy","msg":"Starting to serve","Address":"127.0.0.1:8888"}
{"level":"info","ts":1583491755.4792037,"logger":"controller-runtime.manager","msg":"starting metrics server","path":"/metrics"}
{"level":"info","ts":1583491755.6265934,"logger":"controller-runtime.controller","msg":"Starting Controller","controller":"templateservicebroker-controller"}
{"level":"info","ts":1583491755.7272847,"logger":"controller-runtime.controller","msg":"Starting workers","controller":"templateservicebroker-controller","worker count":1}
{"level":"info","ts":1583491810.160501,"logger":"logging_event_handler","msg":"[playbook task]","name":"template-service-broker","namespace":"openshift-template-service-broker","gvk":"osb.openshift.io/v1, Kind=TemplateServiceBroker","event_type":"playbook_on_task_start","job":"6129484611666145821","EventData.Name":"Validation"}
{"level":"info","ts":1583491810.2125328,"logger":"logging_event_handler","msg":"[playbook task]","name":"template-service-broker","namespace":"openshift-template-service-broker","gvk":"osb.openshift.io/v1, Kind=TemplateServiceBroker","event_type":"playbook_on_task_start","job":"6129484611666145821","EventData.Name":"Run template-service-broker role"}
{"level":"info","ts":1583491810.6452549,"logger":"logging_event_handler","msg":"[playbook task]","name":"template-service-broker","namespace":"openshift-template-service-broker","gvk":"osb.openshift.io/v1, Kind=TemplateServiceBroker","event_type":"playbook_on_task_start","job":"6129484611666145821","EventData.Name":"template-service-broker : Set tsb objects state=present"}
{"level":"info","ts":1583491814.2237854,"logger":"proxy","msg":"Cache miss: /v1, Kind=ServiceAccount, openshift-template-service-broker/apiserver"}
{"level":"info","ts":1583491814.2310069,"logger":"proxy","msg":"Injecting owner reference"}
{"level":"info","ts":1583491814.2313461,"logger":"proxy","msg":"Watching child resource","kind":"/v1, Kind=ServiceAccount","enqueue_kind":"osb.openshift.io/v1, Kind=TemplateServiceBroker"}
{"level":"info","ts":1583491814.231387,"logger":"controller-runtime.controller","msg":"Starting EventSource","controller":"templateservicebroker-controller","source":"kind source: /v1, Kind=ServiceAccount"}
{"level":"info","ts":1583491815.2407959,"logger":"proxy","msg":"Injecting owner reference"}
{"level":"info","ts":1583491816.2548344,"logger":"proxy","msg":"Injecting owner reference"}
{"level":"info","ts":1583491817.6494408,"logger":"proxy","msg":"Cache miss: apps.openshift.io/v1, Kind=DeploymentConfig, openshift-template-service-broker/apiserver"}
{"level":"info","ts":1583491817.6613183,"logger":"proxy","msg":"Injecting owner reference"}
{"level":"info","ts":1583491817.661743,"logger":"proxy","msg":"Watching child resource","kind":"apps.openshift.io/v1, Kind=DeploymentConfig","enqueue_kind":"osb.openshift.io/v1, Kind=TemplateServiceBroker"}
{"level":"info","ts":1583491817.6617777,"logger":"controller-runtime.controller","msg":"Starting EventSource","controller":"templateservicebroker-controller","source":"kind source: apps.openshift.io/v1, Kind=DeploymentConfig"}
{"level":"info","ts":1583491818.7241127,"logger":"proxy","msg":"Cache miss: /v1, Kind=ConfigMap, openshift-template-service-broker/apiserver-config"}
{"level":"info","ts":1583491818.7352583,"logger":"proxy","msg":"Injecting owner reference"}
{"level":"info","ts":1583491818.7355833,"logger":"proxy","msg":"Watching child resource","kind":"/v1, Kind=ConfigMap","enqueue_kind":"osb.openshift.io/v1, Kind=TemplateServiceBroker"}
{"level":"info","ts":1583491818.7356167,"logger":"controller-runtime.controller","msg":"Starting EventSource","controller":"templateservicebroker-controller","source":"kind source: /v1, Kind=ConfigMap"}
{"level":"info","ts":1583491819.991706,"logger":"proxy","msg":"Cache miss: /v1, Kind=Service, openshift-template-service-broker/apiserver"}

Comment 11 errata-xmlrpc 2020-05-04 11:43:15 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:0581


Note You need to log in before you can comment on or make changes to this bug.