Description of problem: Version-Release number of the following components: ansible-2.4.2.0-2.el7.noarch openshift-ansible-3.4.164-1.git.0.82c4c34.el7.noarch # openshift version openshift v3.4.1.44.49 kubernetes v1.4.0+776c994 etcd 3.1.0-rc.0 How reproducible: Always Steps to Reproduce: 1. trigger a rpm install (1 master + 1 node + 1nfs) 2. 3. Actual results: installation is completed successfully, but docker-registry is not deployed successfully. # oc get po NAME READY STATUS RESTARTS AGE docker-registry-3-1f5af 0/1 CrashLoopBackOff 1 14s docker-registry-3-deploy 1/1 Running 0 16s # oc logs docker-registry-3-1f5af /bin/sh: /dockerregistry: No such file or directory # oc get dc docker-registry -o yaml apiVersion: v1 kind: DeploymentConfig metadata: creationTimestamp: 2018-01-25T06:43:00Z generation: 9 labels: docker-registry: default name: docker-registry namespace: default resourceVersion: "6724" selfLink: /oapi/v1/namespaces/default/deploymentconfigs/docker-registry uid: fc56c9e9-019a-11e8-bd01-fa163ea5f7ee spec: replicas: 1 selector: docker-registry: default strategy: resources: {} rollingParams: intervalSeconds: 1 maxSurge: 25% maxUnavailable: 25% timeoutSeconds: 600 updatePeriodSeconds: 1 type: Rolling template: metadata: creationTimestamp: null labels: docker-registry: default spec: containers: - env: - name: REGISTRY_HTTP_ADDR value: :5000 - name: REGISTRY_HTTP_NET value: tcp - name: REGISTRY_HTTP_SECRET value: YmsoeuuYi+qqaxk6Le2v1uCRk2VTQ7QaXrKPG9wqev8= - name: REGISTRY_MIDDLEWARE_REPOSITORY_OPENSHIFT_ENFORCEQUOTA value: "false" - name: REGISTRY_HTTP_TLS_CERTIFICATE value: /etc/secrets/registry.crt - name: REGISTRY_HTTP_TLS_KEY value: /etc/secrets/registry.key image: brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/openshift3/ose-docker-registry:v3.4.1.44.49 imagePullPolicy: IfNotPresent livenessProbe: failureThreshold: 3 httpGet: path: /healthz port: 5000 scheme: HTTPS initialDelaySeconds: 10 periodSeconds: 10 successThreshold: 1 timeoutSeconds: 5 name: registry ports: - containerPort: 5000 protocol: TCP readinessProbe: failureThreshold: 3 httpGet: path: /healthz port: 5000 scheme: HTTPS periodSeconds: 10 successThreshold: 1 timeoutSeconds: 5 resources: requests: cpu: 100m memory: 256Mi securityContext: privileged: false terminationMessagePath: /dev/termination-log volumeMounts: - mountPath: /registry name: registry-storage - mountPath: /etc/secrets name: volume-j4d94 dnsPolicy: ClusterFirst nodeSelector: registry: enabled role: node restartPolicy: Always securityContext: {} serviceAccount: registry serviceAccountName: registry terminationGracePeriodSeconds: 30 volumes: - name: registry-storage persistentVolumeClaim: claimName: regpv-claim - name: volume-j4d94 secret: defaultMode: 420 secretName: registry-certificates test: false triggers: - type: ConfigChange status: conditions: - lastTransitionTime: 2018-01-25T06:43:00Z message: Deployment config does not have minimum availability. status: "False" type: Available - lastTransitionTime: 2018-01-25T10:22:29Z message: Replication controller "docker-registry-3" has failed progressing reason: ProgressDeadlineExceeded status: "False" type: Progressing details: causes: - type: Manual message: manual change latestVersion: 3 observedGeneration: 9 Expected results: docker-registry is deployed successfully. Additional info: Please attach logs from ansible-playbook with the -vvv flag
Created attachment 1385999 [details] installation log with inventory embeded
I suspect at this point v3.4.1.44.39 is a bad build, reaching out to CI/CD team.
The Cmd for the image is bad, need to fix the image. .49 uses /dockerregistry but .38 uses /usr/bin/dockerregistry docker inspect registry.reg-aws.openshift.com/openshift3/ose-docker-registry:v3.4.1.44.49 "Cmd": [ "/bin/sh", "-c", "DOCKER_REGISTRY_URL=${DOCKER_REGISTRY_SERVICE_HOST}:${DOCKER_REGISTRY_SERVICE_PORT} /dockerregistry ${REGISTRY_CONFIGURATION_PATH}" ], docker inspect registry.reg-aws.openshift.com/openshift3/ose-docker-registry:v3.4.1.44.38 "Cmd": [ "/bin/sh", "-c", "DOCKER_REGISTRY_URL=${DOCKER_REGISTRY_SERVICE_HOST}:${DOCKER_REGISTRY_SERVICE_PORT} /usr/bin/dockerregistry ${REGISTRY_CONFIGURATION_PATH}" ],
A new build to address this has just been completed. Details below: OpenShift Version: v3.4.1.44.50-1 Puddle: http://download-node-02.eng.bos.redhat.com/rcm-guest/puddles/RHAOS/AtomicOpenShift/3.4/v3.4.1.44.50_2018-01-25.2 - Mirror: https://mirror.openshift.com/enterprise/enterprise-3.4/v3.4.1.44.50_2018-01-25.2 - Images have been built for this puddle - Images have been pushed to registry.reg-aws.openshift.com:443 (Get pull acceess [1]) [1] https://github.com/openshift/ops-sop/blob/master/services/opsregistry.asciidoc#using-the-registry-manually-using-rh-sso-user Brew: - Openshift: https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=15104747 - OpenShift Ansible: https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=15104889 Jenkins job: https://buildvm.openshift.eng.bos.redhat.com:8443/job/aos-cd-builds/job/build%252Fose/160/ ===Atomic OpenShift changelog snippet=== * Thu Jan 25 2018 Jenkins CD Merge Bot <smunilla> 3.4.1.44.50 - Better reconcile with original dist-git Dockerfiles (adammhaile) * Wed Jan 24 2018 Jenkins CD Merge Bot <smunilla> 3.4.1.44.49 - * Fri Jan 05 2018 Jenkins CD Merge Bot <smunilla> 3.4.1.44.48 - * Wed Jan 03 2018 Jenkins CD Merge Bot <smunilla> 3.4.1.44.47 - * Thu Dec 28 2017 Jenkins CD Merge Bot <smunilla> 3.4.1.44.46 - * Tue Dec 26 2017 Jenkins CD Merge Bot <smunilla> 3.4.1.44.45 - * Thu Dec 21 2017 Jenkins CD Merge Bot <smunilla> 3.4.1.44.44 - * Wed Dec 20 2017 Jenkins CD Merge Bot <smunilla> 3.4.1.44.43 - * Fri Dec 15 2017 Jenkins CD Merge Bot <smunilla> 3.4.1.44.42 - * Wed Dec 13 2017 Jenkins CD Merge Bot <smunilla> 3.4.1.44.41 - * Fri Dec 08 2017 Jenkins CD Merge Bot <smunilla> 3.4.1.44.40 - * Wed Dec 06 2017 Jenkins CD Merge Bot <smunilla> 3.4.1.44.39 - * Fri Nov 10 2017 Jenkins CD Merge Bot <smunilla> 3.4.1.44.38 - * Thu Nov 09 2017 Jenkins CD Merge Bot <smunilla> 3.4.1.44.37 - * Tue Nov 07 2017 Jenkins CD Merge Bot <smunilla> 3.4.1.44.36 - Test for bug 1487408 (obulatov) - Pruning should take all the images into account (agladkov) - UPSTREAM: 43375: Set permission for volume subPaths (ppospisi) * Thu Nov 02 2017 Jenkins CD Merge Bot <smunilla> 3.4.1.44.35 - * Tue Oct 31 2017 Jenkins CD Merge Bot <smunilla> 3.4.1.44.34 - * Thu Oct 26 2017 Jenkins CD Merge Bot <smunilla> 3.4.1.44.33 - * Tue Oct 24 2017 Jenkins CD Merge Bot <smunilla> 3.4.1.44.32 - * Thu Oct 19 2017 Jenkins CD Merge Bot <smunilla> 3.4.1.44.31 - UPSTREAM: 48613: proxy/userspace: honor listen IP address as host IP if given (jtanenba) - Print more details when network diagnostics test setup fails (rpenta) - Bug 1481147 - Fix default pod image for network diagnostics (rpenta) * Tue Oct 17 2017 Jenkins CD Merge Bot <smunilla> 3.4.1.44.30 - * Fri Oct 13 2017 Jenkins CD Merge Bot <smunilla> 3.4.1.44.29 - * Thu Oct 12 2017 Jenkins CD Merge Bot <smunilla> 3.4.1.44.28 - Retry image stream updates when pruning images (maszulik) * Tue Oct 10 2017 Jenkins CD Merge Bot <smunilla> 3.4.1.44.27 - UPSTREAM: 53135: Fixed counting of unbound PVCs towards limit of attached volumes. (jsafrane) * Fri Oct 06 2017 Jenkins CD Merge Bot <smunilla> 3.4.1.44.26 - * Thu Oct 05 2017 Jenkins CD Merge Bot <smunilla> 3.4.1.44.25 - UPSTREAM: 48613: proxy/userspace: honor listen IP address as host IP if given (dcbw) * Tue Oct 03 2017 Jenkins CD Merge Bot <smunilla> 3.4.1.44.24 - * Thu Sep 28 2017 Jenkins CD Merge Bot <smunilla> 3.4.1.44.23 - * Tue Sep 26 2017 Jenkins CD Merge Bot <smunilla> 3.4.1.44.22 - UPSTREAM: google/cadvisor: 1700: Reduce log spam when unable to get network stats (sjenning) * Tue Sep 26 2017 Jenkins CD Merge Bot <smunilla> 3.4.1.44.21 - UPSTREAM: 49640: Run mount in its own systemd scope (hchen) ...truncated... ===OpenShift Ansible changelog snippet=== * Thu Jan 25 2018 Jenkins CD Merge Bot <smunilla> 3.4.165-1 - * Wed Jan 24 2018 Jenkins CD Merge Bot <smunilla> 3.4.164-1 - * Fri Jan 05 2018 Jenkins CD Merge Bot <smunilla> 3.4.163-1 - * Wed Jan 03 2018 Jenkins CD Merge Bot <smunilla> 3.4.162-1 - * Thu Dec 28 2017 Jenkins CD Merge Bot <smunilla> 3.4.161-1 - * Tue Dec 26 2017 Jenkins CD Merge Bot <smunilla> 3.4.160-1 - * Thu Dec 21 2017 Jenkins CD Merge Bot <smunilla> 3.4.159-1 - * Wed Dec 20 2017 Jenkins CD Merge Bot <smunilla> 3.4.158-1 - * Fri Dec 15 2017 Jenkins CD Merge Bot <smunilla> 3.4.157-1 - * Wed Dec 13 2017 Jenkins CD Merge Bot <smunilla> 3.4.156-1 - * Fri Dec 08 2017 Jenkins CD Merge Bot <smunilla> 3.4.155-1 - * Wed Dec 06 2017 Jenkins CD Merge Bot <smunilla> 3.4.154-1 - Place-holder for prerequisites.yml (mgugino) * Fri Nov 10 2017 Jenkins CD Merge Bot <smunilla> 3.4.153-1 - * Thu Nov 09 2017 Jenkins CD Merge Bot <smunilla> 3.4.152-1 - * Tue Nov 07 2017 Jenkins CD Merge Bot <smunilla> 3.4.151-1 - * Thu Nov 02 2017 Jenkins CD Merge Bot <smunilla> 3.4.150-1 - Remove master_count restriction. (abutcher) * Tue Oct 31 2017 Jenkins CD Merge Bot <smunilla> 3.4.149-1 - Make openshift-ansible depend on all subpackages (sdodson) * Thu Oct 26 2017 Jenkins CD Merge Bot <smunilla> 3.4.148-1 - * Tue Oct 24 2017 Jenkins CD Merge Bot <smunilla> 3.4.147-1 - Fix undefined variable for master upgrades (mgugino) * Thu Oct 19 2017 Jenkins CD Merge Bot <smunilla> 3.4.146-1 - * Tue Oct 17 2017 Jenkins CD Merge Bot <smunilla> 3.4.145-1 - Ensure controllerConfig.serviceServingCert is correctly set during upgrade. (abutcher) * Fri Oct 13 2017 Jenkins CD Merge Bot <smunilla> 3.4.144-1 - Bump version to 3.4.143-1 (smunilla) - Ensure docker service status actually changes (mgugino) - Add PartOf to docker systemd service unit. (mgugino) - Ensure docker is restarted when iptables is restarted (mgugino) * Fri Oct 06 2017 Jenkins CD Merge Bot <smunilla> 3.4.141-1 - Bumping version to avoid conflict (smunilla) - Ensure docker service started prior to credentials (mgugino) - Fix origin repo deployment (mgugino) - Fix registry auth variable (mgugino) - Fixes #4809 (edu) - master: Default to false for l_bind_docker_reg_auth (smilner) * Tue Sep 26 2017 Jenkins CD Merge Bot <smunilla> 3.4.137-1 - * Tue Sep 26 2017 Jenkins CD Merge Bot <smunilla> 3.4.136-1 - Fix registry_auth logic for upgrades (mgugino) - Remove default value for oreg_url (mgugino) - Fix registry auth task ordering (mgugino) - Update openshift-ansible-centos-paas-sig.repo (31991104+Blizzkid.github.com) * Fri Sep 15 2017 Scott Dodson <sdodson> 3.4.135-1 - Fix: authenticated registry support for containerized hosts (mgugino) - Remove Jinja2 templating delimiter (rteague) - Backport updated ansible_syntax checks (rteague) - Add independent registry auth support (mgugino) * Tue Sep 05 2017 Jenkins CD Merge Bot <smunilla> 3.4.134-1 - * Tue Aug 29 2017 Jenkins CD Merge Bot <smunilla> 3.4.133-1 - Backport recent tox test updates (rteague)
Verified this issue with openshift v3.4.1.44.50, and PASS.
Closed per comment 5
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2018:1134