Bug 1948524 - 'DeploymentUpdated' Updated Deployment.apps/downloads -n openshift-console because it changed message is printed every minute
Summary: 'DeploymentUpdated' Updated Deployment.apps/downloads -n openshift-console be...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Management Console
Version: 4.8
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ---
: 4.8.0
Assignee: Jakub Hadvig
QA Contact: Yadan Pei
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-04-12 11:11 UTC by Yadan Pei
Modified: 2021-07-27 22:59 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-07-27 22:59:25 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
two lines are identical (1.21 MB, image/png)
2021-04-19 03:31 UTC, Yadan Pei
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Github openshift console-operator pull 525 0 None open Bug 1948524: Remove ResyncEvery method from the operator sync loop 2021-04-12 13:53:16 UTC
Github openshift console-operator pull 531 0 None open Bug 1948524: Pull route health check into a standalone controller 2021-04-20 13:11:52 UTC
Red Hat Product Errata RHSA-2021:2438 0 None None None 2021-07-27 22:59:46 UTC

Description Yadan Pei 2021-04-12 11:11:34 UTC
Description of problem:
'DeploymentUpdated' Updated Deployment.apps/downloads -n openshift-console because it changed message is printed every minute even though there is no change to deployment/downloads 

Version-Release number of selected component (if applicable):
4.8.0-0.nightly-2021-04-09-222447

How reproducible:
Always

Steps to Reproduce:
1. Check console-operator loglevel, it is 'Normal' by default
$ oc get console.operator cluster -o json | jq .spec
{
  "logLevel": "Normal",
  "managementState": "Managed",
  "operatorLogLevel": "Normal"
}

2. check console-operator logs
$ oc logs -f console-operator-8576455b75-lw6sp
I0412 10:59:42.705936       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-console-operator", Name:"console-operator", UID:"936c6a6b-8506-4150-8a28-fc9524f62709", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'DeploymentUpdated' Updated Deployment.apps/downloads -n openshift-console because it changed
I0412 11:00:42.693097       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-console-operator", Name:"console-operator", UID:"936c6a6b-8506-4150-8a28-fc9524f62709", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'DeploymentUpdated' Updated Deployment.apps/downloads -n openshift-console because it changed
I0412 11:01:42.731774       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-console-operator", Name:"console-operator", UID:"936c6a6b-8506-4150-8a28-fc9524f62709", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'DeploymentUpdated' Updated Deployment.apps/downloads -n openshift-console because it changed
I0412 11:02:42.701367       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-console-operator", Name:"console-operator", UID:"936c6a6b-8506-4150-8a28-fc9524f62709", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'DeploymentUpdated' Updated Deployment.apps/downloads -n openshift-console because it changed
I0412 11:03:42.697722       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-console-operator", Name:"console-operator", UID:"936c6a6b-8506-4150-8a28-fc9524f62709", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'DeploymentUpdated' Updated Deployment.apps/downloads -n openshift-console because it changed
I0412 11:04:42.698476       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-console-operator", Name:"console-operator", UID:"936c6a6b-8506-4150-8a28-fc9524f62709", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'DeploymentUpdated' Updated Deployment.apps/downloads -n openshift-console because it changed
I0412 11:05:42.702502       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-console-operator", Name:"console-operator", UID:"936c6a6b-8506-4150-8a28-fc9524f62709", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'DeploymentUpdated' Updated Deployment.apps/downloads -n openshift-console because it changed

Actual results:
2. we can see the message `type: 'Normal' reason: 'DeploymentUpdated' Updated Deployment.apps/downloads -n openshift-console because it changed` is printed every minute, but deployment/downloads in openshift-console namespace doesn't have any changes

$ oc describe deployment downloads -n openshift-console
Name:                   downloads
Namespace:              openshift-console
CreationTimestamp:      Mon, 12 Apr 2021 18:35:21 +0800
Labels:                 app=console
                        component=downloads
Annotations:            deployment.kubernetes.io/revision: 1
                        operator.openshift.io/spec-hash: 8dc46a469d0f67c0a53335bea7fce1c81eeda4bbc53fcdc52a1cca57e4953396
Selector:               app=console,component=downloads
Replicas:               2 desired | 2 updated | 2 total | 2 available | 0 unavailable
StrategyType:           RollingUpdate
MinReadySeconds:        0
RollingUpdateStrategy:  25% max unavailable, 25% max surge
Pod Template:
  Labels:  app=console
           component=downloads
  Containers:
   download-server:
    Image:      quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:96a0062f501db298247b38f012b6a4a18bfd26e5d7e9284af824d5b3630eef43
    Port:       8080/TCP
    Host Port:  0/TCP
    Command:
      /bin/sh
    Args:
      -c
      cat <<EOF >>/tmp/serve.py
      import errno, http.server, os, re, signal, socket, sys, tarfile, tempfile, threading, time, zipfile
    .........
      cpu:        10m
      memory:     50Mi
    Liveness:     http-get http://:8080/ delay=0s timeout=1s period=10s #success=1 #failure=3
    Readiness:    http-get http://:8080/ delay=0s timeout=1s period=10s #success=1 #failure=3
    Environment:  <none>
    Mounts:       <none>
  Volumes:        <none>
Conditions:
  Type           Status  Reason
  ----           ------  ------
  Available      True    MinimumReplicasAvailable
  Progressing    True    NewReplicaSetAvailable
OldReplicaSets:  <none>
NewReplicaSet:   downloads-7968849f55 (2/2 replicas created)
Events:
  Type    Reason             Age   From                   Message
  ----    ------             ----  ----                   -------
  Normal  ScalingReplicaSet  34m   deployment-controller  Scaled up replica set downloads-7968849f55 to 2


Expected results:
2. didn't see many updates from deployment/downloads event, so maybe we should omit the message if no changes to deployment/downloads

Additional info:

Comment 2 Yadan Pei 2021-04-15 08:17:45 UTC
wait on new build to verify the fix

Comment 3 Yadan Pei 2021-04-19 03:25:16 UTC
1. When operator log level is Normal, we can still see the message 'DeploymentUpdated' Updated Deployment.apps/downloads -n openshift-console because it changed' is printed every minute

I0419 01:01:47.156339       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-console-operator", Name:"console-operator", UID:"3373b1cc-a5e8-4338-ba29-386895d84714", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'DeploymentUpdated' Updated Deployment.apps/downloads -n openshift-console because it changed
I0419 01:02:47.152570       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-console-operator", Name:"console-operator", UID:"3373b1cc-a5e8-4338-ba29-386895d84714", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'DeploymentUpdated' Updated Deployment.apps/downloads -n openshift-console because it changed
I0419 01:03:46.901871       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-console-operator", Name:"console-operator", UID:"3373b1cc-a5e8-4338-ba29-386895d84714", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'DeploymentUpdated' Updated Deployment.apps/downloads -n openshift-console because it changed
I0419 01:03:46.958629       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-console-operator", Name:"console-operator", UID:"3373b1cc-a5e8-4338-ba29-386895d84714", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'DeploymentUpdated' Updated Deployment.apps/downloads -n openshift-console because it changed
I0419 01:03:47.089711       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-console-operator", Name:"console-operator", UID:"3373b1cc-a5e8-4338-ba29-386895d84714", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'DeploymentUpdated' Updated Deployment.apps/downloads -n openshift-console because it changed
I0419 01:03:47.214744       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-console-operator", Name:"console-operator", UID:"3373b1cc-a5e8-4338-ba29-386895d84714", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'DeploymentUpdated' Updated Deployment.apps/downloads -n openshift-console because it changed


2. Set operator log level to Debug to see why Deployment.apps/downloads changed and what have been changed
$ oc patch consoles.operator.openshift.io cluster -p '{"spec":{"logLevel": "Debug"}}' --type merge
$ oc get console.operator cluster -o json | jq .spec
{
  "logLevel": "Normal",
  "managementState": "Managed",
  "operatorLogLevel": "Debug"
}

3. check console-operator logs
.....
I0419 03:04:49.540965       1 operator.go:177] started syncing operator "cluster" (2021-04-19 03:04:49.540958518 +0000 UTC m=+11464.799763016)
I0419 03:04:49.542563       1 controller.go:83] console is in a managed state: syncing service
I0419 03:04:49.542563       1 controller.go:101] console is in a managed state: syncing ConsoleCliDownloads custom resources
I0419 03:04:49.544080       1 controller.go:114] console is in a managed state: syncing route
I0419 03:04:49.552036       1 route.go:151] route "downloads" ingress 'default' found and admitted, host: downloads-openshift-console.apps.qe-groupd-0419.qe.devcluster.openshift.com
I0419 03:04:49.553547       1 apps.go:154] Deployment "openshift-console/downloads" changes: {"spec":{"progressDeadlineSeconds":null,"revisionHistoryLimit":null,"strategy":{"rollingUpdate":null,"type":null},"template":{"spec":{"containers":[{"args":["-c","cat \u003c\u003cEOF \u003e\u003e/tmp/serve.py\nimport errno, http.server, os, re, signal, socket, sys, tarfile, tempfile, threading, time, zipfile\n\nsignal.signal(signal.SIGTERM, lambda signum, frame: sys.exit(0))\n\ndef write_index(path, message):\n\twith open(path, 'wb') as f:\n\t\tf.write('\\n'.join([\n\t\t\t'\u003c!doctype html\u003e',\n\t\t\t'\u003chtml lang=\"en\"\u003e',\n\t\t\t'\u003chead\u003e',\n\t\t\t'  \u003cmeta charset=\"utf-8\"\u003e',\n\t\t\t'\u003c/head\u003e',\n\t\t\t'\u003cbody\u003e',\n\t\t\t'  {}'.format(message),\n\t\t\t'\u003c/body\u003e',\n\t\t\t'\u003c/html\u003e',\n\t\t\t'',\n\t\t]).encode('utf-8'))\n\n# Launch multiple listeners as threads\nclass Thread(threading.Thread):\n\tdef __init__(self, i, socket):\n\t\tthreading.Thread.__init__(self)\n\t\tself.i = i\n\t\tself.socket = socket\n\t\tself.daemon = True\n\t\tself.start()\n\n\tdef run(self):\n\t\thttpd = http.server.HTTPServer(addr, http.server.SimpleHTTPRequestHandler, False)\n\n\t\t# Prevent the HTTP server from re-binding every handler.\n\t\t# https://stackoverflow.com/questions/46210672/\n\t\thttpd.socket = self.socket\n\t\thttpd.server_bind = self.server_close = lambda self: None\n\n\t\thttpd.serve_forever()\n\ntemp_dir = tempfile.mkdtemp()\nprint('serving from {}'.format(temp_dir))\nos.chdir(temp_dir)\nfor arch in ['amd64']:\n\tos.mkdir(arch)\n\tfor operating_system in ['linux', 'mac', 'windows']:\n\t\tos.mkdir(os.path.join(arch, operating_system))\nfor arch in ['arm64', 'ppc64le', 's390x']:\n\tos.mkdir(arch)\n\tfor operating_system in ['linux']:\n\t\tos.mkdir(os.path.join(arch, operating_system))\ncontent = ['\u003ca href=\"oc-license\"\u003elicense\u003c/a\u003e']\nos.symlink('/usr/share/openshift/LICENSE', 'oc-license')\n\nfor arch, operating_system, path in [\n\t\t('amd64', 'linux', '/usr/share/openshift/linux_amd64/oc'),\n\t\t('amd64', 'mac', '/usr/share/openshift/mac/oc'),\n\t\t('amd64', 'windows', '/usr/share/openshift/windows/oc.exe'),\n\t\t('arm64', 'linux', '/usr/share/openshift/linux_arm64/oc'),\n\t\t('ppc64le', 'linux', '/usr/share/openshift/linux_ppc64le/oc'),\n\t\t('s390x', 'linux', '/usr/share/openshift/linux_s390x/oc'),\n\t\t]:\n\tbasename = os.path.basename(path)\n\ttarget_path = os.path.join(arch, operating_system, basename)\n\tos.symlink(path, target_path)\n\tbase_root, _ = os.path.splitext(basename)\n\tarchive_path_root = os.path.join(arch, operating_system, base_root)\n\twith tarfile.open('{}.tar'.format(archive_path_root), 'w') as tar:\n\t\ttar.add(path, basename)\n\twith zipfile.ZipFile('{}.zip'.format(archive_path_root), 'w') as zip:\n\t\tzip.write(path, basename)\n\tcontent.append('\u003ca href=\"{0}\"\u003eoc ({1} {2})\u003c/a\u003e (\u003ca href=\"{0}.tar\"\u003etar\u003c/a\u003e \u003ca href=\"{0}.zip\"\u003ezip\u003c/a\u003e)'.format(target_path, arch, operating_system))\n\nfor root, directories, filenames in os.walk(temp_dir):\n\troot_link = os.path.relpath(temp_dir, os.path.join(root, 'child')).replace(os.path.sep, '/')\n\tfor directory in directories:\n\t\twrite_index(\n\t\t\tpath=os.path.join(root, directory, 'index.html'),\n\t\t\tmessage='\u003cp\u003eDirectory listings are disabled.  See \u003ca href=\"{}\"\u003ehere\u003c/a\u003e for available content.\u003c/p\u003e'.format(root_link),\n\t\t)\n\nwrite_index(\n\tpath=os.path.join(temp_dir, 'index.html'),\n\tmessage='\\n'.join(\n\t\t['\u003cul\u003e'] +\n\t\t['  \u003cli\u003e{}\u003c/li\u003e'.format(entry) for entry in content] +\n\t\t['\u003c/ul\u003e']\n\t),\n)\n\n# Create socket\n# IPv6 should handle IPv4 passively so long as it is not bound to a\n# specific address or set to IPv6_ONLY\n# https://stackoverflow.com/questions/25817848/python-3-does-http-server-support-ipv6\ntry:\n\taddr = ('::', 8080)\n\tsock = socket.socket(socket.AF_INET6, socket.SOCK_STREAM)\nexcept socket.error as err:\n\t# errno.EAFNOSUPPORT is \"socket.error: [Errno 97] Address family not supported by protocol\"\n\t# When IPv6 is disabled, socket will bind using IPv4.\n\tif err.errno == errno.EAFNOSUPPORT:\n\t\taddr = ('', 8080)\n\t\tsock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)\n\telse:\n\t\traise    \nsock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)\nsock.bind(addr)\nsock.listen(5)\n\n[Thread(i, socket=sock) for i in range(100)]\ntime.sleep(9e9)\nEOF\nexec python3 /tmp/serve.py"],"command":["/bin/sh"],"image":"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5186b9d01721c1620d86697661ebf6cf77e2eafeb36a04022ffd998a266e256d","imagePullPolicy":"IfNotPresent","livenessProbe":{"httpGet":{"path":"/","port":8080,"scheme":"HTTP"}},"name":"download-server","ports":[{"containerPort":8080,"name":"http","protocol":"TCP"}],"readinessProbe":{"failureThreshold":3,"httpGet":{"path":"/","port":8080,"scheme":"HTTP"}},"resources":{"requests":{"cpu":"10m","memory":"50Mi"}},"terminationMessagePolicy":"FallbackToLogsOnError"}],"dnsPolicy":null,"restartPolicy":null,"schedulerName":null,"securityContext":null}}}}
I0419 03:04:49.558219       1 operator.go:226] console is in a managed state.
I0419 03:04:49.558236       1 sync_v400.go:49] running sync loop 4.0.0
I0419 03:04:49.559111       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-console-operator", Name:"console-operator", UID:"3373b1cc-a5e8-4338-ba29-386895d84714", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'DeploymentUpdated' Updated Deployment.apps/downloads -n openshift-console because it changed
I0419 03:04:49.560270       1 controller.go:238] oc-cli-downloads consoleclidownloads custom resource exists and is in the correct state
I0419 03:04:49.567101       1 controller.go:238] odo-cli-downloads consoleclidownloads custom resource exists and is in the correct state
I0419 03:04:49.575825       1 route.go:151] route "console" ingress 'default' found and admitted, host: console-openshift-console.apps.qe-groupd-0419.qe.devcluster.openshift.com
I0419 03:04:49.716210       1 request.go:600] Waited for 173.588778ms due to client-side throttling, not priority and fairness, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-console/services/console
I0419 03:04:49.915787       1 request.go:600] Waited for 350.439485ms due to client-side throttling, not priority and fairness, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-config-managed/configmaps/console-config
I0419 03:04:50.115877       1 request.go:600] Waited for 539.983342ms due to client-side throttling, not priority and fairness, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-console/configmaps/trusted-ca-bundle
I0419 03:04:50.316273       1 request.go:600] Waited for 595.89126ms due to client-side throttling, not priority and fairness, request: DELETE:https://172.30.0.1:443/api/v1/namespaces/openshift-console/services/console-redirect
I0419 03:04:50.516321       1 request.go:600] Waited for 596.286631ms due to client-side throttling, not priority and fairness, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-console/configmaps/default-ingress-cert
I0419 03:04:50.716077       1 request.go:600] Waited for 580.503652ms due to client-side throttling, not priority and fairness, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-console/configmaps/default-ingress-cert
I0419 03:04:50.916249       1 request.go:600] Waited for 387.632779ms due to client-side throttling, not priority and fairness, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-config-managed/configmaps/monitoring-shared-config
I0419 03:04:51.115923       1 request.go:600] Waited for 195.14678ms due to client-side throttling, not priority and fairness, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-console/configmaps/console-config
I0419 03:04:51.291187       1 reflector.go:530] k8s.io/apiserver/pkg/server/dynamiccertificates/configmap_cafile_content.go:206: Watch close - *v1.ConfigMap total 0 items received
I0419 03:04:51.316210       1 request.go:600] Waited for 195.914569ms due to client-side throttling, not priority and fairness, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-console/configmaps/service-ca
I0419 03:04:51.320114       1 sync_v400.go:377] service-ca configmap exists and is in the correct state
I0419 03:04:51.515537       1 request.go:600] Waited for 195.291585ms due to client-side throttling, not priority and fairness, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-console/configmaps/trusted-ca-bundle
I0419 03:04:51.530145       1 sync_v400.go:408] trusted-ca-bundle configmap exists and is in the correct state
I0419 03:04:51.530178       1 sync_v400.go:478] no custom logo configured
I0419 03:04:51.716255       1 request.go:600] Waited for 185.98762ms due to client-side throttling, not priority and fairness, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-console/configmaps/default-ingress-cert
I0419 03:04:51.872297       1 reflector.go:530] k8s.io/apiserver/pkg/authentication/request/headerrequest/requestheader_controller.go:172: Watch close - *v1.ConfigMap total 0 items received
I0419 03:04:51.915368       1 request.go:600] Waited for 195.304457ms due to client-side throttling, not priority and fairness, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-console/secrets/console-oauth-config
I0419 03:04:51.919254       1 route.go:151] route "console" ingress 'default' found and admitted, host: console-openshift-console.apps.qe-groupd-0419.qe.devcluster.openshift.com
I0419 03:04:51.944787       1 sync_v400.go:135] -----------------------
I0419 03:04:51.944808       1 sync_v400.go:136] sync loop 4.0.0 resources updated: false
I0419 03:04:51.944820       1 sync_v400.go:137] -----------------------
I0419 03:04:51.944832       1 sync_v400.go:166] sync_v400: updating console status
I0419 03:04:51.944840       1 route.go:151] route "console" ingress 'default' found and admitted, host: console-openshift-console.apps.qe-groupd-0419.qe.devcluster.openshift.com
I0419 03:04:51.944848       1 metrics.go:47] metric console_url host transition: https://console-openshift-console.apps.qe-groupd-0419.qe.devcluster.openshift.com to https://console-openshift-console.apps.qe-groupd-0419.qe.devcluster.openshift.com
I0419 03:04:52.115328       1 request.go:600] Waited for 170.423789ms due to client-side throttling, not priority and fairness, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-config-managed/configmaps/console-public
I0419 03:04:52.119180       1 sync_v400.go:183] sync loop 4.0.0 complete
I0419 03:04:52.119198       1 operator.go:218] finished syncing operator "cluster" (27.293µs)
.......
I0419 03:06:47.223586       1 apps.go:154] Deployment "openshift-console/downloads" changes: {"spec":{"progressDeadlineSeconds":null,"revisionHistoryLimit":null,"strategy":{"rollingUpdate":null,"type":null},"template":{"spec":{"containers":[{"args":["-c","cat \u003c\u003cEOF \u003e\u003e/tmp/serve.py\nimport errno, http.server, os, re, signal, socket, sys, tarfile, tempfile, threading, time, zipfile\n\nsignal.signal(signal.SIGTERM, lambda signum, frame: sys.exit(0))\n\ndef write_index(path, message):\n\twith open(path, 'wb') as f:\n\t\tf.write('\\n'.join([\n\t\t\t'\u003c!doctype html\u003e',\n\t\t\t'\u003chtml lang=\"en\"\u003e',\n\t\t\t'\u003chead\u003e',\n\t\t\t'  \u003cmeta charset=\"utf-8\"\u003e',\n\t\t\t'\u003c/head\u003e',\n\t\t\t'\u003cbody\u003e',\n\t\t\t'  {}'.format(message),\n\t\t\t'\u003c/body\u003e',\n\t\t\t'\u003c/html\u003e',\n\t\t\t'',\n\t\t]).encode('utf-8'))\n\n# Launch multiple listeners as threads\nclass Thread(threading.Thread):\n\tdef __init__(self, i, socket):\n\t\tthreading.Thread.__init__(self)\n\t\tself.i = i\n\t\tself.socket = socket\n\t\tself.daemon = True\n\t\tself.start()\n\n\tdef run(self):\n\t\thttpd = http.server.HTTPServer(addr, http.server.SimpleHTTPRequestHandler, False)\n\n\t\t# Prevent the HTTP server from re-binding every handler.\n\t\t# https://stackoverflow.com/questions/46210672/\n\t\thttpd.socket = self.socket\n\t\thttpd.server_bind = self.server_close = lambda self: None\n\n\t\thttpd.serve_forever()\n\ntemp_dir = tempfile.mkdtemp()\nprint('serving from {}'.format(temp_dir))\nos.chdir(temp_dir)\nfor arch in ['amd64']:\n\tos.mkdir(arch)\n\tfor operating_system in ['linux', 'mac', 'windows']:\n\t\tos.mkdir(os.path.join(arch, operating_system))\nfor arch in ['arm64', 'ppc64le', 's390x']:\n\tos.mkdir(arch)\n\tfor operating_system in ['linux']:\n\t\tos.mkdir(os.path.join(arch, operating_system))\ncontent = ['\u003ca href=\"oc-license\"\u003elicense\u003c/a\u003e']\nos.symlink('/usr/share/openshift/LICENSE', 'oc-license')\n\nfor arch, operating_system, path in [\n\t\t('amd64', 'linux', '/usr/share/openshift/linux_amd64/oc'),\n\t\t('amd64', 'mac', '/usr/share/openshift/mac/oc'),\n\t\t('amd64', 'windows', '/usr/share/openshift/windows/oc.exe'),\n\t\t('arm64', 'linux', '/usr/share/openshift/linux_arm64/oc'),\n\t\t('ppc64le', 'linux', '/usr/share/openshift/linux_ppc64le/oc'),\n\t\t('s390x', 'linux', '/usr/share/openshift/linux_s390x/oc'),\n\t\t]:\n\tbasename = os.path.basename(path)\n\ttarget_path = os.path.join(arch, operating_system, basename)\n\tos.symlink(path, target_path)\n\tbase_root, _ = os.path.splitext(basename)\n\tarchive_path_root = os.path.join(arch, operating_system, base_root)\n\twith tarfile.open('{}.tar'.format(archive_path_root), 'w') as tar:\n\t\ttar.add(path, basename)\n\twith zipfile.ZipFile('{}.zip'.format(archive_path_root), 'w') as zip:\n\t\tzip.write(path, basename)\n\tcontent.append('\u003ca href=\"{0}\"\u003eoc ({1} {2})\u003c/a\u003e (\u003ca href=\"{0}.tar\"\u003etar\u003c/a\u003e \u003ca href=\"{0}.zip\"\u003ezip\u003c/a\u003e)'.format(target_path, arch, operating_system))\n\nfor root, directories, filenames in os.walk(temp_dir):\n\troot_link = os.path.relpath(temp_dir, os.path.join(root, 'child')).replace(os.path.sep, '/')\n\tfor directory in directories:\n\t\twrite_index(\n\t\t\tpath=os.path.join(root, directory, 'index.html'),\n\t\t\tmessage='\u003cp\u003eDirectory listings are disabled.  See \u003ca href=\"{}\"\u003ehere\u003c/a\u003e for available content.\u003c/p\u003e'.format(root_link),\n\t\t)\n\nwrite_index(\n\tpath=os.path.join(temp_dir, 'index.html'),\n\tmessage='\\n'.join(\n\t\t['\u003cul\u003e'] +\n\t\t['  \u003cli\u003e{}\u003c/li\u003e'.format(entry) for entry in content] +\n\t\t['\u003c/ul\u003e']\n\t),\n)\n\n# Create socket\n# IPv6 should handle IPv4 passively so long as it is not bound to a\n# specific address or set to IPv6_ONLY\n# https://stackoverflow.com/questions/25817848/python-3-does-http-server-support-ipv6\ntry:\n\taddr = ('::', 8080)\n\tsock = socket.socket(socket.AF_INET6, socket.SOCK_STREAM)\nexcept socket.error as err:\n\t# errno.EAFNOSUPPORT is \"socket.error: [Errno 97] Address family not supported by protocol\"\n\t# When IPv6 is disabled, socket will bind using IPv4.\n\tif err.errno == errno.EAFNOSUPPORT:\n\t\taddr = ('', 8080)\n\t\tsock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)\n\telse:\n\t\traise    \nsock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)\nsock.bind(addr)\nsock.listen(5)\n\n[Thread(i, socket=sock) for i in range(100)]\ntime.sleep(9e9)\nEOF\nexec python3 /tmp/serve.py"],"command":["/bin/sh"],"image":"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5186b9d01721c1620d86697661ebf6cf77e2eafeb36a04022ffd998a266e256d","imagePullPolicy":"IfNotPresent","livenessProbe":{"httpGet":{"path":"/","port":8080,"scheme":"HTTP"}},"name":"download-server","ports":[{"containerPort":8080,"name":"http","protocol":"TCP"}],"readinessProbe":{"failureThreshold":3,"httpGet":{"path":"/","port":8080,"scheme":"HTTP"}},"resources":{"requests":{"cpu":"10m","memory":"50Mi"}},"terminationMessagePolicy":"FallbackToLogsOnError"}],"dnsPolicy":null,"restartPolicy":null,"schedulerName":null,"securityContext":null}}}}
.....
I0419 03:07:47.224208       1 apps.go:154] Deployment "openshift-console/downloads" changes: {"spec":{"progressDeadlineSeconds":null,"revisionHistoryLimit":null,"strategy":{"rollingUpdate":null,"type":null},"template":{"spec":{"containers":[{"args":["-c","cat \u003c\u003cEOF \u003e\u003e/tmp/serve.py\nimport errno, http.server, os, re, signal, socket, sys, tarfile, tempfile, threading, time, zipfile\n\nsignal.signal(signal.SIGTERM, lambda signum, frame: sys.exit(0))\n\ndef write_index(path, message):\n\twith open(path, 'wb') as f:\n\t\tf.write('\\n'.join([\n\t\t\t'\u003c!doctype html\u003e',\n\t\t\t'\u003chtml lang=\"en\"\u003e',\n\t\t\t'\u003chead\u003e',\n\t\t\t'  \u003cmeta charset=\"utf-8\"\u003e',\n\t\t\t'\u003c/head\u003e',\n\t\t\t'\u003cbody\u003e',\n\t\t\t'  {}'.format(message),\n\t\t\t'\u003c/body\u003e',\n\t\t\t'\u003c/html\u003e',\n\t\t\t'',\n\t\t]).encode('utf-8'))\n\n# Launch multiple listeners as threads\nclass Thread(threading.Thread):\n\tdef __init__(self, i, socket):\n\t\tthreading.Thread.__init__(self)\n\t\tself.i = i\n\t\tself.socket = socket\n\t\tself.daemon = True\n\t\tself.start()\n\n\tdef run(self):\n\t\thttpd = http.server.HTTPServer(addr, http.server.SimpleHTTPRequestHandler, False)\n\n\t\t# Prevent the HTTP server from re-binding every handler.\n\t\t# https://stackoverflow.com/questions/46210672/\n\t\thttpd.socket = self.socket\n\t\thttpd.server_bind = self.server_close = lambda self: None\n\n\t\thttpd.serve_forever()\n\ntemp_dir = tempfile.mkdtemp()\nprint('serving from {}'.format(temp_dir))\nos.chdir(temp_dir)\nfor arch in ['amd64']:\n\tos.mkdir(arch)\n\tfor operating_system in ['linux', 'mac', 'windows']:\n\t\tos.mkdir(os.path.join(arch, operating_system))\nfor arch in ['arm64', 'ppc64le', 's390x']:\n\tos.mkdir(arch)\n\tfor operating_system in ['linux']:\n\t\tos.mkdir(os.path.join(arch, operating_system))\ncontent = ['\u003ca href=\"oc-license\"\u003elicense\u003c/a\u003e']\nos.symlink('/usr/share/openshift/LICENSE', 'oc-license')\n\nfor arch, operating_system, path in [\n\t\t('amd64', 'linux', '/usr/share/openshift/linux_amd64/oc'),\n\t\t('amd64', 'mac', '/usr/share/openshift/mac/oc'),\n\t\t('amd64', 'windows', '/usr/share/openshift/windows/oc.exe'),\n\t\t('arm64', 'linux', '/usr/share/openshift/linux_arm64/oc'),\n\t\t('ppc64le', 'linux', '/usr/share/openshift/linux_ppc64le/oc'),\n\t\t('s390x', 'linux', '/usr/share/openshift/linux_s390x/oc'),\n\t\t]:\n\tbasename = os.path.basename(path)\n\ttarget_path = os.path.join(arch, operating_system, basename)\n\tos.symlink(path, target_path)\n\tbase_root, _ = os.path.splitext(basename)\n\tarchive_path_root = os.path.join(arch, operating_system, base_root)\n\twith tarfile.open('{}.tar'.format(archive_path_root), 'w') as tar:\n\t\ttar.add(path, basename)\n\twith zipfile.ZipFile('{}.zip'.format(archive_path_root), 'w') as zip:\n\t\tzip.write(path, basename)\n\tcontent.append('\u003ca href=\"{0}\"\u003eoc ({1} {2})\u003c/a\u003e (\u003ca href=\"{0}.tar\"\u003etar\u003c/a\u003e \u003ca href=\"{0}.zip\"\u003ezip\u003c/a\u003e)'.format(target_path, arch, operating_system))\n\nfor root, directories, filenames in os.walk(temp_dir):\n\troot_link = os.path.relpath(temp_dir, os.path.join(root, 'child')).replace(os.path.sep, '/')\n\tfor directory in directories:\n\t\twrite_index(\n\t\t\tpath=os.path.join(root, directory, 'index.html'),\n\t\t\tmessage='\u003cp\u003eDirectory listings are disabled.  See \u003ca href=\"{}\"\u003ehere\u003c/a\u003e for available content.\u003c/p\u003e'.format(root_link),\n\t\t)\n\nwrite_index(\n\tpath=os.path.join(temp_dir, 'index.html'),\n\tmessage='\\n'.join(\n\t\t['\u003cul\u003e'] +\n\t\t['  \u003cli\u003e{}\u003c/li\u003e'.format(entry) for entry in content] +\n\t\t['\u003c/ul\u003e']\n\t),\n)\n\n# Create socket\n# IPv6 should handle IPv4 passively so long as it is not bound to a\n# specific address or set to IPv6_ONLY\n# https://stackoverflow.com/questions/25817848/python-3-does-http-server-support-ipv6\ntry:\n\taddr = ('::', 8080)\n\tsock = socket.socket(socket.AF_INET6, socket.SOCK_STREAM)\nexcept socket.error as err:\n\t# errno.EAFNOSUPPORT is \"socket.error: [Errno 97] Address family not supported by protocol\"\n\t# When IPv6 is disabled, socket will bind using IPv4.\n\tif err.errno == errno.EAFNOSUPPORT:\n\t\taddr = ('', 8080)\n\t\tsock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)\n\telse:\n\t\traise    \nsock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)\nsock.bind(addr)\nsock.listen(5)\n\n[Thread(i, socket=sock) for i in range(100)]\ntime.sleep(9e9)\nEOF\nexec python3 /tmp/serve.py"],"command":["/bin/sh"],"image":"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5186b9d01721c1620d86697661ebf6cf77e2eafeb36a04022ffd998a266e256d","imagePullPolicy":"IfNotPresent","livenessProbe":{"httpGet":{"path":"/","port":8080,"scheme":"HTTP"}},"name":"download-server","ports":[{"containerPort":8080,"name":"http","protocol":"TCP"}],"readinessProbe":{"failureThreshold":3,"httpGet":{"path":"/","port":8080,"scheme":"HTTP"}},"resources":{"requests":{"cpu":"10m","memory":"50Mi"}},"terminationMessagePolicy":"FallbackToLogsOnError"}],"dnsPolicy":null,"restartPolicy":null,"schedulerName":null,"securityContext":null}}}}


Save all the lines which print 'Deployment "openshift-console/downloads" changes' and compare between them, nothing is diffferent except for the timestamp. It seems nothing in deployment/downloads is changing , but we report it changed 

Assigning back for further discussion, this is checked against 4.8.0-0.nightly-2021-04-18-101412

Comment 4 Yadan Pei 2021-04-19 03:31:10 UTC
sorry typo, the command to change operator log level is: oc patch consoles.operator.openshift.io cluster -p '{"spec":{"operatorLogLevel": "Normal"}}' --type merge

and the differentiate result between them see screenshot

Comment 5 Yadan Pei 2021-04-19 03:31:46 UTC
Created attachment 1773159 [details]
two lines are identical

Comment 7 Yadan Pei 2021-05-08 06:50:22 UTC


1. Check console-operator log when operatorLogLevel is Normal 
2. Set operator log level to Debug and check console-operator log
$ oc patch consoles.operator.openshift.io cluster -p '{"spec":{"operatorLogLevel": "Debug"}}' --type merge


when no changes to deployment/downloads, there isn't message `type: 'Normal' reason: 'DeploymentUpdated' Updated Deployment.apps/downloads -n openshift-console because it changed` printed

Verified on 4.8.0-0.nightly-2021-05-07-075528

Comment 10 errata-xmlrpc 2021-07-27 22:59:25 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.8.2 bug fix and security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2021:2438


Note You need to log in before you can comment on or make changes to this bug.