Bug 1748149 - Metadata unnormal for debug nodes pod logs
Summary: Metadata unnormal for debug nodes pod logs
Keywords:
Status: CLOSED DUPLICATE of bug 1746968
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Logging
Version: 4.2.0
Hardware: Unspecified
OS: Unspecified
unspecified
low
Target Milestone: ---
: ---
Assignee: Jeff Cantrill
QA Contact: Anping Li
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-09-03 02:19 UTC by Anping Li
Modified: 2019-09-09 03:49 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-09-09 03:49:04 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Anping Li 2019-09-03 02:19:04 UTC
Description of problem:
We observed some unnormal filed in the record for some special containers created by 'oc debug nodes/${node_name}'.
1) No container_image and container_image_id
2) The hostname is localhost.localdomain


Version-Release number of selected component (if applicable):
4.2

How reproducible:
sometimes

Steps to Reproduce:


1. oc new-project test1
2. debug nodes and run shell command in the pod
   oc debug nodes/${node_name}
3. Search the records in ES

Actual results:
#1) The unnormal records
sh-4.2$ es_util --query=project.*/_search?size=1 |python -m json.tool
{
    "_shards": {
        "failed": 0,
        "skipped": 0,
        "successful": 12,
        "total": 12
    },
    "hits": {
        "hits": [
            {
                "_id": "Y2E0YmRlNDctYTgzYS00MjZjLThjMmMtZWE5NDEzYmE2NjZh",
                "_index": "project.weinliu.502c6eac-cd52-11e9-8636-027efc23d69e.2019.09.02",
                "_score": 1.0,
                "_source": {
                    "@timestamp": "2019-09-02T07:22:13.092708+00:00",
                    "docker": {
                        "container_id": "2014c17efa4cb497ed2c8c99695b563e7a69b47c18e67c242ef92154bca27ea6"
                    },
                    "hostname": "localhost.localdomain",
                    "kubernetes": {
                        "container_name": "container-00",
                        "namespace_id": "502c6eac-cd52-11e9-8636-027efc23d69e",
                        "namespace_name": "weinliu",
                        "pod_id": "2014c17efa4cb497ed2c8c99695b563e7a69b47c18e67c242ef92154bca27ea6",
                        "pod_name": "ip-10-0-67-225us-east-2computeinternal-debug"
                    },
                    "level": "info",
                    "message": "openshift-hyperkube-4.1.14-201908290858.git.0.3bd3467.el8.x86_64",
                    "pipeline_metadata": {
                        "collector": {
                            "inputname": "fluent-plugin-systemd",
                            "ipaddr4": "10.128.2.20",
                            "ipaddr6": "fe80::6410:8cff:fe7f:4b35",
                            "name": "fluentd",
                            "received_at": "2019-09-02T07:22:49.282400+00:00",
                            "version": "1.5.1 1.6.0"
                        }
                    },
                    "viaq_msg_id": "Y2E0YmRlNDctYTgzYS00MjZjLThjMmMtZWE5NDEzYmE2NjZh"
                },
                "_type": "com.redhat.viaq.common"
            }
        ],
        "max_score": 1.0,
        "total": 537
    },
    "timed_out": false,
    "took": 7
}

#2) A nomral record
sh-4.2$ es_util --query=project.test1*/_search?size=1 |python -m json.tool
{
    "_shards": {
        "failed": 0,
        "skipped": 0,
        "successful": 3,
        "total": 3
    },
    "hits": {
        "hits": [
            {
                "_id": "YjBhNTk0N2EtYjdiYy00MGM3LTk3N2MtNzY4NmQ5MjBkNjg3",
                "_index": "project.test1.9bbc4436-cd59-11e9-a4f4-026219342cde.2019.09.02",
                "_score": 1.0,
                "_source": {
                    "@timestamp": "2019-09-02T08:19:32.631599+00:00",
                    "docker": {
                        "container_id": "f5c3b99575cc60b7569cd2984ef4c183be693296c2b3dcb6e08b64010bc2ca1c"
                    },
                    "hostname": "ip-10-0-59-243.us-east-2.compute.internal",
                    "kubernetes": {
                        "container_image": "registry.redhat.io/rhel7/support-tools:latest",
                        "container_image_id": "registry.redhat.io/rhel7/support-tools@sha256:466a4b727473281159e380a5866e631ecfab024164a62e4670ee8dc277375072",
                        "container_name": "container-00",
                        "host": "ip-10-0-59-243.us-east-2.compute.internal",
                        "master_url": "https://kubernetes.default.svc",
                        "namespace_id": "9bbc4436-cd59-11e9-a4f4-026219342cde",
                        "namespace_name": "test1",
                        "pod_id": "6272d70d-cd5a-11e9-9785-06a53344b2a0",
                        "pod_name": "ip-10-0-59-243us-east-2computeinternal-debug"
                    },
                    "level": "info",
                    "message": "\u001b[?1034hsh-4.2# chroot /host",
                    "pipeline_metadata": {
                        "collector": {
                            "inputname": "fluent-plugin-systemd",
                            "ipaddr4": "10.130.2.16",
                            "ipaddr6": "fe80::9480:fdff:fe4d:6126",
                            "name": "fluentd",
                            "received_at": "2019-09-02T08:19:52.710859+00:00",
                            "version": "1.5.1 1.6.0"
                        }
                    },
                    "viaq_msg_id": "YjBhNTk0N2EtYjdiYy00MGM3LTk3N2MtNzY4NmQ5MjBkNjg3"
                },
                "_type": "com.redhat.viaq.common"
            }
        ],
        "max_score": 1.0,
        "total": 12
    },
    "timed_out": false,
    "took": 3
}

Expected result:
No missing filed and hostname is nodename.

Comment 1 Anping Li 2019-09-09 03:49:04 UTC
It seems the root cause is BZ1748149.

*** This bug has been marked as a duplicate of bug 1746968 ***


Note You need to log in before you can comment on or make changes to this bug.