Bug 1858239 - `crictl ps --pod <pod_id>` returns no results on OCP CoreOS nodes
Summary: `crictl ps --pod <pod_id>` returns no results on OCP CoreOS nodes
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Node
Version: 4.5
Hardware: All
OS: Linux
unspecified
low
Target Milestone: ---
: 4.6.0
Assignee: Peter Hunt
QA Contact: MinLi
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-07-17 10:40 UTC by Paul Needle
Modified: 2020-10-27 16:16 UTC (History)
4 users (show)

Fixed In Version: cri-o-1.19.0-62.rhaos4.6.git10c7a86.el8
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-10-27 16:15:30 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github cri-o cri-o pull 4033 0 None closed Make filter container list be able to filter short pod IDs 2020-12-14 21:35:12 UTC
Github kubernetes-sigs cri-tools pull 643 0 None closed ps: document --pod <id> must be a full ID 2020-12-14 21:35:42 UTC
Red Hat Product Errata RHBA-2020:4196 0 None None None 2020-10-27 16:16:01 UTC

Description Paul Needle 2020-07-17 10:40:04 UTC
Description of problem:

The output of `crictl ps --help` includes a `--pod` or `-p` option to filter a container listing by Pod ID:

----
[core@master-0 ~]$ sudo crictl ps --help
NAME:
   crictl ps - List containers

USAGE:
   crictl ps [command options] [arguments...]

OPTIONS:
<etc>
   --pod value, -p value     Filter by pod id
<etc>
----

However, the option returns no results when run on OpenShift Container Platform (OCP) 4.5 CoreOS nodes.


Version-Release number of selected component (if applicable):

OCP 4.5 CoreOS nodes running .


How reproducible:

Every time.


Steps to Reproduce:

1. Check cluster and CoreOS versions:

----
[core@master-0 ~]$ oc get clusterversion 
NAME      VERSION   AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.5.2     True        False         35m     Cluster version is 4.5.2

[core@master-0 ~]$ oc debug node/<string>-master-0
Starting pod/<string->master-0-debug ...
To use host binaries, run `chroot /host`
Pod IP: <IP_address>
If you don't see a command prompt, try pressing enter.

sh-4.2# chroot /host

sh-4.4# cat /etc/redhat-release 
Red Hat Enterprise Linux CoreOS release 4.5
----

2. Obtain a Pod ID:

----
sh-4.4# sudo crictl pods | head -2
POD ID              CREATED             STATE               NAME                                                          NAMESPACE                                    ATTEMPT
77d0fcd4ba290       30 seconds ago      Ready               <string>-master-0-debug                      default                                      0
----

3. Filter a list of containers by the Pod ID output in the preceding command. None of the following command formats return any containers:

----
sh-4.4# sudo crictl ps --pod 77d0fcd4ba290
CONTAINER           IMAGE               CREATED             STATE               NAME                ATTEMPT             POD ID
----

----
sh-4.4# sudo crictl ps --pod=77d0fcd4ba290
CONTAINER           IMAGE               CREATED             STATE               NAME                ATTEMPT             POD ID
----

----
sh-4.4# sudo crictl ps -p 77d0fcd4ba290
CONTAINER           IMAGE               CREATED             STATE               NAME                ATTEMPT             POD ID
----

4. Running the `crictl ps` command without the `--pod` option and grepping for the same Pod ID returns a container:

----
sh-4.4# sudo crictl ps | grep '77d0fcd4ba290'
5c598a5a0481d       <registry>           About a minute ago   Running             container-00                                  0                   77d0fcd4ba290
----


Actual results:

The `-p` or `--pod` option in `crictl ps` does not return any containers.


Expected results:

For the option to list any containers related to the given Pod ID.

Comment 1 Peter Hunt 2020-07-17 14:08:56 UTC
it's not a super awesome user interface, but you need to specify the full pod id. What you see in `crictl pods` is truncated. you can find the full pod id with
`crictl inspectp $trunc_id | jq -r .status.id`

if filtering based on the full pod id doesn't work, I will look into it. otherwise, I'm inclined to believe this is not a bug (but maybe a crictl feature request?)

Comment 2 Paul Needle 2020-07-17 15:46:28 UTC
(In reply to Peter Hunt from comment #1)
> it's not a super awesome user interface, but you need to specify the full
> pod id. What you see in `crictl pods` is truncated. you can find the full
> pod id with
> `crictl inspectp $trunc_id | jq -r .status.id`
> 
> if filtering based on the full pod id doesn't work, I will look into it.
> otherwise, I'm inclined to believe this is not a bug (but maybe a crictl
> feature request?)

Thank you Peter for explaining that. I tested this using the non-truncated Pod ID and it worked on an OCP 4.5 node.

My view is:

1. The output of `crictl ps --help` should clarify that the non-truncated Pod ID is required. But, this is still not idea because it is not obvious as to where to find the non-truncated ID without an explanation.

2. For consistency, the `--pod` option should accept the Pod ID as it is output by other `crictl` commands, i.e. truncated.

I'm not sure if that is a feature request. In my view the current option is not consistent and it is not obvious in the help menu as to how to use it.

Paul.

Comment 3 Peter Hunt 2020-07-28 21:00:53 UTC
I have opened a docs pr upstream to solve this

Comment 4 Peter Hunt 2020-07-31 21:31:32 UTC
actually, we are changing behavior in CRI-O (attached PR)

Comment 5 Peter Hunt 2020-08-03 19:40:05 UTC
upstream PR has merged, fix will be in 4.6

Comment 8 MinLi 2020-08-05 03:36:33 UTC
not fixed in version : 4.6.0-0.nightly-2020-08-04-210224

sh-4.4# crictl pods | head -2
POD ID              CREATED             STATE               NAME                                            NAMESPACE                                ATTEMPT
dd778ce2e6163       3 minutes ago       Ready               ip-10-0-207-124us-east-2computeinternal-debug   default                                  0

sh-4.4# crictl ps -p dd778ce2e6163 
CONTAINER           IMAGE               CREATED             STATE               NAME                ATTEMPT             POD ID

sh-4.4# crictl ps -p dd778ce2e6163a804d011985cb95275078f13cd3465d38c391e83ebd4fde6300 (long pod id is ok)
CONTAINER           IMAGE                                                                                                                    CREATED             STATE               NAME                ATTEMPT             POD ID
9deeaaab2865d       quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6011b9ece8fa6e259949c98f0a30a05d9bd314e02e364e42723cebeba55c53c6   4 minutes ago       Running             container-00        0                   dd778ce2e6163

Comment 9 Peter Hunt 2020-08-05 15:08:40 UTC
ah, this is because the machine-os-content hasn't pulled in the correct version yet.

here's the latest machine-os-content: https://releases-rhcos-art.cloud.privileged.psi.redhat.com/contents.html?stream=releases%2Frhcos-4.6&release=46.82.202008032211-0
which has cri-o: 1.19.0-61.rhaos4.6.git79c1228.el8 (https://github.com/cri-o/cri-o/commit/79c122895fe645f6b642cd20ea6eea78cf41d085)

cri-o is fixed in https://github.com/cri-o/cri-o/commit/15e88b8f4492d4d6c3348c2f184239f04635c3d3
which is included in https://brewweb.engineering.redhat.com/brew/buildinfo?buildID=1276422

Comment 11 MinLi 2020-08-07 08:56:23 UTC
verified in version : 4.6.0-0.nightly-2020-08-06-131904

sh-4.4# crictl version 
Version:  0.1.0
RuntimeName:  cri-o
RuntimeVersion:  1.19.0-67.rhaos4.6.gitc964c31.el8-dev
RuntimeApiVersion:  v1alpha1

sh-4.4# crictl pods | head -2
POD ID              CREATED             STATE               NAME                                            NAMESPACE                                ATTEMPT
08a6a6ca2572a       2 minutes ago       Ready               redhat-marketplace-qh8jq                        openshift-marketplace                    0
sh-4.4# crictl ps -p 08a6a6ca2572a
CONTAINER           IMAGE                                                                                                                        CREATED             STATE               NAME                ATTEMPT             POD ID
a86b4cff4c1ea       registry.redhat.io/redhat/certified-operator-index@sha256:39b4589658b766142baef3bd20d21114d8d0571e50c5c38d1a13f248c41c1e68   2 minutes ago       Running             registry-server     0                   08a6a6ca2572a

Comment 13 errata-xmlrpc 2020-10-27 16:15:30 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.6 GA Images), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:4196


Note You need to log in before you can comment on or make changes to this bug.