Bug 1763807 - --since argument for 'oc adm node-logs' command returns an error "error: rhhi-node-master-0 the server rejected our request for an unknown reason"
Summary: --since argument for 'oc adm node-logs' command returns an error "error: rhh...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: oc
Version: 4.2.0
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ---
: 4.3.0
Assignee: Maciej Szulik
QA Contact: zhou ying
URL:
Whiteboard:
Depends On: 1779563
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-10-21 16:13 UTC by Alexander Chuzhoy
Modified: 2020-01-23 11:08 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-01-23 11:08:26 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift oc pull 135 0 'None' closed Bug 1763807: print detailed error to user when getting node logs 2020-09-09 09:18:07 UTC
Github openshift origin pull 24105 0 'None' closed Bug 1763807: UPSTREAM: <carry>: Expose a simple journald shim on the kubelet logs endpoint 2020-09-09 09:18:12 UTC
Github openshift origin pull 24293 0 None closed [release-4.3] Bug 1763807: fix oc adm node logs and add tests 2020-09-09 09:18:07 UTC
Red Hat Product Errata RHBA-2020:0062 0 None None None 2020-01-23 11:08:56 UTC

Description Alexander Chuzhoy 2019-10-21 16:13:09 UTC
Environment:
[kni@rhhi-node-worker-0 ~]$ oc version
Client Version: openshift-clients-4.2.1-201910170058
Server Version: 4.2.0-0.nightly-2019-10-18-063630
Kubernetes Version: v1.14.6+2fdcb45



--since argument for 'oc adm node-logs'  command returns an error "error: rhhi-node-master-0 the server rejected our request for an unknown reason"

How to reproduce:


If we try to supply the --since argument, the following happens:

[kni@rhhi-node-worker-0 ~]$ oc adm node-logs rhhi-node-master-0 --since yesterday
error: rhhi-node-master-0 the server rejected our request for an unknown reason
[kni@rhhi-node-worker-0 ~]$ oc adm node-logs rhhi-node-master-0 --since 16:00:00
error: rhhi-node-master-0 the server rejected our request for an unknown reason




[kni@rhhi-node-worker-0 ~]$ oc adm node-logs --help|grep since
      --since='': Return logs after a specific ISO timestamp or relative date. Only applies to node journal logs.






Note that running  'oc adm node-logs' without the --since argument works fine:

[kni@rhhi-node-worker-0 ~]$ oc adm node-logs rhhi-node-master-0 --tail=1
-- Logs begin at Mon 2019-10-21 15:08:25 UTC, end at Mon 2019-10-21 16:10:50 UTC. --
Oct 21 15:38:43 rhhi-node-master-0 systemd-journald[894]: Journal stopped
-- Logs begin at Mon 2019-10-21 15:08:25 UTC, end at Mon 2019-10-21 16:10:50 UTC. --
Oct 21 16:10:50 rhhi-node-master-0 hyperkube[1808]: I1021 16:10:50.136963    1808 prober.go:125] Liveness probe for "mdns-publisher-rhhi-node-master-0_openshift-kni-infra(7effe58e2fadce03a8b20114634b102d):mdns-publisher" succeeded

Comment 1 Maciej Szulik 2019-10-28 13:05:54 UTC
In your case the problem is the format of --since parameter, see this error:

parameter 'since' is invalid: date must be a relative time of the form '(+|-)[0-9]+(s|m|h|d)' or a date in 'YYYY-MM-DDTHH:MM:SSZ' form

which is hidden when you executed with -v=7.

I've put together https://github.com/openshift/oc/pull/135 which should expose that error to user.

Comment 6 Maciej Szulik 2019-11-25 14:12:52 UTC
https://github.com/openshift/origin/pull/24201 is going to fix the situation in master, 4.3 cherry-pick will follow.

Comment 7 Maciej Szulik 2019-12-02 17:57:58 UTC
https://github.com/openshift/origin/pull/24213 merged, moving to qa.

Comment 9 zhou ying 2019-12-04 09:28:34 UTC
Confirmed with latest version, the error issue has fixed:

[root@dhcp-140-138 ~]# oc version -o yaml 
clientVersion:
  buildDate: "2019-12-03T14:24:28Z"
  compiler: gc
  gitCommit: a82d2ce4efdbfa9836480d8567acccfde19d297b
  gitTreeState: clean
  gitVersion: v4.3.0
  goVersion: go1.12.12
  major: ""
  minor: ""
  platform: linux/amd64
openshiftVersion: 4.3.0-0.nightly-2019-12-03-211441

[root@dhcp-140-138 ~]# oc adm node-logs ip-10-0-137-15.us-east-2.compute.internal --since "16:00:00"
error: the server rejected our request for an unknown reason
  parameter 'since' is invalid: date must be a relative time of the form '(+|-)[0-9]+(s|m|h|d)' or a date in 'YYYY-MM-DDTHH:MM:SSZ' form
[root@dhcp-140-138 ~]# oc adm node-logs ip-10-0-137-15.us-east-2.compute.internal --since yesterday
error: the server rejected our request for an unknown reason
  parameter 'since' is invalid: date must be a relative time of the form '(+|-)[0-9]+(s|m|h|d)' or a date in 'YYYY-MM-DDTHH:MM:SSZ' form

Comment 11 Maciej Szulik 2019-12-04 20:29:06 UTC
From previous comment it looks like it's not working as it should, moving back to dev.

Comment 12 Maciej Szulik 2019-12-04 20:29:18 UTC
*** Bug 1779563 has been marked as a duplicate of this bug. ***

Comment 13 Maciej Szulik 2019-12-12 11:21:59 UTC
This is not blocking 4.3, but we want to fix it in 4.3, moving to .z

Comment 15 zhou ying 2019-12-18 05:27:21 UTC
Confirmed with latest version, the issue has fixed:

[root@dhcp-140-138 ~]# oc version -o yaml 
clientVersion:
  buildDate: "2019-12-13T10:59:49Z"
  compiler: gc
  gitCommit: 6a937dfe56ff26255d09702c69b8406040c14505
  gitTreeState: clean
  gitVersion: v4.3.0
  goVersion: go1.12.12
  major: ""
  minor: ""
  platform: linux/amd64


[root@dhcp-140-138 ~]# oc adm node-logs node/ip-10-0-172-171.us-east-2.compute.internal --since  "2019-12-18 05:17:00.9999"
-- Logs begin at Wed 2019-12-18 01:03:06 UTC, end at Wed 2019-12-18 05:21:56 UTC. --
-- No entries --
-- Logs begin at Wed 2019-12-18 01:03:06 UTC, end at Wed 2019-12-18 05:21:56 UTC. --
Dec 18 05:17:01 ip-10-0-172-171 hyperkube[2053]: I1218 05:17:01.170997    2053 kubelet_pods.go:1328] Generating status for "packageserver-55c9c87665-ljw9h_openshift-operator-lifecycle-manager(b7c41ff7-ced4-4425-8a2f-cf11bb60ab9b)"
....


[root@dhcp-140-138 ~]# oc adm node-logs node/ip-10-0-172-171.us-east-2.compute.internal --since=-2m --until=-1m
-- Logs begin at Wed 2019-12-18 01:03:06 UTC, end at Wed 2019-12-18 05:20:59 UTC. --
-- No entries --
-- Logs begin at Wed 2019-12-18 01:03:06 UTC, end at Wed 2019-12-18 05:20:59 UTC. --
Dec 18 05:18:59 ip-10-0-172-171 hyperkube[2053]: I1218 05:18:59.170951    2053 kubelet_pods.go:1328] Generating status for "openshift-service-catalog-apiserver-operator-7bbb64c9bc-tttj2_openshift-service-catalog-apiserver-operator(ed10d844-4252-4986-91be-3766f5c07193)"

Comment 17 errata-xmlrpc 2020-01-23 11:08:26 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:0062


Note You need to log in before you can comment on or make changes to this bug.