Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1678905

Summary: Systemd commands do not work inside node debug container
Product: OpenShift Container Platform Reporter: Derek Carr <decarr>
Component: NodeAssignee: Seth Jennings <sjenning>
Status: CLOSED NOTABUG QA Contact: Jianwei Hou <jhou>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 4.1.0CC: aos-bugs, jokerman, mmccomas
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-02-26 20:32:42 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Derek Carr 2019-02-19 20:46:46 UTC
Description of problem:
Unable to run systemd commands inside a pod after using `oc debug node`.

Version-Release number of selected component (if applicable):


How reproducible:
Always

Steps to Reproduce:
1. oc debug node/<nodename> --asroot=true
2. systemctl status kubelet

Actual results:
Running in chroot, ignoring request.


Expected results:
Able to see status of host process.


Additional info:
http://0pointer.de/blog/projects/changing-roots

Unclear if this is solvable, or if this would require a bastion/proxy with direct SSH access to host instead.

Comment 2 Seth Jennings 2019-02-19 21:20:06 UTC
This works for me:

$ oc debug node/ip-10-0-129-146.us-west-1.compute.internal
Starting pod/ip-10-0-129-146us-west-1computeinternal-debug ...
If you don't see a command prompt, try pressing enter.
sh-4.2# chroot /host /bin/bash
[root@ip-10-0-129-146 /]# systemctl status kubelet
● kubelet.service - Kubernetes Kubelet
   Loaded: loaded (/etc/systemd/system/kubelet.service; enabled; vendor preset: enabled)
   Active: active (running) since Tue 2019-02-19 16:37:06 UTC; 4h 41min ago
  Process: 4064 ExecStartPre=/bin/rm -f /var/lib/kubelet/cpu_manager_state (code=exited, status=0/SUCCESS)
  Process: 4059 ExecStartPre=/bin/mkdir --parents /etc/kubernetes/manifests (code=exited, status=0/SUCCESS)
 Main PID: 4069 (hyperkube)
   Memory: 204.9M
   CGroup: /system.slice/kubelet.service
           └─4069 /usr/bin/hyperkube kubelet --config=/etc/kubernetes/kubelet.conf --bootstrap-kubeconfig=/etc/kubernetes/kubeconfig --kubeconfig=/var/lib/kubel...

Feb 19 21:15:50 ip-10-0-129-146 hyperkube[4069]: W0219 21:15:50.144862    4069 reflector.go:270] object-"openshift-monitoring"/"telemeter-client-servin... (187424)
Feb 19 21:16:32 ip-10-0-129-146 hyperkube[4069]: W0219 21:16:32.549082    4069 reflector.go:270] object-"openshift-image-registry"/"image-registry-cert... (187857)
Feb 19 21:17:27 ip-10-0-129-146 hyperkube[4069]: W0219 21:17:27.345180    4069 reflector.go:270] object-"openshift-dns"/"dns-default": watch of *v1.Con... (188422)
Feb 19 21:18:07 ip-10-0-129-146 hyperkube[4069]: W0219 21:18:07.929239    4069 reflector.go:270] object-"openshift-monitoring"/"prometheus-adapter-prom... (188859)
Feb 19 21:18:14 ip-10-0-129-146 hyperkube[4069]: W0219 21:18:14.772631    4069 reflector.go:270] object-"openshift-cluster-node-tuning-operator"/"tuned... (188930)
Feb 19 21:18:16 ip-10-0-129-146 hyperkube[4069]: W0219 21:18:16.358845    4069 reflector.go:270] object-"openshift-monitoring"/"csr-controller-ca-bundl... (188945)
Feb 19 21:18:31 ip-10-0-129-146 hyperkube[4069]: W0219 21:18:31.742802    4069 reflector.go:270] object-"openshift-sdn"/"sdn-config": watch of *v1.Conf... (189094)
Feb 19 21:18:45 ip-10-0-129-146 hyperkube[4069]: I0219 21:18:45.703569    4069 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for v...
Feb 19 21:18:45 ip-10-0-129-146 hyperkube[4069]: I0219 21:18:45.703635    4069 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for v...
Feb 19 21:18:45 ip-10-0-129-146 hyperkube[4069]: W0219 21:18:45.930364    4069 reflector.go:270] object-"openshift-monitoring"/"adapter-config": watch ... (189232)
Hint: Some lines were ellipsized, use -l to show in full.