Bug 1707447 - jenkins-slave produce process defunct [ Jenkins "SLAVE" ]
Summary: jenkins-slave produce process defunct [ Jenkins "SLAVE" ]
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: ImageStreams
Version: 3.10.0
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
: 3.10.z
Assignee: Gabe Montero
QA Contact: XiuJuan Wang
URL:
Whiteboard:
Depends On: 1700314 1707448 1718379
Blocks: 1705123
TreeView+ depends on / blocked
 
Reported: 2019-05-07 14:35 UTC by Gabe Montero
Modified: 2019-06-27 16:41 UTC (History)
9 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Cause: Long running jenkins agent/slave pods can experience the defect process phenomenon that we previously observed with the jenkins master Consequence: A lot of defect processes show up in process listings until the pod is terminated. Fix: Employ `dumb-init` as with the openshift/jenkins master image to clean up these defect processes which occur during jenkins job processing. Result: Process listings within agent/slave pods and on the hosts those pods reside no longer include the defunct processes.
Clone Of: 1700314
Environment:
Last Closed: 2019-06-27 16:41:12 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2019:1607 0 None None None 2019-06-27 16:41:23 UTC

Comment 3 Gabe Montero 2019-05-13 14:51:49 UTC 
Turns out we will need https://github.com/openshift/ocp-build-data/pull/121 to merge before we can start getting slave builds at osbs/brew with dumb-init
Comment 4 Gabe Montero 2019-06-10 14:18:53 UTC 
looks like we have new images on brew for this
Comment 6 XiuJuan Wang 2019-06-12 07:47:13 UTC 
verified with 
openshift3/jenkins-agent-nodejs-8-rhel7:v3.10 (v3.10.149)
openshift3/jenkins-agent-maven-35-rhel7:v3.10 (v3.10.149)
openshift3/jenkins-slave-nodejs-rhel7:v3.10 (v3.10.149)
openshift3/jenkins-slave-maven-rhel7:v3.10 (v3.10.149)

Steps:
1. Create jenkins server and maven| nodejs pipeline buildconfigs.
2.Login to jenkins console to set the maven/nodejs pod idle 30 mins
3.Trigger maven and nodejs pipeline builds.
4.Rsh into slave pod when time is almost out.
dumb-init process has cleaned defunct processes, no defunct processes exist.

maven-nc44t                           1/1       Running     0          30m
nodejs-3nxf6                          1/1       Running     0          30m


#  oc rsh  -n xiuwang  nodejs-3nxf6 
sh-4.2$ ps -ef 
UID         PID   PPID  C STIME TTY          TIME CMD
default       1      0  0 07:06 ?        00:00:00 /usr/bin/dumb-init -- /usr/local/bin/run-jnlp-client b4b4586fb94d09888b23ffe144cd18271ae9eb9075483b37a933a77ac5a69786 nodejs-3nxf6
default       7      1  1 07:06 ?        00:00:17 java -XX:+UseParallelGC -XX:MinHeapFreeRatio=5 -XX:MaxHeapFreeRatio=10 -XX:GCTimeRatio=4 -XX:AdaptiveSizePolicyWeight=90 -cp /home/jenkins/remoting.jar hudson.re
default     251      0  0 07:34 ?        00:00:00 /bin/sh
default     260    251  0 07:34 ?        00:00:00 ps -ef

#  oc rsh  -n xiuwang maven-nc44t 
sh-4.2$ ps -ef
UID         PID   PPID  C STIME TTY          TIME CMD
default       1      0  0 07:07 ?        00:00:00 /usr/bin/dumb-init -- /usr/local/bin/run-jnlp-client ce6250144ae1685757b511ca3be8555faeebf1451574103da5e4fcea6cfddbc3 maven-nc44t
default       7      1  1 07:07 ?        00:00:18 java -XX:+UseParallelGC -XX:MinHeapFreeRatio=5 -XX:MaxHeapFreeRatio=10 -XX:GCTimeRatio=4 -XX:AdaptiveSizePolicyWeight=90 -cp /home/jenkins/remoting.jar hudson.re
default     265      0  0 07:37 ?        00:00:00 /bin/sh
default     274    265  0 07:37 ?        00:00:00 ps -ef
Comment 8 errata-xmlrpc 2019-06-27 16:41:12 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:1607
Note You need to log in before you can comment on or make changes to this bug.