Bug 1260963 - goferd accumulates memory when failing to connect to qdrouterd, segfaulting at the end
goferd accumulates memory when failing to connect to qdrouterd, segfaulting a...
Status: CLOSED ERRATA
Product: Red Hat Satellite 6
Classification: Red Hat
Component: katello-agent (Show other bugs)
6.1.1
All Linux
high Severity high (vote)
: 6.1.6
: --
Assigned To: Katello Bug Bin
Tazim Kolhar
: Triaged
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2015-09-08 06:07 EDT by Pavel Moravec
Modified: 2017-02-23 14:46 EST (History)
24 users (show)

See Also:
Fixed In Version: qpid-proton-0.9-11
Doc Type: Bug Fix
Doc Text:
Client systems would have issues connecting to Satellite due to out of memory errors from the qpid subsystem.
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-01-21 02:51:25 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
coredump (11.12 MB, application/x-gzip)
2015-09-08 06:56 EDT, Pavel Moravec
no flags Details


External Trackers
Tracker ID Priority Status Summary Last Updated
Apache JIRA PROTON-1028 None None None Never
Apache JIRA PROTON-1030 None None None Never
Red Hat Knowledge Base (Solution) 1605793 None None None Never

  None (edit)
Description Pavel Moravec 2015-09-08 06:07:22 EDT
Description of problem:
When goferd cant connect to qdrouterd due to whatever reason, it's memory consumption grows over the time. Once it reaches approx. 86MB (per ps output, RSS column), goferd segfaults (abrt does not generate a report, though).


Version-Release number of selected component (if applicable):
python-gofer-2.6.2-2.el7sat.noarch
gofer-2.6.2-2.el7sat.noarch
python-gofer-qpid-2.6.2-2.el7sat.noarch
python-gofer-proton-2.6.2-2.el7sat.noarch


How reproducible:
100%


Steps to Reproduce:
1. (just for faster reproducer, lower delays between connection probes by setting DELAY = 1 and MAX_DELAY = 1 in /usr/lib/python2.7/site-packages/gofer/messaging/adapter/proton/connection.py)

2. (on Satellite)
service qdrouterd stop

3. (on Capsule)
while true; do ps aux | grep python | grep gofer; sleep 5; done


Actual results:
- RSS of goferd process growing over time
- once it reaches approx 86MB, PID of the process changes in the ps outputs and /var/log/messages has:

Sep  8 11:05:05 my-capsule kernel: python[24968]: segfault at 0 ip 00007f2e400f5e70 sp 00007f2e4fffab48 error 4 in libqpid-proton.so.2.0.0[7f2e400c6000+4c000]



Expected results:
- no memory accumulation
- no segfault


Additional info:
running goferd under valgrind, it shows some "definitely lost" bytes. But that's some constant occurring regardless of number of failing connection attempts.

segfault sounds to be caused by proton bug, as coredump backtrace starts with:

#0  pn_reactor_attachments (reactor=reactor@entry=0x0) at /usr/src/debug/qpid-proton-0.9/proton-c/src/reactor/reactor.c:119
#1  0x00007f3a0014b2e3 in _wrap_pn_reactor_attachments (self=<optimized out>, args=<optimized out>)
    at /usr/src/debug/qpid-proton-0.9/proton-c/bindings/python/cprotonPYTHON_wrap.c:23077
..

(will check this later on)
Comment 2 Pavel Moravec 2015-09-08 06:17:46 EDT
Valgrind details about the memory accumulation (reordered from the biggest "memory consumers"):

==25650== 7,406,472 bytes in 339 blocks are possibly lost in loss record 9,376 of 9,376
==25650==    at 0x4C29BFD: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==25650==    by 0xFF6C2D2: CRYPTO_malloc (in /usr/lib64/libcrypto.so.1.0.1e)
==25650==    by 0xFFDB8A3: BUF_MEM_grow (in /usr/lib64/libcrypto.so.1.0.1e)
==25650==    by 0xFCC675D: ssl23_connect (in /usr/lib64/libssl.so.1.0.1e)
==25650==    by 0xFCC7181: ssl23_write (in /usr/lib64/libssl.so.1.0.1e)
==25650==    by 0xFCE5F67: ??? (in /usr/lib64/libssl.so.1.0.1e)
==25650==    by 0xFFDC02A: BIO_write (in /usr/lib64/libcrypto.so.1.0.1e)
==25650==    by 0x229A13B3: ??? (in /usr/lib64/libqpid-proton.so.2.0.0)
==25650==    by 0x229953BB: pn_transport_pending (in /usr/lib64/libqpid-proton.so.2.0.0)
==25650==    by 0x22998AE6: ??? (in /usr/lib64/libqpid-proton.so.2.0.0)
==25650==    by 0x22998B36: ??? (in /usr/lib64/libqpid-proton.so.2.0.0)
==25650==    by 0x22999112: pn_reactor_selectable_transport (in /usr/lib64/libqpid-proton.so.2.0.0)
==25650==

==25650== 5,901,312 bytes in 339 blocks are possibly lost in loss record 9,375 of 9,376
==25650==    at 0x4C29BFD: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==25650==    by 0xFF6C2D2: CRYPTO_malloc (in /usr/lib64/libcrypto.so.1.0.1e)
==25650==    by 0xFFE3B24: ??? (in /usr/lib64/libcrypto.so.1.0.1e)
==25650==    by 0xFFE3C61: BIO_new_bio_pair (in /usr/lib64/libcrypto.so.1.0.1e)
==25650==    by 0x229A10BD: ??? (in /usr/lib64/libqpid-proton.so.2.0.0)
==25650==    by 0x227297A2: ??? (in /usr/lib64/python2.7/site-packages/_cproton.so)
==25650==    by 0x4F15B93: PyEval_EvalFrameEx (in /usr/lib64/libpython2.7.so.1.0)
==25650==    by 0x4F171AC: PyEval_EvalCodeEx (in /usr/lib64/libpython2.7.so.1.0)
==25650==    by 0x4EA4097: ??? (in /usr/lib64/libpython2.7.so.1.0)
==25650==    by 0x4E7F072: PyObject_Call (in /usr/lib64/libpython2.7.so.1.0)
==25650==    by 0x4ED19EC: ??? (in /usr/lib64/libpython2.7.so.1.0)
==25650==    by 0x4ED4E32: ??? (in /usr/lib64/libpython2.7.so.1.0)
==25650==

==25650== 5,901,312 bytes in 339 blocks are possibly lost in loss record 9,374 of 9,376
==25650==    at 0x4C29BFD: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==25650==    by 0xFF6C2D2: CRYPTO_malloc (in /usr/lib64/libcrypto.so.1.0.1e)
==25650==    by 0xFFE3B67: ??? (in /usr/lib64/libcrypto.so.1.0.1e)
==25650==    by 0xFFE3C61: BIO_new_bio_pair (in /usr/lib64/libcrypto.so.1.0.1e)
==25650==    by 0x229A10BD: ??? (in /usr/lib64/libqpid-proton.so.2.0.0)
==25650==    by 0x227297A2: ??? (in /usr/lib64/python2.7/site-packages/_cproton.so)
==25650==    by 0x4F15B93: PyEval_EvalFrameEx (in /usr/lib64/libpython2.7.so.1.0)
==25650==    by 0x4F171AC: PyEval_EvalCodeEx (in /usr/lib64/libpython2.7.so.1.0)
==25650==    by 0x4EA4097: ??? (in /usr/lib64/libpython2.7.so.1.0)
==25650==    by 0x4E7F072: PyObject_Call (in /usr/lib64/libpython2.7.so.1.0)
==25650==    by 0x4ED19EC: ??? (in /usr/lib64/libpython2.7.so.1.0)
==25650==    by 0x4ED4E32: ??? (in /usr/lib64/libpython2.7.so.1.0)
==25650==
==25650== 5,665,368 bytes in 339 blocks are possibly lost in loss record 9,373 of 9,376
==25650==    at 0x4C29BFD: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==25650==    by 0xFF6C2D2: CRYPTO_malloc (in /usr/lib64/libcrypto.so.1.0.1e)
==25650==    by 0xFCC4112: ssl3_setup_read_buffer (in /usr/lib64/libssl.so.1.0.1e)
==25650==    by 0xFCC4278: ssl3_setup_buffers (in /usr/lib64/libssl.so.1.0.1e)
==25650==    by 0xFCC60D0: ssl23_connect (in /usr/lib64/libssl.so.1.0.1e)
==25650==    by 0xFCC7181: ssl23_write (in /usr/lib64/libssl.so.1.0.1e)
==25650==    by 0xFCE5F67: ??? (in /usr/lib64/libssl.so.1.0.1e)
==25650==    by 0xFFDC02A: BIO_write (in /usr/lib64/libcrypto.so.1.0.1e)
==25650==    by 0x229A13B3: ??? (in /usr/lib64/libqpid-proton.so.2.0.0)
==25650==    by 0x229953BB: pn_transport_pending (in /usr/lib64/libqpid-proton.so.2.0.0)
==25650==    by 0x22998AE6: ??? (in /usr/lib64/libqpid-proton.so.2.0.0)
==25650==    by 0x22998B36: ??? (in /usr/lib64/libqpid-proton.so.2.0.0)
==25650==
==25650== 5,613,840 bytes in 339 blocks are possibly lost in loss record 9,372 of 9,376
==25650==    at 0x4C29BFD: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==25650==    by 0xFF6C2D2: CRYPTO_malloc (in /usr/lib64/libcrypto.so.1.0.1e)
==25650==    by 0xFCC421E: ssl3_setup_write_buffer (in /usr/lib64/libssl.so.1.0.1e)
==25650==    by 0xFCC4287: ssl3_setup_buffers (in /usr/lib64/libssl.so.1.0.1e)
==25650==    by 0xFCC60D0: ssl23_connect (in /usr/lib64/libssl.so.1.0.1e)
==25650==    by 0xFCC7181: ssl23_write (in /usr/lib64/libssl.so.1.0.1e)
==25650==    by 0xFCE5F67: ??? (in /usr/lib64/libssl.so.1.0.1e)
==25650==    by 0xFFDC02A: BIO_write (in /usr/lib64/libcrypto.so.1.0.1e)
==25650==    by 0x229A13B3: ??? (in /usr/lib64/libqpid-proton.so.2.0.0)
==25650==    by 0x229953BB: pn_transport_pending (in /usr/lib64/libqpid-proton.so.2.0.0)
==25650==    by 0x22998AE6: ??? (in /usr/lib64/libqpid-proton.so.2.0.0)
==25650==    by 0x22998B36: ??? (in /usr/lib64/libqpid-proton.so.2.0.0)
==25650==
==25650== 5,554,176 bytes in 339 blocks are possibly lost in loss record 9,371 of 9,376
==25650==    at 0x4C29BFD: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==25650==    by 0x229905AD: pn_transport (in /usr/lib64/libqpid-proton.so.2.0.0)
==25650==    by 0x2270C157: ??? (in /usr/lib64/python2.7/site-packages/_cproton.so)
==25650==    by 0x4F15B93: PyEval_EvalFrameEx (in /usr/lib64/libpython2.7.so.1.0)
==25650==    by 0x4F171AC: PyEval_EvalCodeEx (in /usr/lib64/libpython2.7.so.1.0)
==25650==    by 0x4EA4097: ??? (in /usr/lib64/libpython2.7.so.1.0)
==25650==    by 0x4E7F072: PyObject_Call (in /usr/lib64/libpython2.7.so.1.0)
==25650==    by 0x4E8E084: ??? (in /usr/lib64/libpython2.7.so.1.0)
==25650==    by 0x4E7F072: PyObject_Call (in /usr/lib64/libpython2.7.so.1.0)
==25650==    by 0x4F1334B: PyEval_EvalFrameEx (in /usr/lib64/libpython2.7.so.1.0)
==25650==    by 0x4F171AC: PyEval_EvalCodeEx (in /usr/lib64/libpython2.7.so.1.0)
==25650==    by 0x4EA4097: ??? (in /usr/lib64/libpython2.7.so.1.0)
==25650==
..
Comment 3 Pavel Moravec 2015-09-08 06:56:18 EDT
Created attachment 1071299 [details]
coredump

Coredump. (need to have debuginfo for python-2.7.5-16.el7.x86_64 and qpid-proton-0.9-4.el7.x86_64)

bt shows:

(gdb) frame 0
#0  pn_reactor_attachments (reactor=reactor@entry=0x0) at /usr/src/debug/qpid-proton-0.9/proton-c/src/reactor/reactor.c:119
119	  return reactor->attachments;
(gdb) list
114	  }
115	}
116	
117	pn_record_t *pn_reactor_attachments(pn_reactor_t *reactor) {
118	  assert(reactor);
119	  return reactor->attachments;
120	}
121	
122	pn_millis_t pn_reactor_get_timeout(pn_reactor_t *reactor) {
123	  assert(reactor);
(gdb) p reactor
$6 = (pn_reactor_t *) 0x0
(gdb) 


Why the assert on previous line did not work? It cant be due to some concurrent change / memory access rewriting the pointer to null, since:

(gdb) frame 1
#1  0x00007f3a0014b2e3 in _wrap_pn_reactor_attachments (self=<optimized out>, args=<optimized out>)
    at /usr/src/debug/qpid-proton-0.9/proton-c/bindings/python/cprotonPYTHON_wrap.c:23077
23077	    result = (pn_record_t *)pn_reactor_attachments(arg1);
(gdb) p arg1
$7 = (pn_reactor_t *) 0x0
(gdb) 


Can't be the cause some limit was reached (i.e. file descriptors, some AMQP connection/session/whatever pool exhausted)?
Comment 4 Pavel Moravec 2015-09-08 08:22:49 EDT
Btw. when changing ulimit -n (both to lower values or to higher), goferd does _not_ segfault. It just consumes more and more memory. Segfault is present only for ulimit -n 1024
Comment 5 Pavel Moravec 2015-09-08 10:50:11 EDT
(In reply to Pavel Moravec from comment #4)
> Btw. when changing ulimit -n (both to lower values or to higher), goferd
> does _not_ segfault. It just consumes more and more memory. Segfault is
> present only for ulimit -n 1024

I was wrong / too eager.

For ulimit -n approx. 900 or lower, memory growth stops after a while and no segfault

For ulimit -n 900 - 1024, memory growth until 86MB and segfault

For ulimit -n 1100 or more, memory growth until some higher limit (i.e. 90MB for 1100) or OOM killer.


The memory growth and segfault seems to be related to reconnect loop. Since when qdrouterd is started for a while and let goferd to connect to itself (until it is killed again), this resets the above scenario - finally stable memory usage starts to grow again.
Comment 8 Christian Horn 2015-10-19 03:29:26 EDT
(In reply to Pavel Moravec from comment #5)
> For ulimit -n approx. 900 or lower, memory growth stops after a while and no
> segfault
> For ulimit -n 900 - 1024, memory growth until 86MB and segfault
> For ulimit -n 1100 or more, memory growth until some higher limit (i.e. 90MB
> for 1100) or OOM killer.

Might be unrelated, but this reminds me of bz1242406 .
Comment 9 Ken Giusti 2015-10-19 11:25:00 EDT
Hi Pavel,

couple of quick points:

re: assert - that assert statement is removed by the compiler for release builds (NDEBUG is defined in that case)

re: leaks - these leaks look as if they are related to initializing an SSL connection, which is held by the pn_transport_t object.  Likely this indicates the leak involves the connection teardown process.
Comment 10 Pavel Moravec 2015-10-20 08:36:58 EDT
Standalone reproducer using proton reactor & proton only:

(assuming nothing listens on port 12345 on localhost):
(assuming some valid SSL certs/CA are provided to ca_certificate / client_certificate)

#!/usr/bin/python

from time import sleep

from proton import ConnectionException
from proton import SSLDomain, SSLException

from proton.utils import BlockingConnection

import traceback


ca_certificate='/etc/rhsm/ca/katello-default-ca.pem'
client_certificate='/etc/pki/consumer/bundle.pem'
client_key=None
domain = SSLDomain(SSLDomain.MODE_CLIENT)
domain.set_trusted_ca_db(ca_certificate)
domain.set_credentials(
  client_certificate,
  client_key or client_certificate, None)
domain.set_peer_authentication(SSLDomain.VERIFY_PEER)

while True:
  sleep(0.1)
  try:
    conn = BlockingConnection("amqps://localhost:12345", ssl_domain=domain, heartbeat=2)
  except ConnectionException, e:
    print e
    print(traceback.format_exc())
Comment 11 Gordon Sim 2015-10-21 15:49:57 EDT
I've committed two fixes upstream that I believe address the leaks:

https://git-wip-us.apache.org/repos/asf?p=qpid-proton.git;h=c799a29
https://git-wip-us.apache.org/repos/asf?p=qpid-proton.git;h=bbba61a
Comment 12 Ted Ross 2015-10-21 16:48:19 EDT
A fix is available upstream:  https://issues.apache.org/jira/browse/PROTON-1028
Comment 13 Pavel Moravec 2015-10-22 07:45:50 EDT
(In reply to Gordon Sim from comment #11)
> I've committed two fixes upstream that I believe address the leaks:
> 
> https://git-wip-us.apache.org/repos/asf?p=qpid-proton.git;h=c799a29
> https://git-wip-us.apache.org/repos/asf?p=qpid-proton.git;h=bbba61a

I can confirm backporting the them fixes both the standalone reproducer and also goferd reproducer. Thanks!

For backporting attempt: please apply this patch _after_ applying patch for bz1272596 that makes changes in very adjacent areas.
Comment 14 Justin Sherrill 2015-12-09 10:44:16 EST
Moving this to 6.1.z I don't see any reason here why it shouldn't be.  If for some reason the patches cannot be backported we can punt.
Comment 21 Tazim Kolhar 2016-01-18 02:05:44 EST
VERIFIED:

# rpm -qa | grep foreman
ruby193-rubygem-foreman_discovery-2.0.0.23-1.el7sat.noarch
cloud-qe-6.idmqe.lab.eng.bos.redhat.com-foreman-client-1.0-1.noarch
cloud-qe-6.idmqe.lab.eng.bos.redhat.com-foreman-proxy-client-1.0-1.noarch
foreman-gce-1.7.2.50-1.el7sat.noarch
rubygem-hammer_cli_foreman-0.1.4.15-1.el7sat.noarch
foreman-1.7.2.50-1.el7sat.noarch
ruby193-rubygem-foreman_docker-1.2.0.24-1.el7sat.noarch
ruby193-rubygem-foreman-tasks-0.6.15.7-1.el7sat.noarch
rubygem-hammer_cli_foreman_bootdisk-0.1.2.7-1.el7sat.noarch
rubygem-hammer_cli_foreman_docker-0.0.3.10-1.el7sat.noarch
foreman-selinux-1.7.2.17-1.el7sat.noarch
foreman-libvirt-1.7.2.50-1.el7sat.noarch
foreman-compute-1.7.2.50-1.el7sat.noarch
foreman-ovirt-1.7.2.50-1.el7sat.noarch
ruby193-rubygem-foreman-redhat_access-0.2.4-1.el7sat.noarch
rubygem-hammer_cli_foreman_discovery-0.0.1.10-1.el7sat.noarch
ruby193-rubygem-foreman_bootdisk-4.0.2.14-1.el7sat.noarch
foreman-vmware-1.7.2.50-1.el7sat.noarch
rubygem-hammer_cli_foreman_tasks-0.0.3.5-1.el7sat.noarch
foreman-debug-1.7.2.50-1.el7sat.noarch
foreman-proxy-1.7.2.8-1.el7sat.noarch
ruby193-rubygem-foreman_hooks-0.3.7-2.el7sat.noarch
ruby193-rubygem-foreman_gutterball-0.0.1.9-1.el7sat.noarch
foreman-postgresql-1.7.2.50-1.el7sat.noarch
cloud-qe-6.idmqe.lab.eng.bos.redhat.com-foreman-proxy-1.0-1.noarch



steps:
1. setting DELAY = 1 and MAX_DELAY = 1 in /usr/lib/python2.7/site-packages/gofer/messaging/adapter/proton/connection.py

2. (on Satellite)
service qdrouterd stop

3. (on Capsule)
while true; do ps aux | grep python | grep gofer; sleep 5; done
]# while true; do ps aux | grep python | grep gofer; sleep 5; done
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:04 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:04 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:04 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:04 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:04 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:04 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:04 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:04 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:04 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:04 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:04 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:04 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:04 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:04 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:04 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:04 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:04 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:04 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:04 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:04 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:04 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:04 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:04 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:04 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:04 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:04 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:04 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:04 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:04 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:04 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:04 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:04 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:04 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:04 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:04 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:04 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:04 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:04 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:04 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:04 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:04 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:04 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:04 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:04 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:04 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:04 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:04 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:04 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:04 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:04 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:04 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:04 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:04 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:04 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:04 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:04 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:04 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:04 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:04 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.4  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:05 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground
root      8189 39.3  0.2 1939920 44804 ?       Ssl  Jan08 5604:06 python /usr/bin/goferd --foreground

As, per my understanding, there is no memory accumulation and 
goferd process seems to be constant

Also, as I was monitoring /var/log/messages on both sat6 and capsule
did not face any seg fault issue

moving it to verified ...

Thanks !!
Comment 23 errata-xmlrpc 2016-01-21 02:51:25 EST
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2016:0051

Note You need to log in before you can comment on or make changes to this bug.