Note: This bug is displayed in read-only format because
the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Description of problem:
System log files filling up, 100% usage from xinetd on one CPU and 50% from rsyslod and a second CPU.
Version-Release number of selected component (if applicable):
Red Hat Enterprise Linux 6.4
How reproducible:
Easily
Steps to Reproduce:
Make the same TFTP (/etc/xinet.d) setting same as MSC-G.
MSC-G version:
------------------------------------------------------------------
# default: off
# description: The tftp server serves files using the trivial file transfer \
# protocol. The tftp protocol is often used to boot diskless \
# workstations, download configuration files to network-aware printers, \
# and to start the installation process for some operating systems.
service tftp
{
socket_type = dgram
protocol = udp
wait = no
user = root
server = /usr/sbin/in.tftpd
server_args = -s /flx/boot
disable = no
per_source = 11
cps = 100 2
flags = IPv4
}
------------------------------------------------------------------
Compared the MSC-A tftp configure (also enabled), and found the "wait" parameter was set to yes, so the issue didn't occur in MSC-A
Below was the default version of tftp settings:
------------------------------------------------------------------
# default: off
# description: The tftp server serves files using the trivial file transfer \
# protocol. The tftp protocol is often used to boot diskless \
# workstations, download configuration files to network-aware printers, \
# and to start the installation process for some operating systems.
service tftp
{
socket_type = dgram
protocol = udp
wait = yes
user = root
server = /usr/sbin/in.tftpd
server_args = -s /flx/boot
disable = yes
per_source = 11
cps = 100 2
flags = IPv4
}
------------------------------------------------------------------
> Actual results:
System log files filling up, 100% usage from xinetd on one CPU and 50% from rsyslod and a second CPU.
> Expected results:> Additional info:
When restarting the xinitd process, following in omp_daemon.log shortly after
---------------------------------------------------------------------------
-rw------- 1 root root 946691 Oct 27 07:38 syslog
-rw------- 1 root root 430697836 Oct 27 07:38 omp_daemon.log
-rw------- 1 root root 63252677 Oct 27 07:38 daemon.log
---------------------------------------------------------------------------
Oct 26 14:15:57 ompgam xinetd[10912]: xinetd Version 2.3.14 started with libwra!
p loadavg labeled-networking options compiled in.
Oct 26 14:15:57 ompgam xinetd[10912]: Started working: 26 available services
Oct 26 14:20:47 ompgam xinetd[10912]: START: tftp pid=24301 from=135.2.12.198
Oct 26 14:20:47 ompgam xinetd[10912]: file descriptor of service evdo_rpcmdview!
er7061 has been closed
Oct 26 14:20:47 ompgam xinetd[10912]: select reported EBADF but no bad file des!
criptors were found
Oct 26 14:20:47 ompgam xinetd[10912]: 1 descriptors still set
Oct 26 14:20:47 ompgam xinetd[10912]: No active service for file descriptor 25
Oct 26 14:20:47 ompgam xinetd[10912]: 1 descriptors still set
Oct 26 14:20:47 ompgam xinetd[10912]: No active service for file descriptor 25
Oct 26 14:20:47 ompgam xinetd[10912]: 1 descriptors still set
---------------------------------------------------------------------------
ls -l
total 982328
-rwxr-xr-x 1 nobody4 nogroup 62589977 Oct 27 07:33 daemon.log
-rwxr-xr-x 1 nobody4 nogroup 500025 Oct 27 07:33 daemon.log-20150803
-rwxr-xr-x 1 nobody4 nogroup 500007 Oct 27 07:33 daemon.log-20150920
-rwxr-xr-x 1 nobody4 nogroup 507808 Oct 27 07:33 daemon.log-20151118
-rwxr-xr-x 1 nobody4 nogroup 529336 Oct 27 07:33 daemon.log-20160720
-rwxr-xr-x 1 nobody4 nogroup 268008 Oct 27 07:33 daemon.log-20161025.bz2
-rwxr-xr-x 1 nobody4 nogroup 266653 Oct 27 07:33 daemon.log-20161026.bz2
-rwxr-xr-x 1 nobody4 nogroup 267211 Oct 27 07:33 daemon.log-20161027.bz2
-rwxr-xr-x 1 nobody4 nogroup 6267712 Oct 27 07:34 messages
-rwxr-xr-x 1 nobody4 nogroup 429753136 Oct 27 07:32 omp_daemon.log
-rwxr-xr-x 1 nobody4 nogroup 935719 Oct 27 07:33 syslog
---------------------------------------------------------------------------
Running the top command shows that xinetd and rsyslogd are using a lot of CPU
---------------------------------------------------------------------------
top - 07:36:48 up 1 day, 19:47, 50 users, load average: 1.81, 1.84, 1.81
Tasks: 651 total, 2 running, 647 sleeping, 0 stopped, 2 zombie
Cpu(s): 4.6%us, 7.4%sy, 0.0%ni, 87.8%id, 0.2%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 32866944k total, 29447216k used, 3419728k free, 334248k buffers
Swap: 2047996k total, 43324k used, 2004672k free, 27212612k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
10912 root 20 0 22132 980 740 R 99.9 0.0 1032:35 xinetd
3369 root 20 0 245m 4744 888 S 48.6 0.0 1259:34 rsyslogd
9585 root 20 0 267m 9.9m 3092 S 2.0 0.0 171:01.00 pcmdaemon
7678 root 20 0 15564 1876 1004 R 0.7 0.0 7:21.98 top
30040 bgcain 20 0 15536 1872 1004 S 0.7 0.0 7:27.55 top
182 root 39 19 0 0 0 S 0.3 0.0 18:50.61 kipmi0
1 root 20 0 19888 1972 1212 S 0.0 0.0 0:06.75 init
2 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kthreadd
3 root RT 0 0 0 0 S 0.0 0.0 0:00.29 migration/0
4 root 20 0 0 0 0 S 0.0 0.0 0:06.82 ksoftirqd/0
5 root RT 0 0 0 0 S 0.0 0.0 0:00.00 migration/0
6 root RT 0 0 0 0 S 0.0 0.0 0:00.06 watchdog/0
7 root RT 0 0 0 0 S 0.0 0.0 0:03.75 migration/1
8 root RT 0 0 0 0 S 0.0 0.0 0:00.00 migration/1
---------------------------------------------------------------------------
----------------------------------
VMAJOR=1
VMINOR=1
# Add your favorite options here
#OPTS="$OPTS -s 0 -i 10 -r"
# uncomment and modify this to check the state of the AC adapter
#OPTS="$OPTS -a /proc/acpi/ac_adapter/*/state"
# uncomment and modify this to check the system temperature
#OPTS="$OPTS -t /proc/acpi/thermal_zone/*/temperature 75"
----------------------------------
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 44
model name : Intel(R) Xeon(R) CPU L5638 @ 2.00GHz
stepping : 2
cpu MHz : 2000.178
cache size : 12288 KB
physical id : 0
siblings : 6
core id : 0
cpu cores : 6
apicid : 0
initial apicid : 0
fpu : yes
fpu_exception : yes
cpuid level : 11
wp : yes
----------------------------------
#
# This is the master xinetd configuration file. Settings in the
# default section will be inherited by all service configurations
# unless explicitly overridden in the service configuration. See
# xinetd.conf in the man pages for a more detailed explanation of
# these attributes.
defaults
{
# The next two items are intended to be a quick access place to
# temporarily enable or disable services.
#
# enabled =
# disabled =
# Define general logging characteristics.
log_type = SYSLOG daemon info
log_on_failure = HOST
log_on_success = PID HOST DURATION EXIT
# Define access restriction defaults
#
# no_access =
# only_from =
# max_load = 0
cps = 50 10
instances = 50
per_source = 10
# Address and networking defaults
#
# bind =
# mdns = yes
v6only = no
# setup environmental attributes
#
# passenv =
groups = yes
umask = 002
# Generally, banners are not used. This sets up their global defaults
#
# banner =
# banner_fail =
# banner_success =
}
includedir /etc/xinetd.d
----------------------------------
# default: off
# description: The tftp server serves files using the trivial file transfer \
# protocol. The tftp protocol is often used to boot diskless \
# workstations, download configuration files to network-aware printers, \
# and to start the installation process for some operating systems.
service tftp
{
socket_type = dgram
protocol = udp
wait = no
user = root
server = /usr/sbin/in.tftpd
server_args = -s /flx/boot
----------------------------------
Red Hat Enterprise Linux 6 shipped it's last Production 2 phase minor
release, RHEL 6.9, on March 21, 2016. On May 10, 2017, RHEL 6 exits
Production 2 phase and moves into Production 3 phase. For RHEL releases
in Production 3 phase, Red Hat will provide critical-impact security
fixes and urgent priority bug fixes for the last minor release but will
not provide any software enhancements or hardware enablement.
This BZ does not appear to meet the Product 3 phase inclusion criteria
described above so is being closed WONTFIX. If this BZ is critical for
your environment, please open a case in the Red Hat Customer Portal,
https://access.redhat.com, provide a thorough business justification and
ask that the BZ be re-opened for consideration. Please note, only
critical-impact security fixes and urgent priority bug fixes will be
considered, and no software enhancements or hardware enablement will be
performed.
Description of problem: System log files filling up, 100% usage from xinetd on one CPU and 50% from rsyslod and a second CPU. Version-Release number of selected component (if applicable): Red Hat Enterprise Linux 6.4 How reproducible: Easily Steps to Reproduce: Make the same TFTP (/etc/xinet.d) setting same as MSC-G. MSC-G version: ------------------------------------------------------------------ # default: off # description: The tftp server serves files using the trivial file transfer \ # protocol. The tftp protocol is often used to boot diskless \ # workstations, download configuration files to network-aware printers, \ # and to start the installation process for some operating systems. service tftp { socket_type = dgram protocol = udp wait = no user = root server = /usr/sbin/in.tftpd server_args = -s /flx/boot disable = no per_source = 11 cps = 100 2 flags = IPv4 } ------------------------------------------------------------------ Compared the MSC-A tftp configure (also enabled), and found the "wait" parameter was set to yes, so the issue didn't occur in MSC-A Below was the default version of tftp settings: ------------------------------------------------------------------ # default: off # description: The tftp server serves files using the trivial file transfer \ # protocol. The tftp protocol is often used to boot diskless \ # workstations, download configuration files to network-aware printers, \ # and to start the installation process for some operating systems. service tftp { socket_type = dgram protocol = udp wait = yes user = root server = /usr/sbin/in.tftpd server_args = -s /flx/boot disable = yes per_source = 11 cps = 100 2 flags = IPv4 } ------------------------------------------------------------------ > Actual results: System log files filling up, 100% usage from xinetd on one CPU and 50% from rsyslod and a second CPU. > Expected results: > Additional info: When restarting the xinitd process, following in omp_daemon.log shortly after --------------------------------------------------------------------------- -rw------- 1 root root 946691 Oct 27 07:38 syslog -rw------- 1 root root 430697836 Oct 27 07:38 omp_daemon.log -rw------- 1 root root 63252677 Oct 27 07:38 daemon.log --------------------------------------------------------------------------- Oct 26 14:15:57 ompgam xinetd[10912]: xinetd Version 2.3.14 started with libwra! p loadavg labeled-networking options compiled in. Oct 26 14:15:57 ompgam xinetd[10912]: Started working: 26 available services Oct 26 14:20:47 ompgam xinetd[10912]: START: tftp pid=24301 from=135.2.12.198 Oct 26 14:20:47 ompgam xinetd[10912]: file descriptor of service evdo_rpcmdview! er7061 has been closed Oct 26 14:20:47 ompgam xinetd[10912]: select reported EBADF but no bad file des! criptors were found Oct 26 14:20:47 ompgam xinetd[10912]: 1 descriptors still set Oct 26 14:20:47 ompgam xinetd[10912]: No active service for file descriptor 25 Oct 26 14:20:47 ompgam xinetd[10912]: 1 descriptors still set Oct 26 14:20:47 ompgam xinetd[10912]: No active service for file descriptor 25 Oct 26 14:20:47 ompgam xinetd[10912]: 1 descriptors still set --------------------------------------------------------------------------- ls -l total 982328 -rwxr-xr-x 1 nobody4 nogroup 62589977 Oct 27 07:33 daemon.log -rwxr-xr-x 1 nobody4 nogroup 500025 Oct 27 07:33 daemon.log-20150803 -rwxr-xr-x 1 nobody4 nogroup 500007 Oct 27 07:33 daemon.log-20150920 -rwxr-xr-x 1 nobody4 nogroup 507808 Oct 27 07:33 daemon.log-20151118 -rwxr-xr-x 1 nobody4 nogroup 529336 Oct 27 07:33 daemon.log-20160720 -rwxr-xr-x 1 nobody4 nogroup 268008 Oct 27 07:33 daemon.log-20161025.bz2 -rwxr-xr-x 1 nobody4 nogroup 266653 Oct 27 07:33 daemon.log-20161026.bz2 -rwxr-xr-x 1 nobody4 nogroup 267211 Oct 27 07:33 daemon.log-20161027.bz2 -rwxr-xr-x 1 nobody4 nogroup 6267712 Oct 27 07:34 messages -rwxr-xr-x 1 nobody4 nogroup 429753136 Oct 27 07:32 omp_daemon.log -rwxr-xr-x 1 nobody4 nogroup 935719 Oct 27 07:33 syslog --------------------------------------------------------------------------- Running the top command shows that xinetd and rsyslogd are using a lot of CPU --------------------------------------------------------------------------- top - 07:36:48 up 1 day, 19:47, 50 users, load average: 1.81, 1.84, 1.81 Tasks: 651 total, 2 running, 647 sleeping, 0 stopped, 2 zombie Cpu(s): 4.6%us, 7.4%sy, 0.0%ni, 87.8%id, 0.2%wa, 0.0%hi, 0.0%si, 0.0%st Mem: 32866944k total, 29447216k used, 3419728k free, 334248k buffers Swap: 2047996k total, 43324k used, 2004672k free, 27212612k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 10912 root 20 0 22132 980 740 R 99.9 0.0 1032:35 xinetd 3369 root 20 0 245m 4744 888 S 48.6 0.0 1259:34 rsyslogd 9585 root 20 0 267m 9.9m 3092 S 2.0 0.0 171:01.00 pcmdaemon 7678 root 20 0 15564 1876 1004 R 0.7 0.0 7:21.98 top 30040 bgcain 20 0 15536 1872 1004 S 0.7 0.0 7:27.55 top 182 root 39 19 0 0 0 S 0.3 0.0 18:50.61 kipmi0 1 root 20 0 19888 1972 1212 S 0.0 0.0 0:06.75 init 2 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kthreadd 3 root RT 0 0 0 0 S 0.0 0.0 0:00.29 migration/0 4 root 20 0 0 0 0 S 0.0 0.0 0:06.82 ksoftirqd/0 5 root RT 0 0 0 0 S 0.0 0.0 0:00.00 migration/0 6 root RT 0 0 0 0 S 0.0 0.0 0:00.06 watchdog/0 7 root RT 0 0 0 0 S 0.0 0.0 0:03.75 migration/1 8 root RT 0 0 0 0 S 0.0 0.0 0:00.00 migration/1 --------------------------------------------------------------------------- ---------------------------------- VMAJOR=1 VMINOR=1 # Add your favorite options here #OPTS="$OPTS -s 0 -i 10 -r" # uncomment and modify this to check the state of the AC adapter #OPTS="$OPTS -a /proc/acpi/ac_adapter/*/state" # uncomment and modify this to check the system temperature #OPTS="$OPTS -t /proc/acpi/thermal_zone/*/temperature 75" ---------------------------------- processor : 0 vendor_id : GenuineIntel cpu family : 6 model : 44 model name : Intel(R) Xeon(R) CPU L5638 @ 2.00GHz stepping : 2 cpu MHz : 2000.178 cache size : 12288 KB physical id : 0 siblings : 6 core id : 0 cpu cores : 6 apicid : 0 initial apicid : 0 fpu : yes fpu_exception : yes cpuid level : 11 wp : yes ---------------------------------- # # This is the master xinetd configuration file. Settings in the # default section will be inherited by all service configurations # unless explicitly overridden in the service configuration. See # xinetd.conf in the man pages for a more detailed explanation of # these attributes. defaults { # The next two items are intended to be a quick access place to # temporarily enable or disable services. # # enabled = # disabled = # Define general logging characteristics. log_type = SYSLOG daemon info log_on_failure = HOST log_on_success = PID HOST DURATION EXIT # Define access restriction defaults # # no_access = # only_from = # max_load = 0 cps = 50 10 instances = 50 per_source = 10 # Address and networking defaults # # bind = # mdns = yes v6only = no # setup environmental attributes # # passenv = groups = yes umask = 002 # Generally, banners are not used. This sets up their global defaults # # banner = # banner_fail = # banner_success = } includedir /etc/xinetd.d ---------------------------------- # default: off # description: The tftp server serves files using the trivial file transfer \ # protocol. The tftp protocol is often used to boot diskless \ # workstations, download configuration files to network-aware printers, \ # and to start the installation process for some operating systems. service tftp { socket_type = dgram protocol = udp wait = no user = root server = /usr/sbin/in.tftpd server_args = -s /flx/boot ----------------------------------