Bug 1383150

Summary: kernel log message confused when both container and localhost running syslog daemon at rhel6
Product: Red Hat Enterprise Linux 6 Reporter: muahao <muahao>
Component: rhel-server-containerAssignee: Frantisek Kluknavsky <fkluknav>
Status: CLOSED WONTFIX QA Contact: Luwen Su <lsu>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 6.2CC: muahao
Target Milestone: rcFlags: fkluknav: needinfo? (muahao)
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-10-14 19:08:13 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description muahao 2016-10-10 04:23:59 UTC 
**Description**

I got a problem ,If I running a container(running rhel6.2) on "Operating System"(rhel6.2), At the both time container and Host both running syslog-ng or rsyslog, If I trigger some kernel log message at localhost ,the kernel message outputted are confused at localhost ! If I stop the container's syslog-ng/rsyslog,and at localhost I trigger some kernel log messages ,The log message will be output ok

**Steps to reproduce the issue:**

1.

Host's OS version:rhel6.2

container's OS version:rhel6.2

2.

984c3ead3fc3 is a containner running on dnstest07.tbc

Both of them running syslog-ng 

```
[root /home/ahao.mah]
#ps aux | grep syslog
root      95792  0.0  0.0  61748  1140 ?        S    Oct09   0:00 supervising syslog-ng
root      95793  0.0  0.0  76984  3492 ?        Ss   Oct09   0:08 syslog-ng -p /var/run/syslog-ng.pid
```

```
[root@984c3ead3fc3 /]
#ps aux | grep syslog
root       1019  0.0  0.0  46608   804 ?        S    11:32   0:00 supervising syslog-ng
root       1020  0.0  0.0  48844  2532 ?        Ss   11:32   0:00 /sbin/syslog-ng
root       1024  0.0  0.0 103232   868 ?        S+   11:32   0:00 grep syslog
```

trigger some kernel log messages at localhost:

```
[root /home/ahao.mah]
#echo m > /proc/sysrq-trigger
```

Localhost only output a little confused kernel log messages:

```
[root /home/ahao.mah]
#tail /var/log/kern -f

Oct 10 11:33:19 dnstest07.tbc kernel: : [4563236.004706] CPU    9: hi:  186, btch:  31 usd:   0
Oct 10 11:33:19 dnstest07.tbc kernel: : [4563236.018333] CPU   10: hi:  186, btch:  31 usd:   0
Oct 10 11:33:19 dnstest07.tbc kernel: : [4563236.036786] CPU   11: hi:  186, btch:  31 usd:   0
Oct 10 11:33:19 dnstest07.tbc kernel: : [4563236.548 P  2 i 16 th 3 s: 0<>4626090]CU 1:h: 8,bc: 1ud
Oct 10 11:33:19 dnstest07.tbc kernel: : [4563236.395181] CPU    7: hi:  186, btch:  31 usd: 179
Oct 10 11:33:19 dnstest07.tbc kernel: : [4563236.408570] CPU    8: hi:  186, btch:  31 usd: 168
Oct 10 11:33:19 dnstest07.tbc kernel: : [4563236.427218] CPU    9: hi:  186, btch:  31 us:11<>4626452]CU 1:h: 8,bc: 1ud 4
Oct 10 11:33:19 dnstest07.tbc kernel: : [4563236.547812] CPU   16: hi:  186, btch:  31 usd: 139

```

If I stop container's syslog-ng deamon:

```
[root@984c3ead3fc3 /]
#service syslog-ng stop
Shutting down syslog-ng:                                   [  OK  ]

[root@984c3ead3fc3 /]
#ps aux | grep syslog
root       1190  0.0  0.0 103232   872 ?        S+   11:34   0:00 grep syslog

```

trigger some kernel log messages at localhost,outputs is very abnormal:

```
[root /home/ahao.mah]
#echo m > /proc/sysrq-trigger
```

```
[root /home/ahao.mah]
#tail /var/log/kern -f
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563337.128442] SysRq : Show Memory
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563337.134077] Mem-Info:
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563337.138281] Node 0 DMA per-cpu:
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563337.154046] CPU    0: hi:    0, btch:   1 usd:   0
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563337.167591] CPU    1: hi:    0, btch:   1 usd:   0
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563337.186477] CPU    2: hi:    0, btch:   1 usd:   0
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563337.204830] CPU    3: hi:    0, btch:   1 usd:   0
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563337.218328] CPU    4: hi:    0, btch:   1 usd:   0
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563337.237091] CPU    5: hi:    0, btch:   1 usd:   0
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563337.255846] CPU    6: hi:    0, btch:   1 usd:   0
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563337.274430] CPU    7: hi:    0, btch:   1 usd:   0
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563337.287778] CPU    8: hi:    0, btch:   1 usd:   0
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563337.306424] CPU    9: hi:    0, btch:   1 usd:   0
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563337.325115] CPU   10: hi:    0, btch:   1 usd:   0
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563337.343748] CPU   11: hi:    0, btch:   1 usd:   0
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563337.357563] CPU   12: hi:    0, btch:   1 usd:   0
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563337.375867] CPU   13: hi:    0, btch:   1 usd:   0
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563337.394600] CPU   14: hi:    0, btch:   1 usd:   0
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563337.408425] CPU   15: hi:    0, btch:   1 usd:   0
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563337.426972] CPU   16: hi:    0, btch:   1 usd:   0
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563337.445402] CPU   17: hi:    0, btch:   1 usd:   0
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563337.463876] CPU   18: hi:    0, btch:   1 usd:   0
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563337.477447] CPU   19: hi:    0, btch:   1 usd:   0
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563337.496279] CPU   20: hi:    0, btch:   1 usd:   0
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563337.514676] CPU   21: hi:    0, btch:   1 usd:   0
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563337.528193] CPU   22: hi:    0, btch:   1 usd:   0
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563337.547067] CPU   23: hi:    0, btch:   1 usd:   0
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563337.565635] Node 0 DMA32 per-cpu:
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563337.576589] CPU    0: hi:  186, btch:  31 usd: 169
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563337.595159] CPU    1: hi:  186, btch:  31 usd:   0
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563337.613706] CPU    2: hi:  186, btch:  31 usd:   0
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563337.627306] CPU    3: hi:  186, btch:  31 usd:   0
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563337.646101] CPU    4: hi:  186, btch:  31 usd:   0
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563337.664527] CPU    5: hi:  186, btch:  31 usd:   0
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563337.678035] CPU    6: hi:  186, btch:  31 usd:   0
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563337.696881] CPU    7: hi:  186, btch:  31 usd:   0
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563337.715403] CPU    8: hi:  186, btch:  31 usd:   0
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563337.734051] CPU    9: hi:  186, btch:  31 usd:   0
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563337.747667] CPU   10: hi:  186, btch:  31 usd:   0
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563337.766131] CPU   11: hi:  186, btch:  31 usd:   0
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563337.784820] CPU   12: hi:  186, btch:  31 usd:   0
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563337.798526] CPU   13: hi:  186, btch:  31 usd:   0
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563337.817014] CPU   14: hi:  186, btch:  31 usd:   0
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563337.835693] CPU   15: hi:  186, btch:  31 usd:   0
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563337.854353] CPU   16: hi:  186, btch:  31 usd:   0
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563337.868178] CPU   17: hi:  186, btch:  31 usd:   0
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563337.886566] CPU   18: hi:  186, btch:  31 usd:   0
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563337.905185] CPU   19: hi:  186, btch:  31 usd:   0
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563337.923729] CPU   20: hi:  186, btch:  31 usd:   0
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563337.937297] CPU   21: hi:  186, btch:  31 usd:   0
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563337.956120] CPU   22: hi:  186, btch:  31 usd:   0
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563337.974534] CPU   23: hi:  186, btch:  31 usd: 179
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563337.988021] Node 0 Normal per-cpu:
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563338.004401] CPU    0: hi:  186, btch:  31 usd:  13
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563338.017873] CPU    1: hi:  186, btch:  31 usd: 117
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563338.036570] CPU    2: hi:  186, btch:  31 usd: 126
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563338.055173] CPU    3: hi:  186, btch:  31 usd:  36
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563338.073724] CPU    4: hi:  186, btch:  31 usd:  73
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563338.087307] CPU    5: hi:  186, btch:  31 usd: 134
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563338.106173] CPU    6: hi:  186, btch:  31 usd:  28
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563338.124754] CPU    7: hi:  186, btch:  31 usd: 160
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563338.138035] CPU    8: hi:  186, btch:  31 usd: 155
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563338.156728] CPU    9: hi:  186, btch:  31 usd: 110
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563338.175414] CPU   10: hi:  186, btch:  31 usd:  42
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563338.194050] CPU   11: hi:  186, btch:  31 usd: 110
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563338.207767] CPU   12: hi:  186, btch:  31 usd: 128
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563338.226145] CPU   13: hi:  186, btch:  31 usd: 162
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563338.244819] CPU   14: hi:  186, btch:  31 usd:  95
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563338.258458] CPU   15: hi:  186, btch:  31 usd: 172
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563338.277304] CPU   16: hi:  186, btch:  31 usd: 180
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563338.295716] CPU   17: hi:  186, btch:  31 usd:  60
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563338.314379] CPU   18: hi:  186, btch:  31 usd: 158
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563338.327912] CPU   19: hi:  186, btch:  31 usd:  96
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563338.346564] CPU   20: hi:  186, btch:  31 usd: 144
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563338.365186] CPU   21: hi:  186, btch:  31 usd: 139
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563338.383722] CPU   22: hi:  186, btch:  31 usd:  71
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563338.397314] CPU   23: hi:  186, btch:  31 usd: 161
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563338.416104] active_anon:220624 inactive_anon:48 isolated_anon:0
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563338.416105]  active_file:1518776 inactive_file:840430 isolated_file:0
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563338.416106]  unevictable:0 dirty:89 writeback:0 unstable:0
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563338.416106]  free:8955795 slab_reclaimable:390911 slab_unreclaimable:54178
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563338.416107]  mapped:12886 shmem:123 pagetables:2946 bounce:0
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563338.525104] Node 0 DMA free:15748kB min:8kB low:8kB high:12kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15356kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
Oct 10 11:35:01 dnstest07.tbc kernel: : [4563338.648733] lowmem_reserve[]: 0 1947 48407 48407

```


3.


**Output of `docker version`:**

```
[root /home/ahao.mah]
#docker version
Client:
 Version:      1.9.1
 API version:  1.21
 Go version:   go1.4.3
 Git commit:   e21b33a BUILDTIME:2016-04-27 21:17:18
 Built:        Wed Apr 27 13:17:18 UTC 2016
 OS/Arch:      linux/amd64

Server:
 Version:      1.9.1
 API version:  1.21
 Go version:   go1.4.3
 Git commit:   e21b33a BUILDTIME:2016-04-27 21:17:18
 Built:        Wed Apr 27 13:17:18 UTC 2016
 OS/Arch:      linux/amd64
```

**Output of `docker info`:**

```
[root /home/ahao.mah]
#docker info
Containers: 9
Images: 42
Server Version: 1.9.1
Storage Driver: overlay
 Backing Filesystem: extfs
Execution Driver: lxc-0.6.8
Logging Driver: json-file
Kernel Version: 2.6.32-220.23.2.ali878.el6.x86_64
Operating System: <unknown>
CPUs: 24
AllocCPU: [1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]
Total Memory: 47.23 GiB
Name: dnstest07.tbc
ID: XFPL:USUJ:D3R6:FRRO:RJXB:FSM3:SR52:ZR2N:V2OY:KJUP:Q44H:KBK3
Debug mode (server): true
 File Descriptors: 18
 Goroutines: 35
 System Time: 2016-10-10T11:26:13.867348007+08:00
 EventsListeners: 0
 Init SHA1:
 Init Path: /usr/bin/docker
 Docker Root Dir: /home/docker
WARNING: bridge-nf-call-iptables is disabled
WARNING: bridge-nf-call-ip6tables is disabled
```
Comment 2 Frantisek Kluknavsky 2016-10-13 11:22:59 UTC 
Hi,

as far as I know, Red Hat does not ship nor support docker for rhel6 at all. Are you sure you do not have a typo in your bug report?
Comment 3 Frantisek Kluknavsky 2016-10-13 11:25:09 UTC 
(To clarify, we support rhel6 containers running on rhel7 hosts, if I am not mistaken.)
Comment 4 Frantisek Kluknavsky 2016-10-14 19:08:13 UTC 
It is confirmed, running docker on top of rhel6 kernel is not supported. The kernel is too old.