Bug 1330598 - lxc: /proc/meminfo / free 'total' becomes huge (9007199254740991)
Summary: lxc: /proc/meminfo / free 'total' becomes huge (9007199254740991)
Keywords:
Status: NEW
Alias: None
Product: Virtualization Tools
Classification: Community
Component: libvirt
Version: unspecified
Hardware: x86_64
OS: Linux
unspecified
unspecified
Target Milestone: ---
Assignee: Libvirt Maintainers
QA Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-04-26 14:33 UTC by Maxim
Modified: 2018-07-18 14:52 UTC (History)
1 user (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed:
Embargoed:


Attachments (Terms of Use)

Description Maxim 2016-04-26 14:33:20 UTC
Description of problem:
Command "free" inside LXC container show 8PTb (9007199254740991) total memory.

Version-Release number of selected component (if applicable):
libvirt 1.3.2
build from sources with next options
--without qemu \
 --without xen  \
 --without openvz \
 --without vmware \
 --without esx \
 --without parallels \
 --without bhyve \
 --without uml \
 --without vbox \
 --without selinux \
 --with lxc    \
 --with macvtap \
 --without storage-rbd \
 --with capng
Run on CentOS Linux release 7.2.1511 (Core), kernel 3.10.0-327.13.1.el7.x86_64

How reproducible:
permanently

Steps to Reproduce:
1. create and start LXC container with memory limit 1G
2. create on top hardware node systemd service:
# cat   /usr/lib/systemd/system/true.service
[Unit]
Description=simple test

[Service]
Type=simple
ExecStart=/bin/true

[Install]
WantedBy=multi-user.target

3. Enable service first time, disable and start:
# systemctl enable   /usr/lib/systemd/system/true.service
Created symlink from
/etc/systemd/system/multi-user.target.wants/true.service to
/usr/lib/systemd/system/true.service.
# systemctl disable true.service
Removed symlink /etc/systemd/system/multi-user.target.wants/true.service.
# systemctl start true.service

Actual results:
inside container
[root@LXC]# free
              total        used        free      shared  buff/cache   available
Mem:    9007199254740991      193168 9007199254226743       11112      321080 9007199254226743
Swap:             0           0           0

Expected results:
inside container
[root@LXC]# free
              total        used        free      shared  buff/cache   available
Mem:        1048576        7668     1022576       11112       18332     1022576
Swap:       1048576           0     1048576


Additional info:
while broken, PID of libvirt LXC process removed from cgroups.
Example.
Before failure:

[root@node01]# cat
/sys/fs/cgroup/memory/machine.slice/machine-lxc\\x2d7445\\x2dtst\\x2dmxs2.test.scope/memory.limit_in_bytes
1073741824
4) Check list of pid in cgroups (it's IMPOTANT moment):
[root@node01]# cat
/sys/fs/cgroup/memory/machine.slice/machine-lxc\\x2d7445\\x2dtst\\x2dmxs2.test.scope/tasks
7445
7446
7480
7506
7510
7511
7512
7529
7532
7533
7723
7724
8251
8253
10455

First PID 7445 - it's pid of  libvirt process for container:
# ps ax | grep 7445
 7445 ?        Sl     0:00 /usr/libexec/libvirt_lxc --name
tst-mxs2.test --console 21 --security=none --handshake 24 --veth
macvlan5
[root@node01]# virsh list
 Id    Name                           State
----------------------------------------------------
 7445  tst-mxs2.test                running

After failure:
root@node01]# cat
/sys/fs/cgroup/memory/machine.slice/machine-lxc\\x2d7445\\x2dtst\\x2dmxs2.test.scope/tasks
7446
7480
7506
7510
7511
7512
7529
7532
7533
7723
7724
8251
8253

After start disabled systemd service, from task list removed libvirt PID 7445.
It's mean that inside LXC limit real still worked, 7446 - it's PID of
/sbin/init inside container.

Workaround:
 Broken only fuse mount.  It's positive news - process inside container
even in case 8Ptb can't allocate more memory that set in cgroups.
But negative news - that some java based sotfware (as puppetdb in our
case)  plan self strategy based on 8Ptb memory and collapsed after
reach real limit.

Run cglassify or equivalent, example:
[root@node01]# echo 7445 >
/sys/fs/cgroup/memory/machine.slice/machine-lxc\\x2d7445\\x2dtst\\x2dmxs2.test.scope/tasks


Note You need to log in before you can comment on or make changes to this bug.