Bug 1015689

Summary: 'usage' attribute defaults to bytes when mounting ram disk in lxc... documented as kb default
Product: Red Hat Enterprise Linux 7 Reporter: David Vossel <dvossel>
Component: libvirtAssignee: Ján Tomko <jtomko>
Status: CLOSED CURRENTRELEASE QA Contact: Virtualization Bugs <virt-bugs>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 7.0CC: acathrow, adamgrayling, ajia, dallan, dyuan, eblake, fdinitto, lsu
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: libvirt-1.1.1-9.el7 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-06-13 10:47:35 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description David Vossel 2013-10-04 20:55:26 UTC
Description of problem:

http://libvirt.org/formatdomain.html#elementsFilesystems

"source
The resource on the host that is being accessed in the guest. The name attribute must be used with type='template', and the dir attribute must be used with type='mount'. The usage attribute is used with type='ram' to set the memory limit in KB."


    <filesystem type='ram'>
        <source usage='393216'/>
        <target dir='/dev/shm'/>
    </filesystem>

Putting the above config in my lxc xml config results in /dev/shm being allocated only 384kb, not 384mb.  Based on the documentation I would expect 384mb


Version-Release number of selected component (if applicable):

yum list installed | grep libvirt
fence-virtd-libvirt.x86_64             0.3.0-13.el7                    @r7      
libvirt.x86_64                         1.1.1-6.el7                     @r7      
libvirt-client.x86_64                  1.1.1-6.el7                     @r7      
libvirt-daemon.x86_64                  1.1.1-6.el7                     @r7      
libvirt-daemon-config-network.x86_64   1.1.1-6.el7                     @r7      
libvirt-daemon-config-nwfilter.x86_64  1.1.1-6.el7                     @r7      
libvirt-daemon-driver-interface.x86_64 1.1.1-6.el7                     @r7      
libvirt-daemon-driver-lxc.x86_64       1.1.1-6.el7                     @r7      
libvirt-daemon-driver-network.x86_64   1.1.1-6.el7                     @r7      
libvirt-daemon-driver-nodedev.x86_64   1.1.1-6.el7                     @r7      
libvirt-daemon-driver-nwfilter.x86_64  1.1.1-6.el7                     @r7      
libvirt-daemon-driver-qemu.x86_64      1.1.1-6.el7                     @r7      
libvirt-daemon-driver-secret.x86_64    1.1.1-6.el7                     @r7      
libvirt-daemon-driver-storage.x86_64   1.1.1-6.el7                     @r7 

How reproducible:

Define a in-memory filesystem in the lxc config.
 
    <filesystem type='ram'>
        <source usage='393216'/>
        <target dir='/dev/shm'/>
    </filesystem>


Within the live lxc domain, execute 'df', you'll see /dev/shm only has 384kb allocated instead of 384mb.

Comment 2 Eric Blake 2013-10-08 17:55:37 UTC
We must match the existing documentation.  Consistency with all other memory-related parameters, which are also output in k, says that the bug is in treating the value as bytes when it should be handled in k.  The first attempt at a patch:
https://www.redhat.com/archives/libvir-list/2013-October/msg00361.html
doesn't seem right to me, as it tries to reject XML that was previously output by older libvirt, which is not backwards-compatible.

Comment 3 Ján Tomko 2013-10-09 13:22:21 UTC
Another upstream patch attempted:
https://www.redhat.com/archives/libvir-list/2013-October/msg00415.html

Missing units default to KiB, and XML generated by older libvirt is parsed correctly.

Both of them also fix the bug of libvirt not being able to parse the units in its own generated XML correctly.

Comment 4 Ján Tomko 2013-10-09 15:46:54 UTC
The third version of the patch has been pushed upstream:
https://www.redhat.com/archives/libvir-list/2013-October/msg00435.html

commit 3f029fb5319b9dc9cc2fbf8d1ba4505ee9e4b1e3
Author:     Ján Tomko <jtomko>
AuthorDate: 2013-10-09 14:17:13 +0200
Commit:     Ján Tomko <jtomko>
CommitDate: 2013-10-09 17:44:45 +0200

    LXC: Fix handling of RAM filesystem size units
    
    Since 76b644c when the support for RAM filesystems was introduced,
    libvirt accepted the following XML:
    <source usage='1024' unit='KiB'/>
    
    This was parsed correctly and internally stored in bytes, but it
    was formatted as (with an extra 's'):
    <source usage='1024' units='KiB'/>
    When read again, this was treated as if the units were missing,
    meaning libvirt was unable to parse its own XML correctly.
    
    The usage attribute was documented as being in KiB, but it was not
    scaled if the unit was missing. Transient domains still worked,
    because this was balanced by an extra 'k' in the mount options.
    
    This patch:
    Changes the parser to use 'units' instead of 'unit', as the latter
    was never documented (fixing persistent domains) and some programs
    (libvirt-glib, libvirt-sandbox) already parse the 'units' attribute.
    
    Removes the extra 'k' from the tmpfs mount options, which is needed
    because now we parse our own XML correctly.
    
    Changes the default input unit to KiB to match documentation, fixing:
    https://bugzilla.redhat.com/show_bug.cgi?id=1015689

git describe: v1.1.3-77-g3f029fb

Comment 7 Luwen Su 2013-10-18 07:37:29 UTC
Verify with
libvirt-1.1.1-9.el7.x86_64

1.Prepare the lxc xml

<domain type='lxc'>
  <name>test</name>
  <uuid>3c77a62e-b868-4351-b837-c225c0a87592</uuid>
  <memory unit='KiB'>1048576</memory>
  <currentMemory unit='KiB'>1048576</currentMemory>
  <vcpu placement='static'>1</vcpu>
  <resource>
    <partition>/machine</partition>
  </resource>
  <os>
    <type arch='x86_64'>exe</type>
    <init>/bin/sh</init>
  </os>
  <clock offset='utc'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/libexec/libvirt_lxc</emulator>
    <filesystem type='ram' accessmode='passthrough'>
      <source usage='10240' units='KiB'/>
      <target dir='/mnt/mississippi'/>
    </filesystem>
    <filesystem type='ram' accessmode='passthrough'>
      <source usage='1024' units='KiB'/>
      <target dir='/mnt/antananarivo'/>
    </filesystem>
    <filesystem type='ram' accessmode='passthrough'>
      <source usage='4' units='KiB'/>
      <target dir='/mnt/ouagadougou'/>
    </filesystem>
    <filesystem type='ram' accessmode='passthrough'>
      <source usage='393216' units='KiB'/>
      <target dir='/dev/shm'/>
    </filesystem>
    <interface type='network'>
      <mac address='00:16:3e:87:67:8c'/>
      <source network='default'/>
    </interface>
    <console type='pty'>
      <target type='lxc' port='0'/>
    </console>
  </devices>
</domain>

2.
#virsh -c lxc:/// start test

3.
#virt-sandbox-service connect test
sh-4.2# df
Filesystem     1K-blocks    Used Available Use% Mounted on
/dev/sda1       41153856 2131960  36908360   6% /
tmpfs              10240       0     10240   0% /mnt/mississippi
tmpfs               1024       0      1024   0% /mnt/antananarivo
tmpfs                  4       0         4   0% /mnt/ouagadougou
tmpfs             393216       0    393216   0% /dev/shm

4.
If omits the units attribute , or change the unit to B , MiB , it will auto-complete and scaler properly , so Set VERIFIED

Comment 8 Ludek Smid 2014-06-13 10:47:35 UTC
This request was resolved in Red Hat Enterprise Linux 7.0.

Contact your manager or support representative in case you have further questions about the request.

Comment 10 Stormy 2021-03-27 15:40:50 UTC
> The third version of the patch has been pushed upstream:
> https://www.redhat.com/archives/libvir-list/2013-October/msg00435.html
> 
> commit 3f029fb5319b9dc9cc2fbf8d1ba4505ee9e4b1e3
> Author:     Ján Tomko <jtomko>
> AuthorDate: 2013-10-09 14:17:13 +0200
> Commit:     Ján Tomko <jtomko>
> CommitDate: 2013-10-09 17:44:45 +0200
> 
>     LXC: Fix handling of RAM filesystem size units
>     
>     Since 76b644c when the support for RAM filesystems was introduced,
>     libvirt accepted the following XML: https://www.dubaiescortsgirls.me/
>     <source usage='1024' unit='KiB'/>
>     
>     This was parsed correctly and internally stored in bytes, but it
>     was formatted as (with an extra 's'):
>     <source usage='1024' units='KiB'/>
>     When read again, this was treated as if the units were missing,
>     meaning libvirt was unable to parse its own XML correctly.
>     
>     The usage attribute was documented as being in KiB, but it was not
>     scaled if the unit was missing. Transient domains still worked,
>     because this was balanced by an extra 'k' in the mount options.
>     
>     This patch:
>     Changes the parser to use 'units' instead of 'unit', as the latter
>     was never documented (fixing persistent domains) and some programs
>     (libvirt-glib, libvirt-sandbox) already parse the 'units' attribute.
>     
>     Removes the extra 'k' from the tmpfs mount options, which is needed
>     because now we parse our own XML correctly.
>     
>     Changes the default input unit to KiB to match documentation, fixing:
>     https://bugzilla.redhat.com/show_bug.cgi?id=1015689
> 
> git describe: v1.1.3-77-g3f029fb

Another upstream patch attempted:
https://www.redhat.com/archives/libvir-list/2013-October/msg00415.html

Missing units default to KiB, and XML generated by older libvirt is parsed correctly.

Both of them also fix the bug of libvirt not being able to parse the units in its own generated XML correctly.