RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1982164 - Podman volumes with size options
Summary: Podman volumes with size options
Keywords:
Status: CLOSED ERRATA
Alias: None
Deadline: 2022-02-28
Product: Red Hat Enterprise Linux 8
Classification: Red Hat
Component: podman
Version: 8.4
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: beta
: ---
Assignee: Daniel Walsh
QA Contact: Joy Pu
URL:
Whiteboard:
Depends On:
Blocks: 1186913
TreeView+ depends on / blocked
 
Reported: 2021-07-14 11:03 UTC by Ashish Reddy
Modified: 2023-12-07 23:11 UTC (History)
12 users (show)

Fixed In Version: podman-3.4
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2022-05-10 13:27:31 UTC
Type: Bug
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2022:1762 0 None None None 2022-05-10 13:27:47 UTC

Description Ashish Reddy 2021-07-14 11:03:08 UTC
Description of problem:
When a volume is created with podman and provide arguments regarding the size, it is not applied to the volume and size of the volume can be used up to the limit of the file system

Version-Release number of selected component (if applicable):


How reproducible:
Easy

Steps to Reproduce:
1. Create a volume with size option 
   podman volume create --opt o=size=800m test

2. Attach it to a container and get inside the container
   podman run -it -v /var/lib/containers/storage/volumes/test/_data:/tmp registry.access.redhat.com/ubi8

3. Now inside the container create files that are greater than the size of the volume specified.
  [root@6116d536672a /]# cd /tmp/
  [root@6116d536672a tmp]# dd if=/dev/zero of=file.txt count=1024 bs=1048576
  1024+0 records in
  1024+0 records out
  1073741824 bytes (1.1 GB, 1.0 GiB) copied, 1.80914 s, 594 MB/s
  [root@6116d536672a tmp]# ls -lrth
  total 1.0G
  -rw-r--r--. 1 root root 1.0G Jul 12 10:08 file.txt
  [root@6116d536672a tmp]# 
  Even though the volume size is 800MB, files of size greater than 800MB can be created.


Actual results:
Volume size should be honored by podman 

Expected results:
Even though the volume size is 800MB, files of size greater than 800MB can be created.

Additional info:
The following are the detailed tests performed:

Case 1:

ENV: RHEL 8.4, podman-3.0.2

Create a volume 

# podman volume create --opt o=size=800m test
# podman volume inspect test
[
    {
         "Name": "test",
         "Driver": "local",
         "Mountpoint": "/var/lib/containers/storage/volumes/test/_data",
         "CreatedAt": "2021-07-12T01:55:03.697904246-04:00",
         "Labels": {},
         "Scope": "local",
         "Options": {
             "o": "size=800m"
        }
    }
]

Attach it to a container and get inside the container

# podman run -it --rm -v /var/lib/containers/storage/volumes/test/_data:/tmp registry.access.redhat.com/ubi8
[root@6116d536672a /]# cd /tmp/
[root@6116d536672a tmp]# dd if=/dev/zero of=file.txt count=1024 bs=1048576
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 1.80914 s, 594 MB/s
[root@6116d536672a tmp]# ls -lrth
total 1.0G
-rw-r--r--. 1 root root 1.0G Jul 12 10:08 file.txt
[root@6116d536672a tmp]# 

Even though the volume size is 800MB, files of size greater than 800MB can be created.


Case 2: 

Volumes are created on a separate LVM partition created for volumes

ENV: RHEL 8.4, podman-3.0.2

Attached a new disk to VM and created an LVM partition on the disk

The new partition is used as volume_path in /usr/share/containers/containers.conf , so as to use this LVM for the volume creation.

# cat /usr/share/containers/containers.conf | grep volume_path | grep -v ^#
volume_path = "/volumes"
[root@vm254-61 ~]# 
# podman info | grep -i volume
volumePath: /volumes

Now repeated the above test case.

The same result happened. Volumes are created on this mount path and we can extend the volume size up to the limit of the LVM size(up to the limit of File system) even though the size is specified during the creation.


Both the cases are tested in podman-3.0.2 and podman-1.6.4

I have performed the Case 1 in Docker of RHEL 7.9 also and it is reproducible

Comment 1 Tom Sweeney 2021-07-14 13:32:27 UTC
Paul, can you take a look at this please?

Comment 2 Daniel Walsh 2021-07-14 23:45:10 UTC
This should go to Nalin, or Giuseppe or Me.  The issue here is whether or not the disk that the volume was created on, has quota.  Also whether we can enforce quota on the root user.

Comment 3 Daniel Walsh 2021-07-15 00:10:10 UTC
Actually on second thought, this has nothing to do with containers/storage but has to do with container volume storage.  So assigning to Matt.

Should only work on XFS.

Can use code from ~/storage/drivers/quota/

Something like:

q= quota.NewControl(path)
q.SetQuota(path, quota.Quota{Size: size});

Might work if the file system that path is mounted on is mounted with pquota.

Comment 4 Derrick Ornelas 2021-07-21 13:40:43 UTC
(In reply to Ashish Reddy from comment #0)
> 
> Steps to Reproduce:
> 1. Create a volume with size option 
>    podman volume create --opt o=size=800m test
> 
> 2. Attach it to a container and get inside the container
>    podman run -it -v /var/lib/containers/storage/volumes/test/_data:/tmp
> registry.access.redhat.com/ubi8
> 

Is this actually a sane way to mount a named volume to a container?  




(In reply to Daniel Walsh from comment #3)
> 
> Might work if the file system that path is mounted on is mounted with pquota.

By default RHEL 8 doesn't mount the rootfs with quotas enabled.  Maybe it would be good to also add a warning when a user tries to specify size with a local fs that isn't mounted with pquote, pjquota, etc.?

Comment 5 Matthew Heon 2021-07-21 13:53:13 UTC
Derrick - very good catch. You're entirely correct, that is very much not sane.

The correct way would be to use `-v test:/tmp` instead. Explanation: these are Podman-managed volumes, which provide guarantees that they cannot be removed until they are not in use by any container. We do this via reference-counting, which does not work if you manually mount the volume's data directory - so `podman volume rm` will quite happily let someone remove that volume while your container is using it. Furthermore, mounting the volume like this breaks any volume that requires a mount command (e.g. NFS volumes, or volume plugins) as we don't know a container is using the mount and will not perform said mount. Please mount named volumes created via `volume create` into containers using their names to avoid this.

Comment 6 Ashish Reddy 2021-07-28 06:16:20 UTC
Hello Matthew and Derrick,

Sorry for not attaching the output properly. I have done the tests also with "-v test:/tmp" option. Please find the output below:

~~~
[root@vm251-225 ~]# podman volume create --opt o=size=1g test
test
[root@vm251-225 ~]# 
[root@vm251-225 ~]# 
[root@vm251-225 ~]# podman volume inspect test 
[
    {
        "Name": "test",
        "Driver": "local",
        "Mountpoint": "/var/lib/containers/storage/volumes/test/_data",
        "CreatedAt": "2021-07-27T01:19:34.166111936-04:00",
        "Labels": {},
        "Scope": "local",
        "Options": {}
    }
]
[root@vm251-225 ~]# podman run -it -v test:/tmp registry.access.redhat.com/ubi8 /bin/bash
Trying to pull registry.access.redhat.com/ubi8:latest...
Getting image source signatures
Checking if image destination supports signatures
Copying blob a50df8fd88fe done  
Copying blob 1cadda38f72d done  
Copying config 0ced1c7c9b done  
Writing manifest to image destination
Storing signatures
[root@99a5d22825da /]#  
[root@99a5d22825da /]# cd /tmp/
[root@99a5d22825da tmp]# dd if=/dev/zero of=file.txt count=1024 bs=1048576
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 1.19697 s, 897 MB/s
[root@99a5d22825da tmp]# 
[root@99a5d22825da tmp]# dd if=/dev/zero of=file2.txt count=1024 bs=1048576
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 1.24617 s, 862 MB/s
[root@99a5d22825da tmp]# 
[root@99a5d22825da tmp]# 
[root@99a5d22825da tmp]# dd if=/dev/zero of=file.txt count=1024 bs=1048576
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 1.2484 s, 860 MB/s
[root@99a5d22825da tmp]# dd if=/dev/zero of=file3.txt count=1024 bs=1048576
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 1.30469 s, 823 MB/s
[root@99a5d22825da tmp]# 
total 4.0G
-rw-r--r--. 1 root root 1.0G Jul 27 05:20 file.txt
-rw-r--r--. 1 root root 1.0G Jul 27 05:20 file2.txt
-rw-r--r--. 1 root root 1.0G Jul 27 05:21 file3.txt
-rwx------. 1 root root  701 Jul 21 00:55 ks-script-eipzyf1a
-rwx------. 1 root root  291 Jul 21 00:55 ks-script-t5il97pv
[root@99a5d22825da tmp]# 
~~~


In this case also, the size option is not being honored. Is there any other way to specify the size for volumes.

Comment 12 Daniel Walsh 2021-08-12 19:24:59 UTC
container-storage.conf now has a size and inodes which apply to each containers rootfs.  So it will follow the rules. 

Note: The size/inodes is per mountpount/volume.  Currently we have no way to say a Container or Pod has a total limit of size of inodes.

They are always rootfs (size/inodes) + foreach(vollume) { volume(size/inodes))

Comment 14 Daniel Walsh 2021-08-19 10:27:41 UTC
podman 3.3-rc2 should be available and built for RHEL8.

Comment 22 Daniel Walsh 2021-12-29 11:44:12 UTC
Looks like podman-3.4.4 has support for this.

Comment 26 Joy Pu 2022-01-17 12:01:57 UTC
Test with podman-3.4.5-0.4.module+el8.6.0+13880+7c3afcf8.x86_64 it works as expected.

It can report error message when not config disk quotas.
# podman volume create --opt o=size=800m test
Error: Volume options size and inodes not supported. Filesystem does not support Project Quota
And works as expect when setup disk quotas:
#  podman volume create --opt o=size=1g test
test
# podman inspect test
[
    {
        "Name": "test",
        "Driver": "local",
        "Mountpoint": "/var/lib/containers/storage/volumes/test/_data",
        "CreatedAt": "2022-01-17T06:55:37.094941027-05:00",
        "Labels": {},
        "Scope": "local",
        "Options": {
            "SIZE": "1g",
            "o": "size=1g"
        }
    }
]
#  podman run -it -v test:/tmp registry.access.redhat.com/ubi8 /bin/bash
[root@a8c83429c8b3 /]# cd /tmp/
[root@a8c83429c8b3 tmp]# ls
ks-script-1rw57zdt  ks-script-g73hk9vn
[root@a8c83429c8b3 tmp]# dd if=/dev/zero of=file.txt count=1024 bs=2M
dd: error writing 'file.txt': No space left on device
477+0 records in
476+0 records out
998244352 bytes (998 MB, 952 MiB) copied, 2.46736 s, 405 MB/s
[root@a8c83429c8b3 tmp]# exit
exit

So set this bug to verified.

Comment 28 errata-xmlrpc 2022-05-10 13:27:31 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: container-tools:rhel8 security, bug fix, and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:1762

Comment 29 Xenico 2023-11-16 04:54:16 UTC Comment hidden (spam)
Comment 30 Xenico 2023-11-16 04:55:27 UTC Comment hidden (spam)
Comment 31 farare 2023-11-18 04:00:59 UTC Comment hidden (spam)

Note You need to log in before you can comment on or make changes to this bug.