1551486 – QEMU image locking needn't double open fd number (i.e. drop file-posix.c:s->lock_fd)

RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1551486 - QEMU image locking needn't double open fd number (i.e. drop file-posix.c:s->lock_fd)

Summary: QEMU image locking needn't double open fd number (i.e. drop file-posix.c:s->l...

Keywords:
Status:	CLOSED ERRATA
Alias:	None
Product:	Red Hat Enterprise Linux 7
Classification:	Red Hat
Component:	qemu-kvm-rhev
Sub Component:
Version:	7.6
Hardware:	Unspecified
OS:	Unspecified
Priority:	medium
Severity:	medium
Target Milestone:	rc
Target Release:	---
Assignee:	Hanna Czenczek
QA Contact:	Tingting Mao
Docs Contact:
URL:
Whiteboard:
Duplicates (1):	1636279 (view as bug list)
Depends On:
Blocks:	1526313 1624734 1651787 1694148
TreeView+	depends on / blocked

Reported:	2018-03-05 09:32 UTC by Ping Li
Modified:	2019-08-22 09:19 UTC (History)
CC List:	12 users (show)
Fixed In Version:	qemu-kvm-rhev-2.12.0-23.el7
Doc Type:	If docs needed, set a value
Doc Text:
Clone Of:
Clones:	1694148 (view as bug list)
Environment:
Last Closed:	2019-08-22 09:18:46 UTC
Target Upstream Version:
Embargoed:

Attachments	(Terms of Use)
trace log (3.04 MB, application/gzip) 2019-02-25 03:07 UTC, Tingting Mao	no flags	Details
View All

Links
System	ID	Private	Priority	Status	Summary	Last Updated
Red Hat Knowledge Base (Solution)	3639881	0	None	None	None	2018-12-05 15:57:08 UTC
Red Hat Product Errata	RHSA-2019:2553	0	None	None	None	2019-08-22 09:19:41 UTC

Description Ping Li 2018-03-05 09:32:40 UTC

Description of problem:
Since qemu-kvm-rhev-2.10.0-3.el7, image locking is enabled by default. It makes qemu to operate one more file descriptor per file. When create a vmdk image with large size, hundreds of extent raw images will be created. However the default maximum number of open file descriptors is 1024. The limit FDs will make qemu fail to open the vmdk image.
It will helpful to document this to make user know clear about it.

Version-Release number of selected component (if applicable):
qemu-kvm-rhev-2.10.0-21.el7

How reproducible:
100%

Steps to Reproduce:
1. Check the default limit of file descriptors for usr
# ulimit -n
1024

2. Create 1TB vmdk iamge
# qemu-img create -f vmdk -o subformat=twoGbMaxExtentFlat test.vmdk 1T
Formatting 'test.vmdk', fmt=vmdk size=1099511627776 compat6=off hwversion=undefined subformat=twoGbMaxExtentFlat

3. Get image info
# qemu-img info test.vmdk
qemu-img: Could not open 'test.vmdk': Could not open 'test-f507.vmdk': Too many open files

4. Increase limit of file descriptors
# ulimit -n 2048

5. Get image info
# qemu-img info test.vmdk
image: test.vmdk
file format: vmdk
virtual size: 1.0T (1099511627776 bytes)
disk size: 20K
Format specific information:
cid: 121892516
parent cid: 4294967295
create type: twoGbMaxExtentFlat
extents:
[0]:
virtual size: 2147483648
filename: test-f001.vmdk
format: FLAT
...
[511]:
virtual size: 2147483648
filename: test-f512.vmdk
format: FLAT

Actual results:

Expected results:

Additional info:

Comment 2 Kevin Wolf 2018-08-10 12:32:13 UTC

Fam, you mentioned in an upstream discussion that we don't technically need s->lock_fd any more [1]. So maybe instead of documenting that we need the double amount of file descriptors, we can just avoid that?

In any case, congratulations, you just won a bug. :-)

[1] http://lists.gnu.org/archive/html/qemu-devel/2018-03/msg07412.html

Comment 4 Ademar Reis 2018-12-04 21:37:05 UTC

Apparently fixed upstream:
http://lists.gnu.org/archive/html/qemu-block/2018-10/msg00364.html

(should this BZ be a dupe of bug 1636279?)

Comment 5 Hanna Czenczek 2018-12-05 15:48:59 UTC

This bug was reported earlier, so I'd say the other is the duplicate. :-)

In any case, in addition http://lists.nongnu.org/archive/html/qemu-block/2018-11/msg00513.html is required as a fix on top of that series.

Comment 6 Hanna Czenczek 2018-12-05 15:57:09 UTC

*** Bug 1636279 has been marked as a duplicate of this bug. ***

Comment 8 Miroslav Rezanina 2019-02-12 08:28:17 UTC

Fix included in qemu-kvm-rhev-2.12.0-23.el7

Comment 9 Tingting Mao 2019-02-25 03:01:11 UTC

Verify this bug as below.


Tested with:
qemu-kvm-rhev-2.12.0-23.el7


Scenario 1

# ulimit -n
1024

# qemu-img create -f vmdk -o subformat=twoGbMaxExtentFlat test.vmdk 1T
Formatting 'test.vmdk', fmt=vmdk size=1099511627776 compat6=off hwversion=undefined subformat=twoGbMaxExtentFlat

# qemu-img info test.vmdk 
image: test.vmdk
file format: vmdk
virtual size: 1.0T (1099511627776 bytes)
disk size: 20K
Format specific information:
    cid: 3975726499
    parent cid: 4294967295
    create type: twoGbMaxExtentFlat
    extents:
        [0]:
            virtual size: 2147483648
            filename: test-f001.vmdk
            format: FLAT
        [1]:
            virtual size: 2147483648
            filename: test-f002.vmdk
            format: FLAT
        [2]:
            virtual size: 2147483648
            filename: test-f003.vmdk
            format: FLAT
......
......
        [509]:
            virtual size: 2147483648
            filename: test-f510.vmdk
            format: FLAT
        [510]:
            virtual size: 2147483648
            filename: test-f511.vmdk
            format: FLAT
        [511]:
            virtual size: 2147483648
            filename: test-f512.vmdk
            format: FLAT


Scenario 2

Steps as https://bugzilla.redhat.com/show_bug.cgi?id=1636279#c1

After checking the trace log, there is just one opened FD for data.img file as expected. And the trace log will be attached, thanks.

Comment 10 Tingting Mao 2019-02-25 03:07:21 UTC

Created attachment 1538276 [details]
trace log

Comment 17 errata-xmlrpc 2019-08-22 09:18:46 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2019:2553

Note You need to log in before you can comment on or make changes to this bug.