Bug 1339777

Summary: ioprocess keep open file on shared storage after touching or truncating a file
Product: [Fedora] Fedora Reporter: Nir Soffer <nsoffer>
Component: ioprocessAssignee: Nir Soffer <nsoffer>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: high Docs Contact:
Priority: unspecified    
Version: rawhideCC: bronhaim, dougsland, nsoffer, smizrahi
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: ioprocess-0.16.1-1.fc24 ioprocess-0.16.1-1.fc23 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1371634 (view as bug list) Environment:
Last Closed: 2016-07-05 05:00:35 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1339780, 1371634    
Attachments:
Description Flags
Output of lsof -p <pid> on all ioprocess processes none

Description Nir Soffer 2016-05-25 20:33:54 UTC
Created attachment 1161615 [details]
Output of lsof -p <pid> on all ioprocess processes

Description of problem:

The first time IOProcess.truncate() or IOProcess.touch() is called, ioprocess
keep the file open and will never close it.

Typically when using with vdsm, ioprocess will keep the __DIRECT_IO_TEST__ 
file open on shared storage, since the first thing vdsm does is touching
this file.

This is an example output from lsof:

$ lsof -p 23805
COMMAND     PID USER   FD   TYPE DEVICE SIZE/OFF     NODE NAME
...
ioprocess 23805 vdsm    0w   REG   0,39        0 13369358 /rhev/data-center/mnt/10.35.0.179:_home_storage__domains_domain2/__DIRECT_IO_TEST__ (10.35.0.179:/home/storage_domains/domain2)
...

When putting host to maintenance, ioprocess still keeps same file open:

COMMAND     PID USER   FD   TYPE DEVICE SIZE/OFF     NODE NAME
...
ioprocess 23805 vdsm    0w   REG   0,39        0 13369358 /__DIRECT_IO_TEST__
...

Keeping open files on shared storage in maintenance mode may cause trouble
on some shared file systems.

Version-Release number of selected component (if applicable):
0.5.0

The defect was introduce in this commit:

commit 7dec019602137186908fdad624e1cc7d1faf4001
Author:     Yeela Kaplan <ykaplan>
AuthorDate: Sun May 18 17:28:51 2014 +0300
Commit:     Yeela Kaplan <ykaplan>
CommitDate: Sun May 25 19:21:40 2014 +0300

    Add missing functionality to exported functions
    
    touch
    truncatefile

How reproducible:
Always

Steps to Reproduce:
1. Invoke truncate or touch
2. Check open files using lsof -p <ioprocess pid>

Actual results:
File remain open forever.

Expected results:
Truncated or touched file close after the operation.

Workaround:

If the host is in maintenance mode, killing ioprocess will safely close the 
open file.

Comment 1 Nir Soffer 2016-05-25 20:42:48 UTC
A fix is available for testing here:
http://koji.fedoraproject.org/koji/taskinfo?taskID=14249832

Comment 2 Yaniv Kaul 2016-06-15 06:18:06 UTC
Should the bug move to POST?

Comment 3 Fedora Update System 2016-06-16 08:48:56 UTC
ioprocess-0.16.1-1.fc24 has been submitted as an update to Fedora 24. https://bodhi.fedoraproject.org/updates/FEDORA-2016-dfdd83c234

Comment 4 Fedora Update System 2016-06-16 09:07:51 UTC
ioprocess-0.16.1-1.fc23 has been submitted as an update to Fedora 23. https://bodhi.fedoraproject.org/updates/FEDORA-2016-f2bcab1a73

Comment 5 Fedora Update System 2016-06-16 15:55:02 UTC
ioprocess-0.16.1-1.fc24 has been pushed to the Fedora 24 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2016-dfdd83c234

Comment 6 Fedora Update System 2016-06-17 18:55:00 UTC
ioprocess-0.16.1-1.fc23 has been pushed to the Fedora 23 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2016-f2bcab1a73

Comment 7 Fedora Update System 2016-07-05 05:00:28 UTC
ioprocess-0.16.1-1.fc24 has been pushed to the Fedora 24 stable repository. If problems still persist, please make note of it in this bug report.

Comment 8 Fedora Update System 2016-07-05 08:25:48 UTC
ioprocess-0.16.1-1.fc23 has been pushed to the Fedora 23 stable repository. If problems still persist, please make note of it in this bug report.

Comment 9 Nir Soffer 2016-08-29 15:22:53 UTC
The request from comment 2 was addressed ages ago.