This service will be undergoing maintenance at 00:00 UTC, 2017-10-23 It is expected to last about 30 minutes
Bug 221330 - kernel dm-multipath: stall on resume if noflush is used
kernel dm-multipath: stall on resume if noflush is used
Status: CLOSED ERRATA
Product: Red Hat Enterprise Linux 5
Classification: Red Hat
Component: kernel (Show other bugs)
5.0
All Linux
medium Severity medium
: ---
: ---
Assigned To: Milan Broz
Corey Marthaler
: OtherQA
Depends On:
Blocks: 228988 227613
  Show dependency treegraph
 
Reported: 2007-01-03 16:03 EST by Jun'ichi NOMURA
Modified: 2013-02-28 23:04 EST (History)
15 users (show)

See Also:
Fixed In Version: RHBA-2007-0959
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2007-11-07 14:18:21 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
Reproduction test script (1.60 KB, application/x-gzip)
2007-01-03 16:03 EST, Jun'ichi NOMURA
no flags Details

  None (edit)
Description Jun'ichi NOMURA 2007-01-03 16:03:18 EST
Description of problem:
dm-multipath sometimes stalls on resume if noflush is used

Version-Release number of selected component:
kernel-2.6.18-1.2839.el5
(should occur with 2.6.18-1.2725.el5 or later)

How reproducible:
Always

Steps to Reproduce:
 1. Create a multipath device by dmsetup
 2. Fail all paths during mkfs
 3. Do dmsetup suspend --noflush and load new map with healthy paths
 4. Do dmsetup resume
 (Attached test script will do the above steps.)

Actual results:
  dmsetup resume stalls

Expected results:
  dmsetup resume shouldn't stall

Additional info:
  How to run the script:
    # modprobe dm-mod
    # modprobe dm-multipath
    # modprobe dm-round-robin
    # tar zxf noflush-stall-test.tgz
    # cd noflush-stall-test
    # ./run.sh

  The backtrace of dmsetup resume will look like:
    PID: 3940   TASK: f75dd000  CPU: 1   COMMAND: "dmsetup"
     #0 [f58a9d3c] schedule at c060f021
     #1 [f58a9da4] __mutex_lock_slowpath at c060fa95
     #2 [f58a9dc0] .text.lock.mutex (via mutex_lock) at c060fad3
     #3 [f58a9dcc] dm_swap_table at f88b833b
     #4 [f58a9de8] dev_suspend at f88bad8e
     #5 [f58a9e00] ctl_ioctl at f88bb5b3
     #6 [f58a9f6c] do_ioctl at c047e57d
     #7 [f58a9f84] vfs_ioctl at c047e7db
     #8 [f58a9fa0] sys_ioctl at c047e839
     #9 [f58a9fb8] system_call at c0403fc4
        EAX: ffffffda  EBX: 00000003  ECX: c134fd06  EDX: 0825df18 
        DS:  007b      ESI: 4ed41e40  ES:  007b      EDI: 4ed3dc8b 
        SS:  007b      ESP: bfd3bb14  EBP: bfd3bc78 
        CS:  0073      EIP: 0030f402  ERR: 00000036  EFLAGS: 00000246 

  Sometimes, the stall occurs in dmsetup suspend in a similar way.

  What happens is that:
    1 Someone does fsync() on the block dev
      holding inode->i_sem and possibly I_LOCK is set
    2 The fsync write is blocked by the log failure
    3 dm_suspend() tries to bdget() and waits forever on the inode
      being unlocked.
      Or, in the middle of dm_resume(), __bind() tries to get
      inode->i_sem to do __set_size() and waits forever

  The bug is in the common code of noflush feature.
  So it could occur on other targets if they start supporting noflush feature.
Comment 1 Jun'ichi NOMURA 2007-01-03 16:03:18 EST
Created attachment 144740 [details]
Reproduction test script
Comment 3 Jun'ichi NOMURA 2007-01-03 16:17:12 EST
Sorry, the scenario description was for the case of mirror,
the feature which is not there.

For multipath, the scenario is similar. That is:
    1 Someone does fsync() on the block dev
      holding inode->i_sem and possibly I_LOCK is set
    2 The fsync write is blocked by all-paths-down and queue_if_no_path
    3 dm_suspend() tries to bdget() and waits forever on the inode
      being unlocked.
      Or, in the middle of dm_resume(), __bind() tries to get
      inode->i_sem to do __set_size() and waits forever

The test script should be able to cover both cases.
Comment 5 RHEL Product and Program Management 2007-01-11 05:24:49 EST
This request was evaluated by Red Hat Product Management for inclusion in a Red
Hat Enterprise Linux maintenance release.  Product Management has requested
further review of this request by Red Hat Engineering, for potential
inclusion in a Red Hat Enterprise Linux Update release for currently deployed
products.  This request is not yet committed for inclusion in an Update
release.
Comment 7 Don Zickus 2007-04-23 17:46:19 EDT
in 2.6.18-16.el5
Comment 9 John Poelstra 2007-08-24 01:13:57 EDT
A fix for this issue should have been included in the packages contained in the
most recent snapshot (partners.redhat.com) for RHEL5.1.  

Requested action: Please verify that your issue is fixed as soon as possible to
ensure that it is included in this update release.

After you (Red Hat Partner) have verified that this issue has been addressed,
please perform the following:
1) Change the *status* of this bug to VERIFIED.
2) Add *keyword* of PartnerVerified (leaving the existing keywords unmodified)

If this issue is not fixed, please add a comment describing the most recent
symptoms of the problem you are having and change the status of the bug to FAILS_QA.

More assistance: If you cannot access bugzilla, please reply with a message to
Issue Tracker and I will change the status for you.  If you need assistance
accessing ftp://partners.redhat.com, please contact your Partner Manager.
Comment 10 Jun'ichi NOMURA 2007-08-28 15:52:21 EDT
Confirmed the testcase in comment#1 doesn't reproduce the problem.
Comment 12 errata-xmlrpc 2007-11-07 14:18:21 EST
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on the solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHBA-2007-0959.html

Note You need to log in before you can comment on or make changes to this bug.