This request was evaluated by Red Hat Product Management for
inclusion in the current release of Red Hat Enterprise Linux.
Because the affected component is not scheduled to be updated
in the current release, Red Hat is unfortunately unable to
address this request at this time. Red Hat invites you to
ask your support representative to propose this request, if
appropriate and relevant, in the next release of Red Hat
Enterprise Linux. If you would like it considered as an
exception in the current release, please ask your support
representative.

This request was evaluated by Red Hat Product Management for
inclusion in the current release of Red Hat Enterprise Linux.
Because the affected component is not scheduled to be updated
in the current release, Red Hat is unfortunately unable to
address this request at this time. Red Hat invites you to
ask your support representative to propose this request, if
appropriate and relevant, in the next release of Red Hat
Enterprise Linux. If you would like it considered as an
exception in the current release, please ask your support
representative.

This request was erroneously denied for the current release of
Red Hat Enterprise Linux.  The error has been fixed and this
request has been re-proposed for the current release.

This is because of the lvm2-monitor script being enabled by default.
Disabling it should allow the system to shutdown properly.
# chkconfig lvm2-monitor off

Actually, the real reason is an inversion in shutdown order of lvm2-monitor and iscsi.  lvm2-monitor it shutting down really late, which is after iscsi and a lot of other things have been torn down:
# ls /etc/rc.d/rc6.d/*lvm2*
/etc/rc.d/rc6.d/K99lvm2-monitor
# ls /etc/rc.d/rc6.d/*iscsi*
/etc/rc.d/rc6.d/K88iscsi  /etc/rc.d/rc6.d/K89iscsid

This looks wrong.

lvm2-monitor is required for systems with mirrors.

Just as activation of all the LVs on the system is done in several steps with other things happening in between, shutdown should be similar.

Which part of the monitor script is causing the problem?

It should be late, after iscsi, yes, I'd have thought.

But what is deactivating LVs on iscsi devs before iscsi is shut down?

I testd to shift shutdown sequence "K87XXX, K88XXX, K89XXX, K90XXX" and change K99lvm2-monitor to K87lvm2-monitor.

I verified shutdown sequence is suceeded on that environment.

Besides, what kind of order should it be?

(In reply to comment #7)
> But what is deactivating LVs on iscsi devs before iscsi is shut down?

I do not think anything is. I think we just hit a similar problem with software fcoe and lvm.

I'll switch this to initscripts rather than kernel for now for co-ordination of any necessary changes, but expect that other components may need to change their own scripts too.

Not sure what initscripts itself has to do with this - wouldn't this be in the specific fcoe/iscsi/lvm2-monitor init scripts?

Moving back - none of the scripts in question are in initscripts.

Does anyone know if this bug will make it into RHEL 6 SP 1?  Thanks.

you could try adding

defaults {
  ...
  queue_without_daemon no
  ...
}

to /etc/multipath.conf

to fix the queue_if_no_paths problem.  That disables queuing whenever the multipathd daemon is not running.

Did the suggestion in comment #15 help to address the problem reported? The shutdown should not hang with that setting used.

A script to deactivate any remaining LVs/VGs should still be provided though with the exception of the LV on which root fs and any system fs resides (together with unmounting any filesystems using them and any layered devices/mappings underneath). I'll try to complete the script using existing lsblk output that shows the device tree already based on information found in sysfs. The lsblk needs a small patch to make the output more suitable for parsing...

I've put together a Kbase article covering this issue, and a couple workarounds.  The current public version is here: https://access.redhat.com/kb/docs/DOC-60763

How is this "NOTABUG"? 

The Kbase article says "This is a known issue and a fix is being worked on"

We already have a bug tracking this problem, but it's marked as private as per customer's request. I've opened a public one with 6.4 as target - bug #800801. Please add any additional comments there. Thanks.