RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 2140997 - Fail to convert physical host with floppy disk via latest virt-p2v iso
Summary: Fail to convert physical host with floppy disk via latest virt-p2v iso
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat Enterprise Linux 9
Classification: Red Hat
Component: virt-v2v
Version: 9.2
Hardware: x86_64
OS: Unspecified
medium
medium
Target Milestone: rc
: ---
Assignee: Laszlo Ersek
QA Contact: Virtualization Bugs
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2022-11-08 11:35 UTC by tingting zheng
Modified: 2022-11-15 14:34 UTC (History)
8 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2022-11-10 09:40:24 UTC
Type: Bug
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker RHELPLAN-138687 0 None None None 2022-11-08 14:44:57 UTC

Description tingting zheng 2022-11-08 11:35:12 UTC
Description of problem:
Fail to convert physical host via latest virt-p2v iso.

Version-Release number of selected component (if applicable):
livecd-p2v-202210261136-v1.42.3.iso
libguestfs-1.48.4-2.el9.x86_64
virt-v2v-2.0.7-6.el9.x86_64
libvirt-8.9.0-2.el9.x86_64
qemu-kvm-7.1.0-4.el9.x86_64

How reproducible:
100%

Steps to Reproduce:
1. Boot physical host via latest virt-p2v iso.
2. Input virt-v2v conversion server ip and connect to the server.
3. Choose both local disk and floppy disk.
4. Start conversion.

Actual results:
Error shows as below and the conversion fails.
libguestfs: trace: v2v: launch = -1 (error)
^[[1;31mvirt-v2v: error: libguestfs error: could not create appliance through libvirt.

Try running qemu directly without libvirt using this environment variable:
export LIBGUESTFS_BACKEND=direct

Original error from libvirt: internal error: qemu unexpectedly closed the monitor: 2022-11-07T07:25:39.336626Z qemu-kvm: -blockdev {"driver":"nbd","server":{"type":"unix","path":"/tmp/v2v.qklUzY/in1"},"node-name":"libvirt-2-storage","cache":{"direct":false,"no-flush":true},"auto-read-only":true,"discard":"unmap"}: Failed to connect to '/tmp/v2v.qklUzY/in1': Connection refused [code=1 int1=-1]^[[0m

Expected results:
The conversion can be finished successfully.

Additional info:
logs:
http://fileshare.englab.nay.redhat.com/pub/section3/libvirtmanual/tzheng//virt-p2v-20221108-kcpahkjq/
http://fileshare.englab.nay.redhat.com/pub/section3/libvirtmanual/tzheng//journalctl.log

Comment 1 Richard W.M. Jones 2022-11-08 11:51:34 UTC
The virt-p2v system journal reveals the real cause which is:

    Nov 08 04:34:59 dell-per750-29.lab.eng.pek2.redhat.com launch-virt-p2v[1818]: nbdkit: file[1]: error: open: /dev/sda: No medium found

In other words it's a SCSI floppy with no medium.  (The conversion might have
worked if a floppy disk had been present!)

We should be able to detect this in virt-p2v.

Comment 2 mxie@redhat.com 2022-11-08 13:08:57 UTC
This is related bug 1374629 - virt-p2v sees LCDRIVE & virtual floppy as hard disks, they are in fact removable

Comment 3 Laszlo Ersek 2022-11-08 14:38:04 UTC
Tingting,

can you please share the contents of:

  /sys/block/sda/removable

?

Thank you.

Comment 4 tingting zheng 2022-11-09 05:39:45 UTC
(In reply to Laszlo Ersek from comment #3)
> Tingting,
> 
> can you please share the contents of:
> 
>   /sys/block/sda/removable
> 
> ?
> 
> Thank you.

#cat /sys/block/sda/removable
1

Comment 5 Laszlo Ersek 2022-11-09 07:54:03 UTC
Here's how the kernel sets "/sys/block/sda/removable" to 1, for our purposes:

(1) Generic SCSI layer:

in scsi_add_lun() [drivers/scsi/scsi_scan.c], bit#7 in byte#1 of the INQUIRY response is translated to the "scsi_device.removable" field (which is a single bit).

		sdev->type = (inq_result[0] & 0x1f);
		sdev->removable = (inq_result[1] & 0x80) >> 7;

Per SCSI spec (SCSI Primary Commands / SPC-4), "Table 137 — Standard INQUIRY data format", this bit is "RMB" -- "A removable medium (RMB) bit set to zero indicates that the medium is not removable. A RMB bit set to one indicates that the medium is removable."

Note that a "floppy disk" is not reported as such in the inquiry data; for example, "Peripheral device type" does not distinguish floppies from "Direct access block device (e.g., magnetic disk)" or "Simplified direct-access device (e.g., magnetic disk)".

(2) SCSI disk layer:

in sd_probe() [drivers/scsi/sd.c], the "scsi_device.removable" field is translated to the GENHD_FL_REMOVABLE flag in the "gendisk.flags" field:

	if (sdp->removable) {
		gd->flags |= GENHD_FL_REMOVABLE;

(
Side comment: this decision is logged in the same function as well:

	sd_printk(KERN_NOTICE, sdkp, "Attached SCSI %sdisk\n",
		  sdp->removable ? "removable " : "");
)

GENHD_FL_REMOVABLE is defined as follows [include/linux/blkdev.h]:

 * ``GENHD_FL_REMOVABLE``: indicates that the block device gives access to
 * removable media.  When set, the device remains present even when media is not
 * inserted.  Shall not be set for devices which are removed entirely when the
 * media is removed.

(Side comment: GENHD_FL_REMOVABLE is also set for devices that are "genuinely" floppy drives. Refer to floppy_alloc_disk() [drivers/block/floppy.c].)

(Side comment: GENHD_FL_REMOVABLE is also set for SCSI CD-ROMs, but those are handled elsewhere: see sr_probe() [drivers/scsi/sr.c]:

	disk->flags |= GENHD_FL_REMOVABLE | GENHD_FL_NO_PART;

CD-ROMs are exposed under different device node names to userspace -- namely, not sd* but sr*, or the more recent scd*. And we treat sr* separately from sd* in virt-p2v's find_all_disks() [disks.c]. Therefore, in virt-p2v, we will not conflate SCSI floppies with SCSI CD-ROMs.)

(3) sysfs layer:

The GENHD_FL_REMOVABLE flag is formatted as 1 vs. 0 in disk_removable_show() [block/genhd.c]:

	return sprintf(buf, "%d\n",
		       (disk->flags & GENHD_FL_REMOVABLE ? 1 : 0));

where disk_removable_show() belongs to:

static DEVICE_ATTR(removable, 0444, disk_removable_show, NULL);


So, end-to-end, /sys/block/sd*/removable is 1 precisely if the device is a removable media SCSI disk device (and that excludes SCSI CR-ROMs, as those don't have sd* device node names), which is good enough for virt-p2v to just skip the device.

Comment 6 Laszlo Ersek 2022-11-09 12:26:47 UTC
For another data point, it seems that USB sticks have "removable" set as well. This looks like a violation of GENHD_FL_REMOVABLE: the comment on the flag says "Shall not be set for devices which are removed entirely when the media is removed". USB sticks do not enable the user to remove their "media" separately, the stick can only be unplugged whole-sale, *but* then the entire device node disappears too. So that's a kernel issue.

This is a problem for virt-p2v. If we filtered out such a USB storage device because it exposed "removable=1", we'd prevent virt-p2v from converting such OSes that are installed on external USB disks -- even large ones, like portable, USB attached, half-TB large SSDs.

So another step is needed (refer to comment 1): try to open the device, and look for ENOMEDIUM. Basically try to trigger the issue (gracefully) that would otherwise break the "file" plugin of nbdkit.

Note that with this, if the host has a SCSI floppy, and there *is* a floppy disk inserted, virt-p2v is going to treat (and convert) that disk as a *normal* SCSI disk. I think it's actually fine: from the inside, there is no way to tell such a floppy drive apart from a larger/faster/actual SCSI disk.

Comment 7 Laszlo Ersek 2022-11-09 12:39:39 UTC
I wonder if iSCSI disks are also presented as "removable". If so, then trying to open the device (just to see if ENOMEDIUM can be triggered) will cause a small performance penalty.

Comment 8 Laszlo Ersek 2022-11-09 12:58:58 UTC
IMO the ENOMEDIUM idea is supported by the kernel source itself; see sd_open() in "drivers/scsi/sd.c":

        /*
         * If the drive is empty, just let the open fail.
         */
        retval = -ENOMEDIUM;
        if (sdev->removable && !sdkp->media_present && !(mode & FMODE_NDELAY))
                goto error_out;

Comment 9 Laszlo Ersek 2022-11-09 13:58:42 UTC
[p2v PATCH] disks.c: skip SCSI floppy drives with no disk inserted
Message-Id: <20221109135222.12387-1-lersek>
(not visible in the list archive just yet, but it was "queued as 4E648492B17" by smtp.corp.redhat.com)

Tingting, the ISO image including the above patch is at <http://lacos.interhost.hu/exclude-floppies-rhbz-2140997/b19895a5acd1/livecd-p2v-202211091434.iso>, the sha256 sum is b0666a9140b03e12829982179bf7da2ac5477737fb53760d2e8c527d8a2bf55a.

Can you please download the image and test it on your machine? The expected behavior is:

- (soft) iSCSI disks are listed as before (after appropriate setup in the XTerm window, of course),
- local hard disks are listed just as before,
- USB drives (USB-connected SSDs, and USB flash sticks) are listed just as before,
- the empty SCSI floppy drive is not listed at all.

Thanks!

Comment 10 Laszlo Ersek 2022-11-09 14:16:24 UTC
Tingting, if you have an external USB SD card reader handy, checking that would be nice as well -- with a formatted SD card inserted, the disk should appear (as a fixed disk), and with an SD card not inserted, the disk should not appear at all.

(Pre-patch, I think the device node appears even with a card not inserted.)

Comment 11 tingting zheng 2022-11-10 09:40:24 UTC
Tested with the new virt-p2v iso:
livecd-p2v-202211091434.iso

After launch virt-p2v client, tested the below scenarios:
1.(soft) iSCSI disks are listed as before (after appropriate setup in the XTerm window, of course) - PASS, tested by Vera Wu
2. local hard disks are listed just as before - PASS
3. USB disks are listed just as before - PASS
4. the empty SCSI floppy drive is not listed at all - PASS

For comment 10, as our test machine is located in the lab, I contacted lab team for help, they only have USB disk, so external USB SD card reader is not tested. 

I think it's enough to verify the bug, so close the bug.

Comment 12 Laszlo Ersek 2022-11-10 14:00:50 UTC
Thank you for the testing! Once Rich approves the patch, I'll merge it, add the commit hash here, and also build a new P2V ISO (from the master branch then, with the patch merged). The above ISO was built from the current master branch, plus my proposed (but not yet merged) patch. Normally I'm supposed to test patches extensively before submitting them, but this time I couldn't do it, due to lack of hardware.

Comment 13 Laszlo Ersek 2022-11-15 14:27:44 UTC
Upstream commit 2082cf98add8. (New ISO to follow later.)

Comment 14 Laszlo Ersek 2022-11-15 14:34:46 UTC
Actually the ISO doesn't need rebuilding -- while the commit hash has changed (from b19895a5acd1 to 2082cf98add8), the tree contents remain identical. So I've just moved the ISO to a "less experimental" location:

  http://lacos.interhost.hu/livecd-p2v-202211091434.iso

Sha256 (unchanged, relative to comment 9): b0666a9140b03e12829982179bf7da2ac5477737fb53760d2e8c527d8a2bf55a.


Note You need to log in before you can comment on or make changes to this bug.