Bug 2077649

The documentation text flag should only be set after 'doc text' field is provided. Please provide the documentation text and set the flag to '?' again.

ovirt-imageio supports can write to any storage supported by qemu-nbd.
The issue is in engine, responsible for orchestration of image transfers.

The problem is that engine does not support image transfer to managed
block storage domains. Adding support for MBS disks should not be hard,
it is just missing feature.

With the current system the only way to upload images is to create a data
domain using another type of storage (e.g. NFS), and upload the images
to the data domain, and move the disk to the MBS domain.

To implement this support for MBS disks, we need:

Engine UI:
- Prevent upload of qcow2 disks to MBS domain, since we don't support converting
  the format during the upload from the UI.

Engine backend:
- Instead of preparing the image before the upload, attach the disk to the host
- Instead of tearing down the image after the upload, detach the disk to the host

Vdsm:
- Starting NBD server in assumes an image (PDIV) add support for MBS volume
  (basically attached volume at /run/vdsm/managedvolume/uuid)

JC, coping an image from data storage domain (for example the master domain) to MBS
should work (but of course never tested with NVMe/tcp storage.

Please report another bug for this, and include complete vdsm log showing the copy
flow. We should see the logs attaching the volume to the host, and the copy_data
logs. Grepping for the volume UUID in the log should show the right log file(s).

Vdsm API need to change to support MBS volume. Currently is expects
PDIV:

    NBDServerConfig: &NBDServerConfig
        added: '4.3' 
        description: NBD server configuration.
        name: NBDServerConfig
        properties:
        -   description: The storage domain UUID. 
            name: sd_id 
            type: *UUID 

        -   description: The image UUID. 
            name: img_id
            type: *UUID 

        -   description: The volume UUID. 
            name: vol_id
            type: *UUID 

        -   defaultvalue: false 
            description: Allow read only access.
            name: readonly
            type: boolean

        -   defaultvalue: false 
            description: Pass discard requests to underlying filesystem. By
                default discard requests are ignored. Recommended when
                uploading to sparse volume.
            name: discard
            type: boolean

        -   defaultvalue: false 
            description: Try to keep the volume sparse by converting zeroes in
              write commands to optimized write zeroes commands. If discard is
              enabled, zeroes writes deallocate space. By default zeroes are
              not detected. Recommended when uploading to a sparse volume.
            name: detect_zeroes
            type: boolean
            added: '4.5' 

        -   defaultvalue: true
            description: If true, expose the backing chain of volume using 
                qcow2 format. If false, expose only the specified volume.
            name: backing_chain
            type: boolean

        -   defaultvalue: null
            description: If set, the NBD server exports the specified dirty 
              bitmap. NBD client can use the dirty bitmap to download changed
              blocks when a VM is powered off.
            name: bitmap
            type: *UUID 

We need to add support for external volumes, similar to the way we support 
external volumes in SDM.copy_data(). However in this case we don't have 
an endpoint. We can deprecate "sd_id", "img_id", "vol_id" and add "endpoint"
which can use the DIVEndpoint and ExternalEndpoint like SDM.copy_data. Engine
can use the endpoint API with MBS volume, and the older API for images.

Hi Nir,

created https://bugzilla.redhat.com/show_bug.cgi?id=2077649 for the copy failure.

Included in the new BZ the UUIDs for localdomain, MBS domain, source and target image.

Look for line 37542 for the beginning of the copy sequence

Extract below
37542 2022-04-21 15:30:01,899-0400 INFO  (jsonrpc/3) [api.storage] START attach_volume(vol_id='f7b88e89-14b8-4384-ac3a-76a29c465bf1', connection_info={'driver_volume_type': 'lightos', 'data': {'lightos_nodes': {'10.100.10.101': {'target_portal': '10.100.10.101', 'target_port': 8009, 'transport_type': 'tcp'}, '10.100.10      .103': {'target_portal': '10.100.10.103', 'target_port': 8009, 'transport_type': 'tcp'}, '10.100.10.102': {'target_portal': '10.100.10.102', 'target_port': 8009, 'transport_type': 'tcp'}}, 'uuid': '3ae21b9e-a59d-4be7-b44f-80d1462d4d94', 'subsysnqn': 'nqn.2016-01.com.lightbitslabs:uuid:e42ee459-90d6-481f-a096-63aa      95248ba1', 'access_mode': 'rw'}}) from=::ffff:10.20.60.35,48428, flow_id=2d9862bf (api:48)
37543 2022-04-21 15:30:01,900-0400 INFO  (jsonrpc/3) [storage.managevolumedb] Adding volume f7b88e89-14b8-4384-ac3a-76a29c465bf1 connection_info={'driver_volume_type': 'lightos', 'data': {'lightos_nodes': {'10.100.10.101': {'target_portal': '10.100.10.101', 'target_port': 8009, 'transport_type': 'tcp'}, '10.100.10.103'      : {'target_portal': '10.100.10.103', 'target_port': 8009, 'transport_type': 'tcp'}, '10.100.10.102': {'target_portal': '10.100.10.102', 'target_port': 8009, 'transport_type': 'tcp'}}, 'uuid': '3ae21b9e-a59d-4be7-b44f-80d1462d4d94', 'subsysnqn': 'nqn.2016-01.com.lightbitslabs:uuid:e42ee459-90d6-481f-a096-63aa95248      ba1', 'access_mode': 'rw'}} (managedvolumedb:181)

Best regards
JC

(In reply to Nir Soffer from comment #3)
> JC, coping an image from data storage domain (for example the master domain)
> to MBS
> should work (but of course never tested with NVMe/tcp storage.
> 
> Please report another bug for this, and include complete vdsm log showing
> the copy
> flow. We should see the logs attaching the volume to the host, and the
> copy_data
> logs. Grepping for the volume UUID in the log should show the right log
> file(s).

Bug 2077666 tracks the copy issue.

This sounds like a reasonable change for upstream, filed https://github.com/oVirt/ovirt-engine/issues/313 - contribution is welcome
Until then, we better block uploading an image to MBS domain

This bugzilla is included in oVirt 4.5.2 release, published on August 10th 2022.
Since the problem described in this bug report should be resolved in oVirt 4.5.2 release, it has been closed with a resolution of CURRENT RELEASE.
If the solution does not work for you, please open a new bug report.