Bug 1592916

*** Bug 1643326 has been marked as a duplicate of this bug. ***

This bug has not been marked as blocker for oVirt 4.3.0.
Since we are releasing it tomorrow, January 29th, this bug has been re-targeted to 4.3.1.

I believe this cannot be tested with an updated sanlock?

We are far from QA, I don't know why
this moved to modified.

This bug is not ready yet. Looks like the CI scripts are confused.

This cannot be ready for 4.3.2, moving to 4.3.3.

*** Bug 1364869 has been marked as a duplicate of this bug. ***

Hi Tal,

Looks like automation bot moved this bug to MODIFIED a few days ago but this bug has many patches still in POST .
Please move back to POST and retarget to 4.3 zstream (currently targeted to 4.3.3)

Hi Tal,

Once more the automation bot moved this bug to MODIFIED(the bug has many patches still in POST)

Please move it back to POST.

Hi,

Commit https://github.com/oVirt/ovirt-engine/commit/e2e04e82351a6aea9c160b0da967de7ed39e5f24#diff-6db200f01d6157a55a94e3797f9724c7
which changed in oVirt v4.3+ default Storage Domain version to V5 has caused reapperance of BUG https://bugzilla.redhat.com/show_bug.cgi?id=1669606

Currently I'm not able to migrate Hosted Engine to new oVirt host with backup-restore procedure:

# hosted-engine --deploy --restore-from-file=/root/HE_BCK_2019.05.28.bck

At storage selection I setup NFS share from company's NAS storage. Each try has failed:

          Please specify the storage you would like to use (glusterfs, iscsi, fc, nfs)[nfs]: nfs
          Please specify the nfs version you would like to use (auto, v3, v4, v4_1)[auto]: v4
          Please specify the full shared storage connection path to use (example: host:/path): mynfsserver:/he_nas
          If needed, specify additional mount options for the connection to the hosted-engine storagedomain (example: rsize=32768,wsize=32768) []:
[ INFO  ] Creating Storage Domain
[ INFO  ] TASK [ovirt.hosted_engine_setup : Execute just a specific set of steps]
...
[ INFO  ] TASK [ovirt.hosted_engine_setup : Activate storage domain]
[ ERROR ] Error: Fault reason is "Operation Failed". Fault detail is "[Storage format is unsupported]". HTTP response code is 400.
[ ERROR ] fatal: [localhost]: FAILED! => {"changed": false, "msg": "Fault reason is \"Operation Failed\". Fault detail is \"[Storage format is unsupported]\". HTTP response code is 400."}
          Please specify the storage you would like to use (glusterfs, iscsi, fc, nfs)[nfs]:

		  
In Hosted Engine VM in /var/log/ovirt-engine/engine.log the error logs are:
2019-05-28 19:25:13,357+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-1) [cba462c2-1d73-4978-89fd-f6db40162601] EVENT_ID: IRS_BROKER_COMMAND_FAILURE(10,803), VDSM command AttachStorageomainVDS failed: Domain version `5` is unsupported by this version of VDSM: ''
2019-05-28 19:25:13,357+02 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.AttachStorageDomainVDSCommand] (default task-1) [cba462c2-1d73-4978-89fd-f6db40162601] Command 'AttachStorageDomainVDSCommand( AttachStorageDomainVDSCommandarameters:{storagePoolId='4669438a-3f99-11e9-aef4-00163e3912a0', ignoreFailoverLimit='false', storageDomainId='51ef1a02-271e-4e27-96e8-dfa1886917c7'})' execution failed: IRSGenericException: IRSErrorException: Domain version `5` is nsupported by this version of VDSM: ''
2019-05-28 19:25:13,358+02 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.AttachStorageDomainVDSCommand] (default task-1) [cba462c2-1d73-4978-89fd-f6db40162601] FINISH, AttachStorageDomainVDSCommand, return: , log id: 580e7243
2019-05-28 19:25:13,358+02 ERROR [org.ovirt.engine.core.bll.storage.domain.AttachStorageDomainToPoolCommand] (default task-1) [cba462c2-1d73-4978-89fd-f6db40162601] Command 'org.ovirt.engine.core.bll.storage.domain.AttachStorageDomanToPoolCommand' failed: EngineException: org.ovirt.engine.core.vdsbroker.irsbroker.IrsOperationFailedNoFailoverException: IRSGenericException: IRSErrorException: Domain version `5` is unsupported by this version of VDSM: '' (Failed ith error UnsupportedDomainVersion and code 394)
2019-05-28 19:25:13,367+02 INFO  [org.ovirt.engine.core.bll.CommandCompensator] (default task-1) [cba462c2-1d73-4978-89fd-f6db40162601] Command [id=0e8d4f68-9e59-4944-a3d1-89b591ff811c]: Compensating NEW_ENTITY_ID of org.ovirt.engin.core.common.businessentities.StoragePoolIsoMap; snapshot: StoragePoolIsoMapId:{storagePoolId='4669438a-3f99-11e9-aef4-00163e3912a0', storageId='51ef1a02-271e-4e27-96e8-dfa1886917c7'}.
2019-05-28 19:25:13,397+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-1) [cba462c2-1d73-4978-89fd-f6db40162601] EVENT_ID: USER_ATTACH_STORAGE_DOMAIN_TO_POOL_FAILED(963), Failed to attah Storage Domain hosted_storage to Data Center *******. (User: admin@internal-authz)
2019-05-28 19:25:13,407+02 INFO  [org.ovirt.engine.core.bll.storage.domain.AttachStorageDomainToPoolCommand] (default task-1) [cba462c2-1d73-4978-89fd-f6db40162601] Lock freed to object 'EngineLock:{exclusiveLocks='[51ef1a02-271e-4e7-96e8-dfa1886917c7=STORAGE]', sharedLocks=''}'
2019-05-28 19:25:13,407+02 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default task-1) [] Operation Failed: [Storage format is unsupported]

All packages are at most recent versions.

(In reply to Szymon Madej from comment #12)
Please file a new bug, or reopen bug 1669606 if you think that this is the same bug.

This is RFE for supporting 4k storage, not the place for tracking bugs, even if they
are related to 4k related code.

Since sanlock packages are now available as per  bug 1690511, is this feature ready to be tested

*** Bug 1720923 has been marked as a duplicate of this bug. ***

*** Bug 1711054 has been marked as a duplicate of this bug. ***

(In reply to Sahina Bose from comment #15)
> Since sanlock packages are now available as per  bug 1690511, is this
> feature ready to be tested

Not yet, sanlock is required but we need to add 4k support in:
- vdsm (in progress, mostly done)
- engine (mostly done)
- ovirt-imageio (work not started yet)
- hosted engine (status unknown)

When all parts are done, we can start testing RHHI.

We depend on ioprocess 1.2.1. Should be released for 4.3.6.

We depend also on 4k support in ovirt-imageio (bug 1729809).

INFO: Bug status wasn't changed from MODIFIED to ON_QA due to the following reason:

[Open patch attached]

For more info please contact: infra

Testing on Fedora 29 hosts and gluster storage show that qemu does not detect
the block size of the underlying storage, so provisioning VMs on gluster 4k
storage fails. The issue is tracked in bug 1737256.

We can work around the issue described in bug 1737256 by adding <blockio> element
to libvirt xml (see https://gerrit.ovirt.org/c/102308/). With this we can provision
a VM, but it will not boot. The issue is tracked in bug 1737268.

To test with gluster 4k storage, you need to enable gluster 4k support in vdsm.

Create a drop-in configuration file:
    
$ cat /etc/vdsm/vdsm.conf.d/gluster.conf
[gluster]
enable_4k_storage = true
    
And restart vdsm to load the new configuration.

With tiny qemu patch:
https://lists.nongnu.org/archive/html/qemu-block/2019-08/msg00133.html

I can provision VMs on gluster 4k storage when using fuse mount.

I don't think it will fix libgfapi, need to test.

(In reply to Nir Soffer from comment #25)
> With tiny qemu patch:
> https://lists.nongnu.org/archive/html/qemu-block/2019-08/msg00133.html
> 
> I can provision VMs on gluster 4k storage when using fuse mount.
> 
> I don't think it will fix libgfapi, need to test.

We don't need libgfapi support - don't bother.

(In reply to Yaniv Kaul from comment #26)
> We don't need libgfapi support - don't bother.

RHHI does not use libgfapi?

(In reply to Nir Soffer from comment #27)
> (In reply to Yaniv Kaul from comment #26)
> > We don't need libgfapi support - don't bother.
> 
> RHHI does not use libgfapi?

Never, because not all features worked well with libgfapi (live migration? don't remember right now).

Another issue with qemu is copying disks. qemu-img convert fails to reading from
gluster 4k storage because of alignment issue. With the same fix for bug 1737256,
the read issue if fixed. However there is another issue with detecting block
size of the target image, causing writes to gluster 4k storage to fail.
The issue is tracked in bug 1738657.

We can workaround this issue in oVirt by using the -n option to qemu convert.
We will provide a workaround in the next 4.3.6 build.

(In reply to Nir Soffer from comment #29)
> We can workaround this issue in oVirt by using the -n option to qemu convert.
> We will provide a workaround in the next 4.3.6 build.

But if we use the -n option, image preallocation during convert will be broken.
The issue is tracked in bug 1738721.

Tested with RHV 4.3.8 + RHGS 3.5.1 ( glusterfs-6.0-29.el7rhgs )

1. All the RHHI-V core cases are run with 4K disks as well as with 4K VDO devices.
2. Deployment and functionally works good without any issues

Tested with RHV 4.3.8 + RHGS 3.5.1 ( glusterfs-6.0-29.el7rhgs )

1. All the RHHI-V core cases are run with 4K disks as well as with 4K VDO devices.
2. Deployment and functionally works good without any issues

*** Bug 1574681 has been marked as a duplicate of this bug. ***

This bugzilla is included in oVirt 4.3.8 release, published on January 27th 2020.

Since the problem described in this bug report should be
resolved in oVirt 4.3.8 release, it has been closed with a resolution of CURRENT RELEASE.

If the solution does not work for you, please open a new bug report.