Bug 1022961 - Gluster: running a VM from a gluster domain should use gluster URI instead of a fuse mount
Summary: Gluster: running a VM from a gluster domain should use gluster URI instead of...
Status: CLOSED CURRENTRELEASE
Alias: None
Product: vdsm
Classification: oVirt
Component: General
Version: ---
Hardware: Unspecified
OS: Unspecified
high
high with 6 votes vote
Target Milestone: ovirt-4.1.5
: 4.19.27
Assignee: Denis Chaplygin
QA Contact: SATHEESARAN
URL:
Whiteboard:
Keywords: Improvement
: 1175800 1177776 (view as bug list)
Depends On: 1032370 1181466 1247521 1247933 1397870
Blocks: oVirt_Hosted_Engine_GlusterFS Generic_Hyper_Converged_Host 1201355 1225425 1322852 Gluster-HC-3 1488995
TreeView+ depends on / blocked
 
Reported: 2013-10-24 11:15 UTC by Aharon Canan
Modified: 2019-04-28 11:14 UTC (History)
40 users (show)

(edit)
This release adds libgfapi support to the Manager and VDSM. libgfapi provides virtual machines with faster access to their images, stored on a Gluster volume, compared to a fuse interface. With the 'LibgfApi' data center feature enabled, or 'lubgfapi_supported' cluster-level feature enabled, virtual machines will access their images, stored on Gluster volumes, directly via libgfapi.
Clone Of:
: 1488995 (view as bug list)
(edit)
Last Closed: 2017-08-23 08:05:26 UTC
ylavi: ovirt-4.1+
rule-engine: exception+
ylavi: requirements_defined?
ylavi: planning_ack+
sabose: devel_ack+
sasundar: testing_ack+


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
oVirt gerrit 21059 None None None Never
oVirt gerrit 41899 master MERGED core: Gluster disk as network device 2017-07-23 14:06 UTC
oVirt gerrit 44061 master MERGED virt: enable glusterfs access through libgfapi interface 2017-07-19 09:05 UTC
oVirt gerrit 56906 None None None 2016-09-22 07:51 UTC
oVirt gerrit 71457 master MERGED engine: Enable per cluster setting for gfapi access 2017-07-23 14:06 UTC
oVirt gerrit 73585 master MERGED storage: Add LibgfApi support as additional feature 2017-07-19 09:05 UTC
oVirt gerrit 73709 master MERGED storage: Add index support for blockcommit 2017-05-22 13:22 UTC
oVirt gerrit 74189 master MERGED storage: Add volume_path method 2017-03-23 16:42 UTC
oVirt gerrit 76362 ovirt-4.1 MERGED errors: Introduce the common.errors module 2017-05-30 07:04 UTC
oVirt gerrit 76363 ovirt-4.1 MERGED errors: Use new errors.Base class 2017-05-30 07:04 UTC
oVirt gerrit 76364 ovirt-4.1 MERGED storage: Add volume_path method 2017-05-30 07:04 UTC
oVirt gerrit 76365 ovirt-4.1 MERGED storage: add parse_volume_chain method 2017-05-30 07:03 UTC
oVirt gerrit 76366 ovirt-4.1 MERGED storage: extracted path_to_vol_id 2017-05-30 07:03 UTC
oVirt gerrit 76367 ovirt-4.1 MERGED storage: Added tests for parse_volume_chain covering error handling. 2017-06-07 08:03 UTC
oVirt gerrit 76979 ovirt-4.1 MERGED storage: extracted get_snapshot_xml 2017-06-13 14:05 UTC
oVirt gerrit 76980 ovirt-4.1 MERGED storage: Added tests for get_snapshot_xml 2017-06-14 08:20 UTC
oVirt gerrit 76981 ovirt-4.1 MERGED storage: Replaced strings with constants in get_disk_snapshot_xml 2017-06-14 09:17 UTC
oVirt gerrit 77023 ovirt-4.1 MERGED vm: Remove unneeded return value handling 2017-06-14 09:26 UTC
oVirt gerrit 77024 ovirt-4.1 MERGED common: Made UnsupportedOperation subclass of a ContextException 2017-06-14 09:26 UTC
oVirt gerrit 77025 ovirt-4.1 MERGED storage: Made get_snapshot_xml a member of Drive. 2017-06-14 09:38 UTC
oVirt gerrit 77026 ovirt-4.1 MERGED storage: Made get_snapshot_xml use _getSourceXml 2017-06-14 09:46 UTC
oVirt gerrit 77332 ovirt-4.1 MERGED storage: Add index support for blockcommit 2017-06-23 16:01 UTC
oVirt gerrit 77527 ovirt-4.1 MERGED storage: Add tests for creating snapshot xml for network drive 2017-06-23 16:02 UTC
oVirt gerrit 77528 ovirt-4.1 MERGED storage: Add support for parsing drive xml for network drives 2017-06-29 07:13 UTC
oVirt gerrit 77630 ovirt-4.1 MERGED cmdutils: Fix message format 2017-06-01 11:05 UTC
oVirt gerrit 77641 master ABANDONED storage: Implemented caching of Drive's diskType detection results. 2017-06-05 11:17 UTC
oVirt gerrit 77642 master ABANDONED storage: Made disk type a constructor parameter of Drive. 2017-06-05 12:19 UTC
oVirt gerrit 77643 master MERGED storage: Made diskType a real property. 2017-06-29 15:24 UTC
oVirt gerrit 77935 master MERGED storage: Added tests for diskType 2017-06-29 15:24 UTC
oVirt gerrit 77936 master MERGED storage: Added retrieval of bricks participating in gluster volume 2017-07-18 11:02 UTC
oVirt gerrit 78557 ovirt-4.1 MERGED virt: Set correct diskType for snapshot target drive 2017-06-29 07:13 UTC
oVirt gerrit 78558 ovirt-4.1 MERGED vm: Block commit call sould use drive name when using indexes. 2017-07-04 07:29 UTC
oVirt gerrit 78559 ovirt-4.1 MERGED virt: Added network drive support to _changeDisk 2017-07-04 07:39 UTC
oVirt gerrit 78837 master MERGED storage: Added disk type change logging 2017-07-03 19:52 UTC
oVirt gerrit 78937 ovirt-4.1 MERGED storage: Added tests for diskType 2017-07-05 06:32 UTC
oVirt gerrit 78938 ovirt-4.1 MERGED storage: Made diskType a real property. 2017-07-05 06:32 UTC
oVirt gerrit 78991 ovirt-4.1 MERGED storage: Added disk type change logging 2017-07-05 06:32 UTC
oVirt gerrit 79555 ovirt-4.1 MERGED storage: Added retrieval of bricks participating in gluster volume 2017-07-27 11:47 UTC
oVirt gerrit 79604 ovirt-4.1 MERGED virt: enable glusterfs access through libgfapi interface 2017-07-27 11:47 UTC
oVirt gerrit 79605 ovirt-4.1 MERGED storage: Add LibgfApi support as additional feature 2017-07-27 11:47 UTC
oVirt gerrit 79745 ovirt-engine-4.1 MERGED core: Gluster disk as network device 2017-08-02 15:58 UTC
oVirt gerrit 79746 ovirt-engine-4.1 MERGED engine: Enable per cluster setting for gfapi access 2017-08-02 15:58 UTC
oVirt gerrit 79785 master MERGED config: Made LibGfApi configurable using engine-config 2017-08-01 13:20 UTC
oVirt gerrit 79800 ovirt-4.1 MERGED Revert "Revert "storage: Made diskType a real property."" 2017-07-27 11:46 UTC
oVirt gerrit 79801 ovirt-4.1 MERGED Revert "Revert "storage: Added disk type change logging"" 2017-07-27 11:47 UTC
oVirt gerrit 79802 ovirt-4.1 MERGED storage: Handle inaccessible storage when creating a drive 2017-07-27 11:47 UTC
oVirt gerrit 79814 ovirt-4.1 MERGED tests: Add failing test when storage is not available 2017-07-27 11:47 UTC
oVirt gerrit 79895 master MERGED engine: libgfapi feature and cluster level feature have different names. 2017-08-01 13:20 UTC
oVirt gerrit 80061 ovirt-engine-4.1 MERGED config: Made LibGfApi configurable using engine-config 2017-08-02 15:58 UTC
oVirt gerrit 80062 ovirt-engine-4.1 MERGED engine: libgfapi feature and cluster level feature have different names. 2017-08-02 15:58 UTC
Red Hat Bugzilla 1017289 None None None Never

Internal Trackers: 1017289

Comment 3 Aharon Canan 2013-10-27 11:59:37 UTC
following comment #1, 
Can't reproduce using is13 as we couldn't run VM on glusterFS using that build.

Comment 8 R P Herrold 2013-11-13 15:59:26 UTC
The comment was made in the ovirt weekly meeting today:

10:54 < abaron> mburns: note that problem is not libgfapi per se but
                rather lack of support in libvirt for snapshots over
                libgfapi


but I do not see a Depends or link to a RFE in libvirt for it ... does one exist?

Comment 16 Sandro Bonazzola 2015-04-23 11:50:54 UTC
Looks like the root cause for gluster storage domain disappearing is that the mount process is executed by vdsm with the same CGroup so the restart of vdsm cause systemd to kill also the mount process.

If you can drop the mount it will solve also bug #1201355

Comment 17 Sandro Bonazzola 2015-04-24 12:09:43 UTC
Dropping dependency on 1017289 which has been closed wontfix.
VDSM won't provide 3.6 support on EL6 so no reason to be blocked by an EL6 wontfix bug while the EL7 has been closed current release.

Moving 1017289 to see also, just for reference.

Comment 18 Sandro Bonazzola 2015-04-30 06:43:57 UTC
*** Bug 1177776 has been marked as a duplicate of this bug. ***

Comment 19 Yaniv Lavi 2015-07-16 04:07:47 UTC
should this be on MODIFIED?
Is there anything left to do from the RHEV side?

Comment 20 Allon Mureinik 2015-07-16 09:03:14 UTC
(In reply to Yaniv Dary from comment #19)
> should this be on MODIFIED?
> Is there anything left to do from the RHEV side?
Yes - to write code that makes the VM use the URI, test it and merge it :-)

Comment 21 Andrejs Baulins 2015-09-21 18:51:22 UTC
Is it possible to get some clarifications?

As far as I understand, will be implemented in 4.0.0
Currently, gluster URI is not able to be used by vdsm, disk always be accessed via mount point, not as network block device.
Practically, it means that FUSE overhead still affects VMs.

Why, in that case, oVirt feature page says feature is implemented?
Just gives instructions, how to avoid https://bugzilla.redhat.com/show_bug.cgi?id=1181466

oVirt page: http://www.ovirt.org/Features/GlusterFS_Storage_Domain

May be, it is possible to run VMs in recommended way, but there is some workaround how to run VM with GlusterFS URI (with only one gluster volume to avoid https://bugzilla.redhat.com/show_bug.cgi?id=1247521 )?

Is it true?
If yes, is it possible to provide instruction for that to be tried?

Comment 22 Allon Mureinik 2015-09-21 20:53:01 UTC
(In reply to Andrejs Baulins from comment #21)
> Is it possible to get some clarifications?
> 
> As far as I understand, will be implemented in 4.0.0
> Currently, gluster URI is not able to be used by vdsm, disk always be
> accessed via mount point, not as network block device.
> Practically, it means that FUSE overhead still affects VMs.
> 
> Why, in that case, oVirt feature page says feature is implemented?
> Just gives instructions, how to avoid
> https://bugzilla.redhat.com/show_bug.cgi?id=1181466
> 
> oVirt page: http://www.ovirt.org/Features/GlusterFS_Storage_Domain
> 
> May be, it is possible to run VMs in recommended way, but there is some
> workaround how to run VM with GlusterFS URI (with only one gluster volume to
> avoid https://bugzilla.redhat.com/show_bug.cgi?id=1247521 )?
> 
> Is it true?
> If yes, is it possible to provide instruction for that to be tried?

I believe the feature page is out of date.
This feature was implemented in 3.3, but reverted before its GA as it broke snapshots.

In any event, it is not available in the current oVirt release.

Comment 23 SATHEESARAN 2015-09-22 02:50:16 UTC
> 
> In any event, it is not available in the current oVirt release.

Is it planned for future ?
Using libgfapi ( QEMU native driver for GlusterFS ) has lots of performance improvements over fuse mounted gluster volume.

Comment 24 Yaniv Lavi 2015-09-22 06:38:36 UTC
(In reply to SATHEESARAN from comment #23)
> > 
> > In any event, it is not available in the current oVirt release.
> 
> Is it planned for future ?
> Using libgfapi ( QEMU native driver for GlusterFS ) has lots of performance
> improvements over fuse mounted gluster volume.

This is why this ticket exists. To track this change.

Comment 25 Andrejs Baulins 2015-09-22 07:02:40 UTC
I believe, these 2 issues must be linked:
https://bugzilla.redhat.com/show_bug.cgi?id=1175800
with status "depends" or "related"

Comment 26 Andrejs Baulins 2015-09-22 10:37:19 UTC
Updated wiki page with current status and main open issues found.
Reader can follow dependent issues himself in order to know current status.

Comment 27 Yaniv Lavi 2016-03-08 10:34:35 UTC
The storage team do not perform any work on HC and the priority of this item will be very low outside the HC use case. We support the Gluster team in their work (review patches and design reviews, for example) and looking to get more req to perform additional work on HC from our side. Currently, however (and unfortunately), nothing is planned from our side. Moving this back.

Comment 28 Sahina Bose 2016-03-31 12:44:18 UTC
*** Bug 1175800 has been marked as a duplicate of this bug. ***

Comment 29 Sahina Bose 2016-03-31 12:56:30 UTC
Reassigning release based on Comment 7 of Bug 1175800, and unblocking on the qemu multiple host requirement, as what we have in functionality would be similar to fuse access now.

Opened a new RFE (Bug 1322852) to add additional host support in gfapi access.

Comment 30 acrow 2016-03-31 12:57:36 UTC
(In reply to Yaniv Dary from comment #27)
> The storage team do not perform any work on HC and the priority of this item
> will be very low outside the HC use case. We support the Gluster team in
> their work (review patches and design reviews, for example) and looking to
> get more req to perform additional work on HC from our side. Currently,
> however (and unfortunately), nothing is planned from our side. Moving this
> back.

I was not trying to do an HC setup when I ran into this issue. It also affects non-HC setups that just happen to have a GlusterFS storage domain.

Comment 31 Sandro Bonazzola 2016-05-02 09:51:04 UTC
Moving from 4.0 alpha to 4.0 beta since 4.0 alpha has been already released and bug is not ON_QA.

Comment 33 Yaniv Lavi 2016-05-23 13:14:44 UTC
oVirt 4.0 beta has been released, moving to RC milestone.

Comment 34 Yaniv Lavi 2016-05-23 13:18:52 UTC
oVirt 4.0 beta has been released, moving to RC milestone.

Comment 36 acrow 2016-05-29 20:38:47 UTC
Can anyone tell me why this still has [HC] in the title? The issue affects ovirt installations using a completely separate glusterfs cluster too.

GlusterFS requirements for use as on oVirt storage domain need to be made much clearer IMHO, eg is replica 3 really required, replica 2 with arbiter, erasure coding. and so on.

Coming into this as a newbie would be very confusing.

Comment 38 Yaniv Lavi 2016-09-22 07:54:58 UTC
Can you please update the bugs with the bug url and cleanup the patch list in this bug?

Comment 40 SATHEESARAN 2017-08-21 13:45:11 UTC
Tested with vdsm-4.19.28-1.el7ev.x86_64

when the engine is configured to use 'libgfapi', the VM disks are accessed via gfapi access mechanism and it works good.


Note You need to log in before you can comment on or make changes to this bug.