Today, you cannot have hosts with local storage in the "Shared" Datacenter, and you cannot have more than one host in the "Local" datacenter. Having multiple hypervisor, with local storage but while sharing other attributes such as networks, or even a mix of local and shared storage, is a basic feature. In bug 1052318, which added the possibility to use several types of shared storage types in a "Shared" datacenter, there was a comment that the feature I am referring too is tracked under bug 757291 (which is private). For some reason, bug 757291 was marked as a duplicate of bug 1052318, which is not the same scenario. As said, this is a basic feature and oVirt should probably provide it (other visualization solutions provides it - But we want to use KVM :) ).
This would be a great capability to have in oVirt; the ability to have both shared and local storage simultaneously. There are other major hypervisors currently, which support both simultaneously. Any way this can be targeted for 3.6 & 4.0 releases?
(In reply to Charlie Inglese from comment #1) > This would be a great capability to have in oVirt; the ability to have both > shared and local storage simultaneously. There are other major hypervisors > currently, which support both simultaneously. > > Any way this can be targeted for 3.6 & 4.0 releases? We don't add features to 3.6, and for 4.0 - probably a bit late to target it. Note that it is dependant on another feature (bug 1302185 ) which is already targeted to a future release.
*** Bug 1331481 has been marked as a duplicate of this bug. ***
Bug 1302185 offers a limited solution here - it allows you to attach domains of "shared" types to a local DC, and convert a local DC to a shared one if it doesn't have any local domains. This will be delivered in oVirt 4.1. The further step required here is to break down the limitation that all hosts in the DC must be able to access all the domains. We're working on it, but we aren't there yet.
*** Bug 1337554 has been marked as a duplicate of this bug. ***
*** Bug 1144494 has been marked as a duplicate of this bug. ***
*** Bug 1453112 has been marked as a duplicate of this bug. ***
Single node gluster volume can be used as storage domain with oVirt 4.2. Moving this to ON_QA
But, as I understand, gluster is a shared type of storage? The case is about having BOTH shared and local storage available within the cluster.
(In reply to Sergei from comment #10) > But, as I understand, gluster is a shared type of storage? > The case is about having BOTH shared and local storage available within the > cluster. With gluster, you can use the local storage to create a gluster volume with a single brick - and use this as storage domain. Yes, the volume is shared storage in that it can be accessed by all hosts in the cluster. Yaniv, do you want to add any more details here?
I think, this is wrong interpretation of initial request. It is not about inventing a workaround, it about a way to use storage systems. Example: converged storage systems are able to create "local" disk on each hypervisor, but this disk is in fact a shared resource (assume, it's storage system's magic). When server writes to this disk - it writes to shared storage resource, which is accessible by each system within cluster. Think about it as NFS share, mounted on each hypervisor, but which is using other than NFS protocol. There's no need to create additional gluster overhead on usch share - you will kill the performance. All that is necessary - is to allow all hosts to believe, that they really have shared storage, despit eof the fact that it looks like local folder.
(In reply to Sergei from comment #12) > I think, this is wrong interpretation of initial request. It is not about > inventing a workaround, it about a way to use storage systems. > > Example: converged storage systems are able to create "local" disk on each > hypervisor, but this disk is in fact a shared resource (assume, it's storage > system's magic). When server writes to this disk - it writes to shared > storage resource, which is accessible by each system within cluster. Think > about it as NFS share, mounted on each hypervisor, but which is using other > than NFS protocol. > > There's no need to create additional gluster overhead on usch share - you > will kill the performance. All that is necessary - is to allow all hosts to > believe, that they really have shared storage, despit eof the fact that it > looks like local folder. This is what Gluster enables for local storage in oVirt. It is both a local and shared storage provisioned by the manager. The overhead on single brick volume, depending on the workload, is not big. To get manageability you incur some overhead, but this gives you much more function as well.
The overhead is present, as soon as you put two file systems over raw device: 1. xfs filesystem on raw disk 2. Gluster over xfs Are there any estimates, what amount of performance (in terms of latency and iops) will the overhead take? It might happen, that you implement the feature, but nobody uses it, because it's too slow?
Also, additional design question - how is this design with gluster different from setting up an NFS share? Only protocol name changes, all other stays the same. Then, why bothering with gluster?
(In reply to Sergei from comment #15) > Also, additional design question - how is this design with gluster different > from setting up an NFS share? Only protocol name changes, all other stays > the same. Then, why bothering with gluster? I have the same opinion as Sergei. I already use a "local" NFS storage to workaround a missing real local storage feature. It's not optimal but it gives relatively good performance. I don't see any point in using Gluster because my main goals here are the performance and it would add some more overhead and complexity. In my usage, (real) local storage is the only feature that is missing from oVirt.
(In reply to Yoann Laissus from comment #16) > (In reply to Sergei from comment #15) > > Also, additional design question - how is this design with gluster different > > from setting up an NFS share? Only protocol name changes, all other stays > > the same. Then, why bothering with gluster? > > I have the same opinion as Sergei. I already use a "local" NFS storage to > workaround a missing real local storage feature. It's not optimal but it > gives relatively good performance. > > I don't see any point in using Gluster because my main goals here are the > performance and it would add some more overhead and complexity. > > In my usage, (real) local storage is the only feature that is missing from > oVirt. Thanks for that feedback. I recommend you to create a separate RFE for the real 'local' storage feature. This bug will be used to track the efforts taken to use single brick gluster storage volume with Local DC.
Verified with glusterfs-3.8.4. 1. Manage multiple hosts in local DC 2. Created gluster storage domain backed with single brick ( on the local disk of one of the host ) gluster volume. 3. Created few VMs. All looks good.
Why did you write it is tech preview? It is fully supported.
(In reply to Yaniv Lavi from comment #19) > Why did you write it is tech preview? > It is fully supported. The original doc text said it was tech preview. I'll remove that sentence.