Bug 1041737 - [RFE][glance]: Add multifilesystem store to support NFS servers as backend
Summary: [RFE][glance]: Add multifilesystem store to support NFS servers as backend
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-glance
Version: unspecified
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: Upstream M3
: 5.0 (RHEL 7)
Assignee: Flavio Percoco
QA Contact: Dafna Ron
URL: https://blueprints.launchpad.net/glan...
Whiteboard: upstream_milestone_icehouse-3 upstrea...
Depends On:
Blocks: 1081957
TreeView+ depends on / blocked
 
Reported: 2013-12-12 19:06 UTC by RHOS Integration
Modified: 2018-12-05 16:45 UTC (History)
12 users (show)

Fixed In Version: openstack-glance-2014.1-2.el7ost
Doc Type: Enhancement
Doc Text:
Previously, it was not possible to mount all disks to a single directory when configuring multiple NFS servers as a backend using the filesystem store. This was due to the filesystem store only allowing administrators to configure a single directory using the filesystem_store_datadir parameter in the glance-api.conf file. While it is possible to use MHDDFS (a FUSE plug-in: https://romanrm.net/mhddfs), which mounts multiple NFS servers to a single directory, MHDDFS does not allow you to evenly store the data on all the disks. Another major drawback is that it is very difficult to know the number and type of images stored on a disk when one of the disks is broken because the Glance registry stores the location specified in the filesystem_store_datadir parameter. This enhancement fixes the above issues by adding multi-filesystem support to the current filesystem store.
Clone Of:
: 1081957 (view as bug list)
Environment:
Last Closed: 2014-07-08 15:31:48 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHEA-2014:0851 0 normal SHIPPED_LIVE Red Hat Enterprise Linux OpenStack Platform Enhancement - Image Service 2014-07-08 19:22:51 UTC

Description RHOS Integration 2013-12-12 19:06:22 UTC
Cloned from launchpad blueprint https://blueprints.launchpad.net/glance/+spec/glance-multifilesystem-store.

Description:

Description:

Currently if you want to configure multiple nfs server as a backend using filesystem store, you cannot mount all disks to a single directory. filesystem store allows administrator to configure only single directory with filesystem_store_datadir parameter in the glance-api.conf.
It is possible to use mhddfs (fuse plugin :https://romanrm.net/mhddfs) which mounts multiple nfs servers to a single directory but it doesn't allow you to evenly store the data on all the disks. Another major drawback is when one of the disk is broken it is very hard to know how many and what images are stored on that disk as glance registry stores location specified in the filesystem_store_datadir parameter.

With multifilesystem, it is possible to over come above issues.

We are proposing to do following things.
1. Add a new store multifilesystem.
2. Add new parameters in the glance-api.conf

Example

3 nfs servers, exporting 1TB of disks
10.2.3.10
10.2.3.11
10.2.3.12

On Glance server,
Administrator will mount complete export tree as below

10.2.3.10:/export/nfs-partition-1  /var/glance/store-1
10.2.3.11:/export/nfs-partition-1  /var/glance/store-2
10.2.3.12:/export/nfs-partition-1  /var/glance/store-3

Edit glance-api.conf

Add new parameter multifilesystem_store_datadir to configure multiple directories.
#multifilesystem_store_datadir = <directory>:<priority number>
multifilesystem_store_datadir = /var/glance/store-1:100,/var/glance/store-2:200,/var/glance/store-3:300

Priority number: Each directory will be given a priority number.
Priority number with less value will be given top priority. If all directories is given same priority number, then the disk with more available space will be selected for storing image.

Specification URL (additional information):

None

Comment 9 Tzach Shefi 2014-05-13 14:44:57 UTC
launchpad source (and RFE here) includes three errors:

1. This is wrong doesn't work:
multifilesystem_store_datadir = /var/glance/store-1:100,/var/glance/store-2:200,/var/glance/store-3:300

Using this comma separated format -> only one dir is created at this path  "/var/glance/store-1:100,/var/glance/store-2:200,/var/glance/store-3:300" all images are created under this folder, parsing doesn't detect comma. 

The correct format is:  
multifilesystem_store_datadirs= /var/glance/store-1:100    
multifilesystem_store_datadirs= /var/glance/store-2:200
...

2. Note _datadirs instead of  _datadir as appears on launchpad, missing an s. 

3.  filesystem_store_datadir parameter's default value "/var/lib/glance/images" must be cleared when using multifilesystem. This isn't emphasized or clear on conf file description.

Comment 13 errata-xmlrpc 2014-07-08 15:31:48 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHEA-2014-0851.html


Note You need to log in before you can comment on or make changes to this bug.