Created attachment 886436 [details] logs Description of problem: we fail to create an image when using a gluster backend for glance on a rhel7 installation. Version-Release number of selected component (if applicable): [root@orange-vdse ~(keystone_admin)]# cat /etc/redhat-release Red Hat Enterprise Linux Server release 7.0 (Maipo) [root@orange-vdse ~(keystone_admin)]# rpm -qa |grep glance openstack-glance-2014.1-0.4.b3.el7.noarch python-glance-2014.1-0.4.b3.el7.noarch python-glanceclient-0.12.0-1.el7.noarch [root@orange-vdse ~(keystone_admin)]# rpm -qa | grep fuse glusterfs-fuse-3.4.0.59rhs-1.el7.x86_64 fuse-2.9.2-5.el7.x86_64 fuse-libs-2.9.2-5.el7.x86_64 xxx.x.xxx.redhat.com:/gluster-setup-glance on /var/lib/gluster type fuse.glusterfs (rw,relatime,user_id=0,group_id=0,default_permissions,allow_other,max_read=131072) xxx.xxx.xx.redhat.com:/gluster-setup-cinder on /var/lib/cinder/mnt/41f8312f185e18c3a7789258935c4874 type fuse.glusterfs (rw,relatime,user_id=0,group_id=0,default_permissions,allow_other,max_read=131072) How reproducible: Steps to Reproduce: 1. 2. 3. Actual results: Expected results: Additional info: [root@orange-vdse ~(keystone_admin)]# glance image-create --name tcms4755754v1 --disk-format qcow2 --container-format bare --file cirros-0.3.2-x86_64-disk.img --progress [=============================>] 100% +------------------+--------------------------------------+ | Property | Value | +------------------+--------------------------------------+ | checksum | 64d7c1cd2b6f60c92c14662941cb7913 | | container_format | bare | | created_at | 2014-04-15T10:08:40 | | deleted | False | | deleted_at | None | | disk_format | qcow2 | | id | 5a15f399-98d2-4795-84de-ab8161c007c6 | | is_public | False | | min_disk | 0 | | min_ram | 0 | | name | tcms4755754v1 | | owner | c7bff83ba61e47b892aa68f0e902af54 | | protected | False | | size | 13167616 | | status | active | | updated_at | 2014-04-15T10:08:40 | | virtual_size | None | +------------------+--------------------------------------+ [root@orange-vdse ~(keystone_admin)]# glance image-list +--------------------------------------+---------------+-------------+------------------+----------+--------+ | ID | Name | Disk Format | Container Format | Size | Status | +--------------------------------------+---------------+-------------+------------------+----------+--------+ | bde0b1b2-ad30-40a9-ae40-c56ae5aff425 | cirros | qcow2 | bare | 13167616 | active | | f42a5abf-540e-419a-a88b-947817b578e5 | cirros2 | raw | bare | 9761280 | active | | 5a15f399-98d2-4795-84de-ab8161c007c6 | tcms4755754v1 | qcow2 | bare | 13167616 | active | +--------------------------------------+---------------+-------------+------------------+----------+--------+ [root@orange-vdse ~(keystone_admin)]# setenforce 1 [root@orange-vdse ~(keystone_admin)]# glance image-create --name dafna --disk-format qcow2 --container-format bare --file cirros-0.3.2-x86_64-disk.img --progress [=============================>] 100% Request returned failure status. 500 Internal Server Error Failed to upload image 4c54caf5-da10-4515-b0b7-731d755fe49e (HTTP 500) 2014-04-15 13:09:10.781 15602 ERROR glance.store.filesystem [c73cf6b4-8a9b-4a55-9b19-c7d2ba3fd495 48023ec51465475a9b69724a3b0374ec c7bff83ba61e47b892aa68f0e902af54 - - -] Unable to create datadir: /var/lib/gluster/images 2014-04-15 13:09:10.781 15602 WARNING glance.store.base [c73cf6b4-8a9b-4a55-9b19-c7d2ba3fd495 48023ec51465475a9b69724a3b0374ec c7bff83ba61e47b892aa68f0e902af54 - - -] Failed to configure store correctly: Store filesystem could not be con figured correctly. Reason: Unable to create datadir: /var/lib/gluster/images Disabling add method. 2014-04-15 13:09:10.782 15602 DEBUG glance.api.v1.images [c73cf6b4-8a9b-4a55-9b19-c7d2ba3fd495 48023ec51465475a9b69724a3b0374ec c7bff83ba61e47b892aa68f0e902af54 - - -] Setting image 4c54caf5-da10-4515-b0b7-731d755fe49e to status 'saving' _upload /usr/lib/python2.7/site-packages/glance/api/v1/images.py:589 2014-04-15 13:09:10.782 15602 DEBUG glance.registry.client.v1.api [c73cf6b4-8a9b-4a55-9b19-c7d2ba3fd495 48023ec51465475a9b69724a3b0374ec c7bff83ba61e47b892aa68f0e902af54 - - -] Updating image metadata for image 4c54caf5-da10-4515-b0b7-73 1d755fe49e... update_image_metadata /usr/lib/python2.7/site-packages/glance/registry/client/v1/api.py:166 2014-04-15 13:09:10.783 15602 DEBUG glance.common.client [c73cf6b4-8a9b-4a55-9b19-c7d2ba3fd495 48023ec51465475a9b69724a3b0374ec c7bff83ba61e47b892aa68f0e902af54 - - -] Constructed URL: http://0.0.0.0:9191/images/4c54caf5-da10-4515-b0b7-7 31d755fe49e _construct_url /usr/lib/python2.7/site-packages/glance/common/client.py:411 2014-04-15 13:09:10.822 15602 DEBUG glance.registry.client.v1.client [c73cf6b4-8a9b-4a55-9b19-c7d2ba3fd495 48023ec51465475a9b69724a3b0374ec c7bff83ba61e47b892aa68f0e902af54 - - -] Registry request PUT /images/4c54caf5-da10-4515-b0b7-731d 755fe49e HTTP 200 request id req-2b693a23-7d90-44d5-93d6-1ffa1873ffe1 do_request /usr/lib/python2.7/site-packages/glance/registry/client/v1/client.py:114 2014-04-15 13:09:10.823 15602 DEBUG glance.api.v1.images [c73cf6b4-8a9b-4a55-9b19-c7d2ba3fd495 48023ec51465475a9b69724a3b0374ec c7bff83ba61e47b892aa68f0e902af54 - - -] Uploading image data for image 4c54caf5-da10-4515-b0b7-731d755fe49e t o file store _upload /usr/lib/python2.7/site-packages/glance/api/v1/images.py:595 2014-04-15 13:09:10.824 15602 DEBUG oslo.messaging._drivers.amqp [c73cf6b4-8a9b-4a55-9b19-c7d2ba3fd495 48023ec51465475a9b69724a3b0374ec c7bff83ba61e47b892aa68f0e902af54 - - -] UNIQUE_ID is 8afd944ad14a4d84a658b16473f10c9a. _add_unique_id /usr/lib/python2.7/site-packages/oslo/messaging/_drivers/amqp.py:338 2014-04-15 13:09:10.831 15602 ERROR glance.api.v1.upload_utils [c73cf6b4-8a9b-4a55-9b19-c7d2ba3fd495 48023ec51465475a9b69724a3b0374ec c7bff83ba61e47b892aa68f0e902af54 - - -] Failed to upload image 4c54caf5-da10-4515-b0b7-731d755fe49e 2014-04-15 13:09:10.831 15602 TRACE glance.api.v1.upload_utils Traceback (most recent call last): 2014-04-15 13:09:10.831 15602 TRACE glance.api.v1.upload_utils File "/usr/lib/python2.7/site-packages/glance/api/v1/upload_utils.py", line 99, in upload_data_to_store 2014-04-15 13:09:10.831 15602 TRACE glance.api.v1.upload_utils store) 2014-04-15 13:09:10.831 15602 TRACE glance.api.v1.upload_utils File "/usr/lib/python2.7/site-packages/glance/store/__init__.py", line 366, in store_add_to_backend 2014-04-15 13:09:10.831 15602 TRACE glance.api.v1.upload_utils (location, size, checksum, metadata) = store.add(image_id, data, size) 2014-04-15 13:09:10.831 15602 TRACE glance.api.v1.upload_utils File "/usr/lib/python2.7/site-packages/glance/store/base.py", line 123, in add_disabled 2014-04-15 13:09:10.831 15602 TRACE glance.api.v1.upload_utils raise exception.StoreAddDisabled 2014-04-15 13:09:10.831 15602 TRACE glance.api.v1.upload_utils StoreAddDisabled: Configuration for store failed. Adding images to this store is disabled. 2014-04-15 13:09:10.831 15602 TRACE glance.api.v1.upload_utils 2014-04-15 13:09:10.832 15602 DEBUG glance.registry.client.v1.api [c73cf6b4-8a9b-4a55-9b19-c7d2ba3fd495 48023ec51465475a9b69724a3b0374ec c7bff83ba61e47b892aa68f0e902af54 - - -] Updating image metadata for image 4c54caf5-da10-4515-b0b7-731d755fe49e... update_image_metadata /usr/lib/python2.7/site-packages/glance/registry/client/v1/api.py:166 2014-04-15 13:09:10.833 15602 DEBUG glance.common.client [c73cf6b4-8a9b-4a55-9b19-c7d2ba3fd495 48023ec51465475a9b69724a3b0374ec c7bff83ba61e47b892aa68f0e902af54 - - -] Constructed URL: http://0.0.0.0:9191/images/4c54caf5-da10-4515-b0b7-731d755fe49e _construct_url /usr/lib/python2.7/site-packages/glance/common/client.py:411 2014-04-15 13:09:10.876 15602 DEBUG glance.registry.client.v1.client [c73cf6b4-8a9b-4a55-9b19-c7d2ba3fd495 48023ec51465475a9b69724a3b0374ec c7bff83ba61e47b892aa68f0e902af54 - - -] Registry request PUT /images/4c54caf5-da10-4515-b0b7-731d755fe49e HTTP 200 request id req-d7565a65-1bd7-40b4-a17f-67fc5e6ae7df do_request /usr/lib/python2.7/site-packages/glance/registry/client/v1/client.py:114 2014-04-15 13:09:10.877 15602 DEBUG oslo.messaging._drivers.amqp [c73cf6b4-8a9b-4a55-9b19-c7d2ba3fd495 48023ec51465475a9b69724a3b0374ec c7bff83ba61e47b892aa68f0e902af54 - - -] UNIQUE_ID is bf7d7b3f76574442a620751551b4c937. _add_unique_id /usr/lib/python2.7/site-packages/oslo/messaging/_drivers/amqp.py:338 2014-04-15 13:09:10.936 15602 INFO glance.wsgi.server [c73cf6b4-8a9b-4a55-9b19-c7d2ba3fd495 48023ec51465475a9b69724a3b0374ec c7bff83ba61e47b892aa68f0e902af54 - - -] 10.35.104.16 - - [15/Apr/2014 13:09:10] "POST /v1/images HTTP/1.1" 500 293 0.370931 type=AVC msg=audit(1397556520.202:53952): avc: denied { add_name } for pid=15602 comm="glance-api" name="5a15f399-98d2-4795-84de-ab8161c007c6" scontext=system_u:system_r:glance_api_t:s0 tcontext=system_u:object_r:fusefs_t:s0 tclass=di r type=AVC msg=audit(1397556520.202:53952): avc: denied { create } for pid=15602 comm="glance-api" name="5a15f399-98d2-4795-84de-ab8161c007c6" scontext=system_u:system_r:glance_api_t:s0 tcontext=system_u:object_r:fusefs_t:s0 tclass=file type=AVC msg=audit(1397556520.202:53952): avc: denied { write } for pid=15602 comm="glance-api" path="/var/lib/gluster/images/5a15f399-98d2-4795-84de-ab8161c007c6" dev="fuse" ino=11882381175005752851 scontext=system_u:system_r:glance_ api_t:s0 tcontext=system_u:object_r:fusefs_t:s0 tclass=file type=SYSCALL msg=audit(1397556520.202:53952): arch=c000003e syscall=2 success=yes exit=12 a0=32210d0 a1=241 a2=1b6 a3=37342d326438392d items=0 ppid=15574 pid=15602 auid=4294967295 uid=161 gid=161 euid=161 suid=161 fsuid=161 egid=161 sgid =161 fsgid=161 tty=(none) ses=4294967295 comm="glance-api" exe="/usr/bin/python2.7" subj=system_u:system_r:glance_api_t:s0 key=(null) type=MAC_STATUS msg=audit(1397556540.085:53953): enforcing=1 old_enforcing=0 auid=0 ses=8106 type=SYSCALL msg=audit(1397556540.085:53953): arch=c000003e syscall=1 success=yes exit=1 a0=3 a1=7fffc8d429d0 a2=1 a3=7fffc8d42790 items=0 ppid=3990 pid=4296 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts0 ses=810 6 comm="setenforce" exe="/usr/sbin/setenforce" subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 key=(null) type=USER_ACCT msg=audit(1397556541.545:53954): pid=4297 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:accounting acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=succ ess' type=CRED_ACQ msg=audit(1397556541.545:53955): pid=4297 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:setcred acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success' type=LOGIN msg=audit(1397556541.546:53956): pid=4297 uid=0 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 old-auid=4294967295 auid=0 old-ses=4294967295 ses=8110 res=1 type=USER_AVC msg=audit(1397556541.557:53957): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='avc: received setenforce notice (enforcing=1) exe="/usr/lib/systemd/systemd" sauid=0 hostname=? addr=? term inal=?' type=USER_START msg=audit(1397556541.566:53958): pid=4297 uid=0 auid=0 ses=8110 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:session_open acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success' type=CRED_REFR msg=audit(1397556541.566:53959): pid=4297 uid=0 auid=0 ses=8110 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:setcred acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success' type=CRED_DISP msg=audit(1397556542.087:53960): pid=4297 uid=0 auid=0 ses=8110 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:setcred acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success' type=USER_END msg=audit(1397556542.088:53961): pid=4297 uid=0 auid=0 ses=8110 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:session_close acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success' type=AVC msg=audit(1397556550.776:53962): avc: denied { search } for pid=15602 comm="glance-api" name="/" dev="fuse" ino=1 scontext=system_u:system_r:glance_api_t:s0 tcontext=system_u:object_r:fusefs_t:s0 tclass=dir type=SYSCALL msg=audit(1397556550.776:53962): arch=c000003e syscall=4 success=no exit=-13 a0=319a430 a1=7fff788441c0 a2=7fff788441c0 a3=62696c2f7261762f items=0 ppid=15574 pid=15602 auid=4294967295 uid=161 gid=161 euid=161 suid=161 fsuid =161 egid=161 sgid=161 fsgid=161 tty=(none) ses=4294967295 comm="glance-api" exe="/usr/bin/python2.7" subj=system_u:system_r:glance_api_t:s0 key=(null) type=AVC msg=audit(1397556550.779:53963): avc: denied { getattr } for pid=15602 comm="glance-api" path="/var/lib/gluster" dev="fuse" ino=1 scontext=system_u:system_r:glance_api_t:s0 tcontext=system_u:object_r:fusefs_t:s0 tclass=dir type=SYSCALL msg=audit(1397556550.779:53963): arch=c000003e syscall=4 success=no exit=-13 a0=30c3f70 a1=7fff78843fc0 a2=7fff78843fc0 a3=0 items=0 ppid=15574 pid=15602 auid=4294967295 uid=161 gid=161 euid=161 suid=161 fsuid=161 egid=161 s gid=161 fsgid=161 tty=(none) ses=4294967295 comm="glance-api" exe="/usr/bin/python2.7" subj=system_u:system_r:glance_api_t:s0 key=(null) type=AVC msg=audit(1397556550.779:53964): avc: denied { search } for pid=15602 comm="glance-api" name="/" dev="fuse" ino=1 scontext=system_u:system_r:glance_api_t:s0 tcontext=system_u:object_r:fusefs_t:s0 tclass=dir type=SYSCALL msg=audit(1397556550.779:53964): arch=c000003e syscall=83 success=no exit=-13 a0=30c3f70 a1=1ff a2=7f4784798f88 a3=62696c2f7261762f items=0 ppid=15574 pid=15602 auid=4294967295 uid=161 gid=161 euid=161 suid=161 fsuid=161 egi d=161 sgid=161 fsgid=161 tty=(none) ses=4294967295 comm="glance-api" exe="/usr/bin/python2.7" subj=system_u:system_r:glance_api_t:s0 key=(null) type=AVC msg=audit(1397556550.779:53965): avc: denied { search } for pid=15602 comm="glance-api" name="/" dev="fuse" ino=1 scontext=system_u:system_r:glance_api_t:s0 tcontext=system_u:object_r:fusefs_t:s0 tclass=dir type=SYSCALL msg=audit(1397556550.779:53965): arch=c000003e syscall=4 success=no exit=-13 a0=30c3f70 a1=7fff788441c0 a2=7fff788441c0 a3=62696c2f7261762f items=0 ppid=15574 pid=15602 auid=4294967295 uid=161 gid=161 euid=161 suid=161 fsuid :
Created attachment 888198 [details] audit.log
#============= glance_api_t ============== allow glance_api_t fusefs_t:dir rw_dir_perms; allow glance_api_t fusefs_t:file manage_file_perms; These permissions maybe can be cleaned up by using rw_fir_perms and manage_file_perms What do you think Miroslav?
I think this needs to shipped in selinux policy
Yes. We need to add commit 67fec5977c3cf6370933180adf151fcefc29f050 Author: Miroslav Grepl <mgrepl> Date: Thu Apr 24 15:03:43 2014 +0200 Add glance_use_fusefs() boolean
Will be a part of RHEL6.6. Is it needed also for RHEL6.5?
RHEL 6.6 is fine.
Hey Dafna, can you do 'ls -lZ /var/lib/gluster' ? I'm curious as to what context that directory has - and if it's what it should be.
that setup no longer exists but I ran it on a new rhel7+RDO setup installed by Tzach today: [root@puma31 ~]# ls -lZ /mnt/gluster/ drwxr-xr-x glance glance ? images drwxr-xr-x glance glance ? instance
This selinux boolean (glance_use_fusefs) is included in current selinux-policy releases.