Bug 1430536

Description of problem:
RGW: using swift CLI create/update container ACL failed with keystone 

Version-Release number of selected component (if applicable):
Red Hat Ceph Storage 1.3.3 - 0.94.9-9.el7cp
Red Hat Openstack Platform 8

How reproducible:
Always

Steps to Reproduce:
1. Integrate RGW with OpenStack keystone
2. Trying to set the ACL
3. Setting ACL fails

Able to reproduce in the test environment.

Additional info:
=======================

We had some known issues:

http://tracker.ceph.com/issues/15976
http://tracker.ceph.com/issues/16694 
Fixed in jewel: 10.2.3 

But these both the bugs tell that the issue is only in updating the container(post) ACL but it works well while creating the container with ACL.

I am only able to set read ACL, write ACL is not working with keystone integration.

Without keystone integration, all works fine.

Test cases and test results:
--------------------------------

- Without keystone integration:


# swift -A http://radosgw1.gsslab.com/auth/v2 -U testuser:swift -K 'i5dF5PdRIVonV26l6xZfNbm7BSI4qh8tDZzYtzGc' post test-bucket -w "testuser2"
# swift -A http://radosgw1.gsslab.com/auth/v2 -U testuser:swift -K 'i5dF5PdRIVonV26l6xZfNbm7BSI4qh8tDZzYtzGc' stat test-bucket
                      Account: v1
                    Container: test-bucket
                      Objects: 2
                        Bytes: 41487824
                     Read ACL:
                    Write ACL: testuser2 <===================================
                      Sync To:
                     Sync Key:
                Accept-Ranges: bytes
             X-Storage-Policy: default-placement
X-Container-Bytes-Used-Actual: 41492480
                  X-Timestamp: 1485992096.00000
                   X-Trans-Id: tx000000000000000000006-0058b8972e-1c4986e-default
                 Content-Type: text/plain; charset=utf-8


# swift -A http://radosgw1.gsslab.com/auth/v2 -U testuser:swift -K 'i5dF5PdRIVonV26l6xZfNbm7BSI4qh8tDZzYtzGc' post test-bucket -w "testuser2" -r "testuser2"
#  swift -A http://radosgw1.gsslab.com/auth/v2 -U testuser:swift -K 'i5dF5PdRIVonV26l6xZfNbm7BSI4qh8tDZzYtzGc' stat test-bucket
                      Account: v1
                    Container: test-bucket
                      Objects: 2
                        Bytes: 41487824
                     Read ACL: testuser2 <===========
                    Write ACL: testuser2 <=============
                      Sync To:
                     Sync Key:
                Accept-Ranges: bytes
                   X-Trans-Id: tx000000000000000000013-0058b89a59-1c81fab-default
             X-Storage-Policy: default-placement
X-Container-Bytes-Used-Actual: 41492480
                  X-Timestamp: 1485992096.00000
                 Content-Type: text/plain; charset=utf-8



All looks good.
--------------------------

http://docs.ceph.com/docs/master/radosgw/keystone/

[client.rgw.radosgw1]
host = radosgw1
keyring = /etc/ceph/ceph.client.radosgw.keyring
log file = /var/log/radosgw/client.radosgw.gateway.log
rgw_frontends = "civetweb port=80"
#rgw_enable_usage_log = true
rgw keystone url = http://192.168.128.27:35357
rgw keystone admin user = swift
rgw keystone admin password = 80746f03e9584805
rgw keystone admin tenant = services
rgw keystone token cache size = 1000
rgw keystone revocation interval = 600
rgw keystone accepted roles = Member,admin
rgw s3 auth use keystone = true
nss db path = /var/ceph/nss


# openstack endpoint create --region RegionOne \
>      --publicurl   "http://radosgw1.redhat.com:80/swift/v1" \
>      --adminurl    "http://radosgw1.redhat.com:80/swift/v1" \
>      --internalurl "http://radosgw1.redhat.com:80/swift/v1" \
>      swift
+--------------+---------------------------------------------------+
| Field        | Value                                             |
+--------------+---------------------------------------------------+
| adminurl     | http://radosgw1.redhat.com:80/swift/v1 |
| id           | 44047368c03148a7815f8307d995584b                  |
| internalurl  | http://radosgw1.redhat.com:80/swift/v1 |
| publicurl    | http://radosgw1.redhat.com:80/swift/v1 |
| region       | RegionOne                                         |
| service_id   | 3d763eaafd5a458cbdd2dcca8b696b97                  |
| service_name | swift                                             |
| service_type | object-store                                      |
+--------------+---------------------------------------------------+


mkdir /var/ceph/nss

openssl x509 -in /etc/keystone/ssl/certs/ca.pem -pubkey | \
        certutil -d /var/ceph/nss -A -n ca -t "TCu,Cu,Tuw"
openssl x509 -in /etc/keystone/ssl/certs/signing_cert.pem -pubkey | \
        certutil -A -d /var/ceph/nss -n signing_cert -t "P,P,P"

# swift post test-bucket4
[root@dell-per630-5 ~(keystone_admin)]# swift list 
test-bucket4

# swift upload test-bucket4 packstack-remove.sh
packstack-remove.sh


# swift stat test-bucket4
                      Account: v1
                    Container: test-bucket4
                      Objects: 1
                        Bytes: 1159
                     Read ACL:
                    Write ACL:
                      Sync To:
                     Sync Key:
                Accept-Ranges: bytes
             X-Storage-Policy: default-placement
X-Container-Bytes-Used-Actual: 4096
                  X-Timestamp: 1488992278.00000
                   X-Trans-Id: tx000000000000000000019-0058c0399e-1ca3f90-default
                 Content-Type: text/plain; charset=utf-8


# swift list test-bucket4
packstack-remove.sh


# radosgw-admin bucket stats --bucket=test-bucket4
{
    "bucket": "test-bucket4",
    "pool": ".rgw.buckets",
    "index_pool": ".rgw.buckets.index",
    "id": "default.30031760.1",
    "marker": "default.30031760.1",
    "owner": "9330794e3e614cc084d0aca90c919837",
    "ver": "0#3",
    "master_ver": "0#0",
    "mtime": "2017-03-08 22:27:58.000000",
    "max_marker": "0#",
    "usage": {
        "rgw.main": {
            "size_kb": 2,
            "size_kb_actual": 4,
            "num_objects": 1
        }
    },
    "bucket_quota": {
        "enabled": false,
        "max_size_kb": -1,
        "max_objects": -1
    }
}


# cat keystonerc_swift 
unset OS_SERVICE_TOKEN
export OS_USERNAME=swift
export OS_PASSWORD=80746f03e9584805
export OS_AUTH_URL=http://192.168.128.27:5000/v2.0
export PS1='[\u@\h \W(keystone_swift)]\$ '

export OS_TENANT_NAME=services
export OS_REGION_NAME=RegionOne

# swift post test-bucket5
[root@dell-per630-5 ~(keystone_swift)]# swift list
test-bucket5
[root@dell-per630-5 ~(keystone_swift)]# swift upload test-bucket5 packstack-remove.sh
packstack-remove.sh
[root@dell-per630-5 ~(keystone_swift)]# swift list test-bucket5
packstack-remove.sh

# swift post -w "*:*" test-bucket5
[root@dell-per630-5 ~(keystone_swift)]# swift stat test-bucket5
                      Account: v1
                    Container: test-bucket5
                      Objects: 1
                        Bytes: 1159
                     Read ACL:
                    Write ACL:
                      Sync To:
                     Sync Key:
                Accept-Ranges: bytes
             X-Storage-Policy: default-placement
X-Container-Bytes-Used-Actual: 4096
                  X-Timestamp: 1488993222.00000
                   X-Trans-Id: tx000000000000000000014-0058c03cf4-1c77945-default
                 Content-Type: text/plain; charset=utf-8

2017-03-08 22:53:59.506398 7f518efed700  1 ====== starting new request req=0x7f51c8021fa0 =====
2017-03-08 22:53:59.602991 7f518efed700  0 validated token: services:swift expires: 1488997440
2017-03-08 22:53:59.612915 7f518efed700  1 ====== req done req=0x7f51c8021fa0 http_status=204 ======
2017-03-08 22:53:59.612960 7f518efed700  1 civetweb: 0x7f51c80056c0: 10.74.128.27 - - [08/Mar/2017:22:53:59 +0530] "POST /swift/v1/test-bucket5 HTTP/1.1" 204 0 - python-swiftclient-2.6.0


# openstack user create testswift --password testswift
+----------+----------------------------------+
| Field    | Value                            |
+----------+----------------------------------+
| email    | None                             |
| enabled  | True                             |
| id       | 25ff852798de400891613b271320d2b7 |
| name     | testswift                        |
| username | testswift                        |
+----------+----------------------------------+

https://docs.openstack.org/developer/swift/overview_acl.html
https://www.swiftstack.com/docs/cookbooks/swift_usage/container_acl.html

# swift post www --read-acl ".r:*" --write-acl "*:*"
[root@dell-per630-5 ~(keystone_swift)]# swift stat www
                      Account: v1
                    Container: www
                      Objects: 0
                        Bytes: 0
                     Read ACL: .r:*
                    Write ACL:
                      Sync To:
                     Sync Key:
                Accept-Ranges: bytes
                   X-Trans-Id: tx000000000000000000026-0058c078d3-1ca4db2-default
             X-Storage-Policy: default-placement
X-Container-Bytes-Used-Actual: 0
                  X-Timestamp: 1489008843.00000
                 Content-Type: text/plain; charset=utf-8


If we check above only read ACL is set.