Bug 2269428

Worked with Eran, and found a solution. I was getting confused about bucket policies and lifecycle configurations. I got it working, but this isn't documented by us, this was in [1] (we should document this or I can create a KCS for it):

[system:admin/openshift-storage  root ~]$ aws s3api put-bucket-lifecycle-configuration --bucket test123-febe138a-6632-41c3-a54c-3d3adbd2b57a --lifecycle-configuration file://retention.json --endpoint-url https://s3-openshift-storage.apps.ocp4.cluster.lab --no-verify-ssl
/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py:1052: InsecureRequestWarning: Unverified HTTPS request is being made to host 's3-openshift-storage.apps.ocp4.cluster.lab'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings
  InsecureRequestWarning,
[system:admin/openshift-storage  root ~]$ 
[system:admin/openshift-storage  root ~]$ 
[system:admin/openshift-storage  root ~]$ 
[system:admin/openshift-storage  root ~]$ aws s3api get-bucket-lifecycle-configuration --bucket test123-febe138a-6632-41c3-a54c-3d3adbd2b57a  --endpoint-url https://s3-openshift-storage.apps.ocp4.cluster.lab --no-verify-ssl
/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py:1052: InsecureRequestWarning: Unverified HTTPS request is being made to host 's3-openshift-storage.apps.ocp4.cluster.lab'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings
  InsecureRequestWarning,
{
    "Rules": [
        {
            "Expiration": {
                "Days": 1
            },
            "ID": "id-1",
            "Filter": {
                "And": {
                    "Prefix": "myprefix",
                    "Tags": [
                        {
                            "Key": "mytagkey1",
                            "Value": "mytagvalue1"
                        },
                        {
                            "Key": "mytagkey2",
                            "Value": "mytagvalue2"
                        }
                    ]
                }
            },
            "Status": "Enabled"
        }
    ]
}

However, trying to deploy a bucket policy using [2] still fails:

[system:admin/openshift-storage  root ~]$ aws --endpoint  https://s3-openshift-storage.apps.ocp4.cluster.lab --no-verify-ssl s3api put-bucket-policy --bucket test123-febe138a-6632-41c3-a54c-3d3adbd2b57a --policy bucket-pol.json
/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py:1052: InsecureRequestWarning: Unverified HTTPS request is being made to host 's3-openshift-storage.apps.ocp4.cluster.lab'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings
  InsecureRequestWarning,

An error occurred (InvalidRequest) when calling the PutBucketPolicy operation: SOAP requests must be made over an HTTPS connection.
[system:admin/openshift-storage  root ~]$ cat bucket-pol.json
{
    "Version": "NewVersion",
    "Statement": [
        {
            "Sid": "Example",
            "Effect": "Allow",
            "Principal": [
                    "john.doe"
            ],
            "Action": [
                "s3:GetObject"
            ],
            "Resource": [
                "arn:aws:s3:::john_bucket"
            ]
        }
    ]
}

To avoid any confusion, this doc BZ will be used to update the command in [2] on how to apply a bucket policy to a noobaa bucket. Please reach out with any questions.

[1] https://docs.aws.amazon.com/AmazonS3/latest/userguide/how-to-set-lifecycle-configuration-intro.html
[2] https://access.redhat.com/documentation/en-us/red_hat_openshift_data_foundation/4.14/html-single/managing_hybrid_and_multicloud_resources/index#using-bucket-policies

More testing... I found the issue and source of my confusion...

[system:admin/openshift-storage  root ~]$ noobaa account create kelson --allow_bucket_create=true 
INFO[0000] ✅ Exists: NooBaa "noobaa"                    
INFO[0000] ❌ Invalid: NooBaaAccount "kelson": NooBaaAccount.noobaa.io "kelson" is invalid: metadata.name: Invalid value: "kelson": a lowercase RFC 1123 subdomain must consist of lower case alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character (e.g. 'example.com', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*') 
FATA[0000] ❌ Could not create noobaaAccount "kelson" in Namespace "openshift-storage" (conflict) 
[system:admin/openshift-storage  root ~]$ noobaa account create kelson --allow_bucket_create=true 
INFO[0000] ✅ Exists: NooBaa "noobaa"                    
INFO[0000] ✅ Created: NooBaaAccount "kelson"            
INFO[0000]                                              
INFO[0000] NOTE:                                        
INFO[0000]   - This command has finished applying changes to the cluster. 
INFO[0000]   - From now on, it only loops and reads the status, to monitor the operator work. 
INFO[0000]   - You may Ctrl-C at any time to stop the loop and watch it manually. 
INFO[0000]                                              
INFO[0000] NooBaaAccount Wait Ready:                    
INFO[0000] ⏳ NooBaaAccount "kelson" Phase is "": waiting... 
INFO[0003] ✅ NooBaaAccount "kelson" Phase is Ready      
INFO[0003]                                              
INFO[0003]                                              
INFO[0003] ✅ Exists: NooBaaAccount "kelson"             
INFO[0003] ✅ NooBaaAccount "kelson" Phase is Ready      

# NooBaaAccount spec:
allow_bucket_creation: true
default_resource: noobaa-default-backing-store

INFO[0003] ✅ Exists: Secret "noobaa-account-kelson"     
Connection info:
  AWS_ACCESS_KEY_ID      : ****
  AWS_SECRET_ACCESS_KEY  : ****

[system:admin/openshift-storage  root ~]$ oc get noobaaaccount
NAME     PHASE   AGE
kelson   Ready   12s
[system:admin/openshift-storage  root ~]$ vim bucket-policy.json 
[system:admin/openshift-storage  root ~]$ aws --endpoint https://s3-openshift-storage.apps.ocp4.cluster.lab --no-verify-ssl s3api put-bucket-policy --bucket test123-febe138a-6632-41c3-a54c-3d3adbd2b57a --policy file://bucket-policy.json
/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py:1052: InsecureRequestWarning: Unverified HTTPS request is being made to host 's3-openshift-storage.apps.ocp4.cluster.lab'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings
  InsecureRequestWarning,
[system:admin/openshift-storage  root ~]$ cat bucket-policy.json 
{
    "Version": "NewVersion",
    "Statement": [
        {
            "Sid": "Example",
            "Effect": "Allow",
            "Principal": [
                    "kelson"
            ],
            "Action": [
                "s3:GetObject"
            ],
            "Resource": [
                "arn:aws:s3:::test123-febe138a-6632-41c3-a54c-3d3adbd2b57a"
            ]
        }
    ]
}

Seems to have worked! Confirming... 

[system:admin/openshift-storage  root ~]$ noobaa account create kelson-test --allow_bucket_create=true 
INFO[0000] ✅ Exists: NooBaa "noobaa"                    
INFO[0000] ✅ Created: NooBaaAccount "kelson-test"       
INFO[0000]                                              
INFO[0000] NOTE:                                        
INFO[0000]   - This command has finished applying changes to the cluster. 
INFO[0000]   - From now on, it only loops and reads the status, to monitor the operator work. 
INFO[0000]   - You may Ctrl-C at any time to stop the loop and watch it manually. 
INFO[0000]                                              
INFO[0000] NooBaaAccount Wait Ready:                    
INFO[0000] ⏳ NooBaaAccount "kelson-test" Phase is "": waiting... 
INFO[0003] ✅ NooBaaAccount "kelson-test" Phase is Ready 
INFO[0003]                                              
INFO[0003]                                              
INFO[0003] ✅ Exists: NooBaaAccount "kelson-test"        
INFO[0003] ✅ NooBaaAccount "kelson-test" Phase is Ready 

# NooBaaAccount spec:
allow_bucket_creation: true
default_resource: noobaa-default-backing-store

INFO[0003] ✅ Exists: Secret "noobaa-account-kelson-test" 
Connection info:
  AWS_ACCESS_KEY_ID      : ****
  AWS_SECRET_ACCESS_KEY  : ****
[system:admin/openshift-storage  root ~]$ NOOBAA_ACCESS_KEY=$(kubectl get secret noobaa-account-kelson-test -n openshift-storage -o json | jq -r '.data.AWS_ACCESS_KEY_ID|@base64d')
[system:admin/openshift-storage  root ~]$ NOOBAA_SECRET_KEY=$(kubectl get secret noobaa-account-kelson-test -n openshift-storage -o json | jq -r '.data.AWS_SECRET_ACCESS_KEY|@base64d')
[system:admin/openshift-storage  root ~]$ s3 ls
/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py:1052: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings
  InsecureRequestWarning,
[system:admin/openshift-storage  root ~]$ s3 ls s3://test123-febe138a-6632-41c3-a54c-3d3adbd2b57a
/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py:1052: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings
  InsecureRequestWarning,

An error occurred (AccessDenied) when calling the ListObjectsV2 operation: Access Denied
[system:admin/openshift-storage  root ~]$ 

Doing this testing made me realize where my confusion lay. In this section in [1], we give a command using ' --policy BucketPolicy':

```
 Using AWS S3 client, use the put-bucket-policy command to apply the bucket policy to your S3 bucket:

# aws --endpoint ENDPOINT --no-verify-ssl s3api put-bucket-policy --bucket MyBucket --policy BucketPolicy

    Replace ENDPOINT with the S3 endpoint.
    Replace MyBucket with the bucket to set the policy on.
    Replace BucketPolicy with the bucket policy JSON file.

    Add --no-verify-ssl if you are using the default self signed certificates.
```

which won't work. However, in the 'For example' section we provide the proper way to run it using '--policy file://BucketPolicy':

```
    For example:

    # aws --endpoint https://s3-openshift-storage.apps.gogo44.noobaa.org --no-verify-ssl s3api put-bucket-policy -bucket MyBucket --policy file://BucketPolicy
```

Can we update this doc to have the first command using '--policy file://BucketPolicy'? Also, since having a valid user account is required for the `Principle` section of the json file, we should add 'Creating a user in the Multicloud Object Gateway' to the  'Prerequisites' section... something like:

```
Prerequisites

    A running OpenShift Data Foundation Platform.
    Access to the Multicloud Object Gateway (MCG), see Chapter 2, Accessing the Multicloud Object Gateway with your applications 
    A valid user account (what's being used in the Principal section of the json file that houses the policy definitions) is required when deploying a bucket policy, see 7.3. Creating a user in the Multicloud Object Gateway to create a user account
```

[1] https://access.redhat.com/documentation/en-us/red_hat_openshift_data_foundation/4.14/html-single/managing_hybrid_and_multicloud_resources/index#using-bucket-policies

Good catch!
Moving to docs