Description of problem (please be detailed as possible and provide log snippests): Hi, SOAP doesn't allow for non https connections to update bucket policies: ``` InsecureRequestWarning,An error occurred (InvalidRequest) when calling the PutBucketPolicy operation: SOAP requests must be made over an HTTPS connection. ``` According to ,[1] we say to use: ``` # aws --endpoint ENDPOINT --no-verify-ssl s3api put-bucket-policy --bucket MyBucket --policy BucketPolicy ``` running this results in ``` [system:admin/openshift-storage root ~]$ aws --endpoint https://s3-openshift-storage.apps.ocp4.cluster.lab:443 s3api put-bucket-policy --bucket test123-febe138a-6632-41c3-a54c-3d3adbd2b57a --policy retention-policy.json --no-verify-ssl /usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py:1052: InsecureRequestWarning: Unverified HTTPS request is being made to host 's3-openshift-storage.apps.ocp4.cluster.lab'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings InsecureRequestWarning,An error occurred (InvalidRequest) when calling the PutBucketPolicy operation: SOAP requests must be made over an HTTPS connection. ``` Without using the --no-verify-ssl flag I get a cert issue: ``` [system:admin/openshift-storage root ~]$ aws --endpoint https://s3-openshift-storage.apps.ocp4.cluster.lab:443 s3api put-bucket-policy --bucket test123-febe138a-6632-41c3-a54c-3d3adbd2b57a --policy retention-policy.jsonSSL validation failed for https://s3-openshift-storage.apps.ocp4.cluster.lab:443/test123-febe138a-6632-41c3-a54c-3d3adbd2b57a?policy [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:897) ``` So, SOAP requires you to use SSL, which the docs state explicitly not to, why? Am I missing something? Seems the docs are incorrect are ignore the fact applying a bucket policy requires to you use HTTPS [1]https://access.redhat.com/documentation/en-us/red_hat_openshift_data_foundation/4.14/html-single/managing_hybrid_and_multicloud_resources/index#using-bucket-policies
Worked with Eran, and found a solution. I was getting confused about bucket policies and lifecycle configurations. I got it working, but this isn't documented by us, this was in [1] (we should document this or I can create a KCS for it): [system:admin/openshift-storage root ~]$ aws s3api put-bucket-lifecycle-configuration --bucket test123-febe138a-6632-41c3-a54c-3d3adbd2b57a --lifecycle-configuration file://retention.json --endpoint-url https://s3-openshift-storage.apps.ocp4.cluster.lab --no-verify-ssl /usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py:1052: InsecureRequestWarning: Unverified HTTPS request is being made to host 's3-openshift-storage.apps.ocp4.cluster.lab'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings InsecureRequestWarning, [system:admin/openshift-storage root ~]$ [system:admin/openshift-storage root ~]$ [system:admin/openshift-storage root ~]$ [system:admin/openshift-storage root ~]$ aws s3api get-bucket-lifecycle-configuration --bucket test123-febe138a-6632-41c3-a54c-3d3adbd2b57a --endpoint-url https://s3-openshift-storage.apps.ocp4.cluster.lab --no-verify-ssl /usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py:1052: InsecureRequestWarning: Unverified HTTPS request is being made to host 's3-openshift-storage.apps.ocp4.cluster.lab'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings InsecureRequestWarning, { "Rules": [ { "Expiration": { "Days": 1 }, "ID": "id-1", "Filter": { "And": { "Prefix": "myprefix", "Tags": [ { "Key": "mytagkey1", "Value": "mytagvalue1" }, { "Key": "mytagkey2", "Value": "mytagvalue2" } ] } }, "Status": "Enabled" } ] } However, trying to deploy a bucket policy using [2] still fails: [system:admin/openshift-storage root ~]$ aws --endpoint https://s3-openshift-storage.apps.ocp4.cluster.lab --no-verify-ssl s3api put-bucket-policy --bucket test123-febe138a-6632-41c3-a54c-3d3adbd2b57a --policy bucket-pol.json /usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py:1052: InsecureRequestWarning: Unverified HTTPS request is being made to host 's3-openshift-storage.apps.ocp4.cluster.lab'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings InsecureRequestWarning, An error occurred (InvalidRequest) when calling the PutBucketPolicy operation: SOAP requests must be made over an HTTPS connection. [system:admin/openshift-storage root ~]$ cat bucket-pol.json { "Version": "NewVersion", "Statement": [ { "Sid": "Example", "Effect": "Allow", "Principal": [ "john.doe" ], "Action": [ "s3:GetObject" ], "Resource": [ "arn:aws:s3:::john_bucket" ] } ] } To avoid any confusion, this doc BZ will be used to update the command in [2] on how to apply a bucket policy to a noobaa bucket. Please reach out with any questions. [1] https://docs.aws.amazon.com/AmazonS3/latest/userguide/how-to-set-lifecycle-configuration-intro.html [2] https://access.redhat.com/documentation/en-us/red_hat_openshift_data_foundation/4.14/html-single/managing_hybrid_and_multicloud_resources/index#using-bucket-policies
More testing... I found the issue and source of my confusion... [system:admin/openshift-storage root ~]$ noobaa account create kelson --allow_bucket_create=true INFO[0000] ✅ Exists: NooBaa "noobaa" INFO[0000] ❌ Invalid: NooBaaAccount "kelson": NooBaaAccount.noobaa.io "kelson" is invalid: metadata.name: Invalid value: "kelson": a lowercase RFC 1123 subdomain must consist of lower case alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character (e.g. 'example.com', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*') FATA[0000] ❌ Could not create noobaaAccount "kelson" in Namespace "openshift-storage" (conflict) [system:admin/openshift-storage root ~]$ noobaa account create kelson --allow_bucket_create=true INFO[0000] ✅ Exists: NooBaa "noobaa" INFO[0000] ✅ Created: NooBaaAccount "kelson" INFO[0000] INFO[0000] NOTE: INFO[0000] - This command has finished applying changes to the cluster. INFO[0000] - From now on, it only loops and reads the status, to monitor the operator work. INFO[0000] - You may Ctrl-C at any time to stop the loop and watch it manually. INFO[0000] INFO[0000] NooBaaAccount Wait Ready: INFO[0000] ⏳ NooBaaAccount "kelson" Phase is "": waiting... INFO[0003] ✅ NooBaaAccount "kelson" Phase is Ready INFO[0003] INFO[0003] INFO[0003] ✅ Exists: NooBaaAccount "kelson" INFO[0003] ✅ NooBaaAccount "kelson" Phase is Ready # NooBaaAccount spec: allow_bucket_creation: true default_resource: noobaa-default-backing-store INFO[0003] ✅ Exists: Secret "noobaa-account-kelson" Connection info: AWS_ACCESS_KEY_ID : **** AWS_SECRET_ACCESS_KEY : **** [system:admin/openshift-storage root ~]$ oc get noobaaaccount NAME PHASE AGE kelson Ready 12s [system:admin/openshift-storage root ~]$ vim bucket-policy.json [system:admin/openshift-storage root ~]$ aws --endpoint https://s3-openshift-storage.apps.ocp4.cluster.lab --no-verify-ssl s3api put-bucket-policy --bucket test123-febe138a-6632-41c3-a54c-3d3adbd2b57a --policy file://bucket-policy.json /usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py:1052: InsecureRequestWarning: Unverified HTTPS request is being made to host 's3-openshift-storage.apps.ocp4.cluster.lab'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings InsecureRequestWarning, [system:admin/openshift-storage root ~]$ cat bucket-policy.json { "Version": "NewVersion", "Statement": [ { "Sid": "Example", "Effect": "Allow", "Principal": [ "kelson" ], "Action": [ "s3:GetObject" ], "Resource": [ "arn:aws:s3:::test123-febe138a-6632-41c3-a54c-3d3adbd2b57a" ] } ] } Seems to have worked! Confirming... [system:admin/openshift-storage root ~]$ noobaa account create kelson-test --allow_bucket_create=true INFO[0000] ✅ Exists: NooBaa "noobaa" INFO[0000] ✅ Created: NooBaaAccount "kelson-test" INFO[0000] INFO[0000] NOTE: INFO[0000] - This command has finished applying changes to the cluster. INFO[0000] - From now on, it only loops and reads the status, to monitor the operator work. INFO[0000] - You may Ctrl-C at any time to stop the loop and watch it manually. INFO[0000] INFO[0000] NooBaaAccount Wait Ready: INFO[0000] ⏳ NooBaaAccount "kelson-test" Phase is "": waiting... INFO[0003] ✅ NooBaaAccount "kelson-test" Phase is Ready INFO[0003] INFO[0003] INFO[0003] ✅ Exists: NooBaaAccount "kelson-test" INFO[0003] ✅ NooBaaAccount "kelson-test" Phase is Ready # NooBaaAccount spec: allow_bucket_creation: true default_resource: noobaa-default-backing-store INFO[0003] ✅ Exists: Secret "noobaa-account-kelson-test" Connection info: AWS_ACCESS_KEY_ID : **** AWS_SECRET_ACCESS_KEY : **** [system:admin/openshift-storage root ~]$ NOOBAA_ACCESS_KEY=$(kubectl get secret noobaa-account-kelson-test -n openshift-storage -o json | jq -r '.data.AWS_ACCESS_KEY_ID|@base64d') [system:admin/openshift-storage root ~]$ NOOBAA_SECRET_KEY=$(kubectl get secret noobaa-account-kelson-test -n openshift-storage -o json | jq -r '.data.AWS_SECRET_ACCESS_KEY|@base64d') [system:admin/openshift-storage root ~]$ s3 ls /usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py:1052: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings InsecureRequestWarning, [system:admin/openshift-storage root ~]$ s3 ls s3://test123-febe138a-6632-41c3-a54c-3d3adbd2b57a /usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py:1052: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings InsecureRequestWarning, An error occurred (AccessDenied) when calling the ListObjectsV2 operation: Access Denied [system:admin/openshift-storage root ~]$ Doing this testing made me realize where my confusion lay. In this section in [1], we give a command using ' --policy BucketPolicy': ``` Using AWS S3 client, use the put-bucket-policy command to apply the bucket policy to your S3 bucket: # aws --endpoint ENDPOINT --no-verify-ssl s3api put-bucket-policy --bucket MyBucket --policy BucketPolicy Replace ENDPOINT with the S3 endpoint. Replace MyBucket with the bucket to set the policy on. Replace BucketPolicy with the bucket policy JSON file. Add --no-verify-ssl if you are using the default self signed certificates. ``` which won't work. However, in the 'For example' section we provide the proper way to run it using '--policy file://BucketPolicy': ``` For example: # aws --endpoint https://s3-openshift-storage.apps.gogo44.noobaa.org --no-verify-ssl s3api put-bucket-policy -bucket MyBucket --policy file://BucketPolicy ``` Can we update this doc to have the first command using '--policy file://BucketPolicy'? Also, since having a valid user account is required for the `Principle` section of the json file, we should add 'Creating a user in the Multicloud Object Gateway' to the 'Prerequisites' section... something like: ``` Prerequisites A running OpenShift Data Foundation Platform. Access to the Multicloud Object Gateway (MCG), see Chapter 2, Accessing the Multicloud Object Gateway with your applications A valid user account (what's being used in the Principal section of the json file that houses the policy definitions) is required when deploying a bucket policy, see 7.3. Creating a user in the Multicloud Object Gateway to create a user account ``` [1] https://access.redhat.com/documentation/en-us/red_hat_openshift_data_foundation/4.14/html-single/managing_hybrid_and_multicloud_resources/index#using-bucket-policies
Good catch! Moving to docs