Description of problem: I am trying to install RedHat Advanced Cluster Management on an OCP 4.6 with Openshift Container Storage 4.6 in place. I am having trouble with the Observability portion. I am not sure what values to put in for the thanos-object-storage.yaml. The OCS deployment was setup following the instructions at: https://access.redhat.com/documentation/en-us/red_hat_openshift_container_storage/4.6/html/deploying_openshift_container_storage_using_bare_metal_infrastructure/index Using the following provided thanos-object-storage.yaml syntax we see thanos-store-shard pods failing to run due to: level=info ts=2021-01-19T02:39:11.128223396Z caller=http.go:64 service=http/server component=store msg="internal server is shutting down" err="bucket store initial sync: sync block: BaseFetcher: iter bucket: The specified bucket does not exist." apiVersion: v1 kind: Secret metadata: name: thanos-object-storage type: Opaque stringData: thanos.yaml: | type: s3 config: bucket: observability-<random-string> endpoint: s3.openshift-storage.svc:443 insecure: false access_key: <key value> secret_ket: <key value> http_config: insecure_skip_verify: true Version-Release number of selected component (if applicable): 2.1.x How reproducible: Steps to Reproduce: 1. 2. 3. Actual results: Expected results: Additional info: Steps for thanos configuration on OCS do not appear to be publicly available at this time.
From the receiver and store, we can see the clear message is `stat s3 object: The specified bucket does not exist.` Firstly, I would like the client to have a try to list buckets If AWS S3 CLI is the application, the following command will list buckets in OCS: AWS_ACCESS_KEY_ID=5x0dCVopWD6B0pOGnR6i AWS_SECRET_ACCESS_KEY=bxcJi19PFOgYx5JbK6tSxRVZINscOgP7Bl75jH4m aws --endpoint https://noobaa-mgmt-openshift-storage.apps.chefist.cp.fyre.ibm.com --no-verify-ssl s3 ls Secondly, I found some errors in noobaa-operator-7fb8694476-cb5n4 ``` ./logs/openshift-storage/noobaa-core-0.log:Jan-23 18:52:43.505 [WebServer/36] [ERROR] CONSOLE:: RPC._on_request: ERROR srv bucket_api.read_bucket_sdk_info reqid 7871@wss://noobaa-mgmt.openshift-storage.svc:443(xwo3qp.zzzzm) connid ws://[::ffff:10.254.12.21]:39616/(14v0h84c) Error: No such bucket: obc-open-cluster-management-observability-observability-thanos ./logs/openshift-storage/noobaa-core-0.log:Jan-23 18:52:58.033 [WebServer/36] [ERROR] CONSOLE:: RPC._on_request: ERROR srv bucket_api.read_bucket_sdk_info reqid 7875@wss://noobaa-mgmt.openshift-storage.svc:443(xwo3qp.zzzzm) connid ws://[::ffff:10.254.12.21]:39616/(14v0h84c) Error: No such bucket: obc-open-cluster-management-observability-observability-thanos ./logs/openshift-storage/noobaa-core-0.log:Jan-23 18:52:58.084 [WebServer/36] [ERROR] CONSOLE:: RPC._on_request: ERROR srv bucket_api.read_bucket_sdk_info reqid 7876@wss://noobaa-mgmt.openshift-storage.svc:443(xwo3qp.zzzzm) connid ws://[::ffff:10.254.12.21]:39616/(14v0h84c) Error: No such bucket: obc-open-cluster-management-observability-observability-thanos ./logs/openshift-storage/noobaa-core-0.log:Jan-23 18:52:59.588 [WebServer/36] [ERROR] CONSOLE:: RPC._on_request: ERROR srv bucket_api.read_bucket_sdk_info reqid 7877@wss://noobaa-mgmt.openshift-storage.svc:443(xwo3qp.zzzzm) connid ws://[::ffff:10.254.12.21]:39616/(14v0h84c) Error: No such bucket: obc-open-cluster-management-observability-observability-thanos ```
BTW, if `observability-deadbeef` exists, may have a try with the endpoint `noobaa-mgmt-openshift-storage.apps.chefist.cp.fyre.ibm.com`
@jayoung Can we close this issue?
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 500 days