Bug 1919690 - Issues with Thanos Observability Install on Bare Metal OCS [NEEDINFO]
Summary: Issues with Thanos Observability Install on Bare Metal OCS
Keywords:
Status: CLOSED INSUFFICIENT_DATA
Alias: None
Product: Red Hat Advanced Cluster Management for Kubernetes
Classification: Red Hat
Component: Core Services / Observability
Version: rhacm-2.1.z
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
: ---
Assignee: Chunlin Yang
QA Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-01-24 18:15 UTC by James Young
Modified: 2021-02-26 19:21 UTC (History)
1 user (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-02-26 19:21:24 UTC
Target Upstream Version:
Embargoed:
ming: needinfo+
chuyang: needinfo? (jayoung)


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github open-cluster-management backlog issues 8809 0 None None None 2021-02-22 14:28:03 UTC

Description James Young 2021-01-24 18:15:40 UTC
Description of problem:
I am trying to install RedHat Advanced Cluster Management on an OCP 4.6 with Openshift Container Storage 4.6 in place.  I am having trouble with the Observability portion. I am not sure what values to put in for the thanos-object-storage.yaml.

The OCS deployment was setup following the instructions  at: 
https://access.redhat.com/documentation/en-us/red_hat_openshift_container_storage/4.6/html/deploying_openshift_container_storage_using_bare_metal_infrastructure/index

Using the following provided thanos-object-storage.yaml syntax we see thanos-store-shard pods failing to run due to: 
level=info ts=2021-01-19T02:39:11.128223396Z caller=http.go:64 service=http/server component=store msg="internal server is shutting down" err="bucket store initial sync: sync block: BaseFetcher: iter bucket: The specified bucket does not exist."

apiVersion: v1
kind: Secret
metadata:
  name: thanos-object-storage
type: Opaque
stringData:
  thanos.yaml: |
    type: s3
    config:
      bucket: observability-<random-string>
      endpoint: s3.openshift-storage.svc:443
      insecure: false
      access_key: <key value>
      secret_ket: <key value>
      http_config:
        insecure_skip_verify: true


Version-Release number of selected component (if applicable): 2.1.x


How reproducible:


Steps to Reproduce:
1. 
2.
3.

Actual results:


Expected results:


Additional info:
Steps for thanos configuration on OCS do not appear to be publicly available at this time.

Comment 2 Chunlin Yang 2021-01-28 10:03:05 UTC
From the receiver and store, we can see the clear message is `stat s3 object: The specified bucket does not exist.`

Firstly, I would like the client to have a try to list buckets

If AWS S3 CLI is the application, the following command will list buckets in OCS:

AWS_ACCESS_KEY_ID=5x0dCVopWD6B0pOGnR6i
AWS_SECRET_ACCESS_KEY=bxcJi19PFOgYx5JbK6tSxRVZINscOgP7Bl75jH4m
aws --endpoint https://noobaa-mgmt-openshift-storage.apps.chefist.cp.fyre.ibm.com --no-verify-ssl s3 ls

Secondly, I found some errors in noobaa-operator-7fb8694476-cb5n4

```
./logs/openshift-storage/noobaa-core-0.log:Jan-23 18:52:43.505 [WebServer/36] [ERROR] CONSOLE:: RPC._on_request: ERROR srv bucket_api.read_bucket_sdk_info reqid 7871@wss://noobaa-mgmt.openshift-storage.svc:443(xwo3qp.zzzzm) connid ws://[::ffff:10.254.12.21]:39616/(14v0h84c) Error: No such bucket: obc-open-cluster-management-observability-observability-thanos
./logs/openshift-storage/noobaa-core-0.log:Jan-23 18:52:58.033 [WebServer/36] [ERROR] CONSOLE:: RPC._on_request: ERROR srv bucket_api.read_bucket_sdk_info reqid 7875@wss://noobaa-mgmt.openshift-storage.svc:443(xwo3qp.zzzzm) connid ws://[::ffff:10.254.12.21]:39616/(14v0h84c) Error: No such bucket: obc-open-cluster-management-observability-observability-thanos
./logs/openshift-storage/noobaa-core-0.log:Jan-23 18:52:58.084 [WebServer/36] [ERROR] CONSOLE:: RPC._on_request: ERROR srv bucket_api.read_bucket_sdk_info reqid 7876@wss://noobaa-mgmt.openshift-storage.svc:443(xwo3qp.zzzzm) connid ws://[::ffff:10.254.12.21]:39616/(14v0h84c) Error: No such bucket: obc-open-cluster-management-observability-observability-thanos
./logs/openshift-storage/noobaa-core-0.log:Jan-23 18:52:59.588 [WebServer/36] [ERROR] CONSOLE:: RPC._on_request: ERROR srv bucket_api.read_bucket_sdk_info reqid 7877@wss://noobaa-mgmt.openshift-storage.svc:443(xwo3qp.zzzzm) connid ws://[::ffff:10.254.12.21]:39616/(14v0h84c) Error: No such bucket: obc-open-cluster-management-observability-observability-thanos
```

Comment 3 Chunlin Yang 2021-01-28 10:05:16 UTC
BTW, if `observability-deadbeef` exists, may have a try with the endpoint `noobaa-mgmt-openshift-storage.apps.chefist.cp.fyre.ibm.com`

Comment 5 Chunlin Yang 2021-02-18 09:28:17 UTC
@jayoung Can we close this issue?


Note You need to log in before you can comment on or make changes to this bug.