Bug 1763303
| Summary: | Metering aws-billing integration not working | |||
|---|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | Chance Zibolski <chancez> | |
| Component: | Metering Operator | Assignee: | tflannag | |
| Status: | CLOSED ERRATA | QA Contact: | Peter Ruan <pruan> | |
| Severity: | medium | Docs Contact: | ||
| Priority: | high | |||
| Version: | 4.3.0 | CC: | bparees, sd-operator-metering, tflannag | |
| Target Milestone: | --- | |||
| Target Release: | 4.3.0 | |||
| Hardware: | Unspecified | |||
| OS: | Unspecified | |||
| Whiteboard: | ||||
| Fixed In Version: | Doc Type: | If docs needed, set a value | ||
| Doc Text: | Story Points: | --- | ||
| Clone Of: | ||||
| : | 1763305 (view as bug list) | Environment: | ||
| Last Closed: | 2020-01-23 11:08:20 UTC | Type: | Bug | |
| Regression: | --- | Mount Type: | --- | |
| Documentation: | --- | CRM: | ||
| Verified Versions: | Category: | --- | ||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
| Cloudforms Team: | --- | Target Upstream Version: | ||
| Embargoed: | ||||
| Bug Depends On: | ||||
| Bug Blocks: | 1763305 | |||
|
Description
Chance Zibolski
2019-10-18 17:47:07 UTC
The scope of this has grown. There are a few additional issues preventing this feature from working once you rename the reportQuery and reportDataSources to not have the second `-raw` suffix. There's a few other issues: - Quoting of partition values/locations is incorrect - Incorrectly checking database name for AWS billing datasources can cause a panic - aws-billing datsaource is not an input in the aws-ec2-billing-raw query, causing table not found errors when the datasource is still initializing due to lack of input dependency - Managing partitions of HiveTables is not using hive database/schema name when altering partitions, preventing partitions from being managed. - HiveTables spec.fileFormat was incorrectly being ignored, preventing the table format from being correctly specified. Able to read from aws billing account and generate report with the following yamls
==== meteringconfig
apiVersion: metering.openshift.io/v1
kind: MeteringConfig
metadata:
name: "operator-metering"
spec:
unsupportedFeatures:
enableHDFS: true
openshift-reporting:
spec:
awsBillingReportDataSource:
enabled: true
bucket: "osdchargeback"
prefix: "billingreport/chance/"
region: "us-east-1"
storage:
type: "hive"
hive:
type: "hdfs"
hdfs:
namenode: "hdfs-namenode-0.hdfs-namenode:9820"
reporting-operator:
spec:
config:
aws:
secretName: "aws-rh-control-secret"
image:
repository: quay.io/openshift/origin-metering-reporting-operator
# use 4.3 until 4.4 is opened and our repo promotes to 4.4 automatically. All of the fixes for AWS billing should be in 4.3.
tag: "4.3"
presto:
spec:
config:
aws:
secretName: "aws-rh-control-secret"
hive:
spec:
config:
aws:
secretName: "aws-rh-control-secret"
===== report generation input
apiVersion: metering.openshift.io/v1
kind: Report
metadata:
name: pod-memory-request-billing-2020-run-now
spec:
query: "pod-memory-request-aws"
reportingStart: '2020-01-01T00:00:00Z'
reportingEnd: '2020-01-03T00:00:00Z'
runImmediately: true
==== sample output
pruan@MacBook-Pro ~/workspace/bushslicer (master●)$ display_report_using_exposed_route pod-memory-request-billing-run-now [ruby-2.6.3]
period_start period_end pod namespace nodepod_request_memory_byte_seconds pod_memory_usage_percent pod_cost
2019-11-12 00:00:00 +0000 UTC 2020-01-14 00:00:00 +0000 UTC alertmanager-main-0 openshift-monitoring ip-10-0-129-7.us-west-1.compute.internal 764411904000.000000 0.002489
2019-11-12 00:00:00 +0000 UTC 2020-01-14 00:00:00 +0000 UTC alertmanager-main-1 openshift-monitoring ip-10-0-150-139.us-west-1.compute.internal 764411904000.000000 0.002489
2019-11-12 00:00:00 +0000 UTC 2020-01-14 00:00:00 +0000 UTC alertmanager-main-2 openshift-monitoring ip-10-0-132-230.us-west-1.compute.internal 764411904000.000000 0.002489
2019-11-12 00:00:00 +0000 UTC 2020-01-14 00:00:00 +0000 UTC apiserver-8rv2n openshift-apiserver ip-10-0-140-28.us-west-1.compute.internal 679477248000.000000 0.002213
2019-11-12 00:00:00 +0000 UTC 2020-01-14 00:00:00 +0000 UTC apiserver-dd4rm openshift-apiserver ip-10-0-151-173.us-west-1.compute.internal 679477248000.000000 0.002213
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:0062 |