Description of problem: A user is seeing the aws billing reportdatasource for AWS billing correlation use the wrong reportQuery. https://github.com/operator-framework/operator-metering/issues/993 Version-Release number of selected component (if applicable): 4.3.0 How reproducible: Always Steps to Reproduce: 1. Install metering via install scripts or OperatorHub 2. Enable AWS billing datasource 3. Actual results: time="2019-10-17T19:02:58Z" level=error msg="error queuing Report dependents of ReportDataSource pod-memory-request-raw" app=metering component=reportDataSourceWorker error="unable to get dependencies for ReportQueryView ReportDataSource aws-ec2-billing-data-raw-raw: reportquery.metering.openshift.io \"aws-ec2-billing-data-raw-raw\" not found" logID=mL1IelFccR namespace=metering reportDataSource=pod-memory-request-raw Expected results: No errors Additional info:
The scope of this has grown. There are a few additional issues preventing this feature from working once you rename the reportQuery and reportDataSources to not have the second `-raw` suffix. There's a few other issues: - Quoting of partition values/locations is incorrect - Incorrectly checking database name for AWS billing datasources can cause a panic - aws-billing datsaource is not an input in the aws-ec2-billing-raw query, causing table not found errors when the datasource is still initializing due to lack of input dependency - Managing partitions of HiveTables is not using hive database/schema name when altering partitions, preventing partitions from being managed. - HiveTables spec.fileFormat was incorrectly being ignored, preventing the table format from being correctly specified.
Able to read from aws billing account and generate report with the following yamls ==== meteringconfig apiVersion: metering.openshift.io/v1 kind: MeteringConfig metadata: name: "operator-metering" spec: unsupportedFeatures: enableHDFS: true openshift-reporting: spec: awsBillingReportDataSource: enabled: true bucket: "osdchargeback" prefix: "billingreport/chance/" region: "us-east-1" storage: type: "hive" hive: type: "hdfs" hdfs: namenode: "hdfs-namenode-0.hdfs-namenode:9820" reporting-operator: spec: config: aws: secretName: "aws-rh-control-secret" image: repository: quay.io/openshift/origin-metering-reporting-operator # use 4.3 until 4.4 is opened and our repo promotes to 4.4 automatically. All of the fixes for AWS billing should be in 4.3. tag: "4.3" presto: spec: config: aws: secretName: "aws-rh-control-secret" hive: spec: config: aws: secretName: "aws-rh-control-secret" ===== report generation input apiVersion: metering.openshift.io/v1 kind: Report metadata: name: pod-memory-request-billing-2020-run-now spec: query: "pod-memory-request-aws" reportingStart: '2020-01-01T00:00:00Z' reportingEnd: '2020-01-03T00:00:00Z' runImmediately: true ==== sample output pruan@MacBook-Pro ~/workspace/bushslicer (master●)$ display_report_using_exposed_route pod-memory-request-billing-run-now [ruby-2.6.3] period_start period_end pod namespace nodepod_request_memory_byte_seconds pod_memory_usage_percent pod_cost 2019-11-12 00:00:00 +0000 UTC 2020-01-14 00:00:00 +0000 UTC alertmanager-main-0 openshift-monitoring ip-10-0-129-7.us-west-1.compute.internal 764411904000.000000 0.002489 2019-11-12 00:00:00 +0000 UTC 2020-01-14 00:00:00 +0000 UTC alertmanager-main-1 openshift-monitoring ip-10-0-150-139.us-west-1.compute.internal 764411904000.000000 0.002489 2019-11-12 00:00:00 +0000 UTC 2020-01-14 00:00:00 +0000 UTC alertmanager-main-2 openshift-monitoring ip-10-0-132-230.us-west-1.compute.internal 764411904000.000000 0.002489 2019-11-12 00:00:00 +0000 UTC 2020-01-14 00:00:00 +0000 UTC apiserver-8rv2n openshift-apiserver ip-10-0-140-28.us-west-1.compute.internal 679477248000.000000 0.002213 2019-11-12 00:00:00 +0000 UTC 2020-01-14 00:00:00 +0000 UTC apiserver-dd4rm openshift-apiserver ip-10-0-151-173.us-west-1.compute.internal 679477248000.000000 0.002213
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:0062