Bug 1734167

Summary: [ci] [azure] The HAProxy router should serve routes that were created from an ingress
Product: OpenShift Container Platform Reporter: Yu Qi Zhang <jerzhang>
Component: NetworkingAssignee: Dan Mace <dmace>
Networking sub component: router QA Contact: Hongan Li <hongli>
Status: CLOSED DUPLICATE Docs Contact:
Severity: medium    
Priority: medium CC: adahiya, aos-bugs, bperkins, mmasters, xxia
Version: 4.2.0   
Target Milestone: ---   
Target Release: 4.2.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-08-21 00:57:33 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Yu Qi Zhang 2019-07-29 20:24:54 UTC
Description of problem:

The Azure e2e tests fail consistently on:

[Conformance][Area:Networking][Feature:Router] The HAProxy router should serve routes that were created from an ingress [Suite:openshift/conformance/parallel/minimal] 

With error:

fail [github.com/openshift/origin/test/extended/util/url/url.go:82]: Unexpected error:
    <*errors.errorString | 0xc00338c030>: {
        s: "response 0 could not be decoded: invalid character '0' after object key:value pair",
    }
    response 0 could not be decoded: invalid character '0' after object key:value pair
occurred

Example test run with error: https://prow.svc.ci.openshift.org/view/gcs/origin-ci-test/pr-logs/pull/openshift_installer/2098/pull-ci-openshift-installer-master-e2e-azure/91

Version-Release number of selected component (if applicable):

Openshift 4.2

How reproducible:
Always.

Steps to Reproduce:

See: https://prow.svc.ci.openshift.org/pr-history/?org=openshift&repo=installer&pr=2098

Actual results:


Expected results:
Test passes

Additional info:

Comment 1 Dan Mace 2019-08-06 20:45:51 UTC
IMO we don't have enough data to consider this a blocker.

https://ci-search-ci-search-next.svc.ci.openshift.org/chart?search=failed%3A+.*HAProxy+router+should+serve+routes&maxAge=336h&context=2&type=all&name=pull-ci-.*-azure

There is no testgrid data for Azure jobs yet.

Comment 2 Abhinav Dahiya 2019-08-15 00:47:55 UTC
https://testgrid.k8s.io/redhat-openshift-release-informing#redhat-canary-openshift-ocp-installer-e2e-azure-4.2&sort-by-flakiness=

`[Conformance][Area:Networking][Feature:Router] The HAProxy router should support reencrypt to services backed by a serving certificate automatically [Suite:openshift/conformance/parallel/minimal]` is the top flake on Azure that causes test to Fail

Comment 3 Dan Mace 2019-08-15 11:38:15 UTC
(In reply to Abhinav Dahiya from comment #2)
> https://testgrid.k8s.io/redhat-openshift-release-informing#redhat-canary-
> openshift-ocp-installer-e2e-azure-4.2&sort-by-flakiness=
> 
> `[Conformance][Area:Networking][Feature:Router] The HAProxy router should
> support reencrypt to services backed by a serving certificate automatically
> [Suite:openshift/conformance/parallel/minimal]` is the top flake on Azure
> that causes test to Fail

Thanks! I missed this bug when re-evaluating priorities. The new testgrid data for Azure is great. I'll promote this and other Azure master flakes to blockers.

Comment 4 Miciah Dashiel Butler Masters 2019-08-21 00:57:33 UTC
Cause is the same as bug 1743728:

    The DNS provider failed to ensure the record: failed to update dns a record: *.apps.ci-op-xq7nkb8s-282fe.ci.azure.devcluster.openshift.com: dns.RecordSetsClient#CreateOrUpdate: Failure responding to request: StatusCode=403 -- Original Error: autorest/azure: Service returned an error. Status=403 Code="AuthorizationFailed" Message="The client '80fc851c-f9e3-41c2-87f2-3a05358d85f7' with object id '80fc851c-f9e3-41c2-87f2-3a05358d85f7' does not have authorization to perform action 'Microsoft.Network/dnsZones/A/write' over scope '/subscriptions/d38f1e38-4bed-438e-b227-833f997adf6a/resourceGroups/os4-common/providers/Microsoft.Network/dnsZones/ci.azure.devcluster.openshift.com/A/*.apps.ci-op-xq7nkb8s-282fe' or the scope is invalid. If access was recently granted, please refresh your credentials."

*** This bug has been marked as a duplicate of bug 1743728 ***