Created attachment 1594379 [details] Kibana-proxy logs Description of problem: The customer has a cluster deployed in Azure with a LB forwarding requests from port 4401 to 443. The customer has performed the following to redirect properly when requesting the site: oc edit dc logging-kibana - -redirect-url=https://kibana.vycld-preprod-us.dieboldservices.local:4401/oauth2/callback - -redeem-url=https://kubernetes.default.svc/oauth/token - -login-url=https://openshift.vycld-preprod-us.dieboldservices.local:4400/oauth/authorize - -request-logging=true The redirections are working correctly in terms of the correct URL being utilized, however the following message appears after logging in: message "An internal server error occurred" statusCode 500 error "Internal Server Error" Additionally, it appears it is failing due to openshift attempting to resolve the customer's PC ip address(10.39.137.125:50594) as seen below: {"type":"response","@timestamp":"2019-07-26T15:58:11Z","tags":[],"pid":228,"method":"get","statusCode":200,"req":{"url":"/","method":"get","headers":{"host":"kibana.vycld-preprod-us.dieboldservices.local","user-agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:60.0) Gecko/20100101 Firefox/60.0","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8","accept-encoding":"gzip, deflate, br","accept-language":"en-US,en;q=0.5","dnt":"1","forwarded":"for=10.176.135.136;host=kibana.vycld-preprod-us.dieboldservices.local;proto=https;proto-version=","referer":"https://openshift.vycld-preprod-us.dieboldservices.local:4400/","upgrade-insecure-requests":"1","x-forwarded-access-token":"Wsh-DCCh9iqgWdRcMOPwdOpo4IAA0_K6nhPEqoCPZUs","x-forwarded-email":"mauricio . (9F7C228C)@cluster.local","x-forwarded-for":"10.39.137.125:50594, 10.176.135.136, 10.1.3.1","x-forwarded-host":"kibana.vycld-preprod-us.dieboldservices.local","x-forwarded-port":"443","x-forwarded-proto":"https","x-forwarded-user":"mauricio. (9F7C228C)","x-original-host":"kibana.vycld-preprod-us.dieboldservices.local:4401","x-original-url":"/"},"remoteAddress":"127.0.0.1","userAgent":"127.0.0.1","referer":"https://openshift.vycld-preprod-us.dieboldservices.local:4400/"},"res":{"statusCode":200,"responseTime":9,"contentLength":9},"message":"GET / 200 9ms - 9.0B"} Version-Release number of selected component (if applicable) Steps to Reproduce: 1. Port Forward on an external facing lB in this instance 4401 --> 443 2. Edit the deployment config for logging-kibana(redirect-url, redeem-url, login-url) 3. Try to log into kibana on the forwarded port. Actual results: 500 Error Expected results: Expected to see the kibana UI Additional info:
I did not see any in the logs that I uploaded. Would you like me to try to get more verbose logs?
Hello, Did you need any further information? Regards, Dirk Porter
Please check to see if the secrets are still in sync: test "$(oc get secret logging-kibana-proxy -o jsonpath={.data.oauth-secret} | base64 -d)" = "$(oc get oauthclient kibana-proxy -o jsonpath={.secret})";echo $?
One additional thing I thought of was did you also edit the oauthclient to modify the redirect-url? Example of mine: # oc get oauthclient kibana-proxy -o yaml accessTokenMaxAgeSeconds: 604800 apiVersion: oauth.openshift.io/v1 kind: OAuthClient metadata: labels: logging-infra: support name: kibana-proxy redirectURIs: - https://kibana.192.168.100.212.nip.io scopeRestrictions: - literals: - user:info - user:check-access - user:list-projects secret: zNhb9FbtXFPPlnAR9ccshPF41tBKBUFjXxF4VIXFA9uyVJnX6ODEFHIakOGsmMy2 Upon further investigation, we don't set any of those options for the proxy container [1] so how it may behave is untested. Additionally note this change will be reverted by ansible upon any upgrade. [1] https://github.com/openshift/openshift-ansible/blob/release-3.11/roles/openshift_logging_kibana/templates/kibana.j2#L99-L110
Closing since the customer case was closed.
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 1000 days