Bug 1610712

Summary: [W21FD0v0] cookie is not set correctly when router dynamic changes enabled
Product: OpenShift Container Platform Reporter: Hongan Li <hongli>
Component: NetworkingAssignee: Ram Ranganathan <ramr>
Networking sub component: router QA Contact: zhaozhanqi <zzhao>
Status: CLOSED ERRATA Docs Contact:
Severity: medium    
Priority: unspecified CC: aos-bugs, dmace
Version: 3.11.0   
Target Milestone: ---   
Target Release: 3.11.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-10-11 07:23:00 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Hongan Li 2018-08-01 09:05:51 UTC 
Description of problem:
the cookie is not set correctly when router dynamic changes enabled and before a router reloaded.

Version-Release number of selected component (if applicable):
openshift v3.11.0-0.10.0
kubernetes v1.11.0+d4cacc0

How reproducible:
always

Steps to Reproduce:
1. enable dynamic changes
$oc set env dc/router ROUTER_HAPROXY_CONFIG_MANAGER=true
2. create your project, pod, svc and route.
$ oc create -f https://raw.githubusercontent.com/openshift-qe/v3-testfiles/master/routing/caddy-docker.json
$ oc create -f https://raw.githubusercontent.com/openshift-qe/v3-testfiles/master/routing/edge/service_unsecure.json
$ oc expose svc service-unsecure
3. ensure no router reloaded while creating pod, svc and route.
$ oc logs router-x-xxxxx
4. curl the route and check the cookies
$ curl service-unsecure-hongli.apps.0801-co1.qe.rhcloud.com -v

Actual results:
<---snip--->
< Set-Cookie: d4bee9152b34dbe72ebd486a984a2809=; Expires=Thu, 01-Jan-1970 00:00:01 GMT; path=/; HttpOnly
< 
Hello-OpenShift-1 http-8080

Expected results:
the cookie should be set like:
< Set-Cookie: 7f595b55e6c0fda92553c87767c7f503=aab9300b612c8906c41fc6866ff2d442; path=/; HttpOnly
< 
Hello-OpenShift-1 http-8080


Additional info:
annotate the route to trigger a router reloaded, then the cookie will be updated as expected result.
Comment 1 Ram Ranganathan 2018-08-02 19:56:52 UTC 
@hongli have to investigate this - the cookie looks a wee bit odd. But it could be the dynamic api is doing that. Will also check scaling service pods up and down.
Comment 2 Ram Ranganathan 2018-08-06 21:37:57 UTC 
Associated PR: https://github.com/openshift/origin/pull/20557

There are some caveats here - in that the cookie name+value as it will change on a reload because the way it is generated. 

1. The cookie name is based on the backend - which in the initial case will be a blueprint route and after a reload will change to the actual route.
2. The cookie value is based on a hash with a dynamic cookie key which is name+ip based - again the name will change after a reload. Same is true for scaled up aka pods added dynamically.

The cookie will stick till a reload.


Example commands: 
$ oc expose service header-test-insecure
$ curl -vvv --resolve header-test-insecure-default.router.default.svc.cluster.local:80:127.0.0.1 http://header-test-insecure-default.router.default.svc.cluster.local/

>>>> Pre-reload with blueprint: 
Set-Cookie: d4bee9152b34dbe72ebd486a984a2809=bf0bfa1560371104; path=/; HttpOnly

$ # create a re-encrypt route to force a reload

>>>> and post-reload with the actual route: 
Set-Cookie: 7ffbbf18807d0b625cf2fe2e2fe0ccf4=bf6bec819e75437d2ac9b1bb36c385da; path=/; HttpOnly

That cookie name `7ffbbf18807d0b625cf2fe2e2fe0ccf4` or `d4bee9152b34dbe72ebd486a984a2809` is associated with the backend and cannot be  changed via the dynamic configuration API. And there's caveat re: the value as well - for dynamic servers it is based on a hash of the name+ip - which will change on a reload.
Comment 4 Hongan Li 2018-08-23 08:10:16 UTC 
verified in atomic-openshift-3.11.0-0.20.0.git.0.d80d8ad.el7 and the issue has been fixed.

the caveats as described in #Comment 2.
Comment 6 errata-xmlrpc 2018-10-11 07:23:00 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:2652