Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1612719

Summary: Environment variable expansion not working all the time
Product: OpenShift Container Platform Reporter: Sergio G. <sgarciam>
Component: NodeAssignee: Ryan Phillips <rphillips>
Status: CLOSED UPSTREAM QA Contact: DeShuai Ma <dma>
Severity: medium Docs Contact:
Priority: medium    
Version: 3.6.0CC: aos-bugs, jokerman, mmccomas, sgarciam
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-08-13 14:33:30 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
DC definition
none
Pod definition on a failed deployment
none
Pod definition on a successfull deployment
none
RC definition on the previously attached failed deployment none

Description Sergio G. 2018-08-06 07:15:42 UTC
Description of problem:
Variable expansion in environment variables not working properly sometimes

Version-Release number of selected component (if applicable):
3.6

How reproducible:
On occasions - often enough to impact the work.

Steps to Reproduce:
1. oc create secret generic mssql-server-linux --from-literal=SQLSERVER_SA_PASSWORD=admin123
2. oc env dc/redis --from=secret/mssql-server-linux
3. oc env dc/redis CONNECTIONSTRINGS__MIGRATIONADMIN='Server=mssql-server-linux;Database=StatisticalData;User Id=sa;Password=${SQLSERVER_SA_PASSWORD};MultipleActiveResultSets=true;'

Actual results:
90% of times this works properly but other 10% of times the variable is not properly expanded

Expected results:
100% working

Comment 1 Sergio G. 2018-08-07 07:15:56 UTC
The fail / not fail behavior matches with the environment variable order in the pod yaml definition.

For some reason sometimes the variables appear in a different order, being CONNECTIONSTRINGS__MIGRATIONADMIN before than SQLSERVER_SA_PASSWORD in the yaml definition. In that situation, the expansion doesn't work as expected.

Why is that?

Comment 3 Sergio G. 2018-08-07 09:04:52 UTC
Created attachment 1473906 [details]
DC definition

Comment 4 Sergio G. 2018-08-07 09:05:24 UTC
Created attachment 1473907 [details]
Pod definition on a failed deployment

Comment 5 Sergio G. 2018-08-07 09:06:04 UTC
Created attachment 1473908 [details]
Pod definition on a successfull deployment

Comment 6 Sergio G. 2018-08-07 09:54:44 UTC
Created attachment 1473923 [details]
RC definition on the previously attached failed deployment

Comment 7 Sergio G. 2018-08-07 10:04:56 UTC
oc version:
oc v3.9.0+191fece
kubernetes v1.9.1+a0ce1bc657
features: Basic-Auth GSSAPI Kerberos SPNEGO

Server https://ocp.example.local:443
openshift v3.6.173.0.96
kubernetes v1.6.1+5115d708d7

Comment 11 Ryan Phillips 2018-08-08 20:00:43 UTC
I've tracked this down to the environment variables not getting sorted correctly. There is an upstream ticket [1] and PR [2], which has been merged in to 3.7+. I will be verifying the fix in 3.7. An upgrade should fix the problem.

[1] https://github.com/kubernetes/kubernetes/issues/40373
[2] https://github.com/kubernetes/kubernetes/pull/45980

Comment 12 Sergio G. 2018-08-09 06:24:17 UTC
Those are good news, thanks!
If you have a reproducer I'll help to verify.

Comment 13 Ryan Phillips 2018-08-10 20:14:29 UTC
I have been trying today to get a reproducible test case, but it might not be possible with the way the API and Pod deployment works. The unit tests were updated in the PR, and I think we should rely on them.

Comment 14 Sergio G. 2018-08-13 06:35:11 UTC
That's enough to me Ryan, thank you very much. 
I did an automated loop of changing the order 10 times and all of them were fine.

I already told the customer to upgrade or to use the variables in a different way if he doesn't want to upgrade.

This can be closed as far as I'm concern.