Bug 1376565

CAI,

1. Not sure why would you think this is a regression? You can not compare containers with rpms?

2. Why would you allow a container read files on the host file system?

3. Its not hardcoded, those are just default values. You can still override default values.

Giuseppe can provide more info to you here.

(In reply to Avesh Agarwal from comment #1)
> CAI,
> 
> 1. Not sure why would you think this is a regression? You can not compare
> containers with rpms?
It was considered an user-experience regression with regards users to make persist configuration changes. It would be fine if we fix this in documentation by providing an accepted new method.
> 2. Why would you allow a container read files on the host file system?
Ideally, it would be the same way of people using configuration file as in the etcd rpm, so people don't need to change their scripts, documentation and minimize the surprise.

I have been discussing about it with Giuseepe and he is suggesting to implement --env flag with atomic install to pass any env vars to a container for configuration. He is also going to discuss it with Dan Walsh.

I also discussed with him the possibility of mounting a conf file inside /etc between host and containers to have similar experience like rpms, but his concern is that a) running multiple instances of the same container on the host (something I think might not be needed most of the time, but its ok)? b) we also break the possibility to run it as an user container with bubblewrap, and it is something we want to do later.

I have cc'ed to Dan Walsh too to get his views.

The issue here is it that with system containers, we provide a file manifest.json that has default env vars and their default values and only these env vars' values can be can be overridden by --set flag of atomic install.

The issue here is that how to set/provide those env vars (required to configure an application) not part of manifest.json?

Can we just leak the environment variables into the container?

Similar flannel-docker (https://bugzilla.redhat.com/show_bug.cgi?id=1376572) bug has been closed as a not a bug, as per the comments in that bug, similar user experience is not expected. So removing regression and blocker bug flags.

I am not going to argue for flannel-docker, since there seems only 2 config settings to change. However, etcd-docker is worse with lots of settings that does not provide a proper way to configuration them right now.

Thats why I have not closed the bug. The best I can do is based on what is supported by atomic that means to have etcd config variables set at install time via --set flag to atomic.

Also as I said in https://bugzilla.redhat.com/show_bug.cgi?id=1376572 that I am fine with bind mounting /etc/etcd/etcd.conf in container on host so that it can be configure like in rpm but Giuseppe does not like that.

there is a patch here so that it is possible to configure more variables in the etcd container:

https://github.com/aveshagarwal/etcd-container/pull/6

--set can be used to pass environment variables that are accepted by the image, that means having something like: "VARIABLE=$VARIABLE" in the env block of the config.json.template file, which is feed directly to runC.

The manifest.json is used to provide a default value, when the user doesn't specify any, so that the image can have a default configuration and not expect --set for each accepted variable.

I don't see why this is less flexible, both the user and the developer of the image should know what values are accepted.

If we bind mount the file, we won't be able to have multiple instances of etcd running (unless we are going to provide a different conf file for each of them):

e.g. if we don't bind the configuration file, we can use something like this to spin 3 etcd instances:

for i in 0 1 2; do
    ETCD_NAME=$HOSTNAME
    ./atomic install --system \
             --set ETCD_NAME=$ETCD_NAME --set ETCD_ADVERTISE_CLIENT_URLS=http://${ipaddress}:$((2379 + $i + 2)),http://${ipaddress}:$((4001 + $i)) \
             --set ETCD_LISTEN_CLIENT_URLS=http://0.0.0.0:$((2379 + $i * 2)),http://0.0.0.0:$((4001 + $i)) \
             --set ETCD_INITIAL_ADVERTISE_PEER_URLS=http://${ipaddress}:$((2380 + $i * 2)),http://${ipaddress}:$((7001 + $i)) \
             --set ETCD_LISTEN_PEER_URLS=http://0.0.0.0:$((2380 + $i * 2)),http://0.0.0.0:$((7001 + $i)) \
             --set ETCD_INITIAL_CLUSTER=$ETCD_NAME=http://${ipaddress}:$((2380 + $i * 2)),$ETCD_NAME=http://${ipaddress}:$((7001 + $i)) \
             --name etcd$i gscrivano/etcd

    systemctl start etcd$i
done

Hi Avesh, 

If this BZ should be included in the Release Notes, could you update the Doc Text field describing the new feature?

Test with the newest etcd-docker-2.3.7-11 with the method in Comment 10 . And seems it works well. All three etcd service are up and the ports are the same from the command line. So set this to verified. 

Following are the details:

root@qcow2test cloud-user]# systemctl status etcd1 -l
● etcd1.service - Etcd Server
   Loaded: loaded (/etc/systemd/system/etcd1.service; enabled; vendor preset: disabled)
   Active: active (running) since Wed 2016-10-12 09:43:08 UTC; 1min 37s ago
 Main PID: 17859 (runc)
   Memory: 3.5M
   CGroup: /system.slice/etcd1.service
           └─17859 /bin/runc start etcd1

Oct 12 09:43:08 qcow2test.example.org runc[17859]: 2016-10-12 05:43:08.379078 I | raft: newRaft 192ca4121e2f485b [peers: [], term: 2, commit: 959, applied: 0, lastindex: 959, lastterm: 2]
Oct 12 09:43:08 qcow2test.example.org runc[17859]: 2016-10-12 05:43:08.380302 I | etcdserver: starting server... [version: 2.3.7, cluster version: to_be_decided]
Oct 12 09:43:08 qcow2test.example.org runc[17859]: 2016-10-12 05:43:08.427157 N | etcdserver: added local member 192ca4121e2f485b [http://10.73.199.147:2382 http://10.73.199.147:7002] to cluster 22b75ee7369e25e1
Oct 12 09:43:08 qcow2test.example.org runc[17859]: 2016-10-12 05:43:08.427372 N | etcdserver: set the initial cluster version to 2.3
Oct 12 09:43:09 qcow2test.example.org runc[17859]: 2016-10-12 05:43:09.780399 I | raft: 192ca4121e2f485b is starting a new election at term 2
Oct 12 09:43:09 qcow2test.example.org runc[17859]: 2016-10-12 05:43:09.780533 I | raft: 192ca4121e2f485b became candidate at term 3
Oct 12 09:43:09 qcow2test.example.org runc[17859]: 2016-10-12 05:43:09.780546 I | raft: 192ca4121e2f485b received vote from 192ca4121e2f485b at term 3
Oct 12 09:43:09 qcow2test.example.org runc[17859]: 2016-10-12 05:43:09.780571 I | raft: 192ca4121e2f485b became leader at term 3
Oct 12 09:43:09 qcow2test.example.org runc[17859]: 2016-10-12 05:43:09.780585 I | raft: raft.node: 192ca4121e2f485b elected leader 192ca4121e2f485b at term 3
Oct 12 09:43:09 qcow2test.example.org runc[17859]: 2016-10-12 05:43:09.782102 I | etcdserver: published {Name:qcow2test.example.org ClientURLs:[http://10.73.199.147:2382 http://10.73.199.147:4002]} to cluster 22b75ee7369e25e1
[root@qcow2test cloud-user]# systemctl status etcd0 -l
● etcd0.service - Etcd Server
   Loaded: loaded (/etc/systemd/system/etcd0.service; enabled; vendor preset: disabled)
   Active: active (running) since Wed 2016-10-12 09:43:04 UTC; 1min 48s ago
 Main PID: 17780 (runc)
   Memory: 1.6M
   CGroup: /system.slice/etcd0.service
           └─17780 /bin/runc start etcd0

Oct 12 09:43:05 qcow2test.example.org runc[17780]: 2016-10-12 05:43:05.038249 I | etcdserver: starting server... [version: 2.3.7, cluster version: to_be_decided]
Oct 12 09:43:05 qcow2test.example.org runc[17780]: 2016-10-12 05:43:05.040496 N | etcdserver: added local member 5c8d32ee0d3b2932 [http://10.73.199.147:2380 http://10.73.199.147:7001] to cluster b69a16535add2b5
Oct 12 09:43:05 qcow2test.example.org runc[17780]: 2016-10-12 05:43:05.442575 I | raft: 5c8d32ee0d3b2932 is starting a new election at term 1
Oct 12 09:43:05 qcow2test.example.org runc[17780]: 2016-10-12 05:43:05.442687 I | raft: 5c8d32ee0d3b2932 became candidate at term 2
Oct 12 09:43:05 qcow2test.example.org runc[17780]: 2016-10-12 05:43:05.442698 I | raft: 5c8d32ee0d3b2932 received vote from 5c8d32ee0d3b2932 at term 2
Oct 12 09:43:05 qcow2test.example.org runc[17780]: 2016-10-12 05:43:05.442716 I | raft: 5c8d32ee0d3b2932 became leader at term 2
Oct 12 09:43:05 qcow2test.example.org runc[17780]: 2016-10-12 05:43:05.442727 I | raft: raft.node: 5c8d32ee0d3b2932 elected leader 5c8d32ee0d3b2932 at term 2
Oct 12 09:43:05 qcow2test.example.org runc[17780]: 2016-10-12 05:43:05.443996 I | etcdserver: setting up the initial cluster version to 2.3
Oct 12 09:43:05 qcow2test.example.org runc[17780]: 2016-10-12 05:43:05.478681 N | etcdserver: set the initial cluster version to 2.3
Oct 12 09:43:05 qcow2test.example.org runc[17780]: 2016-10-12 05:43:05.478793 I | etcdserver: published {Name:qcow2test.example.org ClientURLs:[http://10.73.199.147:2381 http://10.73.199.147:4001]} to cluster b69a16535add2b5

[root@qcow2test cloud-user]# systemctl status etcd1 -l
● etcd1.service - Etcd Server
   Loaded: loaded (/etc/systemd/system/etcd1.service; enabled; vendor preset: disabled)
   Active: active (running) since Wed 2016-10-12 09:43:08 UTC; 6min ago
 Main PID: 17859 (runc)
   Memory: 3.5M
   CGroup: /system.slice/etcd1.service
           └─17859 /bin/runc start etcd1

Oct 12 09:43:08 qcow2test.example.org runc[17859]: 2016-10-12 05:43:08.379078 I | raft: newRaft 192ca4121e2f485b [peers: [], term: 2, commit: 959, applied: 0, lastindex: 959, lastterm: 2]
Oct 12 09:43:08 qcow2test.example.org runc[17859]: 2016-10-12 05:43:08.380302 I | etcdserver: starting server... [version: 2.3.7, cluster version: to_be_decided]
Oct 12 09:43:08 qcow2test.example.org runc[17859]: 2016-10-12 05:43:08.427157 N | etcdserver: added local member 192ca4121e2f485b [http://10.73.199.147:2382 http://10.73.199.147:7002] to cluster 22b75ee7369e25e1
Oct 12 09:43:08 qcow2test.example.org runc[17859]: 2016-10-12 05:43:08.427372 N | etcdserver: set the initial cluster version to 2.3
Oct 12 09:43:09 qcow2test.example.org runc[17859]: 2016-10-12 05:43:09.780399 I | raft: 192ca4121e2f485b is starting a new election at term 2
Oct 12 09:43:09 qcow2test.example.org runc[17859]: 2016-10-12 05:43:09.780533 I | raft: 192ca4121e2f485b became candidate at term 3
Oct 12 09:43:09 qcow2test.example.org runc[17859]: 2016-10-12 05:43:09.780546 I | raft: 192ca4121e2f485b received vote from 192ca4121e2f485b at term 3
Oct 12 09:43:09 qcow2test.example.org runc[17859]: 2016-10-12 05:43:09.780571 I | raft: 192ca4121e2f485b became leader at term 3
Oct 12 09:43:09 qcow2test.example.org runc[17859]: 2016-10-12 05:43:09.780585 I | raft: raft.node: 192ca4121e2f485b elected leader 192ca4121e2f485b at term 3
Oct 12 09:43:09 qcow2test.example.org runc[17859]: 2016-10-12 05:43:09.782102 I | etcdserver: published {Name:qcow2test.example.org ClientURLs:[http://10.73.199.147:2382 http://10.73.199.147:4002]} to cluster 22b75ee7369e25e1
[root@qcow2test cloud-user]# systemctl status etcd2 -l
● etcd2.service - Etcd Server
   Loaded: loaded (/etc/systemd/system/etcd2.service; enabled; vendor preset: disabled)
   Active: active (running) since Wed 2016-10-12 09:43:11 UTC; 6min ago
 Main PID: 17938 (runc)
   Memory: 1.6M
   CGroup: /system.slice/etcd2.service
           └─17938 /bin/runc start etcd2

Oct 12 09:43:11 qcow2test.example.org runc[17938]: 2016-10-12 05:43:11.612881 I | raft: newRaft f2a8444b36c8fa88 [peers: [], term: 2, commit: 960, applied: 0, lastindex: 960, lastterm: 2]
Oct 12 09:43:11 qcow2test.example.org runc[17938]: 2016-10-12 05:43:11.615260 I | etcdserver: starting server... [version: 2.3.7, cluster version: to_be_decided]
Oct 12 09:43:11 qcow2test.example.org runc[17938]: 2016-10-12 05:43:11.619587 N | etcdserver: added local member f2a8444b36c8fa88 [http://10.73.199.147:2384 http://10.73.199.147:7003] to cluster 2454ad45da800408
Oct 12 09:43:11 qcow2test.example.org runc[17938]: 2016-10-12 05:43:11.619765 N | etcdserver: set the initial cluster version to 2.3
Oct 12 09:43:12 qcow2test.example.org runc[17938]: 2016-10-12 05:43:12.715504 I | raft: f2a8444b36c8fa88 is starting a new election at term 2
Oct 12 09:43:12 qcow2test.example.org runc[17938]: 2016-10-12 05:43:12.715708 I | raft: f2a8444b36c8fa88 became candidate at term 3
Oct 12 09:43:12 qcow2test.example.org runc[17938]: 2016-10-12 05:43:12.715729 I | raft: f2a8444b36c8fa88 received vote from f2a8444b36c8fa88 at term 3
Oct 12 09:43:12 qcow2test.example.org runc[17938]: 2016-10-12 05:43:12.715777 I | raft: f2a8444b36c8fa88 became leader at term 3
Oct 12 09:43:12 qcow2test.example.org runc[17938]: 2016-10-12 05:43:12.715801 I | raft: raft.node: f2a8444b36c8fa88 elected leader f2a8444b36c8fa88 at term 3
Oct 12 09:43:12 qcow2test.example.org runc[17938]: 2016-10-12 05:43:12.720584 I | etcdserver: published {Name:qcow2test.example.org ClientURLs:[http://10.73.199.147:2383 http://10.73.199.147:4003]} to cluster 2454ad45da800408

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2016:2647