Bug 1396925 - [RFE] restructure answer-file behavior
Summary: [RFE] restructure answer-file behavior
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: otopi
Classification: oVirt
Component: Plugins.dialog
Version: master
Hardware: Unspecified
OS: Unspecified
medium
low
Target Milestone: ovirt-4.2.1
: 1.7.6
Assignee: Yedidyah Bar David
QA Contact: Pavel Novotny
URL:
Whiteboard:
Depends On:
Blocks: 1414915 1541947 1551825
TreeView+ depends on / blocked
 
Reported: 2016-11-21 08:38 UTC by Yedidyah Bar David
Modified: 2018-06-26 08:07 UTC (History)
8 users (show)

Fixed In Version: otopi-1.7.6
Doc Type: Enhancement
Doc Text:
Otopi can now optionally write its own answer files, which are simpler to understand compared to tool-specific files written by existing tools that use otopi. The functionality is also different, imitating more closely the behavior without an answer file and answers provided interactively. To generate: engine-setup --otopi-environment=DIALOG/answerFile=str:/tmp/ans1 To use: engine-setup --config-append=/tmp/ans1
Clone Of:
Environment:
Last Closed: 2018-03-29 10:47:04 UTC
oVirt Team: Integration
Embargoed:
ratamir: needinfo-
rule-engine: ovirt-4.2+
grafuls: testing_plan_complete-
ylavi: planning_ack+
rule-engine: devel_ack+
lsvaty: testing_ack+


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
oVirt gerrit 85551 0 'None' MERGED dialog: human: Handle answer files 2021-01-27 06:40:38 UTC
oVirt gerrit 86103 0 'None' MERGED dialog: machine: Handle answer files 2021-01-27 06:41:21 UTC

Description Yedidyah Bar David 2016-11-21 08:38:13 UTC
Currently, otopi does not have a concept of answer files.

engine-setup and ovirt-hosted-engine-setup do, with duplicated code, which behaves like this:
1. In many places of the code, where we want to ask a question, we first check if some specific env key has a value or is still empty.
2. If it's empty, we ask, otherwise we use this value.
3. When we ask, we save the answer in the env with this key.
4. Sometimes the relation is not simple 1-1 but a bit more complicated.
5. Also, in constants.py files, we have an "attribute" per such keys,
called "answerfile", which defaults to False.
6. When we want to write an answer file, we loop over all the constants in all the constants files, and write an otopi-style _configuration_ file.
7. When --config-append=file.conf is used with such a configuration file, otopi populates the environment from its contents.

This has several drawbacks:
1. There is no direct 1-1 relation between questions and answer file keys. So it's difficult for users to edit them. That's also part of the reason we do not officially support that.
2. We risk having different code paths based on whether a question was answered interactively or from an answer file.
3. We have duplicated code for handling this.

Also, otopi already has a notion of a 'name' for each question. But this is not used by the answerfile code. It's used only using the machine dialog (in host-deploy and in cockpit).

Suggested improvement:
1. Add new code to otopi that will handle a new kind of answer files, which will actually be just that - answers to questions. The keys will be the existing question names, the values will be strings.
2. To use them, only the code doing the actual interaction with the user will be changed, mainly dialog.py and related. The code using this (in engine-setup, hosted-engine setup, etc) will be unaware of this - will simply call dialog, and if an answer was provided in the answer file, it will be supplied instead prompting the user.
3. The interaction with the user will also be different from today - today usually nothing is shown for questions that were already answered. The new code (IMO) will behave exactly as if no answer was supplied, will show everything, including the questions and answers, just not prompt.
4. It will also save the answers to the environment, with keys being the question names with some prefix
5. Then the code writing answerfiles will loop over the env keys that have this prefix and write them

We might want/need a transition period in which we write both. We'll probably refine the above plan after we get some experience with the new behavior.

In particular, it might make sense to have the new answer files have the same format and semantics of the existing ones - that is, keys will be env keys (so the prefix + question name), values will be otopi style type:value.

Comment 1 Yedidyah Bar David 2016-12-19 12:49:52 UTC
We should also add code to prevent collisions in question names - to warn/fail if the same name is used for two different questions.

Comment 2 Yedidyah Bar David 2017-12-19 14:08:26 UTC
I think I'll simply save/restore all occurrences of each question/answer, e.g.:

- Engine admin password:

QUESTION/1/OVESETUP_CONFIG_ADMIN_SETUP=str:pass1
QUESTION/2/OVESETUP_CONFIG_ADMIN_SETUP=str:pass2

- Passwords do not match

QUESTION/3/OVESETUP_CONFIG_ADMIN_SETUP=str:pass1
QUESTION/4/OVESETUP_CONFIG_ADMIN_SETUP=str:pass1

- Password is weak:

QUESTION/1/OVESETUP_CONFIG_WEAK_ENGINE_PASSWORD=str:no

- Engine admin password:

QUESTION/5/OVESETUP_CONFIG_ADMIN_SETUP=str:pass1
QUESTION/6/OVESETUP_CONFIG_ADMIN_SETUP=str:pass1

- Password is weak:

QUESTION/2/OVESETUP_CONFIG_WEAK_ENGINE_PASSWORD=str:yes

I think it can express accurately the actual flow the interactive setup went through.

Comment 3 Yedidyah Bar David 2017-12-19 14:13:04 UTC
(In reply to Yedidyah Bar David from comment #2)
> I think it can express accurately the actual flow the interactive setup went
> through.

and also work well in case of bugs, which above actually is - engine-setup gives these two questions the same name:

Engine admin password:
Confirm engine admin password:

Comment 4 Yedidyah Bar David 2018-01-08 08:17:51 UTC
Keeping bug in POST, machine dialog is missing and seems like it's needed.

Comment 5 Yedidyah Bar David 2018-01-08 08:27:00 UTC
To generate:

 engine-setup --otopi-environment=DIALOG/answerFile=str:/tmp/ans1

To use:

 engine-setup --config-append=/tmp/ans1

Compared to files created by engine-setup's --generate-answer ("old-style" answer files):

1. Content will be different. Keys are Questions Names + number (occurrence), not functional entities. Values are (should be) always strings, as that's what users reply to questions.

2. Output of a run with --config-append will be different - it will look almost like an interactive run, as opposed to a run with old-style files, which does not show the questions.

3. In principle, logic can be different too, and that's a potential source of bugs. Some places in the code have assumptions about whether they run with an answer file or not (checking if a value is already supplied), and these places will now always behave as if an answer file was not supplied. We'll have to get some experience about this to know more.

Comment 6 Yedidyah Bar David 2018-01-10 07:55:29 UTC
machine dialog patch merged, moving to modified. Functionality is almost identical to the human dialog. Example use:

engine-setup --otopi-environment="DIALOG/answerFile=str:/root/ans-otopi-1 DIALOG/dialect=str:machine"

- Kill it in the middle of the interaction, or re-use in another machine, or whatever. Then use the generated answer file, works with either dialogs:

engine-setup --otopi-environment="DIALOG/dialect=str:machine" --config-append=/root/ans-otopi-1

engine-setup --config-append=/root/ans-otopi-1

Comment 7 Yedidyah Bar David 2018-01-10 08:04:43 UTC
Some notes/open questions/potential future improvements:

1. Currently, this:

engine-setup --otopi-environment="DIALOG/answerFile=str:/root/ans-otopi-1"

engine-setup --config-append=/root/ans-otopi-1 --otopi-environment="DIALOG/answerFile=str:/root/ans-otopi-2"

Will create ans-otopi-2 with all the contents of ans-otopi-1 (plus whatever new answers provided). It's not completely clear we always want this behavior.

2. We might want to add to the machine dialog functionality to tell the client that we were supposed to ask a question but already got one. Currently this is output as a text note.

3. Code is currently specific to each dialog dialect (machine and human). If we add a new dialect one day (say, something that runs a GUI or whatever), we'll have to adapt it as well. It might make sense to unify the behavior.

Comment 8 Pavel Novotny 2018-03-22 17:12:51 UTC
Verified in
otopi-1.7.7-1.el7ev.noarch
ovirt-engine-setup-4.2.2.4-0.1.el7.noarch
ovirt-engine-setup-plugin-ovirt-engine-4.2.2.4-0.1.el7.noarch


Running 'engine-setup with --otopi-environment="DIALOG/answerFile=str:/tmp/otopi-answers"'
creates otopi-style aswer file /tmp/otopi-answers

Command 'engine-setup --config-append=/tmp/otopi-answers' then loads the answer file as usually.

The functionality is almost the same as with the answer file generated by engine-setup,
with few key differences:

1) Otopi answer file contains the occurrence number of the answer, in case the user entered the answer multiple times, for example, the password confirmation(s) - see Example #1.
This mimics the user behavior more realistically as it replays also the incorrect answers (except invalid values).

2) The questions and answers are shown in the console, except sensitive data - see Example #2 and #3.
Showing the questions & answers during the playback makes it easier for a later inspection than searching the answers only in the answer file.


Example #1: multiple attempts of password entering
~~~ console:
# engine-setup --otopi-environment="DIALOG/answerFile=str:/tmp/otopi-answers"
...
          Engine admin password: 
          Confirm engine admin password: 
[WARNING] Passwords do not match
          Engine admin password: 
          Confirm engine admin password: 
[WARNING] Passwords do not match
          Engine admin password: 
          Confirm engine admin password: 
[WARNING] Passwords do not match
          Engine admin password: 
          Confirm engine admin password: 
[ ERROR ] Invalid value
          Use weak password? (Yes, No) [No]: yes
~~~

~~~ answer file /tmp/otopi-answers:
# OTOPI answer file, generated by human dialog
[environment:default]
...
QUESTION/1/OVESETUP_CONFIG_ADMIN_SETUP=str:123
QUESTION/2/OVESETUP_CONFIG_ADMIN_SETUP=str:1234
QUESTION/3/OVESETUP_CONFIG_ADMIN_SETUP=str:123
QUESTION/4/OVESETUP_CONFIG_ADMIN_SETUP=str:12#
QUESTION/5/OVESETUP_CONFIG_ADMIN_SETUP=str:123
QUESTION/6/OVESETUP_CONFIG_ADMIN_SETUP=str:1233
QUESTION/7/OVESETUP_CONFIG_ADMIN_SETUP=str:123
QUESTION/8/OVESETUP_CONFIG_ADMIN_SETUP=str:123
QUESTION/1/OVESETUP_CONFIG_WEAK_ENGINE_PASSWORD=str:yes
...
~~~


Example #2: visible questions & answers
~~~
# engine-setup --config-append=/tmp/otopi-answers
...
[ INFO  ] Stage: Environment customization
         
          --== PRODUCT OPTIONS ==--
         
          Configure Engine on this host (Yes, No) [Yes]: 
          provided answer: yes
          Configure ovirt-provider-ovn (Yes, No) [Yes]: 
          provided answer: no
          Configure Image I/O Proxy on this host? (Yes, No) [Yes]: 
          provided answer: yes
          Configure WebSocket Proxy on this host (Yes, No) [Yes]: 
          provided answer: yes
~~~


Example #3: (not) showing the sensitive data
~~~
# engine-setup --config-append=/tmp/otopi-answers
...
          --== OVIRT ENGINE CONFIGURATION ==--
         
          Engine admin password: 
          provided answer: (hidden)
          Confirm engine admin password: 
          provided answer: (hidden)
~~~

Comment 9 Yedidyah Bar David 2018-03-22 18:24:25 UTC
Thanks for the verification and the detailed report.

One area you might to inspect further, and perhaps report about and/or open RFEs/bugs about, is when mixing files. Some examples:

1.
engine-setup --generate-answer=f1
engine-setup --config-append=f1 --generate-answer=f2
engine-setup --config-append=f2
engine-setup --config-append=f1 --config-append=f2

vs:

2. 
engine-setup --generate-answer=f1
engine-setup --config-append=f1 --otopi-environment="DIALOG/answerFile=str:f2"
engine-setup --config-append=f2
engine-setup --config-append=f1 --config-append=f2

vs:

3.
engine-setup --otopi-environment="DIALOG/answerFile=str:f1"
engine-setup --config-append=f1 --generate-answer=f2
engine-setup --config-append=f2
engine-setup --config-append=f1 --config-append=f2

Also, all of above, but when killing engine-setup in the middle. And of course you can do this even with more files - say, you started it once, answered some questions, then decided to start over but reuse first file, and again, and again. Or, what happens if each run is in a different version, and the next ones are upgrades. Etc.

Another thing to consider is what should happen with --accept-defaults. Only quite recently I pushed a patch that changes the behavior of this:

https://gerrit.ovirt.org/88278

It's only in master (otopi-1.8), not in 4.2 (otopi-1.7). You might want to try both and decide if you prefer a certain behavior.

Comment 10 Yedidyah Bar David 2018-03-22 18:29:04 UTC
Please also consider that we plan to make --generate-answer in 4.3 identical with --otopi-environment=DIALOG/answerFile , see bug 1518697. Previous comment will still apply, if the saved files f1, f2, etc. were generated in 4.2.

Comment 11 Sandro Bonazzola 2018-03-29 10:47:04 UTC
This bugzilla is included in oVirt 4.2.1 release, published on Feb 12th 2018.

Since the problem described in this bug report should be
resolved in oVirt 4.2.1 release, it has been closed with a resolution of CURRENT RELEASE.

If the solution does not work for you, please open a new bug report.

Comment 12 Sam McLeod 2018-05-14 23:33:32 UTC
(In reply to Sandro Bonazzola from comment #11)
> This bugzilla is included in oVirt 4.2.1 release, published on Feb 12th 2018.
> 
> Since the problem described in this bug report should be
> resolved in oVirt 4.2.1 release, it has been closed with a resolution of
> CURRENT RELEASE.
> 
> If the solution does not work for you, please open a new bug report.

Now that this has been closed as aparently it's been 'resolved' in oVirt 4.2.1 - is it _really_ a resolution, or is that some kind of automatec closure just because a release went out?

Comment 13 Yedidyah Bar David 2018-05-15 07:09:46 UTC
(In reply to Sam McLeod from comment #12)
> (In reply to Sandro Bonazzola from comment #11)
> > This bugzilla is included in oVirt 4.2.1 release, published on Feb 12th 2018.
> > 
> > Since the problem described in this bug report should be
> > resolved in oVirt 4.2.1 release, it has been closed with a resolution of
> > CURRENT RELEASE.
> > 
> > If the solution does not work for you, please open a new bug report.
> 
> Now that this has been closed as aparently

Why apparently? Do you think it's not resolved?

> it's been 'resolved' in oVirt
> 4.2.1 - is it _really_ a resolution, or is that some kind of automatec
> closure just because a release went out?

I guess it was semi-automatic. I guess Sandro's filter picks only bugs in VERIFIED. The move from ON_QA to VERIFIED is (hopefully?) never automatic - always done by a person that verifies.

Also, please note comment 10. Current bug is on otopi, for 4.2. Bug 1518697 is on engine-setup, only for 4.3. I decided to not do this on 4.2 there, feeling it's a rather disruptive change to do inside the z-stream. Feel free to comment there if you feel otherwise.

Comment 14 Sam McLeod 2018-05-15 22:39:35 UTC
(In reply to Yedidyah Bar David from comment #13)

I think I actually slightly misread this yesterday when I was a little surprised to see it had been closed, sorry for that!

Noted re: comment 10, I'll keep half an eye on Bug 1518697


Note You need to log in before you can comment on or make changes to this bug.