Bug 825671

Summary: [RFE] Adding Reloadable Configurations
Product: Red Hat Enterprise Virtualization Manager Reporter: Muli Salem <msalem>
Component: ovirt-engine-configAssignee: Moti Asayag <masayag>
Status: CLOSED WONTFIX QA Contact: Gonza <grafuls>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 3.1.0CC: bazulay, bsettle, didi, iheim, juwu, lpeer, masayag, michal.skrivanek, mmucha, oourfali, pdwyer, rbalakri, Rhev-m-bugs, r.landmann, sgordon, sherold
Target Milestone: ---Keywords: FutureFeature, Reopened
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
URL: http://www.ovirt.org/wiki/Features/ReloadableConfiguration
See Also: https://bugzilla.redhat.com/show_bug.cgi?id=1284409
Whiteboard: infra
Fixed In Version: Doc Type: Enhancement
Doc Text:
What's supported is: As a user, you need to issue this API request to reload the configuration. It will only work for entries annotated with @Reloadable, so we should document such config entries.
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-11-24 09:21:05 EST Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Infra RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
Bug Depends On:    
Bug Blocks: 1268842    

Description Muli Salem 2012-05-28 03:47:31 EDT
Adding the option to reload configurations without restarting JBoss. Will allow admin to reload from the rhevm-config command line.

Feature design in http://www.ovirt.org/wiki/Features/ReloadableConfiguration.
Comment 1 Muli Salem 2012-05-28 08:28:47 EDT
    Technical note added. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    New Contents:
The Reloadable Configurations API will be considered a tech preview in version 3.1 and might not be supported in future versions.
Comment 2 Itamar Heim 2012-05-28 15:45:24 EDT
how/where does user provides credentials to access engine via REST API to perform the reload operation?
(or is another communication channel assumed?)
Comment 3 Itamar Heim 2012-05-28 15:50:26 EDT
(In reply to comment #1)
>     Technical note added. If any revisions are required, please edit the
> "Technical Notes" field
>     accordingly. All revisions will be proofread by the Engineering Content
> Services team.
>     
>     New Contents:
> The Reloadable Configurations API will be considered a tech preview in

if tech preview, the tech preview keyword should be used.
also, tech preview doesn't mean the feature doesn't need qe_ack+ and some sanity testing.

> version 3.1 and might not be supported in future versions.

however the technical note conflicts with the definition of tech preview[1]
(you can't put something into tech preview if it is not going to be supported in a future version)

"A feature can qualify as Technology Preview or TP status if it meets the following conditions:

1. Red Hat will fully support it in subsequent minor or major release.
2. It's described functionality is testable.
3. Red Hat will provide high severity security erratas for the feature."


(although it sounds like there is a process to remove a TP feature under the section "How to drop TP feature")

[1] https://home.corp.redhat.com/node/32279
Comment 4 Andrew Cathrow 2012-05-28 15:55:40 EDT
Muli - we should make sure that this option is marked as tech preview in the config tool so it's clear to users.
Comment 5 lpeer 2012-05-31 06:58:59 EDT
(In reply to comment #2)
> how/where does user provides credentials to access engine via REST API to
> perform the reload operation?
> (or is another communication channel assumed?)

we prompt the user for user/pass
Comment 6 lpeer 2012-05-31 07:04:43 EDT
I change the technical note to emphasis that we do not commit to backwards compatibilty in the API for this feature.
If it is not clear you are welcomed to revise it.
Comment 7 lpeer 2012-05-31 07:04:43 EDT
    Technical note updated. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    Diffed Contents:
@@ -1 +1 @@
-The Reloadable Configurations API will be considered a tech preview in version 3.1 and might not be supported in future versions.+The Reloadable Configurations API will be considered a tech preview in version 3.1, the API may change in the future versions in an incompatible way.
Comment 9 Yaniv Kaul 2012-06-03 10:41:55 EDT
Please indicate what was implemented and how to test this.
Comment 12 Muli Salem 2012-07-05 08:35:40 EDT
The following was implemented:

1. There is a new action in the engine-config tool: --reload. When using this action, all reloadable configurations are loaded from vdc_options to the engine, with no need to restart.

2. A new flag --only-reloadable, to be used in the --list action, to receive a list of only the reloadable configurations.
Comment 13 Andrew Cathrow 2012-07-09 04:54:57 EDT
Moving to rhevm-future - won't make 3.1
Comment 15 Muli Salem 2012-09-09 03:06:31 EDT
Reloading is enabled through the API - already in the code base:

http://gerrit.ovirt.org/#/c/4503/

Remaining gap:

Adding the functionality to the engine-config CLI as explained in Comment 12.
Comment 24 Barak 2014-01-01 12:09:20 EST
moved back to new - to reset the work on this issue
Comment 25 Lior Vernia 2014-11-06 05:05:48 EST
Martin Mucha has submitted patches that to my understanding fix some gaps in the REST implementation, and has verified that it works. Adding the support to engine-config doesn't sound too difficult; maybe we want to get this done for 3.6? Wouldn't this be an easy win?
Comment 26 Oved Ourfali 2014-12-15 07:09:20 EST
(In reply to Lior Vernia from comment #25)
> Martin Mucha has submitted patches that to my understanding fix some gaps in
> the REST implementation, and has verified that it works. Adding the support
> to engine-config doesn't sound too difficult; maybe we want to get this done
> for 3.6? Wouldn't this be an easy win?

Sorry for the late response.
I'm not sure about the current status of this feature.
What gaps were addressed in these patches?
What's missing from the feature after merging these patches?
Comment 27 Martin Mucha 2014-12-16 07:33:14 EST
I wanted options of mine feature to be reloadable without restart. I found that there's command, forcing engine to reload data. But it contained a bug. So I fixed it (now the ReloadConfigurationCommand works), and did minor refactoring. Prior to this change ReloadConfigurationCommand only cause engine to reload data to in-memory cache. But that does not change anything. Only when <something> queries that option again, it will see the new value. Which is insufficient. So as a part of ReloadConfigurationCommand, changed options are now identified, stored in memory, and it's fired event(CDI event) informing all listeners about changed options. It's up to them if they do something with this information. My MAC Pool is probably the sole listener so far and it reinitializes it's internal structures to comply with new settings.
Comment 28 Oved Ourfali 2015-04-12 07:49:51 EDT
So it seems like if you do the following, you can reload the configuration entries that are reloadable:
1. As a developer 
 * you need to have the config entry annotated as @Reloadable.
 * Optionally, you can use the user the events mentioned in Comment #27
2. As a user, you need to issue this API request to reload the configuration.

Seems like that allows to close this RFE. Some other RFEs will need to be opened for specific keys, to make them reloadable, but the infra is there.

Moti - am I accurate?
If so, I'll move the RFE to modified, targeting to 3.6.0.
Comment 29 Moti Asayag 2015-04-12 08:43:07 EDT
(In reply to Oved Ourfali from comment #28)
> So it seems like if you do the following, you can reload the configuration
> entries that are reloadable:
> 1. As a developer 
>  * you need to have the config entry annotated as @Reloadable.
>  * Optionally, you can use the user the events mentioned in Comment #27

Observing updated-config-values events patch wasn't merged, since the approach to notify any subscriber for all of the events doesn't seem optimal to me. In addition, up until now, there wasn't a need for that functionality - hence the decision of which design will serve better config-value-change clients was postponed as well. If there is such need, to allow the system to observe a specific event(s), we need to revive that discussion.

> 2. As a user, you need to issue this API request to reload the configuration.
> 
> Seems like that allows to close this RFE. Some other RFEs will need to be
> opened for specific keys, to make them reloadable, but the infra is there.
> 
> Moti - am I accurate?

The reloadable configuration values can be divided into 2 types: static and dynamic.

The static types are read from the cache each time a certain flow is required to use them. So no change to existing flow from the change in config value perspective.

The dynanic types requires further applicative change, for example, when using a config value to define the thread pool size, and if wishes to dynamically resize it, there is a need to shutdown the existing one and starting a new one with the new value, or the time between monitor threads requires deleting quartz job and creating them again. 

The concept here is a code which is invoked once in the application, based on a certain configuration, cannot be updated itself by changing the config value and refreshing the cache alone. Some code should be executed for that change to take effect (when/if possible).

So currently we support only the 'static' types. Some work should be done to provide the infrastructure for 2, and then it become the developer responsibility to implement its own 'config-value-change' triggered event (as mentioned in the response of the first bullet).

> If so, I'll move the RFE to modified, targeting to 3.6.0.
Comment 32 Oved Ourfali 2015-04-12 09:04:14 EDT
Moving to MODIFIED.
What's supported is:
As a user, you need to issue this API request to reload the configuration.
It will only work for entries annotated with @Reloadable.
Comment 34 Oved Ourfali 2015-04-21 05:17:54 EDT
*** Bug 1204844 has been marked as a duplicate of this bug. ***
Comment 35 Oved Ourfali 2015-04-27 01:52:07 EDT
*** Bug 1057967 has been marked as a duplicate of this bug. ***
Comment 42 Scott Herold 2015-05-28 09:24:18 EDT
Closing based on fact most config items aren't reloadable, and there is no customer requirement for this request.
Comment 43 Michal Skrivanek 2015-11-24 09:14:03 EST
(In reply to Scott Herold from comment #42)
> Closing based on fact most config items aren't reloadable, and there is no
> customer requirement for this request.

I'm sorry, I just don't think so.
There are many options which are working already, plenty of others which can easily be enabled to work correctly. If some of the parameters marked as reloadable today doesn't work then disable them, but I don't think we should generally dismiss this functionality.
Necessity to restart ovirt-engine is one of the most frequent complaints I hear when dealing with these parameters.
There are quite a few ideas and suggestion in earlier comments, it sounds entirely feasible to do so in not-so-distant future