CURRENT STATE engine updates statistics in database. dwh does not know if statistics are valid. as a result dwh pulls engine using http and if engine is up it assumes statistics are valid (updated). PROBLEM IN CURRENT STATE dwh should be decoupled from engine, and monitor only database, the fact that it is initiating http request to the engine is redundant, it requires network privileges/reachability and extra communications. SOLUTION When engine update statistics it should also update a specific column (can be in vdc_options or any) with the timestamp of last statistics update. This way the dwh can determine if statistics are valid or not without interacting directly with the engine.
Setting on future for now. Barak, feel free to move to 3.3 if you decide it's feasible
should't be on every stat update - its a multithreaded stats update. can probably just a thread to update the config every x seconds or something like that, which would only work while engine is up anyway.
(In reply to comment #2) > should't be on every stat update - its a multithreaded stats update. > can probably just a thread to update the config every x seconds or something > like that, which would only work while engine is up anyway. Question if less resources to do this within the stored procedure that updates the stats or perform another query. I am unsure if decoupled interval query is less resources. But never the less, will be a better solution than the existing implementation.
eli - thoughts?
This is in since 3.2. Closing. Yaniv
(In reply to Itamar Heim from comment #4) > eli - thoughts? I think that having a field in the database for that is OK, however , I don't think that this is a configuration value so I would not put that in vdc_options rather in a separate table (engine_status or something like that) that will be a place holder for future needs as well. I like more the a thread in the BLL that will update this field periodically rather than putting that inside the SPs since we will have to put that in multiple places and we may be facing deadlock issues and such stuff. So , another thread doing the update every let say 1 min will be perfect.
(In reply to Eli Mesika from comment #6) > (In reply to Itamar Heim from comment #4) > > eli - thoughts? > > I think that having a field in the database for that is OK, however , I > don't think that this is a configuration value so I would not put that in > vdc_options rather in a separate table (engine_status or something like > that) that will be a place holder for future needs as well. > > I like more the a thread in the BLL that will update this field periodically > rather than putting that inside the SPs since we will have to put that in > multiple places and we may be facing deadlock issues and such stuff. So , > another thread doing the update every let say 1 min will be perfect. This is done via heartbeat async module mperina has implemented. Yaniv