Bug 1034495 - [RFE] Improved upgrade task process.
Summary: [RFE] Improved upgrade task process.
Keywords:
Status: CLOSED DEFERRED
Alias: None
Product: Spacewalk
Classification: Community
Component: Server
Version: 2.1
Hardware: Unspecified
OS: Unspecified
unspecified
low
Target Milestone: ---
Assignee: Tomas Lestach
QA Contact: Red Hat Satellite QA List
URL:
Whiteboard:
Depends On:
Blocks: spacewalk-rfe
TreeView+ depends on / blocked
 
Reported: 2013-11-26 00:18 UTC by Michal Bruncko
Modified: 2020-03-24 13:22 UTC (History)
0 users

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2020-03-24 13:22:57 UTC
Embargoed:


Attachments (Terms of Use)

Description Michal Bruncko 2013-11-26 00:18:45 UTC
Description of problem:
I am wondering if there is way to use serialized (machine-per-machine) upgrade process instead of parallel (all machine at once) process without manual intervention by admin. As I can see from upgrade wizard I can only choose to "Install updates ASAP" or "Schedule upgrade process to specified time", but it does not matter as for both cases upgrade task will begin on all selected machines at once.
Currently two issues can be observed: high resources (of virtualization host/hypervisor) utilization and service unavailability (during installation/upgrade process if affected service/daemon is subject of that upgrade process as well) even if HA is implemented on servers. 

I am wondering about two options with more distributed/moderated utilization like which is available today:

1. serializing installation per virtual machine instead of all-at-once
behavior which is available now. this have advantage mainly in cases -
when two redundant machines are serving same service (i.e. coroporate
directory) and directory process needs to be restarted in oder to
upgrade this package - which cause service unavailability at all as both
virtuals will be in upgrade process at same time.

2. second way can be work similar like with clam.d process upgrade like
we know - it us running on daily basis, but the upgrade process is
executed in random time within next four hour from executing this script
by cron process. So I am talking about choice "Upgrade randomly within
XX hours/minutes per every selected machine from now". This will spread
upgrade process of all selected machines randomly over defined time
interval. And of course the disk utilization will be not so high utilized. 

This proposed improved upgrade process will not replace existing options, but be available as next available option to existing two options "Install updates ASAP" or "Schedule upgrade process to specified time".

Comment 1 Michael Mráka 2014-03-04 09:14:16 UTC
(In reply to Michal Bruncko from comment #0)
> 2. second way can be work similar like with clam.d process upgrade like
> we know - it us running on daily basis, but the upgrade process is
> executed in random time within next four hour from executing this script
> by cron process. So I am talking about choice "Upgrade randomly within
> XX hours/minutes per every selected machine from now". This will spread
> upgrade process of all selected machines randomly over defined time
> interval. And of course the disk utilization will be not so high utilized. 

This is exactly how Spacewalk's actions work. Rhn_check on client contacts  Spacewalk server once within "checking" period (4 hours by default) and the beginning of this period is randomly shifted at server boot. So even if all your servers in datacenter start at the very same moment they won't hit your server at the same moment. On the other hand if you need an immediate reaction you can setup osad on client.

Comment 2 Michal Bruncko 2014-03-05 22:20:58 UTC
> On the other hand if you need an immediate reaction you can setup osad on client.
yes, we are using osad because of another requirement of configuration management. but once osad is used we cannot use mechanism described by you using rhn_check running randomly within 4 hours (on those hosts), right?

so now all planned actions are done immediately on all requested hosts as this operation is pushed through osad to all hosts at once, right?

Comment 3 Michael Mráka 2014-03-07 08:49:24 UTC
> yes, we are using osad because of another requirement of configuration
> management. but once osad is used we cannot use mechanism described by you
> using rhn_check running randomly within 4 hours (on those hosts), right?
>
> so now all planned actions are done immediately on all requested hosts as
> this operation is pushed through osad to all hosts at once, right?

That's correct. Once client uses osad all actions are being run immediately.

Comment 4 Michal Bruncko 2014-03-07 11:35:49 UTC
Are the clients still use Rhn_Check in parallel to osad? Or if the osad is used on clients the rhn_check is not used at all (with checking behavior describe before)?

Comment 5 Michael Mráka 2014-03-14 12:54:45 UTC
Well, let me clarify it a bit more.

It's always rhn_check which picks up scheduled action. It can be launched either by rhnsd every 4 hours (by default) or by osad whenever osad receives notification about new action scheduled on Spacewalk server.

There's no problem to run both rhnsd and osad on the client. Anyway if osad is enabled it's notified immediatelly about new action and runs rhn_check which executes the action. So later when rhnsd wakes up and runs rhn_check there's no action to run.

Comment 6 Michal Bruncko 2014-03-14 18:24:24 UTC
Thanks for clarifications, now it is clear to me. 
Now it seems that we are not able to achieve mass system upgrade using rhnsd instead of using osad (if systems are using osad by default), right?

Comment 7 Michael Mráka 2014-03-17 09:57:44 UTC
> Now it seems that we are not able to achieve mass system upgrade using rhnsd
> instead of using osad (if systems are using osad by default), right?

I don't understand. You can use just rhnsd for mass system upgrade. It just won't happen at the precise scheduled time (but at the next following check).

BTW you could also take a look at 'staging content' feature. When enabled clients can download packages (pre-cache) for update ahead and then at scheduled time just perform update from cache.

Comment 8 Michal Bruncko 2014-03-19 11:39:14 UTC
> I don't understand. You can use just rhnsd for mass system upgrade. It just won't happen at the precise scheduled time (but at the next following check).

I am just asking if I am able to use rhnsd for mass system upgrade even if we are using osad communication between clients and server at the same time. 

If the response is again "yes", then please where is such option to use rhnsd for package upgrading instead of osad?

Comment 9 Michael Mráka 2014-03-21 11:07:39 UTC
> I am just asking if I am able to use rhnsd for mass system upgrade even if
> we are using osad communication between clients and server at the same time. 
> 
> If the response is again "yes", then please where is such option to use
> rhnsd for package upgrading instead of osad?

No. If osad is enabled it's notified immediately and runs rhn_check which picks up action. There's no action left for subsequent rhn_checks (invoked via rhnsd).


Take a look at staging content.

Comment 11 Michael Mráka 2020-03-24 13:22:57 UTC
Spacewalk project as an upstream for Red Hat Satellite 5 product is going to be End Of Life on May 31 2020.

Spacewalk 2.10 has been released as the last release of this project.
https://github.com/spacewalkproject/spacewalk/wiki/ReleaseNotes210

Any new feature will not be included therefore closing remaining RFEs to set expectations properly.


Note You need to log in before you can comment on or make changes to this bug.