Created attachment 1871353 [details] host deploy logs oVirt Engine 4.4.10 installed with mode "Gluster" fails to deploy oVirt Node 4.4.10 with: 2022-04-07 15:06:58 UTC - fatal: [node45el8.lab]: FAILED! => {"changed": true, "cmd": "tuned-adm profile \"\"\n", "delta": "0:00:00.097189", "end": "2022-04-07 15:06:58.364844", "msg": "non-zero return code", "r c": 1, "start": "2022-04-07 15:06:58.267655", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2022-04-07 15:06:58 UTC - { "status" : "OK", "msg" : "", "data" : { "uuid" : "7747bdc2-84e9-4418-98d0-3b84047875c1", "counter" : 292, "stdout" : "fatal: [node45el8.lab]: FAILED! => {\"changed\": true, \"cmd\": \"tuned-adm profile \\\"\\\"\\n\", \"delta\": \"0:00:00.097189\", \"end\": \"2022-04-07 15:06:58.364844\", \"msg\": \"non-zero retu rn code\", \"rc\": 1, \"start\": \"2022-04-07 15:06:58.267655\", \"stderr\": \"\", \"stderr_lines\": [], \"stdout\": \"\", \"stdout_lines\": []}", "start_line" : 280, "end_line" : 281, "runner_ident" : "144dce38-b684-11ec-8d1a-525400bc02b1", "event" : "runner_on_failed", "pid" : 37770, "created" : "2022-04-07T15:06:57.595466", "parent_uuid" : "525400bc-02b1-4e29-5db4-00000000033b", "event_data" : { "playbook" : "ovirt-host-deploy.yml", "playbook_uuid" : "f88a7c22-4144-46d9-8db6-5315b3beb926", "play" : "all", "play_uuid" : "525400bc-02b1-4e29-5db4-000000000006", "play_pattern" : "all", "task" : "Set tune profile", "task_uuid" : "525400bc-02b1-4e29-5db4-00000000033b", "task_action" : "command", "task_args" : "", "task_path" : "/usr/share/ovirt-engine/ansible-runner-service-project/project/roles/ovirt-host-deploy-misc/tasks/tuned_profile.yml:18", "role" : "ovirt-host-deploy-misc", "host" : "node45el8.lab", "remote_addr" : "node45el8.lab", "res" : { "msg" : "non-zero return code", "cmd" : "tuned-adm profile \"\"\n", "stdout" : "", "stderr" : "", "rc" : 1, "start" : "2022-04-07 15:06:58.267655", "end" : "2022-04-07 15:06:58.364844", "delta" : "0:00:00.097189", "changed" : true, "invocation" : { "module_args" : { "_raw_params" : "tuned-adm profile \"\"\n", "_uses_shell" : true, "warn" : true, "stdin_add_newline" : true, "strip_empty_ends" : true, "argv" : null, "chdir" : null, "executable" : null, "creates" : null, "removes" : null, "stdin" : null } }, "stdout_lines" : [ ], "stderr_lines" : [ ], "_ansible_no_log" : false }, "start" : "2022-04-07T15:06:56.944147", "end" : "2022-04-07T15:06:57.595318", "duration" : 0.651171, "ignore_errors" : null, "event_loop" : null, "uuid" : "7747bdc2-84e9-4418-98d0-3b84047875c1"
I'm going to upgrade to 4.5.0 beta and check there too.
Any chance to still have engine.log? It seems like tuned profile for gluster is incorrectly set ...
Created attachment 1871426 [details] engine log please note engine got updated in the meanwhile from original report but logs also include the relevant part related to this bug.
Tested with ovirt-engine 4.5.0.1 with ovirt-node-ng-4.5.0-0.20220407.0+1 same error. Attaching new logs in a few seconds
Created attachment 1871429 [details] ansible logs from 4.5
Created attachment 1871431 [details] engine logs with node 4.5/engine 4.5
Not sure if relevant, but please note I was testing this with oVirt Node running in a VM, not on bare metal.
The profile itself is missing in the command (the command should, for example, look like "cmd" : "tuned-adm profile \"virtual-host\"\n", and in this case it appears as "cmd" : "tuned-adm profile \"\"\n") Please add the extravars file (var/lib/ovirt-engine/ansible-runner/<uuid>/env/extravars), I want to see if variables weren't configured as they should have
In OST I see: "cmd" : "tuned-adm profile \"virtual-host\"\n", ovirt-engine-4.5.1-0.2.master.20220420080218.git6ace5876a5.el8.noarch ovirt-node-ng-4.5.1-0.20220419.0
Sandro, did you switch gluster service on on a default cluster / on an existing cluster? I think in that case (because of UI issue) gluster_tuned_profile is not set. I think it is broken like that for a while...
(In reply to Michal Skrivanek from comment #11) > Sandro, did you switch gluster service on on a default cluster / on an > existing cluster? > > I think in that case (because of UI issue) gluster_tuned_profile is not set. > I think it is broken like that for a while... No, clean gluster only installation using a VM as host.
(In reply to Dana from comment #8) > The profile itself is missing in the command > (the command should, for example, look like "cmd" : "tuned-adm profile > \"virtual-host\"\n", and in this case it appears as "cmd" : "tuned-adm > profile \"\"\n") > Please add the extravars file > (var/lib/ovirt-engine/ansible-runner/<uuid>/env/extravars), I want to see if > variables weren't configured as they should have Sorry, I haven't the environment anymore. But it's easy to reproduce. Let me know if you have trouble reproducing.
it still needs clarification. Installed how? So is it a Default cluster that you do not touch in any way, just add host? Or a newly created cluster with gluster enabled (that does work AFAICT)?
(In reply to Michal Skrivanek from comment #14) > it still needs clarification. > Installed how? So is it a Default cluster that you do not touch in any way, > just add host? Or a newly created cluster with gluster enabled (that does > work AFAICT)? 1) setup a VM with CentOS Stream minimal 2) setup ovirt-engine and when it ask Virt/Gluster/Both select Gluster 3) finish the engine-setup process. 4) setup a VM using oVirt Node 5) attach the VM to the engine default datacenter as created by the engine setup process
thanks. Then it's probably the engine-setup that doesn't create the Default cluster properly. By default we set the profile to NULL in packaging/dbscripts/data/00400_insert_cluster.sql This hasn't changed for a long time, are you sure it worked before? WA would be to create a new cluster
yes, this is how it looks right after engine-setup finishes: engine=# select name,gluster_tuned_profile from cluster; name | gluster_tuned_profile ---------+----------------------- Default | again, if this hasn't changed recently probably not worth a fix
So actually there are 3 separate issues: 1. Default cluster should have set a default gluster tuned profile, when gluster services are enabled during host deploy - Currently default cluster values are hard-coded in database initialization scripts, gluster services are disabled by default and even when administrator allows only gluster services, we don't change any settings in the default cluster 2. Current UI is not able to set gluster tuned profile within Edit Cluster dialog if gluster tuned profile is not defined - That issue prevents properly set gluster tuned profile for the Default cluster after initial installation - As a workaround administrators may create a new cluster, where we allow to specify gluster tuned profile from defined list properly 3. During host deploy we should not try to apply empty or undefined tuned profile - This is fixed by https://github.com/oVirt/ovirt-engine/pull/294 So PR https://github.com/oVirt/ovirt-engine/pull/294 should prevent error raised within this bug, but it doesn't solve the core issue described in items 1. and 2. And as mentioned in Comment 17 we may not have enough resources to properly fix those
This bug report has Keywords: Regression or TestBlocker. Since no regressions or test blockers are allowed between releases, it is also being identified as a blocker for this release. Please resolve ASAP.
no behavior change from 4.4 on gluster being disabled in Default Cluster, this apparently never worked out of the box, a new cluster has to be created for Gluster.
Let's move handling tuned profile from https://github.com/oVirt/ovirt-engine/blob/master/packaging/ansible-runner-service-project/project/roles/ovirt-host-deploy-misc/tasks/tuned_profile.yml#L5 to engine, where conditions would be easier and we can always pass the correct tuned profile
This bugzilla is included in oVirt 4.5.1 release, published on June 22nd 2022. Since the problem described in this bug report should be resolved in oVirt 4.5.1 release, it has been closed with a resolution of CURRENT RELEASE. If the solution does not work for you, please open a new bug report.