Description of problem: Module virt was not changed from 8.2 (or 8.3) to av during host upgrade (basically this doesn't work any more: https://bugzilla.redhat.com/show_bug.cgi?id=1877679) Looks like tasks in ovirt-host-deploy-facts especially related to av are skipped (/usr/share/ovirt-engine/ansible-runner-service-project/project/ovirt-host-upgrade.yml - /usr/share/ovirt-engine/ansible-runner-service-project/project/roles/ovirt-host-deploy-facts/tasks/av-setup.yml) which are performed before failing task it is failing on: - name: Install ovirt-host package if it isn't installed yum: name: ovirt-host state: latest update_cache: yes lock_timeout: 300 tags: - updatecheck - skip_ansible_lint # E403 in /usr/share/ovirt-engine/ansible-runner-service-project/project/roles/ovirt-host-upgrade/tasks/main.yml:8 The upgrade fails on dependency issue: (ansible-runner-service.log) 2022-02-15 18:18:07,777 - runner_service.services.playbook - DEBUG - cb_event_handler event_data={'uuid': 'ae6b6253-b7c4-4407-a815-723cc413843f', 'counter': 98, 'stdout': '', 'start_line': 92, 'end_line': 92, 'runner_ident': '8cbb4646-8e8b-11ec-8f7e-001a4a014700', 'event': 'runner_on_start', 'pid': 13838, 'created': '2022-02-15T18:18:07.776237', 'parent_uuid': '001a4a01-4700-51a3-76c0-00000000001d', 'event_data': {'playbook': 'ovirt-host-upgrade.yml', 'playbook_uuid': '271a64e9-e0ec-4b23-9c87-c5afeb2c7f1d', 'play': 'all', 'play_uuid': '001a4a01-4700-51a3-76c0-00000000000b', 'play_pattern': 'all', 'task': "Install ovirt-host package if it isn't installed", 'task_uuid': '001a4a01-4700-51a3 -76c0-00000000001d', 'task_action': 'yum', 'task_args': '', 'task_path': '/usr/share/ovirt-engine/ansible-runner-service-project/project/roles/ovirt-host-upgrade/tasks/main.yml:8', 'role': 'ovirt-host-upgrade', 'host': '<FQDN_OF_THE_HOST>', 'uuid': 'ae6b6253-b7c4-4407-a815-723cc413843f'}} 2022-02-15 18:18:17,411 - runner_service.services.playbook - DEBUG - cb_event_handler event_data={'uuid': '9038e101-2030-4058-8a3c-0c495ca1a6ee', 'counter': 99, 'stdout': 'fatal: [<FQDN_OF_THE_HOST>]: FAILED! => {"changed": false, "failures": [], "msg": "Depsolve Error occured: \\n Problem: package ovirt-host-dependencies-4.4.9-2.el8ev.x86_64 requires vdsm >= 4.40.90, but none of the providers can be installed\\n - package ovirt-host-4.4.9-2.el8ev.x86_64 requires ovirt-host-dependencies = 4.4.9-2.el8ev, but none of the providers can be installed\\n - package vdsm-4.40.90.4-1.el8ev.x86_64 requires libvirt-daemon-kvm >= 7.6.0-4.module+el8.5.0+12786+c4633d9a, but none of the providers can be installed\\n - package vdsm-4.40.90.2-1.el8ev.x86_64 requires libvirt-daemon-kvm >= 7.6.0-4.module+el8.5.0+12786+c4633d9a, but n one of the providers can be installed\\n - package vdsm-4.40.90.5-1.el8ev.x86_64 requires libvirt-daemon-kvm >= 7.6.0-4.module+el8.5.0+12786+c4633d9a, but none of the providers can be installed\\n - package vdsm-4.40.100.2-1.el8ev.x86_64 requires libvirt-daemon-kvm >= 7.6.0-4.module+el8.5.0+12786+c4633d9a, but none of the providers can be installed\\n - cannot install the best update candidate for package ovirt-host-4.4 .1-4.el8ev.x86_64\\n - package libvirt-daemon-kvm-7.6.0-6.module+el8.5.0+13051+7ddbe958.x86_64 is filtered out by modular filtering", "rc": 1, "results": []}', 'start_line': 92, 'end_line': 93, 'runner_ident': '8cbb4646-8e8b-11ec-8f7e-001a4a014700', 'event': 'runner_on_failed', 'pid': 13838, 'created': '2022-02-15T18:18:17.410133', 'parent_uuid': '001a4a01-4700-51a3-76c0-00000000001d', 'event_data': {'playbook': 'ovirt-ho st-upgrade.yml', 'playbook_uuid': '271a64e9-e0ec-4b23-9c87-c5afeb2c7f1d', 'play': 'all', 'play_uuid': '001a4a01-4700-51a3-76c0-00000000000b', 'play_pattern': 'all', 'task': "Install ovirt-host package if it isn't installed", 'task_uuid': '001a4a01-4700-51a3-76c0-00000000001d', 'task_action': 'yum', 'task_args': '', 'task_path': '/usr/share/ovirt-engine/ansible-runner-service-project/project/roles/ovirt-host-upgrade/tasks/m ain.yml:8', 'role': 'ovirt-host-upgrade', 'host': '<FQDN_OF_THE_HOST>', 'remote_addr': '<FQDN_OF_THE_HOST>', 'res': {'msg': 'Depsolve Error occured: \n Problem: package ovirt-host-dependencies-4.4.9-2.el8ev.x86_64 requires vdsm >= 4.40.90, but none of the providers can be installed\n - package ovirt-host-4.4.9-2.el8ev.x86_64 requires ovirt-host-dependencies = 4.4.9-2.el8ev, bu t none of the providers can be installed\n - package vdsm-4.40.90.4-1.el8ev.x86_64 requires libvirt-daemon-kvm >= 7.6.0-4.module+el8.5.0+12786+c4633d9a, but none of the providers can be installed\n - package vdsm-4.40.90.2-1.el8ev.x86_64 requires libvirt-daemon-kvm >= 7.6.0-4.module+el8.5.0+12786+c4633d9a, but none of the providers can be installed\n - package vdsm-4.40.90.5-1.el8ev.x86_64 requires libvirt-daemon-kvm >= 7.6.0-4.module+el8.5.0+12786+c4633d9a, but none of the providers can be installed\n - package vdsm-4.40.100.2-1.el8ev.x86_64 requires libvirt-daemon-kvm >= 7.6.0-4.module+el8.5.0+12786+c4633d9a, but none of the providers can be installed\n - cannot install the best update candidate for package ovirt-host-4.4.1-4.el8ev.x86_64\n - package libvirt-daemon-kvm-7.6.0-6.module+el8.5.0+13051+7ddbe958.x86_64 is filtered out by modular filtering', 'failures': [], 'results': [], 'rc': 1, 'invocation': {'module_args': {'name': ['ovirt-host'], 'state': 'latest', 'update_cache': True, 'lock_timeout': 300, 'allow_downgrade': False, 'autoremove': False, 'bugfix': False, 'disable_gpg_check': False, 'disable_plugin': [], 'disablerepo': [], 'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude': [], 'installroot': '/', 'install_repoquery ': True, 'install_weak_deps': True, 'security': False, 'skip_broken': False, 'update_only': False, 'validate_certs': True, 'conf_file': None, 'disable_excludes': None, 'download_dir': None, 'list': None, 'releasever': None}}, '_ansible_no_log': False, 'changed': False}, 'start': '2022-02-15T18:18:07.776150', 'end': '2022-02-15T18:18:17.409825', 'duration': 9.633675, 'ignore_errors': None, 'event_loop': None, 'uuid': '9038e 101-2030-4058-8a3c-0c495ca1a6ee'}} After fail: [root@sat-client-01 ~]# yum module list --enabled ... virt 8.2 [e] common Virtualization module ... Version-Release number of selected component (if applicable): upgrade from 8.2 or 8.3 RHEL with engine: ovirt-engine-4.4.10.6-0.1.el8ev.noarch How reproducible: always Steps to Reproduce: 1. Have a host with virt:8.2 or virt:8.3 2. upgrade these hosts to latest 4.4.10
AV change need to happen before upgrading ovirt-host, or it shouldn't be trying o upgrade it to latest
(In reply to Michal Skrivanek from comment #1) > AV change need to happen before upgrading ovirt-host, or it shouldn't be > trying o upgrade it to latest Not true, virt module version is change during the upgrade from RHEL 8.m to RHEL 8.n As discussed on the grooming today, repositories are not correct, that's why upgrade fails
Repositories are correct. Used standard build (which was used a lot of times for installing host) Also I used the same repositories for upgrading from RHEL 8.4 - virt:av - RHV 4.4.8 to RHEL 8.5 virt:av RHV 4.4.10 and there it worked Non-working scenarios: RHEL 8.2 - virt:8.2 - RHV 4.4.2 RHEL 8.3 - virt:8.3 - RHV 4.4.5
upgrade from 4.4(8.5) to 4.5(8.6) is also failing: "stdout" : "fatal: [FQDN]: FAILED! => {\"msg\": \"The conditional check 'ansible_facts.packages['r edhat-release'].version is version(latest_rh_rel, '!=')' failed. The error was: error while evaluating conditional (ansible_facts.pa ckages['redhat-release'].version is version(latest_rh_rel, '!=')): 'list object' has no attribute 'version'\\n\\nThe error appears t o be in '/usr/share/ovirt-engine/ansible-runner-service-project/project/roles/ovirt-host-upgrade/tasks/advanced-virtualization.yml': line 16, column 11, but may\\nbe elsewhere in the file depending on the exact syntax problem.\\n\\nThe offending line appears to be :\\n\\n - block:\\n - name: Reset configuration of advanced virtualization module\\n ^ here\\n\"}",
testing from latest live environment (4.4.10) to latest internal build containing ovirt-engine-setup-4.5.0.4-0.1.el8ev.noarch.rpm upgrade of engine itself looks fine and was successful, I will do some more runs to be sure host upgrade from engine GUI - Check for Updgrades > Upgrade is failing, above message was found in ovirt-host-mgmt-ansible...
I saw in host upgrade log that all tasks related to av-setup are skipped: 2022-02-15 18:17:48 UTC - TASK [ovirt-host-deploy-facts : Reset configuration of advanced virtualization module] *** 2022-02-15 18:17:48 UTC - TASK [ovirt-host-deploy-facts : Find relevant advanced virtualization module version] *** 2022-02-15 18:17:48 UTC - TASK [ovirt-host-deploy-facts : Enable advanced virtualization module] ********* I checked the conditions for execution av-setup tasks, and found that: - not vdsm_preinstalled|bool is not met: 2022-02-15 18:17:48 UTC - TASK [ovirt-host-deploy-facts : Check if vdsm is preinstalled] ***************** 2022-02-15 18:17:48 UTC - ok: [sat-client-02.rhev.lab.eng.brq.redhat.com] "res" : { "changed" : false, "ansible_facts" : { "vdsm_preinstalled" : true }, in Fetch installed packages: "vdsm" : [ { "name" : "vdsm", "version" : "4.40.50.10", "release" : "1.el8ev", "epoch" : null, "arch" : "x86_64", "source" : "rpm" } ], so this is the reason the version is not changed. Does it really matter is vdsm is preinstalled or can we just remove this condition?
in which log, the one from Feb? that's irrelevant. Please fix the _current_ problem, and current change from 4.2/4.3/4.4 to 4.5's virt:rhel Pavol, please attach the log
I attached log from upgrade: ovirt-host-mgmt-ansible-20220427111347-FQDN-d4fdda58-bf4e-4dd9-8576-42f7507bc65e.log I tried to generate sos report or ovirt-log-collctor but in both cases size was too big to attach. I contacted also Dana through chat, and provided her with access to environment, so she can also have full picture of env. Current issue which I have is upgrade 4.4.10 > 4.5, ( attempts for upgrade from 4.4.2 or other previous version will come latter )
(In reply to Michal Skrivanek from comment #9) > in which log, the one from Feb? that's irrelevant. Please fix the _current_ > problem, and current change from 4.2/4.3/4.4 to 4.5's virt:rhel > > Pavol, please attach the log It was in the log from February, but that's the issue (I checked it earlier today). I'm creating a fix for it now
The patch I added includes fixes both for the original bug & the issue Pavol found
Note: I see the docs are already updated with SP1 instructions: https://access.redhat.com/documentation/en-us/red_hat_virtualization/4.4/html-single/upgrade_guide/index#Manually_Updating_Hosts_minor_updates https://access.redhat.com/documentation/en-us/red_hat_virtualization/4.4/html-single/installing_red_hat_virtualization_as_a_self-hosted_engine_using_the_command_line/index#Enabling_the_Red_Hat_Enterprise_Linux_Host_Repositories_SHE_deployment_host So we are good here.
Verified in ovirt-engine-4.5.0.6-0.7.el8ev.noarch Tested host upgrade from 4.4.4 (RHEL 8.3) to 4.5 (RHEL 8.6) Module stream was changed from virt:8.3 to virt:rhel during the upgrade
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: RHV Manager (ovirt-engine) [ovirt-4.5.0] security update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2022:4711