Bug 1913785

Does this reproduce?

Please attach also host-deploy logs.

I checked the host deploy, engine logs, ansible logs, there is nothing other than the error Format specifier '%2b'.
After reinstall the host activated successfully, I believe Petr have an env still with 1 more host broken this way, but reinstalling the host would put it to successful active state.

(In reply to Lukas Svaty from comment #3)
> I checked the host deploy, engine logs, ansible logs, there is nothing other
> than the error Format specifier '%2b'.
> After reinstall the host activated successfully, I believe Petr have an env
> still with 1 more host broken this way, but reinstalling the host would put
> it to successful active state.

If it's not reproducible, please CLOSE WORKSFORME. Otherwise, please attach logs. Thanks!

Environment was destroyed (reprovisioned) by mistake on my part and as this does not reproduce on this env or any other, closing and hope to not see it in the future.

Reopening this bug as I think I hit the issue:

2021-06-22 18:28:33,559+03 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-31) [7bdf6bd3] EVENT_ID: VDS_INSTALL_FAILED(505), Host host_mixed_3 installation failed. Format specifier '%2b'.

Attaching engine log.

Petr, can you pls check if this is indeed the same issue?

Error is actually from ansible runner
msg is not propagated to events (audit log): 'msg': 'Failed to download packages: Cannot download Packages/qemu-kvm-block-curl-5.2.0-16.module

2021-06-22 18:28:30,735 - runner_service.services.playbook - DEBUG - cb_event_handler event_data={'uuid': '0346ca66-fbea-41cc-81a0-f987774eca80', 'counter': 59, 'stdout': 'fatal: [host]: FAILED! => {"changed": false, "msg": "Failed to download packages: Cannot download Packages/qemu-kvm-block-curl-5.2.0-16.module%2bel8.4.0%2b10806%2bb7d97207.x86_64.rpm: All mirrors were tried", "results": []}', 'start_line': 51, 'end_line': 52, 'runner_ident': '62aec6bc-d36e-11eb-8837-001a4a168f00', 'event': 'runner_on_failed', 'p
id': 9947, 'created': '2021-06-22T15:28:30.733731', 'parent_uuid': '001a4a16-8f00-80e0-2c51-0000000001a9', 'event_data': {'playbook': 'ovirt-host-deploy.yml', 'playbook_uuid': 'afc2bc81-d5fc-474a-9632-f436f61d7e4a', 'play': 'all', 'play_uuid': '001a4a16-8f00-80e0-2c51-00000
0000008', 'play_pattern': 'all', 'task': 'Install ovirt-hosted-engine-setup package', 'task_uuid': '001a4a16-8f00-80e0-2c51-0000000001a9', 'task_action': 'yum', 'task_args': '', 'task_path': '/usr/share/ovirt-engine/ansible-runner-service-project/project/roles/ovirt-host-de
ploy-vdsm/tasks/packages.yml:6', 'role': 'ovirt-host-deploy-vdsm', 'host': 'host', 'remote_addr': 'host', 'res': {'msg': 'Failed to download packages: Cannot download Packages/qemu-kvm-block-curl-5.2.0-16.module%2bel
8.4.0%2b10806%2bb7d97207.x86_64.rpm: All mirrors were tried', 'results': [], 'invocation': {'module_args': {'name': ['ovirt-hosted-engine-setup'], 'state': 'present', 'allow_downgrade': False, 'autoremove': False, 'bugfix': False, 'disable_gpg_check': False, 'disable_plugin
': [], 'disablerepo': [], 'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude': [], 'installroot': '/', 'install_repoquery': True, 'install_weak_deps': True, 'security': False, 'skip_broken': False, 'update_cache': False, 'update_only': False, 'validate_
certs': True, 'lock_timeout': 30, 'conf_file': None, 'disable_excludes': None, 'download_dir': None, 'list': None, 'releasever': None}}, '_ansible_no_log': False, 'changed': False}, 'start': '2021-06-22T15:27:52.264033', 'end': '2021-06-22T15:28:30.733528', 'duration': 38.4
69495, 'ignore_errors': None, 'event_loop': None, 'uuid': '0346ca66-fbea-41cc-81a0-f987774eca80'}}
2021-06-22 18:28:30,736 - runner_service.services.playbook - DEBUG - cb_event_handler event_data={'uuid': '4790bd1c-d454-4954-9a83-108b599fd62d', 'counter': 60, 'stdout': '\r\nPLAY RECAP *********************************************************************\r\nhost
 : ok=17   changed=3    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0   \r\n', 'start_line': 52, 'end_line': 56, 'runner_ident': '62aec6bc-d36e-11eb-8837-001a4a168f00', 'event': 'playbook_on_stats', 'pid': 9947, 'created': '2021-06-22T1
5:28:30.736082', 'parent_uuid': 'afc2bc81-d5fc-474a-9632-f436f61d7e4a', 'event_data': {'playbook': 'ovirt-host-deploy.yml', 'playbook_uuid': 'afc2bc81-d5fc-474a-9632-f436f61d7e4a', 'changed': {'host': 3}, 'dark': {}, 'failures': {'host
': 1}, 'ignored': {}, 'ok': {'host': 17}, 'processed': {'host': 1}, 'rescued': {}, 'skipped': {}, 'artifact_data': {}, 'uuid': '4790bd1c-d454-4954-9a83-108b599fd62d'}}
2021-06-22 18:28:30,929 - runner_service.services.playbook - INFO - Playbook ovirt-host-deploy.yml, UUID=62aec6bc-d36e-11eb-8837-001a4a168f00 ended, status=failed
2021-06-22 18:28:30,934 - runner_service.services.playbook - INFO - Playbook ovirt-host-deploy.yml Stats: {'skipped': {}, 'ok': {'host': 17}, 'dark': {}, 'failures': {'host': 1}, 'processed': {'host': 1}, 'changed': {'host': 3}}

It happens only very rarely, decreasing medium

just for the information
ansible host deploy log ends with:

....
      "start" : "2021-06-22T15:27:52.027776",
      "end" : "2021-06-22T15:27:52.099039",
      "duration" : 0.071263,
      "event_loop" : "items",
      "uuid" : "1ab2914a-bf0e-4a09-a32e-416cfc3e9764"
    }
  }
}

2021-06-22 18:27:54 IDT - TASK [ovirt-host-deploy-vdsm : Install ovirt-hosted-engine-setup package] ******
<EOF>

I fixed the format specifier error message, but note, as stated in https://bugzilla.redhat.com/show_bug.cgi?id=1913785#c8, the real issue is about a package that couldn't be downloaded: 
"msg": "Failed to download packages: Cannot download Packages/qemu-kvm-block-curl-5.2.0-16.module%2bel8.4.0%2b10806%2bb7d97207.x86_64.rpm: All mirrors were tried",

This bug report has Keywords: Regression or TestBlocker.
Since no regressions or test blockers are allowed between releases, it is also being identified as a blocker for this release. Please resolve ASAP.

Verified on ovirt-engine-4.4.7.6-0.11.el8ev.noarch

This bugzilla is included in oVirt 4.4.7 release, published on July 6th 2021.

Since the problem described in this bug report should be resolved in oVirt 4.4.7 release, it has been closed with a resolution of CURRENT RELEASE.

If the solution does not work for you, please open a new bug report.