Bug 1801710 - ovirt-imageio service not available on engine host after clean reprovision meaning upload/download operations from UI do not work
Summary: ovirt-imageio service not available on engine host after clean reprovision me...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: ovirt-engine
Classification: oVirt
Component: BLL.Storage
Version: 4.4.0
Hardware: x86_64
OS: Unspecified
high
high
Target Milestone: ovirt-4.4.0
: 4.4.0
Assignee: Vojtech Juranek
QA Contact: Ilan Zuckerman
URL:
Whiteboard:
Depends On: 1862059
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-02-11 13:54 UTC by Yosi Ben Shimon
Modified: 2020-07-30 10:18 UTC (History)
10 users (show)

Fixed In Version: ovirt-engine-4.4.0
Clone Of:
Environment:
Last Closed: 2020-05-20 20:01:18 UTC
oVirt Team: Storage
Embargoed:
pm-rhel: ovirt-4.4+
aoconnor: blocker+


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
oVirt gerrit 108333 0 master MERGED packaging: Add imageio setup plugin 2021-02-01 20:14:04 UTC
oVirt gerrit 108349 0 master MERGED core: Remove ImageProxyAddress engine configuration 2021-02-01 20:14:04 UTC

Description Yosi Ben Shimon 2020-02-11 13:54:40 UTC
Description of problem:
After new reprovision of rhv 4.4, the ovirt-imageio-proxy service is not running.
# rpm -qa | grep imageio-proxy 
returns nothing

Using automation tests, the test failed with the next errors in the engine log:

2020-02-05 01:31:09,660+02 ERROR [org.ovirt.engine.core.bll.storage.disk.image.TransferDiskImageCommand] (default task-129) [5558cd2d-29c0-4f40-b9d3-42ba2a740ea8] Failed to add image ticket to ovirt-imageio-proxy: java.net.ConnectException: Connection refused (Connection refused)
	at java.base/java.net.PlainSocketImpl.socketConnect(Native Method)
	at java.base/java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:399)
	at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:242)
	at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:224)
	at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:403)
	at java.base/java.net.Socket.connect(Socket.java:591)
	at java.base/sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:285)
	at java.base/sun.security.ssl.BaseSSLSocketImpl.connect(BaseSSLSocketImpl.java:173)
	at java.base/sun.net.NetworkClient.doConnect(NetworkClient.java:182)
	at java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:474)
	at java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:569)
	at java.base/sun.net.www.protocol.https.HttpsClient.<init>(HttpsClient.java:265)


..... long stack trace


2020-02-05 01:31:09,661+02 ERROR [org.ovirt.engine.core.bll.storage.disk.image.TransferDiskImageCommand] (default task-129) [5558cd2d-29c0-4f40-b9d3-42ba2a740ea8] Failed to add image ticket to ovirt-imageio-proxy
2020-02-05 01:31:09,689+02 WARN  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-129) [5558cd2d-29c0-4f40-b9d3-42ba2a740ea8] EVENT_ID: TRANSFER_FAILED_TO_ADD_TICKET_TO_PROXY(1,075), Failed to add image ticket to ovirt-imageio-proxy - only transferring directly to ovirt-imageio-daemon is applicable (use 'transfer_url').

Apperantly, the ovirt-imageio-proxy was not installed.

Version-Release number of selected component (if applicable):
ovirt-engine-4.4.0-0.19.master.el7.noarch

How reproducible:
always

Steps to Reproduce:
1. install/update rhv environment
2. execute "rpm -qa | grep imageio"
3.

Actual results:
The ovirt-imageio-proxy is not installed

Expected results:
The ovirt-imageio-proxy should be installed and running

Additional info:

Comment 1 Nir Soffer 2020-02-11 14:10:10 UTC
Engine 4.4. is in the middle of migration to python 3 and imageio proxy
package may not be available until we finish this migration.

The proxy is used only for upload and download from UI, so it should not affect
upload/download tests which should use transfer_url. If you have tests using
only proxy_url they should be changed to test both transfer_url and proxy_url.

Tests using proxy_url should be skipped until we finish the migration to python 3.

Comment 4 Tal Nisan 2020-02-17 15:25:34 UTC
Sandro, can you have a look if the package was dropped by mistake?

Comment 5 Sandro Bonazzola 2020-02-28 12:25:58 UTC
(In reply to Tal Nisan from comment #4)
> Sandro, can you have a look if the package was dropped by mistake?

package has not been built for el8 and is not included in el8 repos.
You can take it from el7: https://resources.ovirt.org/repos/ovirt/tested/master/rpm/el7/noarch/ovirt-imageio-proxy-1.6.3-0.el7.noarch.rpm
but can't be automatically installed unless built for el8.

Comment 6 Michal Skrivanek 2020-03-03 13:45:43 UTC
we won't get it in some time, retargetting for 4.4.2 for now

Comment 8 Jan Zmeskal 2020-03-23 12:48:16 UTC
Just want to expand on what Nir wrote in comment #1:
The proxy is used only for upload and download from UI, so it should not affect
upload/download tests which should use transfer_url. If you have tests using
only proxy_url they should be changed to test both transfer_url and proxy_url.

This is not entirely true, since Ansible ovirt_disk module uses url_proxy as well. See here: https://github.com/ansible/ansible/blob/4e8b240b8b74db2453851f12ec82c7f782289187/lib/ansible/modules/cloud/ovirt/ovirt_disk.py#L377

This affects ovirt_disk module in a way described here: https://bugzilla.redhat.com/show_bug.cgi?id=1802661#c7

Comment 9 Nir Soffer 2020-03-23 13:02:14 UTC
(In reply to Jan Zmeskal from comment #8)
> Just want to expand on what Nir wrote in comment #1:
> The proxy is used only for upload and download from UI, so it should not
> affect
> upload/download tests which should use transfer_url. If you have tests using
> only proxy_url they should be changed to test both transfer_url and
> proxy_url.
> 
> This is not entirely true, since Ansible ovirt_disk module uses url_proxy as
> well. See here:
> https://github.com/ansible/ansible/blob/
> 4e8b240b8b74db2453851f12ec82c7f782289187/lib/ansible/modules/cloud/ovirt/
> ovirt_disk.py#L377

This is ovirt_disk ansible module bug, using outdated code. It should use the code in:
https://github.com/oVirt/ovirt-engine-sdk/blob/master/sdk/examples/upload_disk.py

It can support using a proxy for cases when you don't have access to host, but
the recommended and supported way to upload is via the host directly.

> This affects ovirt_disk module in a way described here:
> https://bugzilla.redhat.com/show_bug.cgi?id=1802661#c7

Should be fixed in that package.

Comment 10 Jan Zmeskal 2020-03-23 13:57:38 UTC
Hi Nir, is that bug tracked somewhere, please?

Comment 11 Nir Soffer 2020-03-23 14:02:26 UTC
(In reply to Jan Zmeskal from comment #10)
> Hi Nir, is that bug tracked somewhere, please?

I don't know about a bug, please file one.

Comment 12 Jan Zmeskal 2020-03-23 14:40:02 UTC
> I don't know about a bug, please file one.

The Ansible ovirt_disk issue is now being tracked here: https://github.com/ansible/ansible/issues/68403
Thank you for the explanation, Nir!

Comment 13 Lukas Svaty 2020-03-24 12:50:59 UTC
due to Regression keyword targeting to 4.4.0

Comment 16 Nir Soffer 2020-04-19 13:59:59 UTC
Moving back to 4.4.0, since the patches looks ready.

Comment 19 Ilan Zuckerman 2020-05-13 12:28:31 UTC
package ovirt-imageio-proxy was completely remove and daemon, which previously run only on the host, can now server also as a proxy, i.e. daemon now run host as well as on engine. The service is called ovirt-imageio.

In terms of packages, we have two packages:
* ovirt-imageio-common - basically imageio library, containing most of thecode
* ovirt-imageio-deamon - contains daemon script, service file etc. Requires ovirt-imageio-common.

Verified on:
rhv-release-4.4.0-35-001.noarch
ovirt-imageio-daemon-2.0.5-0.el8ev.x86_64
ovirt-imageio-common-2.0.5-0.el8ev.x86_64

Service is up and running:

[root@storage-ge-05 ~]#  systemctl status ovirt-imageio
● ovirt-imageio.service - oVirt ImageIO Daemon
   Loaded: loaded (/usr/lib/systemd/system/ovirt-imageio.service; enabled; vendor preset: disabled)
   Active: active (running) since Tue 2020-05-12 17:08:37 IDT; 17h ago
 Main PID: 1246057 (ovirt-imageio)
    Tasks: 3 (limit: 101312)
   Memory: 12.1M
   CGroup: /system.slice/ovirt-imageio.service
           └─1246057 /usr/libexec/platform-python -s /usr/bin/ovirt-imageio


I uploaded a disk image from UI while watching /var/log/ovirt-imageio/daemon.log both on engine and vdsm host.
No ERRORS / WARNS were seen, the process completed successfully.

Comment 20 Sandro Bonazzola 2020-05-20 20:01:18 UTC
This bugzilla is included in oVirt 4.4.0 release, published on May 20th 2020.

Since the problem described in this bug report should be
resolved in oVirt 4.4.0 release, it has been closed with a resolution of CURRENT RELEASE.

If the solution does not work for you, please open a new bug report.

Comment 21 Ilan Zuckerman 2020-07-30 10:18:18 UTC
This bug depends on libvirt bz
https://bugzilla.redhat.com/show_bug.cgi?id=1862059


Note You need to log in before you can comment on or make changes to this bug.