Description of problem: Customer has configured for Image I/O and has repeatedly failed to successfully upload a disk from their client It seems similar to https://bugzilla.redhat.com/show_bug.cgi?id=1576500
Daniel, is that a dup of bug 1576500?
According to the proxy log, seems the webadmin didn't reach the proxy. So it could be a certificate issue or misconfigured proxy address. * Try using "Test Connection" button (in upload dialog), to ensure connectivity with imageio-proxy. If it fails, then download the certificate using the link in the error message, and import it to the browser - a green lock should appear in the address bar if imported correctly (have a look at "Error Handling" section of imageio slides: https://danielerez.github.io/slides-imageio/#/7) * Please ensure that ImageProxyAddress config value is <FQDN of engine>:54323 (using engine-config -g ImageProxyAddress).
When updating the engine version, imageio-proxy should be updated as well (to avoid version miscorrelation). Can you please try to update ovirt-imageio-proxy and restart ovirt-imageio-proxy service afterwards (if you're using latest 4.2.z engine version, then proxy version should be 1.4.3).
All of that is already checked (We ensure at the beginning when support case were opened): - Test connection is OK - Certificate problems are resolved (Green lock in browser) - ImageProxyAdress value is OK in manager I see inside the manager that ovirt-imageio-proxy versions are correct and updated: # rpm -qa|grep ovirt-image ovirt-imageio-common-1.4.2-0.el7ev.noarch ovirt-imageio-proxy-1.4.2-0.el7ev.noarch ovirt-imageio-proxy-setup-1.4.2-0.el7ev.noarch There isn't any other update for these packages: # yum update ovirt-imageio-proxy Loaded plugins: product-id, search-disabled-repos, subscription-manager, versionlock No packages marked for update Manager system is correctly subscribed to suitable channels: # subscription-manager list +-------------------------------------------+ Installed Product Status +-------------------------------------------+ Product Name: Red Hat Virtualization Manager Product ID: 415 Version: 4.2 Arch: x86_64 Status: Subscribed Status Details: Starts: 06/14/2018 Ends: 06/14/2019 Product Name: Red Hat Enterprise Linux Server Product ID: 69 Version: 7.5 Arch: x86_64 Status: Subscribed Status Details: Starts: 06/14/2018 Ends: 06/14/2019 From Red Hat download page -> Red Hat Virtualization -> Packages, I can see there isn't any other update for ovirt-imageio-proxy and latest version is 1.4.2 which we already have installed.
> So what are the 1.3.1.2 version packages mentioned in > https://bugzilla.redhat.com/show_bug.cgi?id=1616019#c12 ? Is it before the > update? We're on 1.4.2 version since we updated to RHV 4.2. So, I don't know where it says we had ovirt-imageio-proxy 1.3.1.2 version. I don't know where has been collected that info. Actual scenario is as I said before.
Could I perhaps connect to the engine environment in any way to debug the issue? Or should I prepare a jar with verbose logging to pinpoint the issue?
Both options are OK. On one side, we can do an online session for you can see for yourself the engine environment. On the other side, you can prepare a jar if you think is suitable for this issue.
(In reply to Daniel Erez from comment #21) > Could I perhaps connect to the engine environment in any way to debug the > issue? Or should I prepare a jar with verbose logging to pinpoint the issue? Daniel, This is weird. The info I have in the log-collector shows this : ovirt-imageio-common-1.3.1.2-0.el7ev.noarch Wed Jun 20 18:32:37 2018 ovirt-imageio-proxy-1.3.1.2-0.el7ev.noarch Wed Jun 20 18:36:22 2018 ovirt-imageio-proxy-setup-1.3.1.2-0.el7ev.noarch Wed Jun 20 18:36:20 2018 I did ask for debug logging to be enabled and the subsequent logs they sent did not show any info at all. It may be a good idea to get another log-collector from the system to verify. Will have them run the test again and obtain all the relevant logs.
(In reply to Anitha Udgiri from comment #23) > (In reply to Daniel Erez from comment #21) > > Could I perhaps connect to the engine environment in any way to debug the > > issue? Or should I prepare a jar with verbose logging to pinpoint the issue? > > Daniel, > This is weird. The info I have in the log-collector shows this : > > ovirt-imageio-common-1.3.1.2-0.el7ev.noarch Wed Jun 20 18:32:37 > 2018 > ovirt-imageio-proxy-1.3.1.2-0.el7ev.noarch Wed Jun 20 18:36:22 > 2018 > ovirt-imageio-proxy-setup-1.3.1.2-0.el7ev.noarch Wed Jun 20 18:36:20 > 2018 > > I did ask for debug logging to be enabled and the subsequent logs they sent > did not show any info at all. > > It may be a good idea to get another log-collector from the system to verify. > Will have them run the test again and obtain all the relevant logs. ok, just in case (as we might have an issue with logs output), can you please also add the output of 'journalctl -u ovirt-imageio-proxy' during the upload.
(In reply to Daniel Erez from comment #24) > (In reply to Anitha Udgiri from comment #23) > > (In reply to Daniel Erez from comment #21) > > > Could I perhaps connect to the engine environment in any way to debug the > > > issue? Or should I prepare a jar with verbose logging to pinpoint the issue? > > > > Daniel, > > This is weird. The info I have in the log-collector shows this : > > > > ovirt-imageio-common-1.3.1.2-0.el7ev.noarch Wed Jun 20 18:32:37 > > 2018 > > ovirt-imageio-proxy-1.3.1.2-0.el7ev.noarch Wed Jun 20 18:36:22 > > 2018 > > ovirt-imageio-proxy-setup-1.3.1.2-0.el7ev.noarch Wed Jun 20 18:36:20 > > 2018 > > > > I did ask for debug logging to be enabled and the subsequent logs they sent > > did not show any info at all. > > > > It may be a good idea to get another log-collector from the system to verify. > > Will have them run the test again and obtain all the relevant logs. > > ok, just in case (as we might have an issue with logs output), can you > please also add the output of 'journalctl -u ovirt-imageio-proxy' during the > upload. Daniel, The versions of inmageio on their rhevm system still show : ovirt-imageio-common-1.3.1.2-0.el7ev.noarch Wed Jun 20 18:32:37 2018 ovirt-imageio-proxy-1.3.1.2-0.el7ev.noarch Wed Jun 20 18:36:22 2018 ovirt-imageio-proxy-setup-1.3.1.2-0.el7ev.noarch Wed Jun 20 18:36:20 2018 Will first ensure that they have the versions GA'd with 4.2.4
(In reply to Anitha Udgiri from comment #27) > (In reply to Daniel Erez from comment #24) > > (In reply to Anitha Udgiri from comment #23) > > > (In reply to Daniel Erez from comment #21) > > > > Could I perhaps connect to the engine environment in any way to debug the > > > > issue? Or should I prepare a jar with verbose logging to pinpoint the issue? > > > > > > Daniel, > > > This is weird. The info I have in the log-collector shows this : > > > > > > ovirt-imageio-common-1.3.1.2-0.el7ev.noarch Wed Jun 20 18:32:37 > > > 2018 > > > ovirt-imageio-proxy-1.3.1.2-0.el7ev.noarch Wed Jun 20 18:36:22 > > > 2018 > > > ovirt-imageio-proxy-setup-1.3.1.2-0.el7ev.noarch Wed Jun 20 18:36:20 > > > 2018 > > > > > > I did ask for debug logging to be enabled and the subsequent logs they sent > > > did not show any info at all. > > > > > > It may be a good idea to get another log-collector from the system to verify. > > > Will have them run the test again and obtain all the relevant logs. > > > > ok, just in case (as we might have an issue with logs output), can you > > please also add the output of 'journalctl -u ovirt-imageio-proxy' during the > > upload. > > Daniel, > The versions of inmageio on their rhevm system still show : > > ovirt-imageio-common-1.3.1.2-0.el7ev.noarch Wed Jun 20 18:32:37 2018 > ovirt-imageio-proxy-1.3.1.2-0.el7ev.noarch Wed Jun 20 18:36:22 2018 > ovirt-imageio-proxy-setup-1.3.1.2-0.el7ev.noarch Wed Jun 20 18:36:20 2018 > > Will first ensure that they have the versions GA'd with 4.2.4 ok, so I'll wait for an updated log-collector. If possible, please update to engine 4.2.5, and imageio 1.4.3.
Created attachment 1483255 [details] SOS Report 2018-09-03 First Part
Created attachment 1483257 [details] SOS Report 2018-09-03 Second Part
Hi Daniel, I've attached sosreport from last 03 of September after restart ovirt imageio daemons and test upload ISO again. It's failing yet. After that, we've upgraded to latest RHV 4 minor version on manager and hosts. Mánager: ovirt-engine-4.2.6.4-0.1.el7ev.noarch ovirt-imageio-proxy-1.4.4-0.el7ev.noarch Hosts: vdsm-4.20.39-1.el7ev.x86_64 ovirt-imageio-daemon-1.4.4-0.el7ev.noarch With same result as before. Regards, M.A. Campos
This bug has not been marked as blocker for oVirt 4.3.0. Since we are releasing it tomorrow, January 29th, this bug has been re-targeted to 4.3.1.
Daniel, any news regarding this bug?
Hi Anitha, Does this issue still reproducing?
(In reply to Tal Nisan from comment #40) > Hi Anitha, > Does this issue still reproducing? Where ? and which version? The last we know is that the Customer faces this problem on 4.2.6 Are you asking if the Customer upgraded to 4.3 and is still facing the issue? Was the root cause found and issue fixed?
Yes, as in bug 1576500, this problem was fixed in 4.3.2
(In reply to Tal Nisan from comment #42) > Yes, as in bug 1576500, this problem was fixed in 4.3.2 Should this bug be closed as duplicate of the other bug you mention then?
sync2jira
*** This bug has been marked as a duplicate of bug 1576500 ***