Bug 1687301

Summary: For verifying ovirt-imageio-proxy cert, engine uses internal truststore instead of https one
Product: [oVirt] ovirt-engine Reporter: Yedidyah Bar David <didi>
Component: BLL.StorageAssignee: Yedidyah Bar David <didi>
Status: CLOSED CURRENTRELEASE QA Contact: Yosi Ben Shimon <ybenshim>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 4.3.1CC: aefrat, bugs, damion.brown, emarcus, rmcswain, tnisan
Target Milestone: ovirt-4.3.2Flags: emarcus: needinfo-
pm-rhel: ovirt-4.3+
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: ovirt-engine-4.3.2.1 Doc Type: Bug Fix
Doc Text:
In this release, the Red Hat Enterprise Virtualization Manager verifies third party Certificate Authority certificates by checking the public https trust store, instead of the internal one, when verifying an ovirt-imageio-proxy certificate.
 Story Points: ---
Clone Of: Environment:
Last Closed: 2019-04-16 13:56:27 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Storage RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
Login CA error none

Description Yedidyah Bar David 2019-03-11 08:50:19 UTC 
Description of problem:

$summary.

This isn't a problem when using the (default) internal CA for apache httpd, but when using a 3rd-party CA, we needlessly require adding the 3rd-party CA cert to the internal truststore.

The imageio-proxy is a web service facing external users, and it should be easy to make it use a 3rd-party CA.

The engine is a client of it, so it should use the external/https truststore for verifying its certificate.

Version-Release number of selected component (if applicable):

Current master, probably forever (since imageio was introduced)

How reproducible:

Always

Steps to Reproduce:
1. Setup an engine, host, storage
2. Change the engine to use a 3rd-party CA for apache httpd [1][2]
3. Change imageio-proxy to use apache's key/cert (bug 1637809)'
4. Import the 3rd-party CA cert to your browser
5. Try "Test Connection" in the dialog to upload an image
6. Upload an image

[1] https://www.ovirt.org/documentation/admin-guide/appe-oVirt_and_SSL.html
[2] https://access.redhat.com/documentation/en-us/red_hat_virtualization/4.2/html/administration_guide/appe-red_hat_enterprise_virtualization_and_ssl#Replacing_the_Manager_CA_Certificate

Actual results:

Test Connection works (step 5), but upload fails (step 6).

Expected results:

Both should work

Additional info:

This appears in engine.log when failing to upload:


2019-03-07 06:10:51,587-05 ERROR
[org.ovirt.engine.core.bll.storage.disk.image.TransferDiskImageCommand]
(EE-ManagedThreadFactory-engineScheduled-Thread-50)
[393beec4-3fb0-4006-a8f9-49005d3fc8b4] Failed to add image ticket to
ovirt-imageio-proxy: javax.net.ssl.SSLHandshakeException:
sun.security.validator.ValidatorException: PKIX path building failed:
sun.security.provider.certpath.SunCertPathBuilderException: unable to
find valid certification path to requested target
        at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
[jsse.jar:1.8.0_191]
        at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1946)
[jsse.jar:1.8.0_191]
        at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:316)
[jsse.jar:1.8.0_191]
        at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:310)
[jsse.jar:1.8.0_191]
        at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1639)
[jsse.jar:1.8.0_191]
        at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:223)
[jsse.jar:1.8.0_191]
        at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1037)
[jsse.jar:1.8.0_191]
        at sun.security.ssl.Handshaker.process_record(Handshaker.java:965)
[jsse.jar:1.8.0_191]
        at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1064)
[jsse.jar:1.8.0_191]
        at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1367)
[jsse.jar:1.8.0_191]
        at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1395)
[jsse.jar:1.8.0_191]
        at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1379)
[jsse.jar:1.8.0_191]
        at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:559)
[rt.jar:1.8.0_191]
        at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185)
[rt.jar:1.8.0_191]
        at sun.net.www.protocol.http.HttpURLConnection.getOutputStream0(HttpURLConnection.java:1334)
[rt.jar:1.8.0_191]
        at sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:1309)
[rt.jar:1.8.0_191]
        at sun.net.www.protocol.https.HttpsURLConnectionImpl.getOutputStream(HttpsURLConnectionImpl.java:259)
[rt.jar:1.8.0_191]
        at org.ovirt.engine.core.bll.storage.disk.image.TransferDiskImageCommand.addImageTicketToProxy(TransferDiskImageCommand.java:837)
[bll.jar:]
        at org.ovirt.engine.core.bll.storage.disk.image.TransferDiskImageCommand.startImageTransferSession(TransferDiskImageCommand.java:763)
[bll.jar:]
        at org.ovirt.engine.core.bll.storage.disk.image.TransferDiskImageCommand.handleImageIsReadyForTransfer(TransferDiskImageCommand.java:452)
[bll.jar:]
        at org.ovirt.engine.core.bll.storage.disk.image.TransferDiskImageCommand.handleInitializing(TransferDiskImageCommand.java:423)
[bll.jar:]
        at org.ovirt.engine.core.bll.storage.disk.image.TransferDiskImageCommand.executeStateHandler(TransferDiskImageCommand.java:358)
[bll.jar:]
        at org.ovirt.engine.core.bll.storage.disk.image.TransferDiskImageCommand.proceedCommandExecution(TransferDiskImageCommand.java:345)
[bll.jar:]
        at org.ovirt.engine.core.bll.storage.disk.image.TransferImageCommandCallback.doPolling(TransferImageCommandCallback.java:21)
[bll.jar:]
        at org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller.invokeCallbackMethodsImpl(CommandCallbacksPoller.java:175)
[bll.jar:]
        at org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller.invokeCallbackMethods(CommandCallbacksPoller.java:109)
[bll.jar:]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[rt.jar:1.8.0_191]
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
[rt.jar:1.8.0_191]
        at org.glassfish.enterprise.concurrent.internal.ManagedScheduledThreadPoolExecutor$ManagedScheduledFutureTask.access$201(ManagedScheduledThreadPoolExecutor.java:383)
[javax.enterprise.concurrent-1.0.jar:]
        at org.glassfish.enterprise.concurrent.internal.ManagedScheduledThreadPoolExecutor$ManagedScheduledFutureTask.run(ManagedScheduledThreadPoolExecutor.java:534)
[javax.enterprise.concurrent-1.0.jar:]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[rt.jar:1.8.0_191]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[rt.jar:1.8.0_191]
        at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_191]
        at org.glassfish.enterprise.concurrent.ManagedThreadFactoryImpl$ManagedThread.run(ManagedThreadFactoryImpl.java:250)
[javax.enterprise.concurrent-1.0.jar:]
        at org.jboss.as.ee.concurrent.service.ElytronManagedThreadFactory$ElytronManagedThread.run(ElytronManagedThreadFactory.java:78)
Caused by: sun.security.validator.ValidatorException: PKIX path
building failed:
sun.security.provider.certpath.SunCertPathBuilderException: unable to
find valid certification path to requested target
        at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:397)
[rt.jar:1.8.0_191]
        at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:302)
[rt.jar:1.8.0_191]
        at sun.security.validator.Validator.validate(Validator.java:262)
[rt.jar:1.8.0_191]
        at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:324)
[jsse.jar:1.8.0_191]
        at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:229)
[jsse.jar:1.8.0_191]
        at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:124)
[jsse.jar:1.8.0_191]
        at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1621)
[jsse.jar:1.8.0_191]
        ... 30 more
Caused by: sun.security.provider.certpath.SunCertPathBuilderException:
unable to find valid certification path to requested target
        at sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141)
[rt.jar:1.8.0_191]
        at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126)
[rt.jar:1.8.0_191]
        at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:280)
[rt.jar:1.8.0_191]
        at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:392)
[rt.jar:1.8.0_191]
        ... 36 more
Comment 4 Yosi Ben Shimon 2019-04-08 10:14:34 UTC 
Created attachment 1553528 [details]
Login CA error
Comment 6 Yosi Ben Shimon 2019-04-11 09:11:21 UTC 
Tested according to the steps above using:
ovirt-engine-4.3.3.2-0.1.el7.noarch
ovirt-imageio-proxy-1.5.1-0.el7ev.noarch

Actual result:
Both engine and imageio-proxy used the 3rd-party certificate.
Upload image succeeded without any exceptions or errors.

Moving to VERIFIED
Comment 9 Sandro Bonazzola 2019-04-16 13:56:27 UTC 
This bugzilla is included in oVirt 4.3.2 release, published on March 19th 2019.

Since the problem described in this bug report should be
resolved in oVirt 4.3.2 release, it has been closed with a resolution of CURRENT RELEASE.

If the solution does not work for you, please open a new bug report.
Comment 10 Robert McSwain 2020-04-13 19:24:36 UTC 
I've got a customer hitting this in RHV 4.2.8. Other than upgrading to 4.3 (which they are unable to do at this time) is there any workaround for this on lower versions of RHV?
Comment 11 Red Hat Bugzilla 2023-09-14 05:25:09 UTC 
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 1000 days