Bug 1729511 - engine-setup fails to upgrade to 4.3 with Unicode characters in CA subject
Summary: engine-setup fails to upgrade to 4.3 with Unicode characters in CA subject
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: ovirt-engine
Version: 4.3.0
Hardware: x86_64
OS: Linux
high
high
Target Milestone: ovirt-4.4.0
: ---
Assignee: Yedidyah Bar David
QA Contact: Petr Matyáš
URL:
Whiteboard:
Depends On:
Blocks: 1733438
TreeView+ depends on / blocked
 
Reported: 2019-07-12 12:50 UTC by Juan Orti
Modified: 2020-08-04 13:20 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
During installation or upgrade to Red Had Virtualization 4.3, engine-setup failed if the PKI Organization Name in the CA certificate included non-ASCII characters. In this release, the upgrade engine-setup process completes successfully.
Clone Of:
: 1733438 (view as bug list)
Environment:
Last Closed: 2020-08-04 13:19:49 UTC
oVirt Team: Integration
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Knowledge Base (Solution) 4281201 0 None None None 2019-07-12 12:50:32 UTC
Red Hat Product Errata RHSA-2020:3247 0 None None None 2020-08-04 13:20:17 UTC
oVirt gerrit 102089 0 'None' MERGED packaging: setup: Write postinstall as binary data 2020-05-08 16:22:22 UTC

Description Juan Orti 2019-07-12 12:50:32 UTC
Description of problem:
While upgrading the engine from 4.1 to 4.3, engine-setup fails with:

2019-07-12 08:12:41,623+0100 DEBUG otopi.context context._executeMethod:145 method exception
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/otopi/context.py", line 132, in _executeMethod
    method['method']()
  File "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-common/base/core/postinstall.py", line 89, in _misc
    otopicons.CoreEnv.MODIFIED_FILES
  File "/usr/lib/python2.7/site-packages/otopi/filetransaction.py", line 151, in __init__
    self._content = self._content.encode("utf-8")
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 415: ordinal not in range(128)
2019-07-12 08:12:41,627+0100 ERROR otopi.context context._executeMethod:154 Failed to execute stage 'Misc configuration': 'ascii' codec can't decode byte 0xc3 in position 415: ordinal not in range(128)


The CA certificate has unicode characters in the organization field of the subject:

# openssl x509 -in /etc/pki/ovirt-engine/ca.pem -text -noout
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 4096 (0x1000)
    Signature Algorithm: sha256WithRSAEncryption
        Issuer: C=US, O=Inval\xC3\xADd, CN=rhvm.example.com.17156
        Validity
            Not Before: Apr 10 10:57:10 2018 GMT
            Not After : Apr  7 10:57:10 2028 GMT
        Subject: C=US, O=Inval\xC3\xADd, CN=rhvm.example.com.17156
[...]


Version-Release number of selected component (if applicable):
ovirt-engine-setup-base-4.3.4.3-0.1.el7.noarch

How reproducible:
Always

Steps to Reproduce:
1. Install 4.1 or older engine with a CA subject with unicode characters
2. Upgrade to 4.3

Actual results:
The upgrade process fail with the mentioned error. The upgrade to 4.2 has also been tested with similar errors.

Expected results:
The engine-setup process should identify this CA subject as invalid and do the renewal automatically.

Additional info:
A host has been upgraded to latest RHVH 4.3 and libvirtd failed to start with this error:

error : virNetTLSContextLoadCACertListFromFile:550 : Unable to import CA certificate list /etc/pki/vdsm/certs/cacert.pem

Comment 1 Yedidyah Bar David 2019-07-17 07:28:41 UTC
(In reply to Juan Orti Alcaine from comment #0)
> Steps to Reproduce:
> 1. Install 4.1 or older engine with a CA subject with unicode characters
> 2. Upgrade to 4.3
> 
> Actual results:
> The upgrade process fail with the mentioned error. The upgrade to 4.2 has
> also been tested with similar errors.
> 
> Expected results:
> The engine-setup process should identify this CA subject as invalid and do
> the renewal automatically.

Not sure what you mean here. If it's invalid, perhaps we should prevent upgrade?
Please explain what you want.

I'll try to reproduce and check logs later on, then might have better insights.
Worst case, we might have to recreate all PKI with new certs and a new organization
name.

Comment 2 Juan Orti 2019-07-17 08:18:39 UTC
For other cases of invalid certificates, engine-setup asks to renew them:

https://access.redhat.com/solutions/1572983

I was speaking with Miguel Martín, and in theory, the X520OrganizationName field can be a utf8String, so maybe it's a problem with the SSL library that should accept those certificates.
https://tools.ietf.org/html/rfc5280#appendix-A

So, if we refuse to accept UTF-8 strings, ideally all the PKI should be recreated and the hosts re-enrolled. If not, the necessary changes should be done to accept the certificates.

Comment 3 Yedidyah Bar David 2019-07-22 09:58:33 UTC
Now reproduced also on clean setup - exactly the same failure happens, during postinstall generation, when supplying a unicode organization.

Comment 4 Yedidyah Bar David 2019-07-22 10:00:19 UTC
And the failure is in python code, unrelated to openssl or anything like that. I wonder how it was created so originally...

Comment 5 Yedidyah Bar David 2019-07-22 10:27:54 UTC
OK, found the reason. It was broken only on otopi-1.8 (oVirt/RHV 4.3), due to this patch:

https://gerrit.ovirt.org/92435

I think the patch is correct in principle, and will probably need to patch the code generating postinstall to pass binary data.

Comment 6 Juan Orti 2019-07-22 13:23:49 UTC
Should I open a different bug for libvirtd not starting with that CA cert?

Comment 7 Yedidyah Bar David 2019-07-23 05:18:46 UTC
(In reply to Juan Orti Alcaine from comment #6)
> Should I open a different bug for libvirtd not starting with that CA cert?

Probably, but I am still not sure what the exact problem was. The specific error in comment 0 is unrelated to ssl or any 3rd-party library - it's all due to oVirt code. Once we fix that, we can try to reproduce the libvirt error. Or, you can try to reproduce with an 4.2 engine (I didn't try). If you do reproduce, and indeed upgrading a host (where the engine is still 4.2) fails as noted, please open a bug. Thanks.

Comment 10 Eli Marcus 2019-10-02 12:38:27 UTC
Hi Didi - here is the updated Doc Text, please review: 

During upgrade to Red Had Virtualization 4.3, engine-setup failed if the PKI Organization Name in the CA certificate included non-ASCII characters.
In this release, the upgrade engine-setup process completes successfully.

Comment 11 Yedidyah Bar David 2019-10-03 05:15:38 UTC
(In reply to Eli Marcus from comment #10)
> Hi Didi - here is the updated Doc Text, please review: 
> 
> During upgrade to Red Had Virtualization 4.3, engine-setup failed if the PKI
> Organization Name in the CA certificate included non-ASCII characters.
> In this release, the upgrade engine-setup process completes successfully.

AFAIR it wasn't specific to upgrade - I think I reproduced this failure also on new setups. Otherwise, it's ok.

Comment 12 Eli Marcus 2019-10-03 12:23:08 UTC
(In reply to Yedidyah Bar David from comment #11)
> (In reply to Eli Marcus from comment #10)
> > Hi Didi - here is the updated Doc Text, please review: 
> > 
> > During upgrade to Red Had Virtualization 4.3, engine-setup failed if the PKI
> > Organization Name in the CA certificate included non-ASCII characters.
> > In this release, the upgrade engine-setup process completes successfully.
> 
> AFAIR it wasn't specific to upgrade - I think I reproduced this failure also
> on new setups. Otherwise, it's ok.

OK, I modified the text: 

"During installation or upgrade to Red Had Virtualization 4.3, engine-setup failed if the PKI Organization Name in the CA certificate included non-ASCII characters.
In this release, the upgrade engine-setup process completes successfully."

Comment 13 Petr Matyáš 2019-11-27 11:22:00 UTC
Verified on ovirt-engine-4.4.0-0.5.master.el7.noarch

Comment 15 RHV bug bot 2019-12-13 13:16:17 UTC
WARN: Bug status (VERIFIED) wasn't changed but the folowing should be fixed:

[Found non-acked flags: '{}', ]

For more info please contact: rhv-devops: Bug status (VERIFIED) wasn't changed but the folowing should be fixed:

[Found non-acked flags: '{}', ]

For more info please contact: rhv-devops

Comment 16 RHV bug bot 2019-12-20 17:45:45 UTC
WARN: Bug status (VERIFIED) wasn't changed but the folowing should be fixed:

[Found non-acked flags: '{}', ]

For more info please contact: rhv-devops: Bug status (VERIFIED) wasn't changed but the folowing should be fixed:

[Found non-acked flags: '{}', ]

For more info please contact: rhv-devops

Comment 17 RHV bug bot 2020-01-08 14:49:38 UTC
WARN: Bug status (VERIFIED) wasn't changed but the folowing should be fixed:

[Found non-acked flags: '{}', ]

For more info please contact: rhv-devops: Bug status (VERIFIED) wasn't changed but the folowing should be fixed:

[Found non-acked flags: '{}', ]

For more info please contact: rhv-devops

Comment 18 RHV bug bot 2020-01-08 15:17:56 UTC
WARN: Bug status (VERIFIED) wasn't changed but the folowing should be fixed:

[Found non-acked flags: '{}', ]

For more info please contact: rhv-devops: Bug status (VERIFIED) wasn't changed but the folowing should be fixed:

[Found non-acked flags: '{}', ]

For more info please contact: rhv-devops

Comment 19 RHV bug bot 2020-01-24 19:51:28 UTC
WARN: Bug status (VERIFIED) wasn't changed but the folowing should be fixed:

[Found non-acked flags: '{}', ]

For more info please contact: rhv-devops: Bug status (VERIFIED) wasn't changed but the folowing should be fixed:

[Found non-acked flags: '{}', ]

For more info please contact: rhv-devops

Comment 21 errata-xmlrpc 2020-08-04 13:19:49 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: RHV Manager (ovirt-engine) 4.4 security, bug fix, and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2020:3247


Note You need to log in before you can comment on or make changes to this bug.