Bug 1472149 - Non Unicode CA present in trusted CA bundle causing issue with Satellite [NEEDINFO]
Non Unicode CA present in trusted CA bundle causing issue with Satellite
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: ca-certificates (Show other bugs)
x86_64 Linux
medium Severity medium
: rc
: ---
Assigned To: Kai Engert (:kaie)
BaseOS QE Security Team
Depends On:
Blocks: 1420851
  Show dependency treegraph
Reported: 2017-07-18 03:40 EDT by Chris Roberts
Modified: 2017-08-14 15:00 EDT (History)
7 users (show)

See Also:
Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2017-08-14 15:00:23 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
kengert: needinfo? (chrobert)
rjh.mokkink: needinfo? (chrobert)

Attachments (Terms of Use)

  None (edit)
Description Chris Roberts 2017-07-18 03:40:53 EDT
Description of problem:
Recently we found through a customer case that having non unicode characters in the trusted CA bundle provided by the ca-certificates package is causing issues with Pulp and Satellite. 

Version-Release number of selected component (if applicable): 


How reproducible:

Steps to Reproduce:
0. Create custom ssl certs for a Satellite and a Capsule
1. Add your CA to the Red Hat Trusted CA bundle
2. Install Red Hat Satellite 6.2 latest with the csr/cert/key/CA trusted bundle.
3. Install a Capsule with the cert/csr/key/CA trusted bundle.
4. Try to sync the Capsule and watch the Python traceback about non unicode characters.

Actual results:
Sync to go through

Expected results:
Lots of errors from Pulp not being able to read them. There is a work-around currently but this could affect quite a few customers and would be bad for the user experience.

Additional info:

We have fixed this on our side with a check added to the katello-certs-check which checks the SSL certs and gives the output to the user how to install them with Satellite, if it detects that there is non unicode characters it will not fail out.

Feel free to needinfo on me if you have any questions about installing Satellite/Capsule or anything else related to this bug. We are requesting to remove the 5 non unicode CA's from the bundle.
Comment 1 Kai Engert (:kaie) 2017-07-19 08:36:27 EDT
From the additional information I have seen, the issue seems to be related to *comment* lines that are contained in CA bundle files.

With a recent update to the ca-certificates package, the PEM bundle files we create in the subdirectories of /etc/pki/ca-trust/extracted/ include comment lines, that show the name of the following certificate data block.

Those comment lines appear to use the UTF-8 character encoding. The UTF-8 encoding is designed to be compatible with tools that are able to process plain text files, because regular ASCII characters are still encoded as ASCII, and only international characters are encoded using multiple bytes, in a way that is designed not to break plain text files processing.

I can see that the original list of CAs that we ship with RHEL already contain several lines with international characters. When using standard command line tools to dump those files, the characters appear to be printed as expected:

cat /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem |grep -i arany
# NetLock Arany (Class Gold) Főtanúsítvány

Any tool that processes PEM files should ignore lines outside the BEGIN CERTIFICATE and END CERTIFICATE blocks.

The tool that is involved here (I assume from the satellite software side) should ignore such lines, which are outside the BEGIN/END blocks, and which are also prefixed with the # character that identify them as comments.

In a text snippet I have been shown, the above line was shown as
# NetLock Arany (Class Gold) Főtanúsítvány

Apparently a tool tried to analyze those comment lines, and interpreted them incorrectly.

Having said that, this bug report says, in order to see the problem, it's necessary to use some custom certificates. This surprises me.

If the tools that process CA bundles are unable to work with these comments containing international characters, then why is the problem only occurring when using custom certificates? I would have expected that tool to fail in general, even with the default CA bundle that we ship, which already contains international characters in comments.

Maybe you could explain how you are processing / installing the custom certificates?

I can think of a potential workaround, which can be used to avoid creating comment lines in the CA bundle files. But whether or not that will help you depends on how you are using and processing the bundle files, which I don't understand yet. Additional descriptions of your processing would be helpful.

Under the assumption that you are processing files from somewhere below the /etc/pki/ca-trust/extracted directory, you could use the following workaround to remove all comment lines (with and without international characters) from those files:

You could manually modify the /usr/bin/update-ca-trust script which is contained in the ca-certificates package, by opening it in a text editor, and you could remove all occurrences of the
Afterwards you would have to run the update-ca-trust command, which will then re-generate all files in the /etc/pki/ca-trust/extracted directory.

The files should then no longer contain comment lines, but only certificate blocks.
Comment 2 Rob Mokkink 2017-07-21 05:18:08 EDT
We encounter these problems on our satellite 6 environment, we have the following workaround, see bugzilla https://bugzilla.redhat.com/show_bug.cgi?id=1449418, comment 9.

We put our public rootca's certs in /etc/pki/ca-trust/source/anchors and then run update-ca-trust.

So it looks like the package "ca-certificates" is correct, but them problem lies in the satellite 6 code.
Comment 3 Kai Engert (:kaie) 2017-07-21 07:56:47 EDT
Should this bug be closed, or do you request a change for the ca-certificates package?
Comment 4 Rob Mokkink 2017-07-21 08:07:50 EDT
(In reply to Kai Engert (:kaie) from comment #3)
> Should this bug be closed, or do you request a change for the
> ca-certificates package?

I don't think i am the one who should decide if it is oke to close this BZ, because we are just a customer who use satellite 6 and have found a workaround for the problem we faced.
Comment 5 Kai Engert (:kaie) 2017-07-21 09:13:45 EDT
Thanks, I confirm my question is for the satellite developers.

Note You need to log in before you can comment on or make changes to this bug.