Bug 1448649

Summary: Avoid the use of boringssl in chromium
Product: [Fedora] Fedora Reporter: Ricardo Martin <rickyepoderi>
Component: chromiumAssignee: Tom "spot" Callaway <tcallawa>
Status: CLOSED CANTFIX QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 25CC: sgallagh, tcallawa, yaneti
Target Milestone: ---Keywords: Reopened
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-11-09 16:04:09 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Ricardo Martin 2017-05-06 17:28:54 UTC
Description of problem:

The chromium package in fedora includes boringssl as an external
library. This makes that OpenSC package (a lot of drivers uses openssl
internally) when integrated in the browser also uses boringssl. This
fact is causing problems (at least with the DNIe Spanish driver). The
library is placed here:

/usr/lib64/chromium-browser/libboringssl.so

It generates a core when a DNIe card is used inside OpenSC (the driver uses X509_verify):

#0  0x00007f794760dda9 in BN_num_bits () at
/usr/lib64/chromium-browser/./libboringssl.so
#1  0x00007f794760dde9 in BN_num_bytes () at
/usr/lib64/chromium-browser/./libboringssl.so
#2  0x00007f794765480d in rsa_default_size () at
/usr/lib64/chromium-browser/./libboringssl.so
#3  0x00007f7947652605 in RSA_size () at
/usr/lib64/chromium-browser/./libboringssl.so
#4  0x00007f79476401cf in pkey_rsa_verify () at
/usr/lib64/chromium-browser/./libboringssl.so
#5  0x00007f794763d012 in EVP_DigestVerifyFinal () at
/usr/lib64/chromium-browser/./libboringssl.so
#6  0x00007f7947659d45 in ASN1_item_verify () at
/usr/lib64/chromium-browser/./libboringssl.so
#7  0x00007f78f670b02b in cwa_verify_icc_certificates
(icc_cert=0x1b5a5b4e40f0, sub_ca_cert=0x1b5a5b605a50, provider=
    0x1b5a5a3cac40, card=0x1b5a5a213000) at cwa14890.c:354
#8  0x00007f78f670b02b in cwa_create_secure_channel
(card=card@entry=0x1b5a5a213000, provider=0x1b5a5a3cac40, flag=flag@entry=1)
    at cwa14890.c:1114
#9  0x00007f78f6709bb3 in dnie_pin_verify
(card=card@entry=0x1b5a5a213000, data=data@entry=0x7f791560b2f0,
tries_left=tries_left@entry=0x1b5a5a8368e4) at card-dnie.c:2181

For example debian decided to link that library statically into the
chromium binary. This way OpenSC works without problem. See this email:

https://lists.debian.org/debian-devel/2016/02/msg00185.html


Version-Release number of selected component (if applicable):

rpm -qa | grep chromium
chromium-libs-57.0.2987.133-1.fc25.x86_64
chromium-libs-media-57.0.2987.133-1.fc25.x86_64
chromium-57.0.2987.133-1.fc25.x86_64


How reproducible:

Always (coredump).


Steps to Reproduce:
1. You need a DNIe card.
2. Add the opensc pkcs#11 library to the .pki/nssdb directory.
3. Just go to settings and view cetificates (after the password is shown
it crashes).


Actual results:

It core dumps because using boringssl instead of openssl.


Expected results:

Working ok (display the certificates in the card and so on).


Additional info:

None

Comment 1 Tom "spot" Callaway 2017-05-08 15:35:28 UTC
I cannot reproduce this because I do not have a DNIe card. However, I do not think it is feasible to:

* Replace boringssl with OpenSSL
* Link _just_ boringssl statically into Chromium
* Link _every_ component statically into Chromium

Additionally, there is nothing in the Debian thread which talks about this issue at all, I do not think it is related.

I'm working on a major release update for Chromium, perhaps that will fix this bug for you.

Comment 2 Ricardo Martin 2017-05-08 16:02:09 UTC
Hi,

Yes, I know you weren't going to reproduce it (you need a card). I'm going to try to explain in a different way.

The boringssl has the same method X509_verify than the openssl library:

nm /usr/lib64/chromium-browser/libboringssl.so | grep X509_verify
00000000000afaa0 T X509_verify
00000000000aaf80 T X509_verify_cert
00000000000a9b30 T X509_verify_cert_error_string

So, when the opensc tries to use the same method X509_verify, the one in libboringssl is used and not the one in openssl (libcrypto.so).

ldd /usr/lib64/libopensc.so.4
	linux-vdso.so.1 (0x00007fff3f7dc000)
	libcrypto.so.10 => /lib64/libcrypto.so.10 (0x00007f9515d5f000)
	...


In debian the libboringssl.so doesn't exist, it is just inside the binary and not exposed to the rest (this nm is done over the symbols in the package chromium-dbg_57.0.2987.98-1~deb8u1_amd64.deb in debian):

nm 06e3acde7124fca936e2a5b2fed8196cc5fb46.debug | grep X509_verify
00000000027cca20 t X509_verify
00000000027c8c30 t X509_verify_cert

So, the difference is that in debian is t (local) and in fedora is T (external). And the opensc uses the one in libboring.so instead the one in libcrypto.so. I don't know if it was on purpose in debian, but in debian you finally use the openssl library.

Thanks!

Comment 3 Tom "spot" Callaway 2017-05-08 19:35:04 UTC
Okay. I understand what you're saying here. What I _do not_ understand is why libopensc.so.4 is resolving X509_verify with /usr/lib64/chromium-browser/libboringssl.so.

On Fedora 26:

[spot@localhost sandbox]$ readelf -d /usr/lib64/libopensc.so.4

Dynamic section at offset 0x1a0ea0 contains 28 entries:
  Tag        Type                         Name/Value
 0x0000000000000001 (NEEDED)             Shared library: [libcrypto.so.1.1]
 0x0000000000000001 (NEEDED)             Shared library: [libz.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libdl.so.2]
 0x0000000000000001 (NEEDED)             Shared library: [libpthread.so.0]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
 0x000000000000000e (SONAME)             Library soname: [libopensc.so.4]

[spot@localhost sandbox]$ ldd /usr/lib64/libopensc.so.4
	linux-vdso.so.1 (0x00007ffd0adb5000)
	libcrypto.so.1.1 => /lib64/libcrypto.so.1.1 (0x00007fa4c4e80000)
	libz.so.1 => /lib64/libz.so.1 (0x00007fa4c4c69000)
	libdl.so.2 => /lib64/libdl.so.2 (0x00007fa4c4a65000)
	libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fa4c4847000)
	libc.so.6 => /lib64/libc.so.6 (0x00007fa4c4476000)
	/lib64/ld-linux-x86-64.so.2 (0x00005577e416f000)

My best guess is that something is happening inside Chromium when OpenSC is used inside the browser environment which is resulting in the symbol confusion.

The problem is that Fedora can't just compile boringssl inside Chromium as static (I tried, it fails), nor can we compile everything static (because that breaks chrome-remote-desktop).

The only other option I can think of would be to rename the functions in boringssl (and all the places they are called in Chromium) to avoid the overlap.

I think you should open this bug with the Chromium upstream and see if they have any advice on how they'd prefer to resolve this issue.

Comment 4 Ricardo Martin 2017-05-09 13:43:58 UTC
Hi,

Now I understand your point of making libboringssl.so dynamic, it is used by other packages.

I think the reason to fail under chromium is just because the libboringssl.so is already loaded (by chromium) and the symbol is there and external (T). It's the same as using LD_PRELOAD, it's already there. I can indeed emulate the problem using LD_PRELOAD using normal opensc tools:

$ pkcs11-tool -I -l
Cryptoki version 2.20
Manufacturer     OpenSC Project
Library          OpenSC smartcard framework (ver 0.16)
Using slot 0 with a present token (0x0)
Logging in to "PIN1 (DNI electrónico)".
Please enter User PIN: 
$ LD_PRELOAD=/usr/lib64/chromium-browser/libboringssl.so pkcs11-tool -I -l
Cryptoki version 2.20
Manufacturer     OpenSC Project
Library          OpenSC smartcard framework (ver 0.16)
Using slot 0 with a present token (0x0)
Logging in to "PIN1 (DNI electrónico)".
Please enter User PIN: 
Segmentation fault (core dumped)

The funny thing is that with chrome everything works, because google distributes an enormous binary of 118MB with everything linked statically except the basics. So official chrome works fine, it's the same case as debian.

I suppose that I'll need to report a bug against boringssl, but for that I need some files that are inside the card to do a test-case (coding to extract the certificates that make it crash). You know, a real pain in the ass. And then I'm not sure they are going to even care about it.

Thanks anyway!

Comment 5 Ricardo Martin 2017-05-10 19:16:26 UTC
bug filed against boringssl:

https://bugs.chromium.org/p/boringssl/issues/detail?id=194

Comment 6 Tom "spot" Callaway 2017-05-15 15:56:25 UTC
I see they didn't like that very much. If you want to try to namespace out the conflict in boringssl, let me know. I'm happy to carry that patch.

Comment 7 Ricardo Martin 2017-05-16 07:57:07 UTC
I think my issue is done on purpose in boringssl, it's like they are improving the performance, the manageability or something in openssl at the cost of not covering all the features. So opensc is just a collateral damage. You can see what they said, they recommend to link boringssl statically and not expose it to the rest of the system for end-user installations (because of the previous fact). I don't know about the method renaming solution, it seems that chromium devs considered it at the beginning but they finally gave up. I still think the easiest solution here is compiling boringssl statically inside chromium but it's up to you guys.

Thanks!

Comment 8 Fedora End Of Life 2017-11-16 18:36:03 UTC
This message is a reminder that Fedora 25 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 25. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as EOL if it remains open with a Fedora  'version'
of '25'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version'
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not
able to fix it before Fedora 25 is end of life. If you would still like
to see this bug fixed and are able to reproduce it against a later version
of Fedora, you are encouraged  change the 'version' to a later Fedora
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's
lifetime, sometimes those efforts are overtaken by events. Often a
more recent Fedora release includes newer upstream software that fixes
bugs or makes them obsolete.

Comment 9 Fedora End Of Life 2017-12-12 10:02:53 UTC
Fedora 25 changed to end-of-life (EOL) status on 2017-12-12. Fedora 25 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.

Comment 10 Stephen Gallagher 2018-11-09 15:40:32 UTC
Reopening, as this remains an issue in Fedora and we now can reproduce it trivially simply with GSSAPI calls (so no special hardware required).

See https://bugzilla.redhat.com/show_bug.cgi?id=1640158

Comment 11 Tom "spot" Callaway 2018-11-09 16:04:09 UTC
See previous comments. We use is_component_build=true to enable the many (many) libaries inside chromium to be built as shared objects. Fedora can't just compile boringssl inside Chromium as static (I tried, it fails), nor can we compile everything static (because that breaks chrome-remote-desktop and makes it impossible to swap in rpmfusion version to enable codecs).

Upstream does not care about these issues, and just wants us to build everything statically.

Hence, CLOSED->CANTFIX.