Bug 1614419 - libreoffice-calc crashes in FIPS mode when handling password protected documents
Summary: libreoffice-calc crashes in FIPS mode when handling password protected documents
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: libreoffice
Version: 7.5
Hardware: Unspecified
OS: Linux
Target Milestone: rc
: ---
Assignee: Caolan McNamara
QA Contact: Desktop QE
Depends On:
TreeView+ depends on / blocked
Reported: 2018-08-09 14:36 UTC by Joe Wright
Modified: 2018-10-30 07:57 UTC (History)
3 users (show)

Fixed In Version: libreoffice-
Doc Type: Bug Fix
Doc Text:
Previously, when LibreOffice was running in an environment with FIPS kernel mode activated, LibreOffice terminated unexpectedly on encrypting Office Open XML documents. A patch has been applied to load a symmetric key that works when FIPS kernel mode is activated, and the described problem no longer occurs.
Clone Of:
Last Closed: 2018-10-30 07:57:29 UTC
Target Upstream Version:

Attachments (Terms of Use)
backtrace (1.04 KB, application/x-gzip)
2018-08-09 14:38 UTC, Joe Wright
no flags Details

System ID Priority Status Summary Last Updated
Red Hat Bugzilla 1461450 None CLOSED Corosync hangs on secauth with FIPS enabled 2019-09-12 16:47:40 UTC
Red Hat Knowledge Base (Solution) 3558731 None None None 2018-08-09 14:50:35 UTC
Red Hat Product Errata RHSA-2018:3054 None None None 2018-10-30 07:57:33 UTC

Internal Links: 1461450

Description Joe Wright 2018-08-09 14:36:50 UTC
Description of problem:

Version-Release number of selected component (if applicable):
- libreoffice-calc-
- nss-3.36.0-5.el7_5
- FIPS mode
- kernel 3.10.0-862.9.1

How reproducible:

Steps to Reproduce:
1. Boot system in FIPS mode
2. Open a new spreadhsheet in Libreoffice-calc
3. Add some random data to a few cells
4. Save as xlsx or ods format, and password protect the document

Actual results:
- crashes in nss code

Expected results:

Additional info:
- reproduced on server and worksation

Comment 2 Joe Wright 2018-08-09 14:38:02 UTC
Created attachment 1474720 [details]

Backtrace from while attempting to save password protected document

Comment 3 Caolan McNamara 2018-08-09 14:51:50 UTC
I guess this might be solved with https://cgit.freedesktop.org/libreoffice/core/commit/?id=0498b983cc62bc37dacd246ed6480563ede470b1

Comment 5 Caolan McNamara 2018-08-09 15:02:28 UTC
regression vs what versions of what component, when did it last work ?

Comment 6 Caolan McNamara 2018-08-09 16:01:38 UTC
In RHEL-7 FIPS mode I can reproduce the crash on saving to xlsx with a password set, and the commit referenced above does turn that from a crash to a warning dialog about the inability to use nss to encrypt the document.

I don't see a save to ods problem however.

Comment 7 Joe Wright 2018-08-09 19:58:48 UTC
The customer is not sure when this changed. The user in this case just migrated, but this should have been working.

Comment 8 Caolan McNamara 2018-08-10 07:35:18 UTC
for the xlxs case the problem is that PK11_ImportSymKey fails and returns null and that's unexpected so libreoffice goes on to crash. I can add the fix that detects PK11_ImportSymKey failure and go on to report inability to save rather than crash, which fixes the crash. Not sure that actually gains the customer a whole pile though, it won't crash, but it won't work either.

Comment 9 Caolan McNamara 2018-08-10 08:46:37 UTC
bug 1461450 has a similar problem and the workaround there works for us too to give functional encryption without a crash, so I could do that.

Comment 10 Caolan McNamara 2018-08-10 09:18:46 UTC
upstreaming that as https://gerrit.libreoffice.org/#/c/58816/ and committed a backport of that to our package, so xlsx encryption now works under FIPS without a crash

Comment 12 Martin Krajnak 2018-08-13 13:04:47 UTC
Works fine with:


Comment 14 errata-xmlrpc 2018-10-30 07:57:29 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.