Bug 1614419

Summary: libreoffice-calc crashes in FIPS mode when handling password protected documents
Product: Red Hat Enterprise Linux 7 Reporter: Joe Wright <jwright>
Component: libreofficeAssignee: Caolan McNamara <caolanm>
Status: CLOSED ERRATA QA Contact: Desktop QE <desktop-qa-list>
Severity: high Docs Contact:
Priority: unspecified    
Version: 7.5CC: alanm, mkrajnak, tpelka
Target Milestone: rcKeywords: Regression
Target Release: ---   
Hardware: Unspecified   
OS: Linux   
Whiteboard:
Fixed In Version: libreoffice-5.3.6.1-19.el7 Doc Type: Bug Fix
Doc Text:
Previously, when LibreOffice was running in an environment with FIPS kernel mode activated, LibreOffice terminated unexpectedly on encrypting Office Open XML documents. A patch has been applied to load a symmetric key that works when FIPS kernel mode is activated, and the described problem no longer occurs.
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-10-30 07:57:29 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
backtrace none

Description Joe Wright 2018-08-09 14:36:50 UTC
Description of problem:


Version-Release number of selected component (if applicable):
- libreoffice-calc-5.3.6.1-10
- nss-3.36.0-5.el7_5
- FIPS mode
- kernel 3.10.0-862.9.1

How reproducible:


Steps to Reproduce:
1. Boot system in FIPS mode
2. Open a new spreadhsheet in Libreoffice-calc
3. Add some random data to a few cells
4. Save as xlsx or ods format, and password protect the document

Actual results:
- crashes in nss code

Expected results:


Additional info:
- reproduced on server and worksation

Comment 2 Joe Wright 2018-08-09 14:38:02 UTC
Created attachment 1474720 [details]
backtrace

Backtrace from while attempting to save password protected document

Comment 3 Caolan McNamara 2018-08-09 14:51:50 UTC
I guess this might be solved with https://cgit.freedesktop.org/libreoffice/core/commit/?id=0498b983cc62bc37dacd246ed6480563ede470b1

Comment 5 Caolan McNamara 2018-08-09 15:02:28 UTC
regression vs what versions of what component, when did it last work ?

Comment 6 Caolan McNamara 2018-08-09 16:01:38 UTC
In RHEL-7 FIPS mode I can reproduce the crash on saving to xlsx with a password set, and the commit referenced above does turn that from a crash to a warning dialog about the inability to use nss to encrypt the document.

I don't see a save to ods problem however.

Comment 7 Joe Wright 2018-08-09 19:58:48 UTC
The customer is not sure when this changed. The user in this case just migrated, but this should have been working.

Comment 8 Caolan McNamara 2018-08-10 07:35:18 UTC
for the xlxs case the problem is that PK11_ImportSymKey fails and returns null and that's unexpected so libreoffice goes on to crash. I can add the fix that detects PK11_ImportSymKey failure and go on to report inability to save rather than crash, which fixes the crash. Not sure that actually gains the customer a whole pile though, it won't crash, but it won't work either.

Comment 9 Caolan McNamara 2018-08-10 08:46:37 UTC
bug 1461450 has a similar problem and the workaround there works for us too to give functional encryption without a crash, so I could do that.

Comment 10 Caolan McNamara 2018-08-10 09:18:46 UTC
upstreaming that as https://gerrit.libreoffice.org/#/c/58816/ and committed a backport of that to our package, so xlsx encryption now works under FIPS without a crash

Comment 12 Martin Krajnak 2018-08-13 13:04:47 UTC
Works fine with:

libreoffice-calc-5.3.6.1-19.el7.x86_64
3.10.0-931.el7.x86_64
nss-3.36.0-5.el7_5.x86_64

Comment 14 errata-xmlrpc 2018-10-30 07:57:29 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2018:3054