Bug 1056224

Summary: Enhance openjdk to allow configuration of replacement CA/TLS trust store.
Product: Red Hat Enterprise Linux 7 Reporter: Kai Engert (:kaie) (inactive account) <kengert>
Component: java-1.8.0-openjdkAssignee: Deepak Bhole <dbhole>
Status: CLOSED WONTFIX QA Contact: OpenJDK QA <java-qa>
Severity: medium Docs Contact:
Priority: medium    
Version: 7.0CC: cww, dbhole, dkochuka, eparis, fred.kubli, gauthier.stephane, gbonocor, jfabriko, jlyle, jshepherd, jvanek, martinsson.patrik, me, pasteur, platform-rfe, sardella, savsingh, stefw, tomek
Target Milestone: rcKeywords: Reopened
Target Release: 7.5   
Hardware: Unspecified   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-11-23 18:25:35 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 894083, 1420851, 1477664    

Description Kai Engert (:kaie) (inactive account) 2014-01-21 17:44:48 UTC
This is an enhancement suggestion inspired by a customer's desire to use a different trust store for TLS root CA certificates.

Although the oracle-java package ships its own trust store, the customer would like to rather use the new systemwide one that can be easily extended, which is available in java file format at
  /etc/pki/ca-trust/extracted/java/cacerts
(see also: man update-ca-trust)

Although it's possible to override the default file by specifying a command line argument at start time
  java -Djavax.net.ssl.trustStore=/etc/pki/ca-trust/extracted/java/cacerts
this is inconvenient to deploy on many computers.

Here's a proposed solution:

As of today, the oracle-java package ships file
  /usr/lib/jvm/java-...-oracle-.../jre/lib/security/cacerts

Instead, if could:
- install the same file as
  /usr/lib/jvm/java-...-oracle-.../jre/lib/security/cacerts.default

- install a symbolic link from 
  /usr/lib/jvm/java-...-oracle-.../jre/lib/security/cacerts
  pointing to 
    /etc/alternatives/javax-net-ssl-truststore.{arch}

- at installation time, run a %post scriptlet which sets up 
    /etc/alternatives/javax-net-ssl-truststore.{arch}
  as an alternative, with the default priority of e.g. 10,
  pointing to 
    /usr/lib/jvm/java-...-oracle-.../jre/lib/security/cacerts.default

The above setup wouldn't cause any notable runtime change.

However, it would enable the customer to perform a one-time configuration to change the cacerts file that will be actually used.

A command line using
  update-alternatives --install 
could be used to install the systemwide trust store with a higher priority, and from that time on it would be used, regardless of future packages upgrades.

(Note, openjdk doesn't require this solution, as it's already using the systemwide store.)

Comment 4 jiri vanek 2014-01-22 16:40:22 UTC
This - /etc/alternatives/javax-net-ssl-truststore.{arch} - do exists already?

The issue is that  we have to ship oracle binaries as they are, without modification. The ranaming is modification. The proposed change is possible for OpenJDK.

Comment 5 Kai Engert (:kaie) (inactive account) 2014-01-22 17:00:14 UTC
(In reply to jiri vanek from comment #4)
> This - /etc/alternatives/javax-net-ssl-truststore.{arch} - do exists already?

no


> The issue is that  we have to ship oracle binaries as they are, without
> modification. The ranaming is modification. 

This isn't a proposal to change a binary.

This simply is a proposal to rename one static file and by default, replace the original filename with a chain of symbolic links, pointing to the same default file.

From my personal point of view, this wouldn't qualify as changing the binary files, but IANAL.


> The proposed change is possible for OpenJDK.

openjdk doesn't need it. openjdk already works as desired.

Comment 7 Patrik Martinsson 2014-01-28 09:26:04 UTC
Hi, 

This proposal makes *a lot* of sense to me. 

Consider the use-case, 
- Develop a in-house java-oracle/sun-application that uses tls. 
- We use our own CA (that is, we use self-signed certificates) 
- To get the java applications to work with self-signed certificates, two alternatives comes to mind, 
  
 1) Import our root-ca into the default keystore, $JAVA_HOME/jre/lib/security/cacerts 
 2) Create our own java-keystore, and point every java-instance to use by "-Djavax.net.ssl.trustStore=/path_to_our_keystore" 

However these two alternatives both have their downsides, 

 1) This method works fine until the java-rpm gets updated. The rpm overwrites the keystore every time it gets updated, thus forcing us to "reimport" the root-ca at every update. We use puppet to trigger this today, but it's inconvenient and kind of a "hack". 

 2 ) This means that we need to inform every developer to change their applications so that it uses a different keystore than the default. This seems just as much of a hack as the first alternative. 

The new centralized CA-trust-store that is being developed has a lot of potential and really makes it easier for sysadmins (as my self) to handle root-ca's system-wide. The "alternatives-system-suggestion" by Kai Engert also makes a lot of sense since it allows sysadmins to override the default keystore without it being overwritten at every update. 

This requires no change to the binary whatsoever, just a change in how the rpm handles the default keystore for java-oracle/sun. 

Best regards, 
Patrik Martinsson, 
Sweden

Comment 8 Sam Richards 2015-01-06 23:18:42 UTC
Just a couple of additional notes:

Kai's proposal importantly maintains the original behaviour of using oracle's supplied cacerts store by default whilst allowing modification for users that require modification of the keystore. This reduces risk for applications that might depend on differences between the oracle trusted roots and those in the 
system-wide store.

The proposal would also match the solution that has been employed for other configuration in the same $JAVA_HOME/jre/lib/security directory:

e.g. 

/usr/lib/jvm/jre-1.7.0-oracle.x86_64/lib/security

blacklist
cacerts
java.policy
java.security
local_policy.jar -> /etc/alternatives/jce_1.7.0_oracle_local_policy.x86_64
US_export_policy.jar -> /etc/alternatives/jce_1.7.0_oracle_us_export_policy.x86_64

In the meantime what would be the recommended workaround to persist changes to the cacerts file through upgrades of the oracle-java rpm?

Comment 9 Jason Shepherd 2015-01-23 05:34:07 UTC
Perhaps an option is to change the spec file for the $JAVA_HOME/jre/lib/security/cacerts

 "%config" to "%config(noreplace)"

So that that file is not updated when the rpm is updated?

Comment 10 Deepak Bhole 2015-02-12 15:07:17 UTC
Hi,

The deployment guide at http://docs.oracle.com/javase/7/docs/technotes/guides/security/jsse/samples/README.txt states that <java-home>/lib/security/jssecacerts should be used for custom certs as desired here.

Is there a use case (as relates to this bug) that the above does not cover?

Comment 11 Deepak Bhole 2015-10-08 18:40:39 UTC
Hi Kai, can we close this bug based on comment #10?

Comment 12 Deepak Bhole 2015-10-08 18:59:23 UTC
*** Bug 1060877 has been marked as a duplicate of this bug. ***

Comment 13 Sam Richards 2015-10-13 08:41:32 UTC
We have deployed site-specific trusted ca certificates based on Deepak's comments that creating a file <java-home>/lib/security/jssecacerts is the appropriate place to put them.

This file isn't replaced by either an Oracle Java 7 (or 8 incidentally) installation/upgrade as provided by this rpm so solves Kai's original use-case (as it does mine).

Comment 14 Kai Engert (:kaie) (inactive account) 2015-10-13 11:28:20 UTC
If I understood correctly, you have decided:

- you don't want to change the Oracle Java configuration to use our central
  CA trust store

- you want Oracle Java to use the trust store that is provided by Oracle

- for extending the Java trust store, you want to rely on the mechanism
  that Oracle Java offers

Given your argument that you aren't allowed to modify Oracle Java, your proposal makes sense to me.

Your approach doesn't implement my proposal, but instead you have decided to keep the Oracle Java trust store completely separate from our systemwide shared trust store.

I suggest that you mark this bug as wontfix.

Comment 15 Deepak Bhole 2015-10-14 17:58:10 UTC
Thanks Kai.

Closing per comment #14.

Comment 16 Giuseppe Bonocore 2015-11-10 15:19:12 UTC
Hello,


I have a customer requesting for the same feature (see also https://bugzilla.redhat.com/show_bug.cgi?id=894083). 

I don't understand what the current suggestion is. 

I do agree with you that using JSSECACERTS should be encouraged, but nevertheless, if we don't hook this to the alternatives subsystem (as Kai suggest), we have to manually set the correct file to point to <java-home>/lib/security/jssecacerts (e.g. manually using RPM triggers and symlinks) and to cleanup the leftover when we uninstall Oracle Java (as it is marked as a configuration file and so is correctly not deleted if was updated)

Am I right or am I missing something?

What is the reason why we cannot hook jssecacerts to alternatives subsystem, since this will not change the default behaviour of Oracle JDK, but just give to sys admins a tool to better manage (and eventually hook to system wide store) CA certificates configuration?

Comment 17 Deepak Bhole 2015-11-11 20:06:57 UTC
Hi Giuseppe,

I am not sure I completely understand. <java-home>/lib/security/jssecacerts is an additional store on top of the default cacerts. cacerts are expected to be managed by the system and ideally it should not be changed. The recommended path to add certs to is <java-home>/lib/security/jssecacerts as that is expected to be user managed. Both cacerts and jssecacerts will be automatically loaded by the JVM each time.

What advantage would having an alternative provide when jssecacerts can be used and is not clobbered by updates?

Comment 18 Giuseppe Bonocore 2015-11-12 14:37:34 UTC
(In reply to Deepak Bhole from comment #17)
> Hi Giuseppe,
> 
> I am not sure I completely understand. <java-home>/lib/security/jssecacerts
> is an additional store on top of the default cacerts. cacerts are expected
> to be managed by the system and ideally it should not be changed. The
> recommended path to add certs to is <java-home>/lib/security/jssecacerts as
> that is expected to be user managed. Both cacerts and jssecacerts will be
> automatically loaded by the JVM each time.
> 
> What advantage would having an alternative provide when jssecacerts can be
> used and is not clobbered by updates?

What my customer is complaining about is that, if they manually create / manage jseecacerts (so create an additional ca certs store), they also need to manually move it every time they upgrade the JVM package (because the location on the filesystem changes), and also remove it after the JVM uninstall, since it will not be removed with RPM uninstall.

Having that file managed by the alternatives subsystem will simplify that, as every update may just leave the file as is (since it is just a link which points to a central location) and may be safely removed with RPM uninstall

Comment 19 Patrik Martinsson 2016-04-27 07:01:43 UTC
(In reply to Giuseppe Bonocore from comment #18)
> (In reply to Deepak Bhole from comment #17)
> > Hi Giuseppe,
> > 
> > I am not sure I completely understand. <java-home>/lib/security/jssecacerts
> > is an additional store on top of the default cacerts. cacerts are expected
> > to be managed by the system and ideally it should not be changed. The
> > recommended path to add certs to is <java-home>/lib/security/jssecacerts as
> > that is expected to be user managed. Both cacerts and jssecacerts will be
> > automatically loaded by the JVM each time.
> > 
> > What advantage would having an alternative provide when jssecacerts can be
> > used and is not clobbered by updates?
> 
> What my customer is complaining about is that, if they manually create /
> manage jseecacerts (so create an additional ca certs store), they also need
> to manually move it every time they upgrade the JVM package (because the
> location on the filesystem changes), and also remove it after the JVM
> uninstall, since it will not be removed with RPM uninstall.
> 
> Having that file managed by the alternatives subsystem will simplify that,
> as every update may just leave the file as is (since it is just a link which
> points to a central location) and may be safely removed with RPM uninstall

That's exactly my main issue that I tried do describe in the beginning of this bug report. I'm not sure if I failed or why the issue doesn't seem to get through. In my world, this should be an issue for every system administrator that manages servers, that runs in house java-applications, that connects to sites that uses certificates issued by an CA that is not in the default 'jseecacerts-store'. 

I still don't see why this is so hard to fix, you seem to miss the whole point of centrally managed servers - that updates java with a custom ca - automatically.

Comment 20 Deepak Bhole 2016-05-03 17:32:32 UTC
The issue with using the main ca store via an alternative is that it cannot account for blacklisted certs. We may update the default store from time to time to not just add, but also blacklist servers. 

By allowing an alternatives based management, we are allowing for a way to link to a potentially compromised store with no easy way for the customer to know that they need to update it. This is the reason why jseecacerts is a better approach IMO.

Jiri, can we using some RPM mechanism like ghost or something to track and copy jssecacerts over? Perhaps we can create an empty one and place it, marking is config(noreplace)? In theory such an empty store file would never change so it shouldn't cause conflicts.

Comment 21 Patrik Martinsson 2016-05-04 09:30:04 UTC
Hi Deepak, 

I don't buy that, neither does the maintainers of openjdk, since they already use the systemwide store, as Kai Engert describes in the first post. 

I think the best way to handle it is obviously to make use of those mechanisms that's already in place (such as alternativs, the central ca-trust store). 

And by the way, is there really a scenario where the "oracle rpm maintaners" suddenly decides that "X certificate needs to be blacklisted" and the rest of the system (ie. all applications etc. that trusts the central ca-trust store) should still trust that CA ? I mean, if a certificate gets blacklisted, surely it would get blacklisted in the whole system, no ? 

// Patrik

Comment 22 jiri vanek 2016-05-04 09:42:22 UTC
Before answering... 
Patrick, Is it better to keep copy of jssecacerts in <java-home>/lib/security/ or to link some known locations?

The approach how rpms will handle it will be a bit different:
 Linking - if there is some location where this jssecacerts file is pernamently located, then indeed RPMs can easily always create link <java-home>/lib/security/jssecacerts->knownLocation.
Openjdk is already linking /etc/pki/java/cacerts. Can we link some /etc/java/jssecacerts? Where /etc/java/jssecacerts do not need exists, but if it does, it will be used. And if it shoudl exists,will be in power of system admin
This is easy and nice to do in specfiles, but sounds like big system change

If Only aceptable place is <java-home>/lib/security/jssecacerts,  then I'm afraid it will get a bit more complicated.
In rhel 6 the %ghost should work, and update should be ok.
In rhel 7 (as this change should follow htis change)  the NVRA install will may it tricky. The file probably can be ghost, but its hadnling during updates  have to go through copy-jdk-configs.

In any case *every* jdk n rhel should adapt to this schema.

Comment 23 Patrik Martinsson 2016-05-04 10:43:06 UTC
(In reply to jiri vanek from comment #22)
> Before answering... 
> Patrick, Is it better to keep copy of jssecacerts in
> <java-home>/lib/security/ or to link some known locations?
> 
> The approach how rpms will handle it will be a bit different:
>  Linking - if there is some location where this jssecacerts file is
> pernamently located, then indeed RPMs can easily always create link
> <java-home>/lib/security/jssecacerts->knownLocation.
> Openjdk is already linking /etc/pki/java/cacerts. Can we link some
> /etc/java/jssecacerts? Where /etc/java/jssecacerts do not need exists, but
> if it does, it will be used. And if it shoudl exists,will be in power of
> system admin
> This is easy and nice to do in specfiles, but sounds like big system change
> 
> If Only aceptable place is <java-home>/lib/security/jssecacerts,  then I'm
> afraid it will get a bit more complicated.
> In rhel 6 the %ghost should work, and update should be ok.
> In rhel 7 (as this change should follow htis change)  the NVRA install will
> may it tricky. The file probably can be ghost, but its hadnling during
> updates  have to go through copy-jdk-configs.
> 
> In any case *every* jdk n rhel should adapt to this schema.


Short answer, I think openjdk is doing it right - *both in rhel 6 and 7*.

Rhel 7x
ls -la /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.71-2.b15.el7_2.x86_64/jre/lib/security/cacerts
lrwxrwxrwx. 1 root root 41 May  4 12:40 /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.71-2.b15.el7_2.x86_64/jre/lib/security/cacerts -> ../../../../../../../etc/pki/java/cacerts

Rhel 6x,
ls -la  /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.65-0.b17.el6_7.x86_64/jre/lib/security/cacerts 
lrwxrwxrwx. 1 root root 41 May  4 12:04 /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.65-0.b17.el6_7.x86_64/jre/lib/security/cacerts -> ../../../../../../../etc/pki/java/cacerts

Just link the certstore to /etc/pki/java/cacerts (or directly to /etc/pki/ca-trust/extracted/java/cacerts)

// Patrik

Comment 24 jiri vanek 2016-05-04 11:43:23 UTC
(In reply to Patrik Martinsson from comment #23)
> (In reply to jiri vanek from comment #22)
> > Before answering... 
> > Patrick, Is it better to keep copy of jssecacerts in
> > <java-home>/lib/security/ or to link some known locations?
> > 
> > The approach how rpms will handle it will be a bit different:
> >  Linking - if there is some location where this jssecacerts file is
> > pernamently located, then indeed RPMs can easily always create link
> > <java-home>/lib/security/jssecacerts->knownLocation.
> > Openjdk is already linking /etc/pki/java/cacerts. Can we link some
> > /etc/java/jssecacerts? Where /etc/java/jssecacerts do not need exists, but
> > if it does, it will be used. And if it shoudl exists,will be in power of
> > system admin
> > This is easy and nice to do in specfiles, but sounds like big system change
> > 
> > If Only aceptable place is <java-home>/lib/security/jssecacerts,  then I'm
> > afraid it will get a bit more complicated.
> > In rhel 6 the %ghost should work, and update should be ok.
> > In rhel 7 (as this change should follow htis change)  the NVRA install will
> > may it tricky. The file probably can be ghost, but its hadnling during
> > updates  have to go through copy-jdk-configs.
> > 
> > In any case *every* jdk n rhel should adapt to this schema.
> 
> 
> Short answer, I think openjdk is doing it right - *both in rhel 6 and 7*.
> 

Yes, it does. But oracle jdk package can not use this approach. It have its own cacerts. And as it is locsed product, we cant legally  change this file(including its handling).

Thats why  jssecacerts come to play.

Comment 25 Patrik Martinsson 2016-05-08 19:10:12 UTC
(In reply to jiri vanek from comment #24)
> (In reply to Patrik Martinsson from comment #23)
> > (In reply to jiri vanek from comment #22)
> > > Before answering... 
> > > Patrick, Is it better to keep copy of jssecacerts in
> > > <java-home>/lib/security/ or to link some known locations?
> > > 
> > > The approach how rpms will handle it will be a bit different:
> > >  Linking - if there is some location where this jssecacerts file is
> > > pernamently located, then indeed RPMs can easily always create link
> > > <java-home>/lib/security/jssecacerts->knownLocation.
> > > Openjdk is already linking /etc/pki/java/cacerts. Can we link some
> > > /etc/java/jssecacerts? Where /etc/java/jssecacerts do not need exists, but
> > > if it does, it will be used. And if it shoudl exists,will be in power of
> > > system admin
> > > This is easy and nice to do in specfiles, but sounds like big system change
> > > 
> > > If Only aceptable place is <java-home>/lib/security/jssecacerts,  then I'm
> > > afraid it will get a bit more complicated.
> > > In rhel 6 the %ghost should work, and update should be ok.
> > > In rhel 7 (as this change should follow htis change)  the NVRA install will
> > > may it tricky. The file probably can be ghost, but its hadnling during
> > > updates  have to go through copy-jdk-configs.
> > > 
> > > In any case *every* jdk n rhel should adapt to this schema.
> > 
> > 
> > Short answer, I think openjdk is doing it right - *both in rhel 6 and 7*.
> > 
> 
> Yes, it does. But oracle jdk package can not use this approach. It have its
> own cacerts. And as it is locsed product, we cant legally  change this
> file(including its handling).
> 
> Thats why  jssecacerts come to play.

Oh, okey. Sorry I missed that part. 

Well, in that case my vote is on 
> Can we link some /etc/java/jssecacerts? Where /etc/java/jssecacerts do not need exists, but if > it does, it will be used. And if it shoudl exists,will be in power of system admin

I think this sound as a good solution. I'll just verify it so I got it right, 
- As a system administrator, I'll create a file /etc/java/jssecacerts (that in turn will be a link to /etc/pki/ca-trust/extracted/java/cacerts) 
- Every java-oracle instance will use that file as it's ca-trust store *if it exists*. And it is up to the system administrator to make sure it exists if they want to make use of it. Of course this behavior should be documented so no confusion will arise.

I think that solution solves the initial problem, at least from my perspective. 

Question though, will the jssecacerts-file-cacertstore (if it exists) be used by *exclusively*, or will it be added in addition to the "original" oracle cacertstore ?

Comment 28 Chris Williams 2017-06-14 20:07:35 UTC
Red Hat Enterprise Linux 6 transitioned to the Production 3 Phase on May 10, 2017.  During the Production 3 Phase, Critical impact Security Advisories (RHSAs) and selected Urgent Priority Bug Fix Advisories (RHBAs) may be released as they become available.
 
The official life cycle policy can be reviewed here:
 
http://redhat.com/rhel/lifecycle
 
This issue does not appear to meet the inclusion criteria for the Production Phase 3 and has been moved to RHEL 7 for consideration in 7.5. RHEL 7 is still in phase 1.

Comment 35 Deepak Bhole 2018-11-23 18:25:35 UTC
For cases where jssecertpath is not a viable options, the following options can specify additional trust stores:
-Djavax.net.ssl.trustStore

Please see https://docs.oracle.com/cd/E19830-01/819-4712/ablqw/index.html for more information.

Given that multiple mechanisms exist to mitigate this request, I don't think adding an additional layer of alternatives configuration is warranted.