Bug 840512 - JON CP test connection not working with valid credentials
JON CP test connection not working with valid credentials
Product: RHQ Project
Classification: Other
Component: Content (Show other bugs)
Unspecified Unspecified
urgent Severity high (vote)
: ---
: RHQ 4.5.0
Assigned To: Simeon Pinder
Mike Foley
Depends On:
Blocks: 844673
  Show dependency treegraph
Reported: 2012-07-16 10:06 EDT by Simeon Pinder
Modified: 2013-09-01 06:07 EDT (History)
2 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 844673 (view as bug list)
Last Closed: 2013-09-01 06:07:22 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)
patch for obfuscation issue with content source. (2.10 KB, text/plain)
2012-07-16 10:51 EDT, Simeon Pinder
no flags Details
Updated patch. (2.27 KB, application/octet-stream)
2012-07-16 15:55 EDT, Simeon Pinder
no flags Details
Patch for ObfuscatedPropertySimple (8.28 KB, patch)
2012-07-19 10:57 EDT, Lukas Krejci
no flags Details | Diff
Obfuscation logging captures. (464.78 KB, text/plain)
2012-07-21 19:20 EDT, Simeon Pinder
no flags Details

  None (edit)
Description Simeon Pinder 2012-07-16 10:06:09 EDT
Description of problem:
Install JON  3.1.0.GA and insert valid CSP credentials, the try to 'Test Connection' or 'Synchronize'. In both cases the UI displays an error message as the valid credentials are not being accepted.

Test failed - failed to connect to the remote repository for [JBoss CP Patch Feed] - check the configuration and make sure the remote repository is up and reachable. Details: java.lang.reflect.InvocationTargetException:null -> org.rhq.enterprise.server.plugin.pc.content.SyncException:Invalid login credentials specified for user [spinder]. Make sure this user has an active account at the CSP and that the password is correct.  

Version-Release number of selected component (if applicable):

How reproducible:
Every time.

Steps to Reproduce:
1. Install fresh JON 3.1.0.GA instance and log into the server.
2. Navigate to Administration/Content/Content Sources and update the default connection with valid CSP credentials.
3. Select the Content Source and attemt to 'Test Connection' or 'Synchronize'.
Actual results:
Failed connection due to invalid credentials.

Expected results:
Valid test results or synchronization.

Additional info:
Comment 1 Simeon Pinder 2012-07-16 10:14:28 EDT
I've narrowed this down to an issue with obfuscation. The problem is visible with the JON 3.1.0.GA release, but does not appear to be in master or in an RHQ instance built off of the JON 3.1.0.GA tag.

The password shows up correctly obfuscated in the database, but is not being correctly unobfuscated for some reason. The relevant JPA annotations applied to ObfuscatedPropertySimple seem to be correct and valid for the non-GA builds so it is still uncertain why this is happening. 

The JON 3.1.0.GA release was not signed so it does not appear that signing has anything to do with this issue.
Comment 2 Simeon Pinder 2012-07-16 10:51:28 EDT
Created attachment 598471 [details]
patch for obfuscation issue with content source.
Comment 3 Simeon Pinder 2012-07-16 15:55:14 EDT
Created attachment 598520 [details]
Updated patch.
Comment 4 Simeon Pinder 2012-07-16 17:57:34 EDT
Pushed to master 39e9490178. This will need to be cherrypicked to the release branch for JON 3.1.1 and for hotfixes pending GSS/Project management input.
Comment 5 Simeon Pinder 2012-07-17 09:25:43 EDT
Note to QA. This has only manifested in JON builds so we'll need another JON build to QA/verify this.
Comment 6 Lukas Krejci 2012-07-19 10:57:49 EDT
Created attachment 599182 [details]
Patch for ObfuscatedPropertySimple

I'm attaching a patch for ObfuscatedPropertySimple that uses a different approach to transparent obfuscation by introducing a transient "clear text" field that acts as the value in memory while letting Hibernate persist the obfuscated "stringValue".

The reason why the previous approach didn't work, I think, is that by changing the value of the persisted field in @Post* method, we effectively made Hibernate mark the property as dirty on each load.

Now, during a simple fetch, Hibernate optionally flushes all the dirty fields, which caused the former @PreUpdate to trigger. But because the fetch (I believe) was not enclosed in a transaction, the @PostUpdate never ran (I believe it runs only during commit to actually update), which left our obfuscated properties with the obfuscated value in memory (which was unexpected by the code).
Comment 7 Simeon Pinder 2012-07-21 19:01:49 EDT
As mentioned by Lukas, we finally figured out what was behind the obfuscation problems. The issue was that the initial implementation for property obfuscation  was based on symmetric annotations (Pre/PostPersist,Pre/PostUpdate) and PostLoad to obfuscate and deobfuscate passwords(always write to db obfuscated, but in memory unobfuscated).  

With a normal load of a Content Source there'd be PostLoad -> PreUpdate -> PostUpdate lifecycle calls as the property value would be cycled from obfuscated to clear text and back and forth. 

However depending upon this persistence lifecycle mechanism, which itself caused additional persistence calls, caused a couple issues:
i)When updating an existing content source you'd go through the PostLoad->PreUpdate->PostUpdate three times. It was consistent but redundant.
ii)On reboot, when there was no commit occurring(but the Content Sources were loaded as part of the Content Provider initialization), the PostUpdate call never ran(See above for Lukas's explanation). 

The solution was to re-implement the Obfuscation logic to:
- do a lot less translation from clear-text <-> obfuscated 
- do away with the usage of symmetric hibernate lifecyle events 
- to always persist the obfuscated string to db but keep transient clear-text version  
- modify the construction/copy logic to handle obfuscation and clear-text content creation
Comment 8 Simeon Pinder 2012-07-21 19:04:01 EDT
Added a fix to the latest ObfuscatedPropertySimple patch attached and added better protection against the situation where unobfuscated passwords may have been persisted to the database because of this bug.  

This is now fixed in master with commit: 0170a5ac909fd412

The fix when applied to an installed JON 3.1.0.GA instance will require the users to manually reset the credentials on their content source instances. This is necessary because there is no way for us to reliably detect whether the existing property was obfuscated or not before this patch was applied.

The fix was reviewed by Lukas and I and additionally I tested the change in master and on a JON 3.1.0.GA instance.  The worst case scenario where a password was persisted in unobfuscated form was also tested. In such a situation we log a message about the offending property but this only a WARN as the credential reset(last step of patch) with properly correct the state.

Leaving this in ON_DEV pending a little more review by Charles.
Comment 9 Simeon Pinder 2012-07-21 19:20:36 EDT
Created attachment 599552 [details]
Obfuscation logging captures.
Comment 10 John Sanda 2012-08-02 14:52:20 EDT
JON 3.1.1 ER1 build is available. Moving to ON_QA.

Comment 11 John Sanda 2012-08-02 21:13:56 EDT
Moving back to ON_DEV. This was incorrectly moved to ON_QA in comment 10.
Comment 12 Charles Crouch 2012-08-06 10:15:43 EDT
This fix needs to be applied to jon311

Simeon, can you raise a new docs bug to cover the credential reset step that is required

We need to test the CSP content source we ship with works in all fresh installs and upgrades: e.g.
-fresh install of jon311: setup works ok
-content source not setup in jon310, upgraded to jon311: setup works ok
-content source setup and working in jon310, upgraded to jon311: continues to work
-content source setup and working in jon301, upgraded to jon311: continues to work
Comment 13 Simeon Pinder 2012-08-06 10:39:26 EDT
BZ for docs bug: https://bugzilla.redhat.com/show_bug.cgi?id=846021
Comment 14 Simeon Pinder 2012-08-30 16:21:30 EDT
Adding logic to fix one more Content Source login related issue that happens just after installation and before reboot because the @PostLoad logic doesn't execute as expected.  See comments 4 and greater of https://bugzilla.redhat.com/show_bug.cgi?id=844673#c4 for more details. 

This is fixed in master with commit :1293a88ef64918b6401bd

Leaving in ON_QA state.
Comment 15 Heiko W. Rupp 2013-09-01 06:07:22 EDT
Bulk closing of items that are on_qa and in old RHQ releases, which are out for a long time and where the issue has not been re-opened since.

Note You need to log in before you can comment on or make changes to this bug.