Bug 788497

Summary: CDS sync fails with the below issue
Product: Red Hat Update Infrastructure for Cloud Providers Reporter: Kedar Bidarkar <kbidarka>
Component: CDSAssignee: James Slagle <jslagle>
Status: CLOSED CURRENTRELEASE QA Contact: wes hayutin <whayutin>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 2.0.2CC: jslagle, kbidarka, sghai
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-03-12 19:40:19 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Attachments:
Description Flags
cds sync issue
none
logs from gofer ssl and certs output none

Description Kedar Bidarkar 2012-02-08 10:08:26 UTC
Description of problem:
[Wed Feb 08 00:42:07 2012] [error] [client 10.12.119.135] Certificate to verify: 
[Wed Feb 08 00:42:07 2012] [error] [client 10.12.119.135] \tsubject=</CN=Red Hat Update Infrastructure>, issuer=</C=IN/ST=MH/L=Pune/O=RH/OU=Eng/CN=Entitlements CA>, subject.as_hash=<2273005625>, issuer.as_hash=<2936995902>, fingerprint=<7505E1A0389DCDFEC8A9D05458DC3A03>, serial=<257>, version=<2>, check_ca=<0>, notBefore=<Feb  7 11:22:52 2012 GMT>, notAfter=<Feb  4 11:22:52 2022 GMT>
[Wed Feb 08 00:42:07 2012] [error] [client 10.12.119.135] Using a CA Chain with 0 cert(s)
[Wed Feb 08 00:42:07 2012] [error] [client 10.12.119.135] Using a CRL Stack with 0 CRL(s)Client certificate did not match the repo consumer CA certificate
[Wed Feb 08 00:42:07 2012] [error] [client 10.12.119.135] user /CN=Red Hat Update Infrastructure: authentication failure for "/pulp/repos/content/dist/rhel/rhui/server/6/6Server/x86_64/optional/os/repodata/repomd.xml": Password Mismatch


Version-Release number of selected component (if applicable):



How reproducible:
during cds sync

Steps to Reproduce:
1.
2.
3.
  
Actual results:
cds sync fails 

Expected results:
cds sync should pass

Additional info:

Comment 1 Kedar Bidarkar 2012-02-08 10:10:57 UTC
Created attachment 560209 [details]
cds sync issue

/var/log/ssl_error_log

Comment 2 Kedar Bidarkar 2012-02-08 10:12:29 UTC
Please let us know if there are any change in the steps. 

Currently we are not using ca-chain certs for testing.

Comment 3 James Slagle 2012-02-09 01:54:16 UTC
Issue was caused by bad merge in repo_cert_utils.py after rebasing on newer pulp.  Fixed in commit 7dba257bcc16c40b72f28d6d1a2a95783907fb0c to pulp rhui branch.

Built into new pulp packages at:
https://brewweb.devel.redhat.com/buildinfo?buildID=198024

I'll ask rel-eng to spin a new ISO tomorrow.

Comment 4 wes hayutin 2012-02-09 16:45:48 UTC
looks fixed in the brew rpms.. wait for brew iso spin to verify

Comment 6 Sachin Ghai 2012-02-10 11:02:48 UTC
Tested  with latest rhui2.0.2 iso ( RHEL-6.2-RHUI-2.0.2-20120209.0-Server-x86_64-DVD1.iso ).

CDS sync is working fine, I synced two CDS nodes with rhel6 repos.

--
------------------------------------------------------------------------------
rhui (cds) => l

-= RHUI Content Delivery Servers =-

  cds2
  cds1

------------------------------------------------------------------------------
--

------------------------------------------------------------------------------
             -= Red Hat Update Infrastructure Management Tool =-


-= CDS Synchronization Status =-

Last Refreshed: 06:01:19
(updated every 5 seconds, ctrl+c to exit)


cds1 ........................................................ [  UP  ]
cds2 ........................................................ [  UP  ]


Next Sync                    Last Sync                    Last Result         
------------------------------------------------------------------------------
cds1
02-10-2012 09:28             02-10-2012 04:36             Success    

cds2
02-10-2012 09:29             02-10-2012 04:36             Success    


                                      Connected: ip-10-124-57-219.ec2.internal
------------------------------------------------------------------------------

Comment 7 Kedar Bidarkar 2012-02-14 06:45:06 UTC
------------------------------------------------------------------------------
             -= Red Hat Update Infrastructure Management Tool =-


-= CDS Synchronization Status =-

Last Refreshed: 01:33:25
(updated every 5 seconds, ctrl+c to exit)


CDS1_173_191 ................................................ [  UP  ]
CDS2_109_224 ................................................ [  UP  ]


Next Sync                    Last Sync                    Last Result         
------------------------------------------------------------------------------
CDS1_173_191
02-14-2012 03:10             02-13-2012 21:12             Success    

CDS2_109_224
02-14-2012 03:10             02-13-2012 21:12             Success    


                                       Connected: ip-10-80-202-84.ec2.internal
------------------------------------------------------------------------------

Comment 8 Kedar Bidarkar 2012-02-15 16:53:16 UTC
After expiration of entitlement ca cert and identity cert, I planned to add other valid certs for both and then the cds sync fails.

I unregistered cds nodes and re-registered them again and still it fails.

from /var/log/httpd/ssl_error_log

[Wed Feb 15 11:33:41 2012] [error] [client 10.224.1.234] Using a CRL Stack with 0 CRL(s)Client certificate did not match the repo consumer CA certificate
[Wed Feb 15 11:33:41 2012] [error] [client 10.224.1.234] user /CN=Red Hat Update Infrastructure: authentication failure for "/pulp/repos/content/dist/rhel/rhui/server/6/6Server/x86_64/os/repodata/repomd.xml": Password Mismatch
[Wed Feb 15 11:33:41 2012] [error] [client 10.100.207.211] Cert verification failed against 1 ca cert(s) and 0 CRL(s)
[Wed Feb 15 11:33:41 2012] [error] [client 10.100.207.211] Current Time: <Wed Feb 15 11:33:41 2012>
[Wed Feb 15 11:33:41 2012] [error] [client 10.100.207.211] Certificate:
[Wed Feb 15 11:33:41 2012] [error] [client 10.100.207.211]     Data:
[Wed Feb 15 11:33:41 2012] [error] [client 10.100.207.211]         Version: 3 (0x2)
[Wed Feb 15 11:33:41 2012] [error] [client 10.100.207.211]         Serial Number: 259 (0x103)
[Wed Feb 15 11:33:41 2012] [error] [client 10.100.207.211]         Signature Algorithm: sha1WithRSAEncryption
[Wed Feb 15 11:33:41 2012] [error] [client 10.100.207.211]         Issuer: C=IN, ST=MH, L=Pune, O=RH, OU=Engg, CN=RHUI Entitlement CA
[Wed Feb 15 11:33:41 2012] [error] [client 10.100.207.211]         Validity
[Wed Feb 15 11:33:41 2012] [error] [client 10.100.207.211]             Not Before: Feb 15 13:28:14 2012 GMT
[Wed Feb 15 11:33:41 2012] [error] [client 10.100.207.211]             Not After : Feb 14 13:28:14 2013 GMT
[Wed Feb 15 11:33:41 2012] [error] [client 10.100.207.211]         Subject: CN=Red Hat Update Infrastructure
[Wed Feb 15 11:33:41 2012] [error] [client 10.100.207.211]         Subject Public Key Info:
[Wed Feb 15 11:33:41 2012] [error] [client 10.100.207.211]             Public Key Algorithm: rsaEncryption


---------------------------------------------------

[Wed Feb 15 11:33:41 2012] [error] [client 10.100.207.211] Certificate to verify:
[Wed Feb 15 11:33:41 2012] [error] [client 10.100.207.211] \tsubject=</CN=Red Hat Update Infrastructure>, issuer=</C=IN/ST=MH/L=Pune/O=RH/OU=Engg/CN=RHUI Entitlement CA>, subject.as_hash=<2273005625>, issuer.as_hash=<1839428531>, fingerprint=<74DC25257EA7EECB53EF199BA8891C7E>, serial=<259>, version=<2>, check_ca=<0>, notBefore=<Feb 15 13:28:14 2012 GMT>, notAfter=<Feb 14 13:28:14 2013 GMT>
[Wed Feb 15 11:33:41 2012] [error] [client 10.100.207.211] Using a CA Chain with 1 cert(s)
[Wed Feb 15 11:33:41 2012] [error] [client 10.100.207.211] \tCA: subject=</C=IN/ST=MH/L=Pune/O=RH/OU=Engg/CN=RHUI Entitlement CA>, issuer=</C=IN/ST=MH/L=Pune/O=RH/OU=Engg/CN=RHUI Entitlement CA>, subject.as_hash=<1839428531>, issuer.as_hash=<1839428531>, fingerprint=<BBB906AD4BE2E7949169E9AEED1C23FF>, serial=<14155745042073364165>, version=<2>, check_ca=<1>, notBefore=<Feb 14 08:07:31 2012 GMT>, notAfter=<Feb 15 08:07:31 2012 GMT>
[Wed Feb 15 11:33:41 2012] [error] [client 10.100.207.211] Using a CRL Stack with 0 CRL(s)Client certificate did not match the repo consumer CA certificate
[Wed Feb 15 11:33:41 2012] [error] [client 10.100.207.211] user /CN=Red Hat Update Infrastructure: authentication failure for "/pulp/repos/content/dist/rhel/rhui/server/6/6Server/x86_64/os/repodata/repomd.xml": Password Mismatch

Comment 9 James Slagle 2012-02-16 23:12:53 UTC
I wasn't able to reproduce this.

I sync'd my CDS successfully.

Then I generated an identity certificate that had expired, and readded the CDS.  I then got the password mismatch error.

I then generated an identity certificate that had not expired, and resynced the CDS successfully (I didn't even have to delete/readd the CDS).

Please try to reproduce the error again after verifying that you've updated the identity certificate.  If it still occurs, can you attach /var/log/gofer/agent.log from the CDS, /var/log/httpd/ssl_error_log from the RHUA, and /etc/pki/rhui/identity.crt from the RHUA.  Thanks.

Comment 10 Kedar Bidarkar 2012-02-17 08:53:49 UTC
After expiration of entitlement ca cert and identity cert, I planned to add
 valid entitlement ca certs and identity cert for both and then the cds sync fails.


Here even the entitlement ca cert was also expired along with identity cert.

Both had a validity for 1 day and upon recreation of both the valid certs 
cds sync fails.

Comment 11 James Slagle 2012-02-17 18:02:02 UTC
I've run through this with both an expired entitlement ca and expired identity certificate and can not reproduce any issue.  When either (or both) are expired, the CDS sync failes, when neither are expired, the CDS sync succeeds.

One thing to note that might be causing the issue:

You can not just replace the identity certificate at /etc/pki/rhui/identity.crt with a hand crafted one.  The certificate can only be created via rhui-manager, this is because there is an extra step that configures every repo to use this certificate for authentication.  The user is never supposed to generate an identity certificate outside of rhui-manager.

Comment 12 James Slagle 2012-02-17 18:03:41 UTC
If you still have the issue please attach the requested files.

/var/log/gofer/agent.log from the CDS, /var/log/httpd/ssl_error_log from the
RHUA, /etc/pki/rhui/identity.crt from the RHUA, /etc/pki/rhui/entitlement-ca.crt from the RHUA.

Comment 13 Kedar Bidarkar 2012-02-22 07:22:48 UTC
I could reproduce this issue.

For all my setups I use

RHUA : ca2.crt
CDS : ca1.crt
Entitlement CA : ca3.crt

Comment 14 Kedar Bidarkar 2012-02-22 07:25:21 UTC
When certs expired, I had created 
a) entitlementca valid for days 1
b) identity cert valid for days 1

the below status after 1 day

[root@ip-10-224-6-47 ~]# rhui-manager status
CDS1_97_178 ................................................. [  UP  ]

CDS1_97_178 ................................................. [  ERROR  ]
CDS2_57_70 .................................................. [  UP  ]

CDS2_57_70 .................................................. [  ERROR  ]

Red Hat Enterprise Linux 6 Server (RPMs) from RHUI (6Server-x86_64) ... [  SUCCESS  ]

Red Hat Enterprise Linux 5 Server from RHUI (RPMs) (5Server-x86_64) ... [  SUCCESS  ]

Identity certificate expiration date = 2012-02-21T13:53:36Z  .......... [  ERROR  ]
Identity certificate at /etc/pki/rhui/identity.crt is expired!

Entitlement CA certificate expiration date = 2012-02-21T13:46:06Z  .... [  ERROR  ]
Entitlement CA certificate at /etc/pki/rhui/entitlement-ca.crt is expired!
82

Comment 15 Kedar Bidarkar 2012-02-22 07:28:46 UTC
After that I moved to /etc/pki/rhui and moved all the certs to a dir old

Which made the rhui-manager prompt me for new entitlement ca cert and identity cert

Created a valid new entitlement ca cert and updated it.

The below status regarding the certs now is 
[root@ip-10-224-6-47 ~]# rhui-manager status
CDS1_97_178 ................................................. [  UP  ]

CDS1_97_178 ................................................. [  ERROR  ]
CDS2_57_70 .................................................. [  UP  ]

CDS2_57_70 .................................................. [  ERROR  ]

Red Hat Enterprise Linux 6 Server (RPMs) from RHUI (6Server-x86_64) ... [  SUCCESS  ]

Red Hat Enterprise Linux 5 Server from RHUI (RPMs) (5Server-x86_64) ... [  SUCCESS  ]

Identity certificate expiration date = 2013-02-21T06:58:28Z  .......... [  OK  ]

Entitlement CA certificate expiration date = 2013-02-21T06:57:41Z  .... [  OK  ]
2

Comment 16 Kedar Bidarkar 2012-02-22 07:29:25 UTC
------------------------------------------------------------------------------
             -= Red Hat Update Infrastructure Management Tool =-


-= CDS Synchronization Status =-

Last Refreshed: 02:28:12
(updated every 5 seconds, ctrl+c to exit)


CDS1_97_178 ................................................. [  UP  ]
CDS2_57_70 .................................................. [  UP  ]


Next Sync                    Last Sync                    Last Result         
------------------------------------------------------------------------------
CDS1_97_178
02-22-2012 03:09             02-22-2012 01:59             Error      

CDS2_57_70
02-22-2012 03:10             02-22-2012 01:59             Error      


                                        Connected: ip-10-224-6-47.ec2.internal

Comment 17 Kedar Bidarkar 2012-02-22 07:35:06 UTC
Created attachment 564853 [details]
logs from gofer ssl and certs output

Tried to sync at 01:59 , but failed , also captured logs for the same time.

Comment 18 James Slagle 2012-02-22 10:21:19 UTC
The problem is the deletion of the identity certificate from /etc/pki/rhui.

From what I can tell, the identity cert generation that happens the very first
time you start rhui-manager is not meant to be a means to regenerate the cert;
only generate it the first time.  That initial generation does not reconfigure
any repos to use the new identity cert because there should be no repos since
it's the very first time you've run rhui-manager.

To restore the RHUI to a functioning state, you could now go into rhui-manager
and use the "i" screen to regenerate the identity cert, and it should fix the
issue.

That being said, I think we can protect against this situation.  The fix is
easy, we just need to reconfigure any repos if they happen to exist.

Comment 19 James Slagle 2012-02-22 10:51:47 UTC
committed to cloude master: a3bf1275b0faeaa93723b057629c530c2d9b9226

Comment 20 James Slagle 2012-02-22 11:05:26 UTC
Built into new rh-rhui-tools package at:
http://download.devel.redhat.com/brewroot/packages/rh-rhui-tools/2.0.57/1.el6/noarch/rh-rhui-tools-2.0.57-1.el6.noarch.rpm

Will request a new iso build later today.

Comment 21 Kedar Bidarkar 2012-02-22 13:24:12 UTC
appears fixed, will be moved to verified with new ISO

------------------------------------------------------------------------------
             -= Red Hat Update Infrastructure Management Tool =-


-= CDS Synchronization Status =-

Last Refreshed: 07:50:00
(updated every 5 seconds, ctrl+c to exit)


CDS1_97_178 ................................................. [  UP  ]
CDS2_57_70 .................................................. [  UP  ]


Next Sync                    Last Sync                    Last Result         
------------------------------------------------------------------------------
CDS1_97_178
02-22-2012 09:09             02-22-2012 07:39             Success    

CDS2_57_70
02-22-2012 09:10             02-22-2012 07:39             Success    


                                        Connected: ip-10-224-6-47.ec2.internal

Comment 23 Kedar Bidarkar 2012-02-24 09:24:55 UTC
------------------------------------------------------------------------------
             -= Red Hat Update Infrastructure Management Tool =-


-= CDS Synchronization Status =-

Last Refreshed: 04:21:42
(updated every 5 seconds, ctrl+c to exit)


CDS1_50_75 .................................................. [  UP  ]
CDS2_27_41 .................................................. [  UP  ]


Next Sync                    Last Sync                    Last Result         
------------------------------------------------------------------------------
CDS1_50_75
02-24-2012 10:09             02-24-2012 04:09             Error      

CDS2_27_41
02-24-2012 07:44             02-24-2012 01:44             Success    


                                        Connected: ip-10-98-9-150.ec2.internal





[root@ip-10-98-9-150 rhui]# rhui-manager status
Previous authentication credentials could not be found. Logging into
the RHUI.

If this is the first time using the RHUI, it is recommended to change
the user's password in the User Management section of RHUI Tools.

RHUI Username: admin
RHUI Password: 
CDS1_50_75 .................................................. [  UP  ]

CDS1_50_75 .................................................. [  SUCCESS  ]
CDS2_27_41 .................................................. [  UP  ]

CDS2_27_41 .................................................. [  SUCCESS  ]

Red Hat Enterprise Linux 6 Server (RPMs) from RHUI (6Server-x86_64) ... [  SUCCESS  ]

Red Hat Update Infrastructure 2.0 (RPMs) (6Server-x86_64) ............. [  SUCCESS  ]

Identity certificate expiration date = 2013-02-23T09:20:09Z  .......... [  OK  ]

Entitlement CA certificate expiration date = 2012-07-25T16:36:58Z  .... [  OK  ]
0

Comment 24 James Slagle 2012-03-12 19:40:19 UTC
Released in RHUI 2.0.2