Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1033333

Summary: Certmonger crashes when trying to decode certificate with invalid data in header
Product: Red Hat Enterprise Linux 7 Reporter: Nalin Dahyabhai <nalin>
Component: certmongerAssignee: Nalin Dahyabhai <nalin>
Status: CLOSED CURRENTRELEASE QA Contact: Kaleem <ksiddiqu>
Severity: high Docs Contact:
Priority: unspecified    
Version: 7.0CC: dpal, jr.aquino, kchamart
Target Milestone: rc   
Target Release: ---   
Hardware: All   
OS: Linux   
URL: https://fedorahosted.org/certmonger/ticket/22
Whiteboard:
Fixed In Version: certmonger-0.69-1.el7 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: 1032760 Environment:
Last Closed: 2014-06-13 11:19:05 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1032760    
Bug Blocks:    

Description Nalin Dahyabhai 2013-11-21 21:22:38 UTC
+++ This bug was initially created as a clone of Bug #1032760 +++

Description of problem:
dogtag-ipa-retrieve-agent-submit imports ipalib.x509, which imports ipapython.ipautil, which imports readline, which prints the control sequence ^[[?1034h

This causes the certifcate that certmonger is handling to be corrupt, which in turn blows up certmonger.

Version-Release number of selected component (if applicable):
libipa_hbac-1.9.2-82.10.el6_4.x86_64
ipa-client-3.0.0-26.el6_4.4.x86_64
ipa-python-3.0.0-26.el6_4.4.x86_64
ipa-server-selinux-3.0.0-26.el6_4.4.x86_64
libipa_hbac-python-1.9.2-82.10.el6_4.x86_64
ipa-server-3.0.0-26.el6_4.4.x86_64
ipa-pki-ca-theme-9.0.3-7.el6.noarch
ipa-admintools-3.0.0-26.el6_4.4.x86_64
ipa-pki-common-theme-9.0.3-7.el6.noarch
python-libs-2.6.6-37.el6_4.x86_64

Steps to Reproduce:
1. getcert resubmit -i <id>
2. wait for certmonger

Actual results:
Certmonger coredumps

Expected results:
Certmonger should resubmit and renew the certificate

Additional info:

--- Additional comment from RHEL Product and Program Management on 2013-11-20 14:23:12 EST ---

Since this bug report was entered in bugzilla and this package is
not scheduled to be updated in the current release, the release
flag has been set to ? to ensure that it is properly evaluated
for the next release.

--- Additional comment from Nalin Dahyabhai on 2013-11-21 09:00:20 EST ---

We get here through a combination of three things:

* When running an external enrollment (or in this case, "enrollment") helper, certmonger isn't clearing the environment.  In particular, when TERM is set to an xterm-like terminal (which happens when the daemon inherits that value due to being started as a child of a shell), the IPA helper's use of the framework (as noted in the description) ends up pulling in the readline module, which outputs a control sequence before it starts.  The development tree now includes a candidate fix for this.

* When reading the certificate that's output by the helper when it returns success, garbage that precedes or follows the certificate isn't stripped out.  While this isn't something that the docs say helpers can expect the daemon to do, for this case we could work around it.  The development tree does this now, but it's only a bandage, and doesn't cover other 

* When the daemon attempts to save the certificate value, which includes noise, it fails.  The state machine hits an assertion failure when it subsequently tries again to save the certificate.  The development tree now includes a fix to give it the behavior which is asserted.

At minimum we'd need to pull in the changes for the first and third parts.

Comment 1 Nalin Dahyabhai 2013-11-21 21:39:40 UTC
I'd like to pull this in, though starting under systemd should already avoid most of the complications of the first part, so it's not as urgent here.

Comment 3 Kaleem 2014-02-03 07:12:55 UTC
How QE should verify (steps) this?

Comment 4 Nalin Dahyabhai 2014-02-04 16:37:39 UTC
The root cause is that the environment isn't cleared when the enrollment helper is run.  To verify that, we need to ensure that the TERM variable isn't set when the helper is run, so that it won't unexpectedly output a terminal initialization string.  We can do that either by examining the process itself while it runs, or by comparing the normal behavior with the behavior we get when we ensure that it _is_ set (though this will only work so long as the helper itself behaves differently).

For the first method, we need to find the helper which is invoked in an attempt to get a new certificate.  Locate the file under /var/lib/certmonger/cas which includes the CA's name (as noted in the 'getcert' output for the certificate we're tracking) in its 'id' field.  If that file also contains a 'ca_external_helper' value, that's the command that the daemon runs to handle interactions with the CA.  Then, while that process is running (it may require a manual 'getcert resubmit' to run it again if it finishes quickly), run
  strings /proc/`pidof COMMAND`/environ | sort
A fixed version of certmonger will always set TERM to "dumb".  One that doesn't have that fix will just pass through the value it was started with.

The second method will only work so long as the helper actually behaves differently.  For example, if you run
  /usr/libexec/certmonger/dogtag-ipa-retrieve-agent-submit | od -t x1c
and see it produce output, then bug #880393 is still present and we can observe what happens when we stop the certmonger service and edit the appropriate CA file so that the helper it invokes is preceded by "env TERM=xterm" or "env TERM=dumb".

Comment 5 Kaleem 2014-03-05 08:40:33 UTC
Verified

IPA and certmonger version:
===========================
[root@nec-em6 ~]# rpm -q ipa-server certmonger
ipa-server-3.3.3-19.el7.x86_64
certmonger-0.70-2.el7.x86_64
[root@nec-em6 ~]# 

TERM variable is set to "dumb" by /usr/libexec/certmonger/dogtag-ipa-retrieve-agent-submit and "getcert resubmit" works fine.

[root@nec-em6 ~]# cat /etc/sysconfig/certmonger 
TERM=xterm-256color
[root@nec-em6 ~]# ps -eaf|grep agent
root      5284  3429  3 03:36 ?        00:00:00 /usr/bin/python -E /usr/libexec/certmonger/dogtag-ipa-retrieve-agent-submit
root      5286  5221  0 03:36 pts/1    00:00:00 grep --color=auto agent
[root@nec-em6 ~]# strings /proc/5284/environ |grep TEMP
[root@nec-em6 ~]# strings /proc/5284/environ |grep TERM
TERM=dumb
[root@nec-em6 ~]# strings /proc/5284/environ
HOME=/var/run/certmonger
PATH=/usr/bin:/bin:/usr/sbin:/sbin
SHELL=/bin/sh
TERM=dumb
TMPDIR=/var/run/certmonger
CERTMONGER_REQ_SUBJECT=CN=IPA RA,O=TESTRELM.TEST
CERTMONGER_OPERATION=SUBMIT
CERTMONGER_CSR=-----BEGIN NEW CERTIFICATE REQUEST-----
MIIC+jCCAeICAQAwKTEWMBQGA1UEChMNVEVTVFJFTE0uVEVTVDEPMA0GA1UEAxMG
SVBBIFJBMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA4EhxGXR5DzUo
ATbyvsyBiiajsWzuJv2ex6KqOs1kc9CWjpJu4IBpNv6IoNBzXNERPgBUABKFz7M0
hb7+JvTV0AxvrogKBK1WdI5UNaPfHVR8i4smyA7YRZgtkfmGN55uJrea5XFL7UF0
JOMAy8OlPC4MES1RrJnFeAdqESiDlEb0be+Vu5KFTroDFbkFb12AtDbSU3k8vXNF
2zeDhMiCsiHv01zIPQmc5Wg7ciD6STGGOzOBhbl7lca62/yR9dCZnCs+WjcJnKn6
c4eguAWJjk02IUwNvL1zrve8L3oS5A37VbtcvI77tB9rCxYZ0GcbJuqqRefC7BFs
GwZNYqlMNQIDAQABoIGLMBYGCSqGSIb3DQEJFDEJEwdpcGFDZXJ0MHEGCSqGSIb3
DQEJDjFkMGIwDgYDVR0PAQEABAQDAgTwMCAGA1UdJQEBAAQWMBQGCCsGAQUFBwMB
BggrBgEFBQcDAjAMBgNVHRMBAf8EAjAAMCAGA1UdDgEBAAQWBBQsT+v6Vz3JvtYa
c8sggFa4aDhDDjANBgkqhkiG9w0BAQsFAAOCAQEA3BmcBdttrbGs9mI2FkdEsqGs
a1xOB9YQ1wUMmNHo46PB9jbJYxjm2XqGfWC+HvpPWmipLnVRSZDloXXBdhGTxKXP
l/v9CkZYV0bD2CEcuZ8nTbWXYRT96HwSBt0L/ms+EJo555wHON3YRtlkNGZ11YHU
Evqw6EIwLubTPl83boQermoIJysogRTVbcW7l9DvBfieGgUMsXjRKnvGfiBfEHhB
UfodJOq0guOz0optZY3Sg011kn6A4cAE+NOxF04fB9t3l/l0owv/7s28aS2ElRH9
7gDumjYoc5rhefKYEwRqyDhERBHBjrkvF7A1vkr8xo/2ZA+1/u4PzVpXAJo1dw==
-----END NEW CERTIFICATE REQUEST-----
CERTMONGER_CA_PROFILE=ipaCert
CERTMONGER_CERTIFICATE=-----BEGIN CERTIFICATE-----
MIIDgzCCAmugAwIBAgIBFDANBgkqhkiG9w0BAQsFADA4MRYwFAYDVQQKEw1URVNU
UkVMTS5URVNUMR4wHAYDVQQDExVDZXJ0aWZpY2F0ZSBBdXRob3JpdHkwHhcNMTQw
MzA0MTc0OTUwWhcNMTYwMjIyMTc0OTUwWjApMRYwFAYDVQQKEw1URVNUUkVMTS5U
RVNUMQ8wDQYDVQQDEwZJUEEgUkEwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEK
AoIBAQDgSHEZdHkPNSgBNvK+zIGKJqOxbO4m/Z7Hoqo6zWRz0JaOkm7ggGk2/oig
0HNc0RE+AFQAEoXPszSFvv4m9NXQDG+uiAoErVZ0jlQ1o98dVHyLiybIDthFmC2R
+YY3nm4mt5rlcUvtQXQk4wDLw6U8LgwRLVGsmcV4B2oRKIOURvRt75W7koVOugMV
uQVvXYC0NtJTeTy9c0XbN4OEyIKyIe/TXMg9CZzlaDtyIPpJMYY7M4GFuXuVxrrb
/JH10JmcKz5aNwmcqfpzh6C4BYmOTTYhTA28vXOu97wvehLkDftVu1y8jvu0H2sL
FhnQZxsm6qpF58LsEWwbBk1iqUw1AgMBAAGjgaYwgaMwHwYDVR0jBBgwFoAUc9MQ
RUDCw1Wqr4Xlpt3PJc/B1/MwUQYIKwYBBQUHAQEERTBDMEEGCCsGAQUFBzABhjVo
dHRwOi8vaHAtZGwzODBwZ2VuOC0wMi12bS0zLnRlc3RyZWxtLnRlc3Q6ODAvY2Ev
b2NzcDAOBgNVHQ8BAf8EBAMCBPAwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsGAQUF
BwMCMA0GCSqGSIb3DQEBCwUAA4IBAQB4P24UJO8wEgEqcP5+LxV91QkjSqtiV46A
k+FsIVYTCH6W1gW68WMUb/o6R1Kz6kKE/Kq0befCQgaXeG7HY9BzOJgnzKN8f5Jm
hCnSxyTSGYd1aOv+HVThU6YzwYH2qzvuCEPtZCehT8dkUucVLmd1y8iFirDyMBLp
87MmLNVRhbVlVJY4RW1zd4nWZvhlWdTmN9fb+WoeRY/n0tRmVKj+hW8wfQFDJTZM
q7+OEbtkXdLpEaowyjmZTgLphSlT0qOOq+BdCNjmPuf/QXcz/pYiJ0la6Ua4KvCP
k4MOx32mFsLLAvyKka80KNGPmdJx2mCPKK/EDvl0RvwgYGy0+mHT
-----END CERTIFICATE-----
[root@nec-em6 ~]#

[root@nec-em6 ~]# getcert list -i 20140304104507
Number of certificates and requests being tracked: 7.
Request ID '20140304104507':
	status: MONITORING
	stuck: no
	key pair storage: type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',token='NSS Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'
	certificate: type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',token='NSS Certificate DB'
	CA: dogtag-ipa-retrieve-agent-submit
	issuer: CN=Certificate Authority,O=TESTRELM.TEST
	subject: CN=IPA RA,O=TESTRELM.TEST
	expires: 2016-02-22 17:49:50 UTC
	key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
	eku: id-kp-serverAuth,id-kp-clientAuth
	pre-save command: 
	post-save command: /usr/lib64/ipa/certmonger/restart_httpd
	track: yes
	auto-renew: yes
[root@nec-em6 ~]# getcert resubmit -i 20140304104507
Resubmitting "20140304104507" to "dogtag-ipa-retrieve-agent-submit".
[root@nec-em6 ~]# 

[root@nec-em6 ~]# getcert list -i 20140304104507
Number of certificates and requests being tracked: 7.
Request ID '20140304104507':
	status: MONITORING
	stuck: no
	key pair storage: type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',token='NSS Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'
	certificate: type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',token='NSS Certificate DB'
	CA: dogtag-ipa-retrieve-agent-submit
	issuer: CN=Certificate Authority,O=TESTRELM.TEST
	subject: CN=IPA RA,O=TESTRELM.TEST
	expires: 2016-02-23 08:28:05 UTC
	key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
	eku: id-kp-serverAuth,id-kp-clientAuth
	pre-save command: 
	post-save command: /usr/lib64/ipa/certmonger/restart_httpd
	track: yes
	auto-renew: yes
[root@nec-em6 ~]#

Comment 6 Ludek Smid 2014-06-13 11:19:05 UTC
This request was resolved in Red Hat Enterprise Linux 7.0.

Contact your manager or support representative in case you have further questions about the request.