RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1992439 - Certmonger certificates stuck in NEED_GUIDANCE
Summary: Certmonger certificates stuck in NEED_GUIDANCE
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: certmonger
Version: 7.9
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: rc
: ---
Assignee: Rob Crittenden
QA Contact: ipa-qe
URL:
Whiteboard:
Depends On:
Blocks: 2001079 2001082
TreeView+ depends on / blocked
 
Reported: 2021-08-11 06:04 UTC by David Sedgmen
Modified: 2022-02-25 04:26 UTC (History)
7 users (show)

Fixed In Version: certmonger-0.78.4-16.el7_9
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 2001079 2001082 (view as bug list)
Environment:
Last Closed: 2021-10-12 15:31:05 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker FREEIPA-7005 0 None None None 2021-10-01 11:05:23 UTC
Red Hat Issue Tracker RHELPLAN-93178 0 None None None 2021-08-11 06:04:35 UTC
Red Hat Knowledge Base (Solution) 6759001 0 None None None 2022-02-25 04:26:31 UTC
Red Hat Product Errata RHBA-2021:3808 0 None None None 2021-10-12 15:31:09 UTC

Description David Sedgmen 2021-08-11 06:04:05 UTC 
Description of problem:
When resubmit certificates through certmonger gets stuck in NEED_GUIDANCE.
This seems to be related to how long the certmonger process has been running. 

After restarting the service there is no issue resubmitting certificates

Version-Release number of selected component (if applicable):
Red Hat Enterprise Linux Server release 7.9 (Maipo)
certmonger-0.78.4-14.el7.x86_64

How reproducible:
Hard to tell, it seems to be based on how long the service has been running. 
In the test environment where the certmonger has been up for 8 months. 
Of the 3 of 8 servers we tried to replicate on, we could replicate it each time.

Steps to Reproduce:
1. Submit certificates to IPA through certmonger
2. Wait an undetermined about of time
3. resubmit the certificates 

Actual results:

Request ID 'haproxy-storage_mgmt-cert':
	status: NEED_GUIDANCE
	stuck: yes
	key pair storage: type=FILE,location='/etc/pki/tls/private/haproxy/overcloud-haproxy-storage_mgmt.key'
	certificate: type=FILE,location='/etc/pki/tls/certs/haproxy/overcloud-haproxy-storage_mgmt.crt'
	CA: IPA
	issuer: CN=Certificate Authority,O=REDHAT.LOCAL
	subject: CN=controller-0.storagemgmt.redhat.local,O=REDHAT.LOCAL
	expires: 2023-04-10 05:18:02 UTC
	dns: overcloud.storagemgmt.redhat.local,controller-0.storagemgmt.redhat.local
	principal name: haproxy/controller-0.storagemgmt.redhat.local
	key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
	eku: id-kp-serverAuth,id-kp-clientAuth
	pre-save command: 
	post-save command: /usr/bin/certmonger-haproxy-refresh.sh reload storage_mgmt
	track: yes
	auto-renew: yes

Expected results:

Request ID 'haproxy-storage_mgmt-cert':
	status: MONITORING
	stuck: no
	key pair storage: type=FILE,location='/etc/pki/tls/private/haproxy/overcloud-haproxy-storage_mgmt.key'
	certificate: type=FILE,location='/etc/pki/tls/certs/haproxy/overcloud-haproxy-storage_mgmt.crt'
	CA: IPA
	issuer: CN=Certificate Authority,O=REDHAT.LOCAL
	subject: CN=controller-0.storagemgmt.redhat.local,O=REDHAT.LOCAL
	expires: 2023-04-10 05:18:02 UTC
	dns: overcloud.storagemgmt.redhat.local,controller-0.storagemgmt.redhat.local
	principal name: haproxy/controller-0.storagemgmt.redhat.local
	key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
	eku: id-kp-serverAuth,id-kp-clientAuth
	pre-save command: 
	post-save command: /usr/bin/certmonger-haproxy-refresh.sh reload storage_mgmt
	track: yes
	auto-renew: yes


Additional info:


From what I can see from the strace is there Bad file descriptor when try to read -1, 0x55f57c3507d0, 8192 right after sending the submitting status to dbus. 

After this is goes into NEED_GUIDANCE
~~~
15104 1628051842.134790 close(1022</var/lib/certmonger/requests/20201112060423.tmp>) = 0 <0.000019>
15104 1628051842.134844 munmap(0x7fde2bcec000, 4096) = 0 <0.000022>
15104 1628051842.134895 rename("/var/lib/certmonger/requests/20201112060423.tmp", "/var/lib/certmonger/requests/20201112060423") = 0 <0.000051>
15104 1628051842.135021 sendmsg(7<UNIX:[63674->62964]>, {msg_name=NULL, msg_namelen=0, msg_iov=[{iov_base="l\4\1\1O\0\0\0\230\34\0\0\214\0\0\0\1\1o\0.\0\0\0/org/fedorahosted/certmonger/requests/Request5\0\0\2\1s\0\37\0\0\0org.freedesktop.
DBus.Properties\0\3\1s\0\21\0\0\0PropertiesChanged\0\0\0\0\0\0\0\10\1g\0\6sa{sv}\0\0\0\0\0", iov_len=160}, {iov_base="#\0\0\0org.fedorahosted.certmonger.request\0\37\0\0\0\0\0\0\0\6\0\0\0status\0\1s\0\0\0\n\0\0\0SUBMITTING\0", iov_len=79}
], msg_iovlen=2, msg_controllen=0, msg_flags=0}, MSG_NOSIGNAL) = 239 <0.000025>
15104 1628051842.135103 epoll_wait(3<anon_inode:[eventpoll]>, [], 1, 4999) = 0 <5.000301>
15104 1628051847.135591 read(-1, 0x55f57c3507d0, 8192) = - <0.000030>
15104 1628051847.135691 close(-1)       = -1 EBADF (Bad file descriptor) <0.000022>
15104 1628051847.135752 wait4(0, 0x55f57c32e0bc, 0, NULL) = -1 ECHILD (No child processes) <0.000018>
15104 1628051847.135837 open("/var/lib/certmonger/requests/20201112060423.tmp", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 1022</var/lib/certmonger/requests/20201112060423.tmp> <0.000130>
15104 1628051847.136067 fstat(1022</var/lib/certmonger/requests/20201112060423.tmp>, {st_dev=makedev(252, 2), st_ino=14349509, st_mode=S_IFREG|0600, st_nlink=1, st_uid=0, st_gid=0, st_blksize=4096, st_blocks=0, st_size=0, st_atime=1628051
847 /* 2021-08-04T04:37:27.135056204+0000 */, st_atime_nsec=135056204, st_mtime=1628051847 /* 2021-08-04T04:37:27.135056204+0000 */, st_mtime_nsec=135056204, st_ctime=1628051847 /* 2021-08-04T04:37:27.135056204+0000 */, st_ctime_nsec=1350
56204}) = 0 <0.000023>
15104 1628051847.136171 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fde2bcec000 <0.000028>
~~~
Comment 4 Rob Crittenden 2021-08-11 15:22:53 UTC 
If your reproducer is still running can you see if /etc/sysconfig/certmonger exists? Hopefully it does and bumps up the debug level a bit.

Otherwise the journal should still include some minor output which could be useful.

An lsof of the pid may be useful as well. It can open and rename files ok but a fd of 1022 seems a bit on the edge.
Comment 5 David Sedgmen 2021-08-12 00:54:01 UTC 
Hi Rob

There is no "/etc/sysconfig/certmonger" also from what I have seen there is not journal output either when we hit the issue. 

Also the issue is once we enable debug we won't be able to replicate again for undetermined amount of time. 

I can't get the lsof of the process from that strace, but I get it from one another one we replicated on.
Comment 11 Rob Crittenden 2021-08-12 14:03:20 UTC 
I can see this leak in a Fedora installation I have which, when I first started poking, had 190 open descriptors, most of which are FIFO.

By using the refresh-ca command I can provoke a leak of 8 file descriptors! The CA being refreshed doesn't seem to matter:

# lsof -p `pidof certmonger` | tail -1
certmonge 2841750 root  229r     FIFO               0,13      0t0  8840919 pipe
# getcert refresh-ca -c local
Data for CA 'local' being refreshed.
# lsof -p `pidof certmonger` | tail -1
certmonge 2841750 root  237r     FIFO               0,13      0t0  8843863 pipe
Comment 12 Rob Crittenden 2021-08-12 16:43:12 UTC 
The leak I was seeing in certmonger-0.79.13-1.fc33 is fixed in certmonger-0.79.14-1.fc34.

Will try to reproduce similar in el7.
Comment 13 Rob Crittenden 2021-08-12 20:33:03 UTC 
Upstream PR https://pagure.io/certmonger/pull-request/218
Comment 14 Rob Crittenden 2021-08-16 15:34:09 UTC 
I should note that this PR eliminates *a* fd leak, I don't have confidence that it eliminates the reported leak which I've yet to reproduce with current upstream.
Comment 19 Rob Crittenden 2021-09-03 17:37:08 UTC 
Fixed upstream
master: b4c090d2e12956a2df6157592839936adf4024f4
Comment 26 Rob Crittenden 2021-09-20 21:21:05 UTC 
This is easiest to verify in an IPA installation.

- yum -y install ipa-server-dns lsof
- ipa-server-install <options>
- lsof -p `pidof certmonger`

In prior releases this would be a number in the 120's or higher. With the fixed version this should be 8.
Comment 27 anuja 2021-09-21 08:13:09 UTC 
Verified using : certmonger-0.78.4-16.el7_9.x86_64
 
[root@master ~]# lsof -p `pidof certmonger`
COMMAND     PID USER   FD      TYPE             DEVICE SIZE/OFF     NODE NAME
..
..
certmonge 12175 root    0r      CHR                1,3      0t0     1028 /dev/null
certmonge 12175 root    1u     unix 0xffff925446911dc0      0t0    50844 socket
certmonge 12175 root    2u     unix 0xffff925446911dc0      0t0    50844 socket
certmonge 12175 root    3u  a_inode               0,10        0     6397 [eventpoll]
certmonge 12175 root    4uW     REG              253,1        0 50356946 /var/lib/certmonger/lock
certmonge 12175 root    5u  a_inode               0,10        0     6397 [eventfd]
certmonge 12175 root    6u  netlink                         0t0    50845 ROUTE
certmonge 12175 root    7u     unix 0xffff925446913300      0t0    50846 socket
certmonge 12175 root    8uW     REG               0,20        6    51813 /run/certmonger.pid
[root@master ~]# rpm -qa ipa-server certmonger
certmonger-0.78.4-16.el7_9.x86_64
ipa-server-4.6.8-5.el7_9.9.x86_64
[root@master ~]# 

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Reproduced using: certmonger-0.78.4-15.el7_9.x86_64
[root@master ~]# lsof -p `pidof certmonger`
COMMAND     PID USER   FD      TYPE             DEVICE SIZE/OFF     NODE NAME
..
..
certmonge 12262 root  mem       REG              253,1   163312     7895 /usr/lib64/ld-2.17.so
certmonge 12262 root    0r      CHR                1,3      0t0     1028 /dev/null
certmonge 12262 root    1u     unix 0xffff8fb0f44f9980      0t0    51775 socket
certmonge 12262 root    2u     unix 0xffff8fb0f44f9980      0t0    51775 socket
certmonge 12262 root    3u  a_inode               0,10        0     6397 [eventpoll]
certmonge 12262 root    4uW     REG              253,1        0 75524016 /var/lib/certmonger/lock
certmonge 12262 root    5u  a_inode               0,10        0     6397 [eventfd]
certmonge 12262 root    6u  netlink                         0t0    52539 ROUTE
certmonge 12262 root    7u     unix 0xffff8fb0f7135dc0      0t0    52540 socket
certmonge 12262 root    8uW     REG               0,20        6    52541 /run/certmonger.pid
certmonge 12262 root    9r     FIFO                0,9      0t0    52542 pipe
certmonge 12262 root   10r     FIFO                0,9      0t0    52547 pipe
certmonge 12262 root   11r     FIFO                0,9      0t0    52613 pipe
certmonge 12262 root   12r     FIFO                0,9      0t0    51785 pipe
certmonge 12262 root   13r     FIFO                0,9      0t0    51821 pipe
certmonge 12262 root   14r     FIFO                0,9      0t0    51790 pipe
certmonge 12262 root   15r     FIFO                0,9      0t0    52697 pipe
certmonge 12262 root   16r     FIFO                0,9      0t0    51792 pipe
certmonge 12262 root   17r     FIFO                0,9      0t0    51823 pipe
certmonge 12262 root   18r     FIFO                0,9      0t0    51794 pipe
certmonge 12262 root   19r     FIFO                0,9      0t0    51941 pipe
certmonge 12262 root   20r     FIFO                0,9      0t0    52576 pipe
certmonge 12262 root   21r     FIFO                0,9      0t0    51825 pipe
certmonge 12262 root   22r     FIFO                0,9      0t0    51796 pipe
certmonge 12262 root   23r     FIFO                0,9      0t0    51943 pipe
certmonge 12262 root   24r     FIFO                0,9      0t0    51798 pipe
certmonge 12262 root   25r     FIFO                0,9      0t0    51827 pipe
certmonge 12262 root   26r     FIFO                0,9      0t0    52578 pipe
certmonge 12262 root   27r     FIFO                0,9      0t0    51945 pipe
certmonge 12262 root   28r     FIFO                0,9      0t0    51800 pipe
certmonge 12262 root   29r     FIFO                0,9      0t0    52615 pipe
certmonge 12262 root   30r     FIFO                0,9      0t0    52580 pipe
certmonge 12262 root   31r     FIFO                0,9      0t0    51947 pipe
certmonge 12262 root   32r     FIFO                0,9      0t0    51802 pipe
certmonge 12262 root   33r     FIFO                0,9      0t0    52617 pipe
certmonge 12262 root   34r     FIFO                0,9      0t0    52582 pipe
certmonge 12262 root   35r     FIFO                0,9      0t0    52700 pipe
certmonge 12262 root   36r     FIFO                0,9      0t0    51804 pipe
certmonge 12262 root   37r     FIFO                0,9      0t0    52619 pipe
certmonge 12262 root   38r     FIFO                0,9      0t0    52584 pipe
certmonge 12262 root   39r     FIFO                0,9      0t0    52704 pipe
certmonge 12262 root   40r     FIFO                0,9      0t0    51806 pipe
certmonge 12262 root   41r     FIFO                0,9      0t0    51831 pipe
certmonge 12262 root   42r     FIFO                0,9      0t0    52586 pipe
certmonge 12262 root   43r     FIFO                0,9      0t0    52706 pipe
certmonge 12262 root   44r     FIFO                0,9      0t0    51808 pipe
certmonge 12262 root   45r     FIFO                0,9      0t0    51833 pipe
certmonge 12262 root   46r     FIFO                0,9      0t0    52588 pipe
certmonge 12262 root   47r     FIFO                0,9      0t0    52760 pipe
certmonge 12262 root   48r     FIFO                0,9      0t0    52590 pipe
certmonge 12262 root   49r     FIFO                0,9      0t0    51835 pipe
certmonge 12262 root   50r     FIFO                0,9      0t0    52592 pipe
certmonge 12262 root   51r     FIFO                0,9      0t0    52004 pipe
certmonge 12262 root   52r     FIFO                0,9      0t0    52594 pipe
certmonge 12262 root   53r     FIFO                0,9      0t0    51838 pipe
certmonge 12262 root   54r     FIFO                0,9      0t0    52596 pipe
certmonge 12262 root   55r     FIFO                0,9      0t0    58388 pipe
certmonge 12262 root   56r     FIFO                0,9      0t0    52598 pipe
certmonge 12262 root   57r     FIFO                0,9      0t0    51842 pipe
certmonge 12262 root   58r     FIFO                0,9      0t0    51813 pipe
certmonge 12262 root   59r     FIFO                0,9      0t0    52006 pipe
certmonge 12262 root   60r     FIFO                0,9      0t0    51815 pipe
certmonge 12262 root   61r     FIFO                0,9      0t0    51844 pipe
certmonge 12262 root   62r     FIFO                0,9      0t0    51817 pipe
certmonge 12262 root   63r     FIFO                0,9      0t0    58105 pipe
certmonge 12262 root   64r     FIFO                0,9      0t0    52602 pipe
certmonge 12262 root   65r     FIFO                0,9      0t0    52626 pipe
certmonge 12262 root   66r     FIFO                0,9      0t0    52604 pipe
certmonge 12262 root   67r     FIFO                0,9      0t0    52008 pipe
certmonge 12262 root   68r     FIFO                0,9      0t0    52606 pipe
certmonge 12262 root   69r     FIFO                0,9      0t0    52630 pipe
certmonge 12262 root   70r     FIFO                0,9      0t0    52608 pipe
certmonge 12262 root   71r     FIFO                0,9      0t0    58364 pipe
certmonge 12262 root   72r     FIFO                0,9      0t0    51848 pipe
certmonge 12262 root   73r     FIFO                0,9      0t0    52010 pipe
certmonge 12262 root   74r     FIFO                0,9      0t0    51850 pipe
certmonge 12262 root   75r     FIFO                0,9      0t0    58390 pipe
certmonge 12262 root   76r     FIFO                0,9      0t0    51855 pipe
certmonge 12262 root   77r     FIFO                0,9      0t0    52012 pipe
certmonge 12262 root   78r     FIFO                0,9      0t0    51861 pipe
certmonge 12262 root   79r     FIFO                0,9      0t0    58500 pipe
certmonge 12262 root   80r     FIFO                0,9      0t0    51863 pipe
certmonge 12262 root   81r     FIFO                0,9      0t0    52014 pipe
certmonge 12262 root   82r     FIFO                0,9      0t0    51866 pipe
certmonge 12262 root   83r     FIFO                0,9      0t0    58107 pipe
certmonge 12262 root   84r     FIFO                0,9      0t0    51868 pipe
certmonge 12262 root   85r     FIFO                0,9      0t0    52016 pipe
certmonge 12262 root   86r     FIFO                0,9      0t0    51874 pipe
certmonge 12262 root   87r     FIFO                0,9      0t0    62188 pipe
certmonge 12262 root   88r     FIFO                0,9      0t0    58109 pipe
certmonge 12262 root   89r     FIFO                0,9      0t0    58366 pipe
certmonge 12262 root   90r     FIFO                0,9      0t0    58111 pipe
certmonge 12262 root   91r     FIFO                0,9      0t0    60849 pipe
certmonge 12262 root   92r     FIFO                0,9      0t0    58113 pipe
certmonge 12262 root   93r     FIFO                0,9      0t0    58368 pipe
certmonge 12262 root   94r     FIFO                0,9      0t0    58115 pipe
certmonge 12262 root   95r     FIFO                0,9      0t0    62238 pipe
certmonge 12262 root   96r     FIFO                0,9      0t0    59394 pipe
certmonge 12262 root   97r     FIFO                0,9      0t0    62190 pipe
certmonge 12262 root   98r     FIFO                0,9      0t0    59404 pipe
certmonge 12262 root   99r     FIFO                0,9      0t0    60883 pipe
certmonge 12262 root  100r     FIFO                0,9      0t0    58527 pipe
certmonge 12262 root  101r     FIFO                0,9      0t0    62192 pipe
certmonge 12262 root  102r     FIFO                0,9      0t0    59406 pipe
certmonge 12262 root  103r     FIFO                0,9      0t0    60914 pipe
certmonge 12262 root  104r     FIFO                0,9      0t0    62194 pipe
certmonge 12262 root  105r     FIFO                0,9      0t0    62240 pipe
certmonge 12262 root  106r     FIFO                0,9      0t0    62196 pipe
certmonge 12262 root  107r     FIFO                0,9      0t0    62273 pipe
certmonge 12262 root  108r     FIFO                0,9      0t0    62198 pipe
certmonge 12262 root  109r     FIFO                0,9      0t0    62242 pipe
certmonge 12262 root  110r     FIFO                0,9      0t0    62200 pipe
certmonge 12262 root  111r     FIFO                0,9      0t0    62479 pipe
certmonge 12262 root  112r     FIFO                0,9      0t0    62244 pipe
certmonge 12262 root  113r     FIFO                0,9      0t0    62275 pipe
certmonge 12262 root  114r     FIFO                0,9      0t0    62246 pipe
certmonge 12262 root  115r     FIFO                0,9      0t0    62481 pipe
certmonge 12262 root  116r     FIFO                0,9      0t0    62248 pipe
certmonge 12262 root  117r     FIFO                0,9      0t0    62277 pipe
certmonge 12262 root  118r     FIFO                0,9      0t0    62258 pipe
certmonge 12262 root  119r     FIFO                0,9      0t0    62483 pipe
certmonge 12262 root  120r     FIFO                0,9      0t0    62279 pipe
certmonge 12262 root  121r     FIFO                0,9      0t0    62485 pipe
certmonge 12262 root  122r     FIFO                0,9      0t0    62281 pipe
certmonge 12262 root  123r     FIFO                0,9      0t0    62487 pipe
certmonge 12262 root  124r     FIFO                0,9      0t0    62283 pipe
certmonge 12262 root  125r     FIFO                0,9      0t0    61432 pipe
certmonge 12262 root  126r     FIFO                0,9      0t0    62285 pipe
certmonge 12262 root  127r     FIFO                0,9      0t0    61436 pipe
certmonge 12262 root  128r     FIFO                0,9      0t0    63640 pipe
certmonge 12262 root  129r     FIFO                0,9      0t0    61438 pipe
certmonge 12262 root  130r     FIFO                0,9      0t0    62547 pipe
certmonge 12262 root  132r     FIFO                0,9      0t0    62549 pipe
certmonge 12262 root  134r     FIFO                0,9      0t0    62551 pipe
certmonge 12262 root  136r     FIFO                0,9      0t0    62553 pipe
certmonge 12262 root  138r     FIFO                0,9      0t0    62555 pipe
certmonge 12262 root  140r     FIFO                0,9      0t0    63642 pipe
certmonge 12262 root  142r     FIFO                0,9      0t0    63644 pipe
[root@master ~]# rpm -qa ipa-server certmonger
ipa-server-4.6.8-5.el7_9.7.x86_64
certmonger-0.78.4-15.el7_9.x86_64
[root@master ~]#
Comment 31 errata-xmlrpc 2021-10-12 15:31:05 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (certmonger bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:3808
Note You need to log in before you can comment on or make changes to this bug.