This bug has been migrated to another issue tracking site. It has been closed here and may no longer be being monitored.

If you would like to get updates for this issue, or to participate in it, you may do so at Red Hat Issue Tracker .
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 2136335 - rhc connect does not consider what state rhcd is in
Summary: rhc connect does not consider what state rhcd is in
Keywords:
Status: CLOSED MIGRATED
Alias: None
Product: Red Hat Enterprise Linux 9
Classification: Red Hat
Component: rhc
Version: 9.0
Hardware: Unspecified
OS: Unspecified
medium
low
Target Milestone: rc
: ---
Assignee: CSI Client Tools Bugs
QA Contact: Red Hat subscription-manager QE Team
URL:
Whiteboard: SCRUB_20221027
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2022-10-20 01:26 UTC by Paul Dudley
Modified: 2024-01-06 04:29 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2023-09-07 13:48:40 UTC
Type: Story
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker   RHEL-2490 0 None Migrated None 2023-09-07 13:48:34 UTC
Red Hat Issue Tracker RHELPLAN-137115 0 None None None 2022-10-20 01:29:50 UTC

Description Paul Dudley 2022-10-20 01:26:16 UTC
Description of problem:
`rhc connect` will report a green circle and report that it 'Activated rhcd' even if rhcd is in a failed state either before or after the connect was run.

For example:
~~~
[root@rhel9new ~]# systemctl list-units --failed
  UNIT                            LOAD   ACTIVE SUB    DESCRIPTION                                   
● rhcd.service                    loaded failed failed Red Hat connector daemon

===

[root@rhel9new ~]# rhc connect
Connecting rhel9new to Red Hat.
This might take a few seconds.

● This system is already connected to Red Hat Subscription Management
● Connected to Red Hat Insights
● Activated the Red Hat connector daemon

Manage your Red Hat connector systems: https://red.ht/connector

===

[root@rhel9new ~]# systemctl list-units --failed
  UNIT                            LOAD   ACTIVE SUB    DESCRIPTION                                   
● rhcd.service                    loaded failed failed Red Hat connector daemon
~~~

Expectations:
Even if `rhc connect` can't report complete status of rhcd, it should at least point in the direction of an issue existing if the service is in a failed state both before and after running it

Comment 1 Link Dupont 2022-10-21 18:09:34 UTC
rhc does check the ActiveState of the unit after starting it.

https://github.com/RedHatInsights/rhc/blob/main/activate.go#L31-L34

>	properties, err := conn.GetUnitProperties(unitName)
>	activeState := properties["ActiveState"]
>	if activeState.(string) != "active" {
>		return fmt.Errorf("error: The unit %v failed to start. Run 'systemctl status %v' for more information", unitName, unitName)
>	}

Can you attach the output of `journalctl -u rhcd`? I'm wondering if the unit is starting successfully, is active for the moment that GetUnitProperties is called, and then exits afterwards.

Comment 2 Paul Dudley 2022-10-21 20:16:26 UTC
Hey Link,

Sure, here are two examples of failures, one where the failure comes 30 seconds after a start due to network issues and another is a config issue that causes the service to go into failed almost immediately:
~~~
From `journalctl -xfeu rhcd`
Oct 21 15:57:29 rhel9new systemd[1]: Started Red Hat connector daemon.
░░ Subject: A start job for unit rhcd.service has finished successfully
░░ Defined-By: systemd
░░ Support: https://access.redhat.com/support
░░ 
░░ A start job for unit rhcd.service has finished successfully.
░░ 
░░ The job identifier is 50490.
Oct 21 15:57:29 rhel9new rhcd[82810]: [rhcd] 2022/10/21 15:57:29 /builddir/build/BUILD/rhc/yggdrasil-0.2.1/cmd/yggd/main.go:160: starting rhcd version 0.2.1
Oct 21 15:57:29 rhel9new rhcd[82810]: [rhcd] 2022/10/21 15:57:29 /builddir/build/BUILD/rhc/yggdrasil-0.2.1/cmd/yggd/main.go:162: attempting to kill any orphaned workers
Oct 21 15:57:29 rhel9new rhcd[82810]: [rhcd] 2022/10/21 15:57:29 /builddir/build/BUILD/rhc/yggdrasil-0.2.1/cmd/yggd/main.go:209: listening on socket: @yggd-dispatcher-tvnKen
Oct 21 15:57:59 rhel9new rhcd[82810]: cannot connect to broker: network Error : dial tcp 10.10.10.10:3128: i/o timeout
Oct 21 15:57:59 rhel9new systemd[1]: rhcd.service: Main process exited, code=exited, status=1/FAILURE
░░ Subject: Unit process exited
░░ Defined-By: systemd
░░ Support: https://access.redhat.com/support
░░ 
░░ An ExecStart= process belonging to unit rhcd.service has exited.
░░ 
░░ The process' exit code is 'exited' and its exit status is 1.
Oct 21 15:57:59 rhel9new systemd[1]: rhcd.service: Failed with result 'exit-code'.
░░ Subject: Unit failed
░░ Defined-By: systemd
░░ Support: https://access.redhat.com/support
░░ 
░░ The unit rhcd.service has entered the 'failed' state with result 'exit-code'.
Oct 21 15:59:16 rhel9new systemd[1]: Started Red Hat connector daemon.
░░ Subject: A start job for unit rhcd.service has finished successfully
░░ Defined-By: systemd
░░ Support: https://access.redhat.com/support
░░ 
░░ A start job for unit rhcd.service has finished successfully.
░░ 
░░ The job identifier is 50846.
Oct 21 15:59:16 rhel9new rhcd[84105]: 2022/10/21 15:59:16 Unable to load TOML file '/etc/rhc/config.toml': inner error:
Oct 21 15:59:16 rhel9new rhcd[84105]: 'Near line 8 (last key parsed 'env'): expected a comma or array terminator ']', but got '\x00' instead'
Oct 21 15:59:16 rhel9new systemd[1]: rhcd.service: Main process exited, code=exited, status=1/FAILURE
░░ Subject: Unit process exited
░░ Defined-By: systemd
░░ Support: https://access.redhat.com/support
░░ 
░░ An ExecStart= process belonging to unit rhcd.service has exited.
░░ 
░░ The process' exit code is 'exited' and its exit status is 1.
Oct 21 15:59:16 rhel9new systemd[1]: rhcd.service: Failed with result 'exit-code'.
░░ Subject: Unit failed
░░ Defined-By: systemd
░░ Support: https://access.redhat.com/support
░░ 
░░ The unit rhcd.service has entered the 'failed' state with result 'exit-code'.
~~~

In testing both of these scenarios by stopping rhcd and letting `rhc connect` start it up, it still retrieves '● Activated the Red Hat connector daemon'

Comment 3 Link Dupont 2022-10-26 16:17:40 UTC
I was able to reproduce this behavior with rhc-0.2.1-9.el9 easily by breaking the syntax of the config file. This causes rhcd to exit.

Steps to Reproduce
==================

1. systemctl disable --now rhcd
2. echo "blah = " >> /etc/rhc/config.toml
3. rhc connect

Expected Results
================

Output should correctly report whether the service started successfully or not.

Actual Results
==============

Output reports the service was successfully activated. If the service subsequently aborts, that exit status is not reported.

Notes
=====

Technically, this is not a bug. rhc is correct when it reports that the 'rhcd' unit was activated. It doesn't check that the service is still running, or that it successfully started up. It just reports "yep, I told systemd to start the unit, and systemd told me it started". However, I agree that it's a flaw in the UX of the 'rhc connect' operation. Users rightly assume that a message to the effect of "Started a systemd service" with a green checkbox next to it should mean that the service is actively running. To fix this, we'll probably need to block the main routine of rhc until a signal is received from systemd indicating that the service is fully up and running. In order to do *that*, we'll need to enhance rhcd to use the systemd notify protocol to accurately report the service's status as "up and running". This has already been filed as a feature enhancement upstream[1].

1: https://github.com/RedHatInsights/yggdrasil/issues/56

Comment 5 RHEL Program Management 2023-09-07 13:08:00 UTC
Issue migration from Bugzilla to Jira is in process at this time. This will be the last message in Jira copied from the Bugzilla bug.

Comment 6 RHEL Program Management 2023-09-07 13:48:40 UTC
This BZ has been automatically migrated to the issues.redhat.com Red Hat Issue Tracker. All future work related to this report will be managed there.

Due to differences in account names between systems, some fields were not replicated.  Be sure to add yourself to Jira issue's "Watchers" field to continue receiving updates and add others to the "Need Info From" field to continue requesting information.

To find the migrated issue, look in the "Links" section for a direct link to the new issue location. The issue key will have an icon of 2 footprints next to it, and begin with "RHEL-" followed by an integer.  You can also find this issue by visiting https://issues.redhat.com/issues/?jql= and searching the "Bugzilla Bug" field for this BZ's number, e.g. a search like:

"Bugzilla Bug" = 1234567

In the event you have trouble locating or viewing this issue, you can file an issue by sending mail to rh-issues.

Comment 7 Red Hat Bugzilla 2024-01-06 04:29:20 UTC
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 120 days


Note You need to log in before you can comment on or make changes to this bug.