Bug 1578792

Summary:	SSA performed on RHEL VM counts duplicate services
Product:	Red Hat CloudForms Management Engine	Reporter:	Jan Zmeskal <jzmeskal>
Component:	SmartState Analysis	Assignee:	Hui Song <hsong>
Status:	CLOSED ERRATA	QA Contact:	Angelina Vasileva <anikifor>
Severity:	medium	Docs Contact:
Priority:	medium
Version:	5.9.0	CC:	cpelland, obarenbo, roliveri, simaishi
Target Milestone:	GA
Target Release:	5.10.0
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	5.10.0.1	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2019-02-07 23:02:55 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	RHEVM	Target Upstream Version:
Embargoed:

Description Jan Zmeskal 2018-05-16 11:59:05 UTC

Description of problem:
The way counting of services is implemented for distros with systemd (such as RHEL >=7) makes it possible to count _links_ as individual services. This leads to a situation where a link and the actual *.service file that is target of such link are counted as two distinct services.

Version-Release number of selected component (if applicable):
CFME 5.9.2.4
manageiq-smartstate-0.2.10

How reproducible:
100 %

Steps to Reproduce:
1. Have a RHV as infrastructure provider. Have a VM on that provider with RHEL >=7.
2. SSH into your RHEL VM. Make sure that there is at least one link in /etc/systemd/system that points to some .service file in /usr/lib/systemd/system. In my example, let's take a look at this one:

# ls -l /etc/systemd/system/dbus-org.freedesktop.NetworkManager.service
lrwxrwxrwx. 1 root root 46 Sep 5 2016 /etc/systemd/system/dbus-org.freedesktop.NetworkManager.service -> /usr/lib/systemd/system/NetworkManager.service

3. Go to CFME UI and perform SSA on your VM. You can see the attached evm.log for my case. Wait for SSA to finish.
4. Go to VM's detail page in CFME UI. Click Configuration -> Init processes.
5. Now try to search for the link path. In my case, it was /etc/systemd/system/dbus-org.freedesktop.NetworkManager.service. I can locate that path among "Init processes"
6. Then try to look for path that the link is pointing to. In my case it was /usr/lib/systemd/system/NetworkManager.service. I can locate that entry in "Init processes" as well.

Actual results:
Both link and its actual target are included among "Init processes" even though that is actually only one service.

Expected results:
I would expect only unique service entries to be included there. Links should not be accounted for.

Additional info:
If you take a look at this module (https://github.com/ManageIQ/manageiq-smartstate/blob/4a746f3552bf35a0514920d79f8b5a46fba2a50a/lib/metadata/linux/LinuxSystemd.rb) you can see that gathering of services during SSA of VM is done by counting numbers of files with .service externsion as far as I can understand. I am just wondering why don't we use command 'systemctl list-unit-files --type=service' there, which I believe is already doing that for us.

I found about this bug while inspecting VM with RHEL7.3, but I believe this is completely independent of specific RHEL version as long as that RHEL is using systemd.

Comment 3 CFME Bot 2018-05-17 20:08:38 UTC

https://github.com/ManageIQ/manageiq-smartstate/pull/73

Comment 4 CFME Bot 2018-05-22 17:54:27 UTC

New commit detected on ManageIQ/manageiq-smartstate/master:

https://github.com/ManageIQ/manageiq-smartstate/commit/79087ebda6f000bb8e5c3da6173f74e419885343
commit 79087ebda6f000bb8e5c3da6173f74e419885343
Author:     hsong-rh <hsong>
AuthorDate: Thu May 17 15:47:12 2018 -0400
Commit:     hsong-rh <hsong>
CommitDate: Thu May 17 15:47:12 2018 -0400

    Fix to remove duplicated records in services list after SSA

    https://bugzilla.redhat.com/show_bug.cgi?id=1578792

 lib/metadata/linux/LinuxSystemd.rb | 10 +
 1 file changed, 10 insertions(+)

Comment 5 Jan Zmeskal 2018-05-23 07:11:43 UTC

(In reply to CFME Bot from comment #4)
> New commit detected on ManageIQ/manageiq-smartstate/master:
> 
> https://github.com/ManageIQ/manageiq-smartstate/commit/
> 79087ebda6f000bb8e5c3da6173f74e419885343
> commit 79087ebda6f000bb8e5c3da6173f74e419885343
> Author:     hsong-rh <hsong>
> AuthorDate: Thu May 17 15:47:12 2018 -0400
> Commit:     hsong-rh <hsong>
> CommitDate: Thu May 17 15:47:12 2018 -0400
> 
>     Fix to remove duplicated records in services list after SSA
> 
>     https://bugzilla.redhat.com/show_bug.cgi?id=1578792
> 
>  lib/metadata/linux/LinuxSystemd.rb | 10 +
>  1 file changed, 10 insertions(+)

I am not sure about the proposed solution. Let's think about a scenario in which user links their own service by using this:
https://www.freedesktop.org/software/systemd/man/systemctl.html#link%20PATH%E2%80%A6
Then even though such service is started after boot, it would be disregarded by SSA.

Comment 6 Hui Song 2018-05-23 14:06:52 UTC

Hi，Jan:

Our SSA works not only for running VMs, but also for those images/templates. This means:
1. We cannot run any system commands to query services;
2. We cannot parse the whole file system to filter out services, for performance reason;

So currently SSA only lists those services located in the default system directories.

Comment 7 Rich Oliveri 2018-05-23 15:32:43 UTC

Aren't links skipped only if we also see the target of the link?

"Link a unit file that is not in the unit file search paths into the unit file search path"

So, if the target isn't in the unit file search path, and the link is, we will count it.

Comment 8 Jan Zmeskal 2018-05-24 09:56:21 UTC

(In reply to Rich Oliveri from comment #7)
> Aren't links skipped only if we also see the target of the link?
> 
> "Link a unit file that is not in the unit file search paths into the unit
> file search path"
> 
> So, if the target isn't in the unit file search path, and the link is, we
> will count it.

If that is the case, then it's of course good. But I am not sure about if target of the link is considered at all. I am however not skilled in Ruby. Is that the way it works, Hui?

Comment 9 Hui Song 2018-05-24 13:13:14 UTC

Yes, Jan. All the links in the unit file search path will be counted.

Comment 11 Jan Zmeskal 2018-09-06 12:10:47 UTC

I followed reproduction steps and verified that links like that are not counted any more. I also created symlink in /usr/lib/systemd/system on tested VM like this:

cd /usr/lib/systemd/system
ln -sf sshd.service something.service

I ran SSA again and made sure that symlink something.service has not been accounted for.

Verified on version: 5.10.0.14

Comment 12 errata-xmlrpc 2019-02-07 23:02:55 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2019:0212