Bug 1499161 - If a container image SmartState Analysis fails, the image is still marked as compliant
Summary: If a container image SmartState Analysis fails, the image is still marked as ...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat CloudForms Management Engine
Classification: Red Hat
Component: Control
Version: 5.8.0
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: GA
: 5.10.0
Assignee: Lucy Fu
QA Contact: juwatts
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-10-06 09:40 UTC by Peter McGowan
Modified: 2019-02-07 23:03 UTC (History)
9 users (show)

Fixed In Version: 5.10.0.25
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-02-07 23:02:54 UTC
Category: ---
Cloudforms Team: CFME Core
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2019:0212 None None None 2019-02-07 23:03:07 UTC

Description Peter McGowan 2017-10-06 09:40:53 UTC
Description of problem:
If the SmartState Analysis of a container image fails, and the default OpenSCAP profile is assigned to the provider, then the container image will still be marked as compliant after the failed scan.

Version-Release number of selected component (if applicable):
5.8.1.5

How reproducible:
Every time


Steps to Reproduce:
1. Assign the default OpenSCAP profile to an OpenShift Container Platform provider
2. Remove the rights for the system:serviceaccount:management-infra:management-admin OpenShift user to be able to create in pods in project "management-infra"
3. Initiate a SmartState Analysis of a container image
4. Confirm that the following is seen in evm.log: "pod creation for [management-infra/manageiq-img-scan-b2b50] failed: [HTTP status code 403, User "system:serviceaccount:management-infra:management-admin" cannot create pods in project "management-infra"]"

Actual results:
The image is still marked as "Compliant as of 1 minute ago"

Expected results:
The image should be marked with a compliance status of 'Verification Failed' or 'Never Verified'


Additional info:

Comment 4 CFME Bot 2018-08-30 20:26:27 UTC
New commits detected on ManageIQ/manageiq/master:

https://github.com/ManageIQ/manageiq/commit/4a0578426aac62197bda2de035bd6a037cbc6910
commit 4a0578426aac62197bda2de035bd6a037cbc6910
Author:     Lucy Fu <lufu@redhat.com>
AuthorDate: Mon Oct 16 14:25:32 2017 -0400
Commit:     Lucy Fu <lufu@redhat.com>
CommitDate: Mon Oct 16 14:25:32 2017 -0400

    Get rid of the condition modifier.

    https://bugzilla.redhat.com/show_bug.cgi?id=1499161

 app/models/condition.rb | 9 +-
 app/models/miq_policy.rb | 1 -
 db/fixtures/miq_policy_sets.yml | 28 +-
 product/policy/built_in_policies.yml | 4 -
 spec/factories/condition.rb | 1 -
 spec/models/condition_spec.rb | 13 +
 6 files changed, 33 insertions(+), 23 deletions(-)


https://github.com/ManageIQ/manageiq/commit/9938c089fe789b95b57fa9512a4fb4e9d64dac67
commit 9938c089fe789b95b57fa9512a4fb4e9d64dac67
Author:     Lucy Fu <lufu@redhat.com>
AuthorDate: Mon Oct 16 14:29:01 2017 -0400
Commit:     Lucy Fu <lufu@redhat.com>
CommitDate: Mon Oct 16 14:29:01 2017 -0400

    Always apply actions when condition is met for compliance policy.

    https://bugzilla.redhat.com/show_bug.cgi?id=1499161

 app/models/miq_action.rb | 21 +-
 app/models/miq_policy.rb | 10 +-
 spec/models/miq_policy_spec.rb | 20 +-
 3 files changed, 31 insertions(+), 20 deletions(-)

Comment 6 CFME Bot 2018-09-10 18:39:16 UTC
New commit detected on ManageIQ/manageiq-schema/master:

https://github.com/ManageIQ/manageiq-schema/commit/9007d08f3559d4b8649be2cf24165d2830b69364
commit 9007d08f3559d4b8649be2cf24165d2830b69364
Author:     Lucy Fu <lufu@redhat.com>
AuthorDate: Mon Oct 16 16:17:45 2017 -0400
Commit:     Lucy Fu <lufu@redhat.com>
CommitDate: Mon Oct 16 16:17:45 2017 -0400

    Get rid of the condition modifier which is not needed and confusing.

    https://bugzilla.redhat.com/show_bug.cgi?id=1499161

 db/migrate/20171031200739_drop_condition_modifier.rb | 31 +
 spec/migrations/20171031200739_drop_condition_modifier_spec.rb | 44 +
 2 files changed, 75 insertions(+)

Comment 7 juwatts 2018-11-01 17:19:11 UTC
Tested in:
5.10.0.22.20181030184024_26956a0

Recreation steps:
1) Removed admin rights for system:serviceaccount:management-infra:management-admin
2) Assigned the default OpenSCAP profile to an OpenShift Container Platform provider
3) Picked a Container image and verified that the status of compliance was "Never Verified"
4)Initiated a SSA scan
5) Observed the task failed with the following message:
pod creation for [management-infra/manageiq-img-scan-420f4] failed

6) Checked the evm log, observed the following:
[----] E, [2018-11-01T13:01:12.631503 #2423:d18f80] ERROR -- : Q-task_id([job_dispatcher]) MIQ(ManageIQ::Providers::Kubernetes::ContainerManager::Scanning::Job#start) pod creation for [management-infra/manageiq-img-scan-43018] failed: [HTTP status code 403, pods is forbidden: User "system:serviceaccount:management-infra:management-admin" cannot create pods in the namespace "management-infra": User "system:serviceaccount:management-infra:management-admin" cannot create pods in project "management-infra"]
[----] I, [2018-11-01T13:01:12.643278 #2423:d18f80]  INFO -- : Q-task_id([job_dispatcher]) MIQ(MiqQueue.put) Message id: [7923],  id: [], Zone: [default], Role: [smartstate], Server: [], MiqTask id: [], Ident: [generic], Target id: [], Instance id: [6], Task id: [43018f3f-7b84-415a-b328-278ef5ceaf99], Command: [Job.signal], Timeout: [600], Priority: [20], State: [ready], Deliver On: [], Data: [], Args: [:abort_job, "pod creation for [management-infra/manageiq-img-scan-43018] failed", "error"]
[----] I, [2018-11-01T13:01:12.643966 #2423:d18f80]  INFO -- : Q-task_id([job_dispatcher]) MIQ(MiqQueue#delivered) Message id: [7922], State: [ok], Delivered in [0.143190521] seconds
[----] I, [2018-11-01T13:01:17.401157 #11013:d18f80]  INFO -- : MIQ(MiqServer#populate_queue_messages) Fetched 1 miq_queue rows for queue_name=generic, wcount=4, priority=200
[----] I, [2018-11-01T13:01:17.673192 #2423:d18f80]  INFO -- : MIQ(MiqPriorityWorker::Runner#get_message_via_drb) Message id: [7923], MiqWorker id: [45], Zone: [default], Role: [smartstate], Server: [], MiqTask id: [], Ident: [generic], Target id: [], Instance id: [6], Task id: [43018f3f-7b84-415a-b328-278ef5ceaf99], Command: [Job.signal], Timeout: [600], Priority: [20], State: [dequeue], Deliver On: [], Data: [], Args: [:abort_job, "pod creation for [management-infra/manageiq-img-scan-43018] failed", "error"], Dequeued in: [5.035763996] seconds
[----] I, [2018-11-01T13:01:17.673400 #2423:d18f80]  INFO -- : Q-task_id([job_dispatcher]) MIQ(MiqQueue#deliver) Message id: [7923], Delivering...
[----] I, [2018-11-01T13:01:17.723190 #2423:d18f80]  INFO -- : Q-task_id([job_dispatcher]) MIQ(MiqQueue.put) Message id: [7924],  id: [], Zone: [default], Role: [automate], Server: [], MiqTask id: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqAeEngine.deliver], Timeout: [3600], Priority: [20], State: [ready], Deliver On: [], Data: [], Args: [{:object_type=>"ContainerImage", :object_id=>320, :attrs=>{:event_type=>"containerimage_scan_complete", "MiqEvent::miq_event"=>787, :miq_event_id=>787, "EventStream::event_stream"=>787, :event_stream_id=>787}, :instance_name=>"Event", :user_id=>1, :miq_group_id=>1, :tenant_id=>1, :automate_message=>nil}]
[----] I, [2018-11-01T13:01:17.763697 #2423:d18f80]  INFO -- : Q-task_id([job_dispatcher]) MIQ(ManageIQ::Providers::Kubernetes::ContainerManager::Scanning::Job#delete_pod) pod management-infra/manageiq-img-scan-43018 not found, skipping delete
[----] E, [2018-11-01T13:01:17.763909 #2423:d18f80] ERROR -- : Q-task_id([job_dispatcher]) MIQ(ManageIQ::Providers::Kubernetes::ContainerManager::Scanning::Job#process_abort) job aborting, pod creation for [management-infra/manageiq-img-scan-43018] failed
[----] I, [2018-11-01T13:01:17.794013 #2423:d18f80]  INFO -- : Q-task_id([job_dispatcher]) MIQ(ManageIQ::Providers::Kubernetes::ContainerManager::Scanning::Job#finish) Dispatch Status is 'finished'
[----] I, [2018-11-01T13:01:17.806036 #2423:d18f80]  INFO -- : Q-task_id([job_dispatcher]) MIQ(ManageIQ::Providers::Kubernetes::ContainerManager::Scanning::Job#process_finished) job finished, pod creation for [management-infra/manageiq-img-scan-43018] failed

7) Checked the compliance status of the image after the failed scan and it was "Compliant as of 3 Minutes Ago"

Moving bug back to ON_DEV due to failed verification

Comment 8 Lucy Fu 2018-11-01 20:45:16 UTC
What is the IP of your appliance? May I take a look at it?
Thanks.

Comment 18 CFME Bot 2018-11-13 02:30:48 UTC
New commits detected on ManageIQ/manageiq/master:

https://github.com/ManageIQ/manageiq/commit/25460e7776aba69d482b2f23d726bfff0bebe751
commit 25460e7776aba69d482b2f23d726bfff0bebe751
Author:     Lucy Fu <lufu@redhat.com>
AuthorDate: Mon Nov 12 11:34:58 2018 -0500
Commit:     Lucy Fu <lufu@redhat.com>
CommitDate: Mon Nov 12 11:34:58 2018 -0500

    Fix OpenSCAP policy name and description.

    https://bugzilla.redhat.com/show_bug.cgi?id=1499161

 db/fixtures/miq_policy_sets.yml | 4 +-
 1 file changed, 2 insertions(+), 2 deletions(-)


https://github.com/ManageIQ/manageiq/commit/5b674f6d6f6ac05649a9d101cfca21f40690def4
commit 5b674f6d6f6ac05649a9d101cfca21f40690def4
Author:     Lucy Fu <lufu@redhat.com>
AuthorDate: Mon Nov 12 11:36:02 2018 -0500
Commit:     Lucy Fu <lufu@redhat.com>
CommitDate: Mon Nov 12 11:36:02 2018 -0500

    Add event containerimage_scan_abort.

    https://bugzilla.redhat.com/show_bug.cgi?id=1499161

 db/fixtures/miq_event_definitions.csv | 1 +
 1 file changed, 1 insertion(+)

Comment 19 CFME Bot 2018-11-13 02:33:31 UTC
New commit detected on ManageIQ/manageiq-providers-kubernetes/master:

https://github.com/ManageIQ/manageiq-providers-kubernetes/commit/2b36305babe882de8f421e58b0fb4cb5d4419fc0
commit 2b36305babe882de8f421e58b0fb4cb5d4419fc0
Author:     Lucy Fu <lufu@redhat.com>
AuthorDate: Mon Nov 12 11:40:28 2018 -0500
Commit:     Lucy Fu <lufu@redhat.com>
CommitDate: Mon Nov 12 11:40:28 2018 -0500

    Raise containerimage_scan_complete event only when scan is finished without error.

    https://bugzilla.redhat.com/show_bug.cgi?id=1499161

 app/models/manageiq/providers/kubernetes/container_manager/scanning/job.rb | 11 +-
 spec/models/manageiq/providers/kubernetes/container_manager/scanning/job_spec.rb | 1 +
 2 files changed, 7 insertions(+), 5 deletions(-)

Comment 20 CFME Bot 2018-11-13 14:10:48 UTC
New commit detected on ManageIQ/manageiq/hammer:

https://github.com/ManageIQ/manageiq/commit/467ee8ee30e9f096fa65987b25e630b6aa1d3e6b
commit 467ee8ee30e9f096fa65987b25e630b6aa1d3e6b
Author:     Greg McCullough <gmccullo@redhat.com>
AuthorDate: Mon Nov 12 21:26:57 2018 -0500
Commit:     Greg McCullough <gmccullo@redhat.com>
CommitDate: Mon Nov 12 21:26:57 2018 -0500

    Merge pull request #18189 from lfu/openscap_1499161

    Fix issues with OpenSCAP policy

    (cherry picked from commit 22baeb0722ef50b69b350f63c8ff6289c43f61d6)

    https://bugzilla.redhat.com/show_bug.cgi?id=1499161

 db/fixtures/miq_event_definitions.csv | 1 +
 db/fixtures/miq_policy_sets.yml | 4 +-
 2 files changed, 3 insertions(+), 2 deletions(-)

Comment 21 CFME Bot 2018-11-13 14:11:58 UTC
New commit detected on ManageIQ/manageiq-providers-kubernetes/hammer:

https://github.com/ManageIQ/manageiq-providers-kubernetes/commit/a5a45aa5a4f77a37d81587083b941b090ee29987
commit a5a45aa5a4f77a37d81587083b941b090ee29987
Author:     Greg McCullough <gmccullo@redhat.com>
AuthorDate: Mon Nov 12 21:31:59 2018 -0500
Commit:     Greg McCullough <gmccullo@redhat.com>
CommitDate: Mon Nov 12 21:31:59 2018 -0500

    Merge pull request #303 from lfu/openscap_1499161

    Raise event containerimage_scan_complete only when scan succeeds.

    (cherry picked from commit 8b01ee3e73acf8a300e8edb2e7bd0ba122552326)

    https://bugzilla.redhat.com/show_bug.cgi?id=1499161

 app/models/manageiq/providers/kubernetes/container_manager/scanning/job.rb | 11 +-
 spec/models/manageiq/providers/kubernetes/container_manager/scanning/job_spec.rb | 1 +
 2 files changed, 7 insertions(+), 5 deletions(-)

Comment 22 juwatts 2018-11-15 19:04:30 UTC
Tested in: 5.10.0.24.20181113213923_03b81fd

Recreation steps:
1) Assigned the default OpenSCAP profile to an OpenShift Container Platform provider
2) Picked a Container image and verified that the status of compliance was "Never Verified"
3) Initiated a SSA scan
4) Scan completes successfully
5) Navigate back to the Container image the scan was run against, the Status was marked as "Never Verified" even though the scan ran and completed successfully. 

Moving this PR back to ON_DEV.

Comment 23 Lucy Fu 2018-11-15 19:30:13 UTC
What is the IP of your appliance?

Comment 26 Beni Paskin-Cherniavsky 2018-11-19 13:26:58 UTC
Question: how should we treat non-RHEL images for which we don't have security definitions?
For this OpenSCAP fails with "Unable to run OpenSCAP: Unable to get RHEL dist number" error.  We don't get security results, but it is a definite answer — this is not some transient error and there's no point re-scanning.
And we do get some non-security SSA info — CFME gets access to image filesystem and collects some info like package versions (I think only for RPM-based images?).

So, is such image compliant or not?

Comment 27 CFME Bot 2018-11-20 12:06:54 UTC
New commit detected on ManageIQ/manageiq-providers-kubernetes/master:

https://github.com/ManageIQ/manageiq-providers-kubernetes/commit/3b923c35fa53045fa6f9a60e2acf31754630ac6c
commit 3b923c35fa53045fa6f9a60e2acf31754630ac6c
Author:     Lucy Fu <lufu@redhat.com>
AuthorDate: Fri Nov 16 11:56:53 2018 -0500
Commit:     Lucy Fu <lufu@redhat.com>
CommitDate: Fri Nov 16 11:56:53 2018 -0500

    Different way to check for job's complete state.

    State finishing does not exist in hammer branch.

    https://bugzilla.redhat.com/show_bug.cgi?id=1499161

 app/models/manageiq/providers/kubernetes/container_manager/scanning/job.rb | 2 +-
 spec/models/manageiq/providers/kubernetes/container_manager/scanning/job_spec.rb | 15 +-
 2 files changed, 14 insertions(+), 3 deletions(-)

Comment 28 Beni Paskin-Cherniavsky 2018-11-20 12:09:20 UTC
Hmm, given the whole purpose of this BZ is "if scanning fails, should not mark compliant", I guess the partial failure from comment 25 should still be considered failure?
If so, still need to fix this case: https://github.com/ManageIQ/manageiq-providers-kubernetes/pull/304#discussion_r234287608

Comment 29 CFME Bot 2018-11-20 14:36:56 UTC
New commit detected on ManageIQ/manageiq-providers-kubernetes/hammer:

https://github.com/ManageIQ/manageiq-providers-kubernetes/commit/3974721e329bb77147f303c687c6adda13abcbf6
commit 3974721e329bb77147f303c687c6adda13abcbf6
Author:     Beni Cherniavsky-Paskin <cben@redhat.com>
AuthorDate: Tue Nov 20 07:05:51 2018 -0500
Commit:     Beni Cherniavsky-Paskin <cben@redhat.com>
CommitDate: Tue Nov 20 07:05:51 2018 -0500

    Merge pull request #304 from lfu/openscap_1499161_2

    Different way to check for scan job's complete state.

    (cherry picked from commit 2454b3d7ccdb6712cf2c3eb3ffe2987df88de94b)

    https://bugzilla.redhat.com/show_bug.cgi?id=1499161

 app/models/manageiq/providers/kubernetes/container_manager/scanning/job.rb | 2 +-
 spec/models/manageiq/providers/kubernetes/container_manager/scanning/job_spec.rb | 15 +-
 2 files changed, 14 insertions(+), 3 deletions(-)

Comment 30 juwatts 2018-12-05 19:55:43 UTC
Tested in: 5.10.0.27.20181128170555_43ed8cb

Verification steps:
Recreation steps:
1) Removed admin rights for system:serviceaccount:management-infra:management-admin
2) Assigned the default OpenSCAP profile to an OpenShift Container Platform provider
3) Picked a Container image and verified that the status of compliance was "Never Verified"
4) Initiated a SSA scan
5) Verified the scan failed: pod creation for [management-infra/manageiq-img-scan-4c1dd] failed
6) Navigate back to the Container image the scan was run against, verified the Status was marked as "Never Verified" 
7) Added admin rights back for system:serviceaccount:management-infra:management-admin
8) Picked a different Container image and verified that the status of compliance was "Never Verified"
9) Initiated a SSA scan
10) Verified the scan was successful
11) Navigate back to the Container image the scan was run against, verified the Status was marked as "Compliant" 
12) Removed admin rights for system:serviceaccount:management-infra:management-admin
13) Rescanned the image from step 8
14) Verified the scan failed: pod creation for [management-infra/manageiq-img-scan-4c1dd] failed
15) Navigate back to the Container image the scan was run against, verified the Status was marked still marked as "Complaint" with the previous time stamp of the successful scan

Comment 31 errata-xmlrpc 2019-02-07 23:02:54 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2019:0212


Note You need to log in before you can comment on or make changes to this bug.