Bug 1801622 - Images in Satellite GCE CR are not the same as Google Cloud
Summary: Images in Satellite GCE CR are not the same as Google Cloud
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Satellite 6
Classification: Red Hat
Component: Compute Resources - GCE
Version: 6.7.0
Hardware: Unspecified
OS: Unspecified
unspecified
high vote
Target Milestone: 6.7.0
Assignee: Kavita
QA Contact: Jitendra Yejare
URL:
Whiteboard:
Depends On:
Blocks: 1721890
TreeView+ depends on / blocked
 
Reported: 2020-02-11 11:35 UTC by Jitendra Yejare
Modified: 2020-04-14 13:28 UTC (History)
8 users (show)

Fixed In Version: tfm-rubygem-foreman_theme_satellite-5.0.1.12-1,foreman-1.24.1.20-1
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-04-14 13:28:43 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Foreman Issue Tracker 29049 Normal Closed GCE should not display deprecated images while creating image 2020-04-07 06:59:08 UTC
Red Hat Product Errata RHSA-2020:1454 None None None 2020-04-14 13:28:53 UTC

Description Jitendra Yejare 2020-02-11 11:35:52 UTC
Description of problem:
The count of available images in Satellite GCE CR is different(more) than whats available in Google Cloud under the images section. The Satellite GCE CR is showing multiple images for any Linux flavor/version, whereas the Google Cloud shows 1 or max 2 for any flavor/version.

Version-Release number of selected component (if applicable):
Satellite 6.7 snap 10

How reproducible:
Always

Steps to Repro:
1. Create a GCE Compute Resource.
2. From UI:
     1. While host provisioning, attempt to create an image in GCE CR from the list of images loaded from Google Cloud.
3. From API/CLI:
     1. List the available images in GCE Compute Resource.

Actual results:
The count of available images in Satellite GCE CR is different(more) than whats available in Google Cloud.

Expected results:
The count of available images in Satellite GCE CR should be the same as available in Google Cloud for creating VM.

Comment 3 Kavita 2020-02-17 11:45:04 UTC
After debugging on this behaviour, there are two things:

1. fog-google library has pre-defined list of projects and user's project from compute-resource gets added to this list.
Reference link - https://github.com/fog/fog-google/blob/master/lib/fog/compute/google/models/images.rb#L11

2. fog-google's images method returns all the images (current + deprecated) ---> (Which can be fixed - will show only current one)

Pre-defined project list:
["centos-cloud",
 "cos-cloud",
 "coreos-cloud",
 "debian-cloud",
 "rhel-cloud",
 "suse-cloud",
 "suse-sap-cloud",
 "ubuntu-os-cloud",
 "windows-cloud",
 "windows-sql-cloud",
 "opensuse-cloud"]

On other hand, Google Compute Engine Console shows ONLY current images not deprecated and showing images from many other projects that are absent in above list.
Example - gce-uefi-images, rhel-sap-cloud projects from shielded/unshielded-images,  ml-images

Reference documents - 
https://cloud.google.com/compute/docs/images
https://cloud.google.com/compute/docs/images#shielded-images
https://cloud.google.com/compute/docs/images#unshielded-images

In my opinion, we should ONLY prepend projects list that provides images for which Satellite supports provisioning. 
In the project "rhel-sap-cloud", images like rhel-7-7-sap-v20200205 are shown.

@Bryan/Marek, I have a query that for host provisioning do we need to support such images (example - rhel-*-sap) in the Satellite?

Comment 4 Marek Hulan 2020-02-17 13:30:04 UTC
Could this be a compute resource configuration? I can imagine few checkboxes - "show deprecated images", "show shielded-images", "show unnshielded-images". If you think checkboxes would be an overkill, I'd display shielded and unshielded, but not deprecated. Potentially we could also add a text filter supporting wildcards. A theme plugin could set the default value to to "rhel-*".

Comment 5 Bryan Kearney 2020-02-17 13:36:58 UTC
I like Marek's suggestion. We should default to only what satellite supports, but give them options to find other things.

Comment 6 Kavita 2020-02-24 10:34:37 UTC
Thank you Marek and Bryan!

I have created two upstream tickets one to remove deprecated images from list and second to add option for users to include shielded/unshielded-images.

i) https://projects.theforeman.org/issues/29050 (marked it as BUG)
ii) https://projects.theforeman.org/issues/29049 (marked it as Feature)

As second ticket (ii) will be an enhancement to the existing functionality, I would suggest to treat it as an RFE and good to track it separately. 
@JItendra, what do you think? 

As a part of this BZ and upstream issue 29050, I have opened up pull-request -
https://github.com/theforeman/foreman/pull/7463
where, it will hide deprecated images and extends Google global projects list with "rhel-sap-cloud" project (as this is without shielded project and ONLY missing one from existing list)
Also, it adds a method to override images list by family. It would be easy to override list in downstream using foreman theme satellite.

Comment 7 Kavita 2020-02-24 10:41:58 UTC
> 
> i) https://projects.theforeman.org/issues/29050 (marked it as BUG)
> ii) https://projects.theforeman.org/issues/29049 (marked it as Feature)
> 
> As second ticket (ii) will be an enhancement to the existing functionality,
> I would suggest to treat it as an RFE and good to track it separately. 
> @JItendra, what do you think? 
> 
> As a part of this BZ and upstream issue **29049, I have opened up pull-request
> -
> https://github.com/theforeman/foreman/pull/7463
> where, it will hide deprecated images and extends Google global projects
> list with "rhel-sap-cloud" project (as this is without shielded project and
> ONLY missing one from existing list)
> Also, it adds a method to override images list by family. It would be easy
> to override list in downstream using foreman theme satellite.

Sorry sequence was wrong. Corrected upstream ticket ids :-)

i) https://projects.theforeman.org/issues/29049 (marked it as BUG)
> ii) https://projects.theforeman.org/issues/29050 (marked it as Feature)
>

Comment 8 Jitendra Yejare 2020-02-26 09:24:17 UTC
Hello Kavita,

As per our discussion last week on the same, I ACK for I and ii issues. Agree to make (i) as a bug and remove depreciated images now as it can be easily done now and (ii) as an enhancement as this will intro a lot of changes to GCE and we don't have enough time to do that in 6.7.0

Comment 9 Mike McCune 2020-03-06 17:38:38 UTC
Jitendra, based on Comment 8, I'm going to move this back ON_QA so we can determine if the original bug is verified and we are following the remaining issues in separate bugs.

If my interpretation of this is wrong, absolutely feel free to move back ton ASSIGNED.

Comment 10 Jitendra Yejare 2020-03-11 10:45:38 UTC
@Kavita,

I could see some windows images listed as well especially 'windows-server' do you think we should altogether remove them regardless of what pool they are coming in?

Also, there are some core-os and cos (container) images which I am not sure should be supported. Thoughts ?

Comment 12 Kavita 2020-03-12 11:23:10 UTC
(In reply to Jitendra Yejare from comment #10)
> @Kavita,
> 
> I could see some windows images listed as well especially 'windows-server'
> do you think we should altogether remove them regardless of what pool they
> are coming in?
> 
> Also, there are some core-os and cos (container) images which I am not sure
> should be supported. Thoughts ?

@Jitendra,


Pull-request for an original issue is not yet merged into an upstream.
Yes for downstream, it is possible to filter images of specific image families by overriding gce methods in foreman_theme_satellite.
But the only point is that these methods will be available after my PR changes. 
so, I would suggest to add a separate BZ to track mentioned change. 
If not in 6.7.0, I will try to get it into 6.7.z.


With PR changes, it will show current available images from all global projects not deprecated one.

Comment 13 Jitendra Yejare 2020-03-13 08:55:03 UTC
Mike, The upstream issue is not yet closed, the issue is still in Ready for Testing state. 

Moving this to ONQA.

Comment 14 Mike McCune 2020-03-17 17:31:28 UTC
Upstream was merged yesterday, moving to POST.

Comment 17 Jitendra Yejare 2020-03-25 14:58:55 UTC
@Kavita,


The depreciated images are still being shown :

CLI(Same from UI):

# hammer compute-resource image available --compute-resource-id 92 | grep centos-6-v20131120
centos-6-v20131120                                   | 11748647391859510935


This image is shown as depreciated from console.google.com, but this is still being fetched in the satellite.

As per your comment 12, this bug fixes the removal of depreciated images that seems not fixed.

Comment 18 Kavita 2020-03-25 15:09:50 UTC
@Jitendra,

Would it be possible for you to share reproducer?
Thanks!

Comment 20 Kavita 2020-03-25 15:22:47 UTC
@Jitendra,

As per my observation that only my foreman_theme_satellite changes are available in snap not my upstream fix (https://projects.theforeman.org/issues/29049).
Somehow it wasn't cherry-picked to the downstream.

@Eric, could you please guide me on this? I am not sure what went wrong here.

Comment 21 Kavita 2020-03-25 15:37:35 UTC
Moving to POST as required changes for BZ are available in upstream for cherry-picking.

Comment 23 Jitendra Yejare 2020-04-03 05:19:37 UTC
Verified !

@ Satellite 6.7 snap 18

Steps:
----
1. Create a GCE Compute Resource.
2. From UI:
     1. While host provisioning, attempt to create an image in GCE CR from the list of images loaded from Google Cloud.
3. From API/CLI:
     1. List the available images in GCE Compute Resource.


Expected results:
As per comment 12, its now showing current available images from all global projects and not the deprecated images.

Comment 24 Jitendra Yejare 2020-04-03 05:21:30 UTC
Unintentionally, CCed apatel. Removing the needinfo now.

Comment 26 errata-xmlrpc 2020-04-14 13:28:43 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2020:1454


Note You need to log in before you can comment on or make changes to this bug.