Bug 1853280 - Content view filter is excluding modules and Packages when published after upgrading the Satellite from 6.6 to 6.7
Summary: Content view filter is excluding modules and Packages when published after up...
Alias: None
Product: Red Hat Satellite
Classification: Red Hat
Component: Content Views
Version: 6.7.0
Hardware: Unspecified
OS: Unspecified
Target Milestone: 6.8.0
Assignee: Partha Aji
QA Contact: Lai
: 1883984 (view as bug list)
Depends On:
TreeView+ depends on / blocked
Reported: 2020-07-02 10:49 UTC by Krutika Kinge
Modified: 2020-10-29 14:23 UTC (History)
10 users (show)

Fixed In Version: tfm-rubygem-katello-3.16.0-1
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1862264 (view as bug list)
Last Closed: 2020-10-27 13:03:46 UTC
Target Upstream Version:

Attachments (Terms of Use)

System ID Priority Status Summary Last Updated
Foreman Issue Tracker 30352 Normal Closed Add 'Module Streams with no Errata' filter 2020-11-30 11:07:27 UTC
Red Hat Knowledge Base (Solution) 5310871 None None None 2020-08-11 10:01:27 UTC
Red Hat Product Errata RHSA-2020:4366 None None None 2020-10-27 13:04:03 UTC

Internal Links: 1122832

Description Krutika Kinge 2020-07-02 10:49:52 UTC
Description of problem:
Content view with Include errata and Package filters is created on Satellite 6.6 Server. When the same content view is published after upgrading the satellite server to 6.7 module stream and packages are missing from the new Content view version.

Version-Release number of selected component (if applicable):
Red Hat Satellite 6.7

How reproducible:

Steps to Reproduce:
1. Create a content view on Satelite 6.6 server with include errata and include rpm filter.
2. Publish a new version of the content view.
3. upgrade the Satellite server to version 6.7
4. Publish a new version of the Same content view.

Actual results:
Package and modules are missing from the content view.

Expected results:
Packages and Modules count should be same as it was on the Satellite 6.6 Server.

Additional info:

Comment 1 Partha Aji 2020-07-02 15:59:23 UTC
Could you elaborate on the specific scenario on the include errata filter and include rpm filter (are you filtering by ids or are you filtering by date?)

With Satellite 6.7 the big difference is that it filters both Modular and Non Modular errata. Satellite 6.5 or 6.6 do not.
* Satellite 6.5 just bases it of rpms it has no idea whether they are modular or not (which may be good enough for the errata case, but might break for regular rpm filtering).  
* Satellite 6.6 is more module stream aware. The idea here is that we do not want to just copy module stream and NOT copy the packages belonging to that module stream. In other words we do not want the user to be able filter out a modular rpm by any filters.
* Satellite 6.7 added the ability to filter out module streams. This adds a new rule to filtering as applied by the rules below.
** The Module Stream filters control the modular RPMs (RPMs that are part of a Module Stream.)
** The Package filters control the non-modular RPMs (RPMs that do not belong to any Module Stream.)
** Errata filters control both.

In your case 6.5 and 6.7 probably have the same behaviour when you jut filter out errata by date. But big difference in 6.5 you can filter out modular rpms via package filters (since it does not know about modular rpms), leaving some of the copied module streams in a broken state. 6.7 will not allow you to filter out modular rpms via package filters.

Satellite 6.6 behaviour while confusing was done with the idea of treating module streams as an independent special entity. Feel free to ping me if this is still unclear :)

Comment 3 Peter Vreman 2020-07-03 10:14:01 UTC
The filters are straightforward:
- include all rpm without errata
- include errata based from 2000-01-01 until 2020-07-03

After i upgraded to Sat6.7 and then published new content views for RHEL8.x suddenly all of my RHEL8.x systems are broken due to missing ~4000 rpms and ~200 errata although the ContetnView parameters are not changed.
For me this is really a big suprise because Sat6.7 is a minor release that had to be backwards compatible with Sat6.6.

Comment 4 Peter Vreman 2020-07-03 10:19:03 UTC
And from user point of view:

An rpm is just an rpm. Independent from the source being a normal repo or in a module stream.

You cannot expect that i as a user understand that in Sat6.7 'rpm' has changed its meaning to match only the non-module-rpms. If that really had to be the case then the name must be renamed to match the new behaviour.

Comment 6 Peter Vreman 2020-07-07 19:33:21 UTC
I have created a dirty hack that reverted new Sat6.7 Feature that had undocumented and not-workaroundable side-effects. The hack just whitelists all modules instead of building a nice filter clause based on errata or selected module stream.

--- lib/katello/util/module_stream_clause_generator.rb.200707-1 2020-05-11 13:14:12.000000000 +0000
+++ lib/katello/util/module_stream_clause_generator.rb  2020-07-07 19:08:55.731946118 +0000
@@ -25,13 +25,14 @@

       def make_module_stream_clauses(repo, filters)
-        content_type = filters.first.content_type
-        clauses = filters.collect do |filter|
-          filter.generate_clauses(repo)
-        end
-        clauses.flatten!
-        clauses.compact!
-        module_stream_clauses_from_content(content_type, clauses) unless clauses.empty?
+#        content_type = filters.first.content_type
+#        clauses = filters.collect do |filter|
+#          filter.generate_clauses(repo)
+#        end
+#        clauses.flatten!
+#        clauses.compact!
+#        module_stream_clauses_from_content(content_type, clauses) unless clauses.empty?
+        whitelist_all_matcher_clause

       def module_stream_clauses_from_content(content_type, clauses)

A cleaner solution i leave for engineering to implement for Sat6.8:
- Create a 'original modules' filter, just like the rpm packages have
- Document in the Release Notes the Changed behavior so users are informed

Comment 7 Oliver Falk 2020-07-08 07:27:30 UTC
 thanks a lot for identifying the code involved in that and providing a preliminary workaround.

Kind regards,

Comment 8 Partha Aji 2020-07-09 02:58:25 UTC
Connecting redmine issue https://projects.theforeman.org/issues/30352 from this bug

Comment 9 Partha Aji 2020-07-09 03:32:19 UTC
I m working on an patch upstream for this. Looks like Module Streams that do not belong to any errata are not getting copied.

Comment 10 Bryan Kearney 2020-07-15 04:01:06 UTC
Moving this bug to POST for triage into Satellite since the upstream issue https://projects.theforeman.org/issues/30352 has been resolved.

Comment 14 Lai 2020-09-10 17:40:46 UTC
Steps to test

1. Sync the rhel8 appstream repo
2. Create a cv and add rhel8 appstream
3. publish
4. Drill into the published version -> Errata
5. Search for "modular = true"
6. Notice the number of modular with errata.
7. Yum content -> filter
8. Create a filter with module stream with include filter
9. Click checkbox for "Include Module stream with no errata"
10. Publish
11. Repeat steps 4-5

No errata shows up

No errata shows up

The filter has only include modules with no errata.

I've also verified that the apidoc does have the two parameters: original_packages and original_module_streams and checked the content_view_controller.rb to see that they are indeed in there.

Marking issue as verified.  6.8.0_14

Comment 16 Justin Sherrill 2020-10-02 00:33:22 UTC
*** Bug 1883984 has been marked as a duplicate of this bug. ***

Comment 19 errata-xmlrpc 2020-10-27 13:03:46 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: Satellite 6.8 release), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.