Red Hat Satellite engineering is moving the tracking of its product development work on Satellite to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "Satellite project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs will be migrated starting at the end of May. If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "Satellite project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/SAT-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 2255546 - Module stream filter adds all modules to all repositories
Summary: Module stream filter adds all modules to all repositories
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Satellite
Classification: Red Hat
Component: Content Views
Version: 6.15.0
Hardware: x86_64
OS: Linux
unspecified
high
Target Milestone: 6.15.0
Assignee: Ian Ballou
QA Contact: Sam Bible
URL:
Whiteboard:
: 2255548 2255549 (view as bug list)
Depends On:
Blocks: 2255548 2255549
TreeView+ depends on / blocked
 
Reported: 2023-12-21 18:49 UTC by Taft Sanders
Modified: 2024-04-23 17:16 UTC (History)
6 users (show)

Fixed In Version: rubygem-katello-4.11.0.5-1
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 2255548 2255549 (view as bug list)
Environment:
Last Closed: 2024-04-23 17:16:36 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
CV repository package counts as seen on webui (74.30 KB, image/png)
2023-12-21 18:49 UTC, Taft Sanders
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Foreman Issue Tracker 37050 0 Normal New Module stream filter adds all modules to all repositories 2024-01-12 18:17:19 UTC
Red Hat Issue Tracker SAT-22114 0 None None None 2023-12-21 18:51:19 UTC
Red Hat Product Errata RHSA-2024:2010 0 None None None 2024-04-23 17:16:39 UTC

Description Taft Sanders 2023-12-21 18:49:11 UTC
Created attachment 2005321 [details]
CV repository package counts as seen on webui

Description of problem:
When publishing a content view containing 1 module containing repository and 1 non-module containing repository, the non-module containing repository is created with all modules from the module containing repository.

Version-Release number of selected component (if applicable):
Satellite 6.15 snap 3

How reproducible:
Every time

Steps to Reproduce:
1. Create a content view containing 1 module and 1 non-module repository (IE: 'Red Hat Enterprise Linux 8 for x86_64 - AppStream RPMs 8' & 'Red Hat Ansible Engine 2 for RHEL 8 x86_64 RPMs'
2. Create a content-view filter for module streams with the following settings:
- Include: true
3. Add a rule to this CV filter to:
- include all module streams
- original-module-streams: false
- Apply to all repositories in the CV: true
4. Publish a new version
5. Observe the package count for the non-module containing repository (IE: 'Red Hat Ansible Engine 2 for RHEL 8 x86_64 RPMs')

Actual results:
# hammer content-view version info --content-view '03676447-dup-test' --id 2 --include-applied-filters true
Id:                     2
Name:                   03676447-dup-test 1.0
Version:                1.0
Description:            
Content View ID:        2
Content View Name:      03676447-dup-test
Content View Label:     03676447-dup-test
Lifecycle Environments: 
 1) Id:    1
    Name:  Library
    Label: Library
Repositories:           
 1) Id:    29
    Name:  Red Hat Enterprise Linux 8 for x86_64 - AppStream RPMs 8
    Label: Red_Hat_Enterprise_Linux_8_for_x86_64_-_AppStream_RPMs_8
 2) Id:    30
    Name:  Red Hat Ansible Engine 2 for RHEL 8 x86_64 RPMs
    Label: Red_Hat_Ansible_Engine_2_for_RHEL_8_x86_64_RPMs
Has Applied Filters:    yes
Applied Filters:        
 1) Id:                      1
    Name:                    module stream lockdown
    Type:                    modulemd
    Inclusion:               yes
    Original packages:       no
    Original module streams: no
    Rules:                   
     1) Id:                     1
        Module stream Id:       2154
        Content view filter Id: 1
     2) Id:                     2
        Module stream Id:       771
        Content view filter Id: 1
--snip--
     749)Id:                     749
        Module stream Id:       23
        Content view filter Id: 1
Dependency Solving:     false
===========================================
[root@bombsat615 ~]# hammer repository info --id 30
Id:                 30
Name:               Red Hat Ansible Engine 2 for RHEL 8 x86_64 RPMs
Label:              Red_Hat_Ansible_Engine_2_for_RHEL_8_x86_64_RPMs
Description:        
Organization:       Default Organization
Red Hat Repository: yes
Content Type:       yum
Content Label:      ansible-2-for-rhel-8-x86_64-rpms
Mirroring Policy:   Complete Mirroring
Url:                https://cdn.redhat.com/content/dist/layered/rhel8/x86_64/ansible/2/os
Publish Via HTTP:   no
Published At:       https://bombsat615.d.sysmgmt.cee.redhat.com/pulp/content/Default_Organization/content_views/03676447-dup-test/1.0/content/dist/layered/rhel8/x86_64/ansible/2/os/
Relative Path:      Default_Organization/content_views/03676447-dup-test/1.0/content/dist/layered/rhel8/x86_64/ansible/2/os
Download Policy:    on_demand
HTTP Proxy:         
    HTTP Proxy Policy: global_default_http_proxy
Product:            
    Id:   213
    Name: Red Hat Ansible Engine
GPG Key:            

Sync:               
    Status: Not Synced
Created:            2023/12/21 17:20:43
Updated:            2023/12/21 17:20:43
Content Counts:     
    Packages:       11183
    Source RPMS:    0
    Package Groups: 0
    Errata:         32
    Module Streams: 749

[root@bombsat615 ~]# hammer repository info --id 28
Id:                 28
Name:               Red Hat Ansible Engine 2 for RHEL 8 x86_64 RPMs
Label:              Red_Hat_Ansible_Engine_2_for_RHEL_8_x86_64_RPMs
Description:        
Organization:       Default Organization
Red Hat Repository: yes
Content Type:       yum
Content Label:      ansible-2-for-rhel-8-x86_64-rpms
Mirroring Policy:   Complete Mirroring
Url:                https://cdn.redhat.com/content/dist/layered/rhel8/x86_64/ansible/2/os
Publish Via HTTP:   no
Published At:       https://bombsat615.d.sysmgmt.cee.redhat.com/pulp/content/Default_Organization/Library/content/dist/layered/rhel8/x86_64/ansible/2/os/
Relative Path:      Default_Organization/Library/content/dist/layered/rhel8/x86_64/ansible/2/os
Download Policy:    on_demand
HTTP Proxy:         
    HTTP Proxy Policy: global_default_http_proxy
Product:            
    Id:   213
    Name: Red Hat Ansible Engine
GPG Key:            

Sync:               
    Status:         Success
    Last Sync Date: about 1 hour
Created:            2023/12/21 17:20:43
Updated:            2023/12/21 17:20:43
Content Counts:     
    Packages:       58
    Source RPMS:    0
    Package Groups: 0
    Errata:         32
    Module Streams: 0


Expected results:
Module content should only be in repositories that contain module content after the publish. 

Additional info:

Comment 1 Taft Sanders 2023-12-21 19:33:01 UTC
A workaround to this issue is to update the CV filter for the module stream to refine the scope of the affected repositories to only the repository that contains module data. The below hammer command will change the affected repository for the rules in the CV filter:
# '7' is the Appstream root repository ID for this test satellite
[root@bombsat615 ~]# hammer content-view filter update --content-view '03676447-dup-test' --name 'module stream lockdown' --repository-ids 7 --organization-id 1
Filter updated.

//Steps to verify the new package count for version 2 of the CV:
[root@bombsat615 ~]# hammer content-view publish --name '03676447-dup-test' --organization-id 1
[..................................................................................................................................................................] [100%]

[root@bombsat615 ~]# hammer content-view version list --content-view '03676447-dup-test' --organization-id 1
---|-----------------------|---------|-------------|-----------------------
ID | NAME                  | VERSION | DESCRIPTION | LIFECYCLE ENVIRONMENTS
---|-----------------------|---------|-------------|-----------------------
3  | 03676447-dup-test 2.0 | 2.0     |             | Library               
2  | 03676447-dup-test 1.0 | 1.0     |             |                       
---|-----------------------|---------|-------------|-----------------------

[root@bombsat615 ~]# hammer content-view version info --id 3 --fields Repositories
Repositories: 
 1) Id:    33
    Name:  Red Hat Enterprise Linux 8 for x86_64 - AppStream RPMs 8
    Label: Red_Hat_Enterprise_Linux_8_for_x86_64_-_AppStream_RPMs_8
 2) Id:    34
    Name:  Red Hat Ansible Engine 2 for RHEL 8 x86_64 RPMs
    Label: Red_Hat_Ansible_Engine_2_for_RHEL_8_x86_64_RPMs

[root@bombsat615 ~]# for i in {33,34}; do hammer repository info --id $i --fields "Content counts","Name"; done
Name:           Red Hat Enterprise Linux 8 for x86_64 - AppStream RPMs 8
Content Counts: 
    Packages:       35090
    Source RPMS:    0
    Package Groups: 58
    Errata:         3424
    Module Streams: 749

Name:           Red Hat Ansible Engine 2 for RHEL 8 x86_64 RPMs
Content Counts: 
    Packages:       58
    Source RPMS:    0
    Package Groups: 0
    Errata:         32
    Module Streams: 0

Comment 2 Ron Lavi 2023-12-27 10:37:46 UTC
*** Bug 2255548 has been marked as a duplicate of this bug. ***

Comment 3 Ron Lavi 2023-12-27 10:37:48 UTC
*** Bug 2255549 has been marked as a duplicate of this bug. ***

Comment 5 Ian Ballou 2024-01-12 18:10:42 UTC
This is only reproducible with dependency solving off -- with dependency solving on, the module streams don't get copied erroneously.

Pulp-rpm has two different copy APIs, one that is more complicated for dependency solving, and one that is quicker for non-dependency solving. The dependency-solving copy API accepts a source repository as input. However, the generic copy api does not take a source repository version as input.

I think this bug is probably fallout from us starting to use the generic copy api.

The fix will involve us filtering out Pulp hrefs to copy that don't already exist in the source repository.

Note: this is not an issue for container content because we use a container-specific copy api for container image filters. The container copy api does accept a source repository.

Comment 6 Ian Ballou 2024-01-12 18:17:17 UTC
Created redmine issue https://projects.theforeman.org/issues/37050 from this bug

Comment 7 Ian Ballou 2024-01-12 18:53:55 UTC
Our code for determining what packages to add from filters does take into account the source repository for all types accept module streams, so the issue isn't as wide-spread as I thought when writing Comment 5.

`content_unit_pulp_ids` in app/models/katello/content_view_module_stream_filter.rb just needs to begin filtering module streams by the source repository.

Comment 9 Ian Ballou 2024-01-24 15:38:18 UTC
Moving to POST since the upstream PR is merged.

Comment 11 Sam Bible 2024-01-25 17:31:02 UTC
Verified on 6.15 - Snap 8

Steps to Verify:
1. Create a content view containing 1 module and 1 non-module repository (IE: 'Red Hat Enterprise Linux 8 for x86_64 - AppStream RPMs 8' & 'Red Hat Ansible Engine 2 for RHEL 8 x86_64 RPMs'
2. Create a content-view filter for module streams with the following settings:
- Include: true
3. Add a rule to this CV filter to:
- include all module streams
- original-module-streams: false
- Apply to all repositories in the CV: true
4. Publish a new version
5. Observe the package count for the non-module containing repository (IE: 'Red Hat Ansible Engine 2 for RHEL 8 x86_64 RPMs')

Expected Results:
The non-module repository should contain no module streams, and match the package count it has outside of the CV

Expected Results:
The non-module repository contains no module streams, and matches the package count it has outside of the CV.

Comment 14 errata-xmlrpc 2024-04-23 17:16:36 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: Satellite 6.15.0 release), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2024:2010


Note You need to log in before you can comment on or make changes to this bug.