Bug 803837

Summary: [RFE] spacewalk-repo-sync should process channel-specific config options, including the includepkgs directive
Product: [Community] Spacewalk Reporter: Seb7 <8pcwh2dhgm>
Component: ServerAssignee: Michael Mráka <mmraka>
Status: CLOSED CURRENTRELEASE QA Contact: Red Hat Satellite QA List <satqe-list>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 1.6CC: dyordano, jpazdziora, kfiresmith, mmraka, shardy
Target Milestone: ---Keywords: Reopened
Target Release: ---   
Hardware: Unspecified   
OS: Linux   
Whiteboard:
Fixed In Version: spacewalk-backend-2.1.1-1 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: 798604 Environment:
Last Closed: 2014-03-04 13:07:26 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 737830, 1069560    
Attachments:
Description Flags
Patch - read include/exclude from conf file. none

Description Seb7 2012-03-15 18:11:51 UTC
+++ This bug was initially created as a clone of Bug #798604 +++

Description of problem:

spacewalk-repo-sync ignores the includepkgs directive in /etc/rhn/spacewalk-repo-sync/yum.conf in order to include packages so that repos can be partially sync'd.

This script takes optional --include/--exclude CLI args, and supports some options specified via /etc/rhn/spacewalk-repo-sync/yum.conf. (See #621166.)

Can this feature please be added to spacewalk as enhancement errata, as it would be very useful in customer environments (e.g for doing partial sync of EPEL)

Version-Release number of selected component (if applicable):
spacewalk-backend-tools-1.6.66-1.el5

How reproducible:
Always

Steps to Reproduce:
Install latest spacewalk 1.6.0, with version of spacewalk-backend-tools mentioned above, note that spacewalk-repo-sync ignores feature discussed above.
  
Actual results:
Starts downloading the entire repo.

Expected results:
Just downloads the packages listed with includepkgs.

Additional info:
The manpage of the released spacewalk-repo-sync (spacewalk-backend-tools-1.6.66-1) doesn't mention --include/--exclude, but the code clearly supports these options. I discovered this after hours of searching (before #798604 was posted). Steve Hardy has added them as per #798604 in commit 55db255cca7effb1c070c3ae1381db3f2989f181.

Comment 1 Jan Pazdziora 2012-03-16 08:12:59 UTC
I cannot reproduce the problem:

# spacewalk-repo-sync -c spacewalk-1.7-x86_64 --include spacewalk-selinux
Repo URL: http://spacewalk.redhat.com/yum/1.7/RHEL/6/x86_64
Packages in repo:               135
Packages passed filter rules:     7
Packages already synced:          0
Packages to sync:                 7
1/7 : spacewalk-backend-server-1.7.38-1.el6-0.noarch
2/7 : spacewalk-admin-1.7.4-1.el6-0.noarch
3/7 : spacewalk-certs-tools-1.7.3-1.el6-0.noarch
4/7 : spacewalk-setup-1.7.9-1.el6-0.noarch
5/7 : spacewalk-config-1.7.2-1.el6-0.noarch
6/7 : spacewalk-backend-1.7.38-1.el6-0.noarch
7/7 : spacewalk-selinux-1.7.2-1.el6-0.noarch
Repo http://spacewalk.redhat.com/yum/1.7/RHEL/6/x86_64 has 0 errata.
Sync completed.
Total time: 0:00:19

Comment 2 Seb7 2012-03-16 12:30:21 UTC
OK, let me clarify then as you are doing it the way I know works, rather than the method listed above, which does not.

Here is my /etc/rhn/spacewalk-repo-sync/yum.conf:

[main]


[epel6-centos6-x86_64]
name=External yum repo - EPEL 6 for CentOS 6 (x86_64)
#baseurl=http://download.fedoraproject.org/pub/epel/6/x86_64
mirrorlist=http://mirrors.fedoraproject.org/mirrorlist?repo=epel-6&arch=x86_64
failovermethod=priority
enabled=1
gpgcheck=1
#gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL
gpgkey=http://download.fedoraproject.org/pub/epel/RPM-GPG-KEY-EPEL-6
# The following directive does not work. You must use --include on the command line. /Seb
includepkgs=ipython byobu libffi python26 python26-libs screen \
            python-genshi \
            python-hashlib \
            python-simplegeneric \
            bzr bzrtools python-pycurl python-paramiko python-crypto \
            python-hwdata

The command I issue is:
# spacewalk-repo-sync --channel=epel6-centos6-x86_64
Repo URL: http://mirrors.fedoraproject.org/mirrorlist?repo=epel-6&arch=x86_64
Packages in repo:              6975
Packages already synced:          8
Packages to sync:              6967
1/6967 : pdfjam-2.08-1.el6-0.noarch

User interrupted process.

Comment 3 Jan Pazdziora 2012-03-16 12:58:38 UTC
I don't think spacewalk-repo-sync processes the channel-specific directives at all, at this moment.

Marking as RFE.

Comment 4 Seb7 2012-03-16 13:15:11 UTC
OK, thanks. It wasn't clear from the Bug 621166 Comment 14 ('You can put yum related options for spacewalk-repo-sync to /etc/rhn/spacewalk-repo-sync/yum.conf.') and the commit messages associated with that comment. I assumed all yum options should be available to put in /etc/rhn/spacewalk-repo-sync/yum.conf.

CCing Michael Mráka who made those commits.

Comment 5 Kodiak Firesmith 2013-03-06 18:45:53 UTC
I can confirm that this is still a problem

Comment 6 Kodiak Firesmith 2013-03-06 18:52:57 UTC
Whoops, hit save too soon.  

I see that this is still missing in RHN Satellite 5.5.  I also see that it's been set as a blocker to 737830, which is very depressing if I interpret correctly that 737830 is some sort of catch-all that isn't associated with any particular future release.  


This functionality would really be nice to have as it would be the simplest way to deal with keeping conflicting packages such as EPEL's nrpe out of our satellite rather than having to do something clumsy like keep our version or NRPE one epoch ahead. 

The basic functionality appears to be there as I can demonstrate that spacewalk-repo-sync reads and uses at least some directives from /etc/rhn/spacewalk-repo-sync/yum.conf (you can break repo-syncs by creating a blank file there), and the code within spacewalk-repo-sync clearly shows the '-e' '--exclude' option.

Comment 7 Jan Pazdziora 2013-03-28 09:22:09 UTC
(In reply to comment #6)
> 
> I see that this is still missing in RHN Satellite 5.5.  I also see that it's

Most any feature is first implemented in upstream (Spacewalk) first.

If you'd like to see this RFE implemented in Satellite, I really recommend you contact your Red Hat support representative to formally track this RFE for the Satellite product.

Comment 8 Michael Mráka 2013-05-03 12:13:27 UTC
Permanent exclude/include lists for spacewalk-repo-sync can be maintained via API introduced in Spacewalk 1.8:
channel.software.addRepoFilter()
channel.software.setRepoFilters()
channel.software.clearRepoFilters()
channel.software.listRepoFilters()

Comment 10 Dimitar Yordanov 2013-07-12 19:38:14 UTC
Created attachment 772849 [details]
Patch - read include/exclude from conf file.

Comment 11 Michael Mráka 2013-07-25 15:01:15 UTC
Patch committed to spacewalk master with two changes -
a) exclude/includepkgs applies only when there's no --include/--exclude commandline option or a filter stored in database,
b) order is first includepkgs then exclude (the same order which yum uses).

commit 439593537cc9a1afa5e3f8b8877c3b124423b406
    803837 - process includepkgs and exclude from yum.conf

Comment 12 Matej Kollar 2014-01-17 12:16:18 UTC
Switching MODIFIED Spacewalk bugs to ON_QA before 2.1 release.

Comment 13 Matej Kollar 2014-03-04 13:07:26 UTC
Spacewalk 2.1 has been released.
https://fedorahosted.org/spacewalk/wiki/ReleaseNotes21

Comment 14 Matej Kollar 2014-03-04 13:08:48 UTC
Spacewalk 2.1 has been released.
https://fedorahosted.org/spacewalk/wiki/ReleaseNotes21