Description of problem: The gzipped groups.xml generated by pulp 2.0 and 2.1 is not compatible with anaconda ver 13.21.195-1.el6 (RHEL 6.4) due to the whiteout/blacklist plugins. See https://bugzilla.redhat.com/show_bug.cgi?id=949992 for more. Although this works fine with yum outside of anaconda, can we please have an option to disable groups.xml compression for compatibility with anaconda?
<jdob> could we just provide both compressed and uncompressed and be done with it? <jmatthews> yeah, we can provide both, that's what I used to do in pulp v1. uncompressed must be labeled 'group' in repomd.xml, and compressed(gzipped) would be labeled "group_gz"
The newer version of modifyrepo provides a flag (--compress) to compress the file being added into the repomd. The code and help indicates that the default is to NOT do the compression. But, when running a test 'modifyrepo comps.xml repodata', it compressed the file by default. Looking at the code in: /usr/share/createrepo/modifyrepo.py:116 an 'or' is used when I think and 'and' was intended. do_compress = False ## Compress the metadata and move it into the repodata --> if self.compress or not mdname.split('.')[-1] in ('gz', 'bz2', 'xz'): do_compress = True mdname += '.' + self.compress_type mdtype = self._get_mdtype(mdname, mdtype) destmd = os.path.join(self.repodir, mdname) if do_compress: newmd = compressOpen(destmd, mode='wb', compress_type=self.compress_type) else: newmd = open(destmd, 'wb')
Filed this BZ for modifyrepo: https://bugzilla.redhat.com/show_bug.cgi?id=950724
Looking back to RHEL 6.2, modifyrepo seems to have always compressed the metadata. However createrepo has and continues to not compress the "group" metadata when included using the -g. This suggests that Pulp started compressing the "group" metadata when we switched from using createrepo to modifyrepo. The incompatibility with anaconda was introduced at this time.
Patched version of modifyrepo carried in the pulp yum repos. build: 2.1.1-0.4.beta
[root@preethi ~]# pulp-admin rpm repo create --repo-id rhel6_2 --feed https://cdn.redhat.com/content/dist/rhel/rhui/server/6/6.2/x86_64/os/ --feed-ca-cert CDN/cdn.redhat.com-chain.crt --feed-cert CDN/1359391926_4512.crt --feed-key CDN/1359391926_4512.key Successfully created repository [rhel6_2] Synced & Published the repo over https & https [root@preethi ~]# pulp-admin repo list --details +----------------------------------------------------------------------+ Repositories +----------------------------------------------------------------------+ Id: rhel6_2 Display Name: rhel6_2 Description: None Content Unit Counts: Distribution: 1 Erratum: 1048 Package Category: 10 Package Group: 202 Rpm: 7281 Notes: Importers: Config: Feed URL: https://cdn.redhat.com/content/dist/rhel/rhui/server/6/6.2/ x86_64/os/ Version: 3 (0x2)ificate: 91:8a:cf:ca:85:00:00:00 Issuer: Signature Algorithm: sha1WithRSAEncryption C=US, ST=North Carolina, L=Raleigh, O=Red Hat, Inc., OU=Red Hat Network, CN=Entitlement Master Validity CA/emailAddress=ca-support Not Afterot Before: Mar 18 11:24:54 2010 GMT Subject: C=US, ST=North:24:54 2030 GMT Carolina, O=Red Hat, Inc., OU=Red Hat Network, CN=Red Hat Entitlement Operations Authority/emailAddress=ca-support Public Key Algorithm:c Key Info: RSA Public Key: (4096 bit) Modulus (4096 bit): X509v3 Subject Keyensions: Identifier: C4:49:78:56:45:9D:1E:2C:61:60:C3:66:4B:58:05:37:2F:03:22:7B X509v3 Authority Key Identifier: keyid:88:4B:A6:45:C4:46:EC:8F:D6:CF:A6:F6:1A:CF:26:3C:90:CC DirName:/C=US/ST=North Carolina/L=Raleigh/O=Red Hat, Inc./OU=Red Hat Network/CN=Entitlement Master CA/emailAddress=ca-support X509v3 Basical:E6:FE:42:28:25:C9:E6:5E X509v3:TRUEstraints: Certificate Sign, CRL Sign SSL CA, S/MIME CA Type: X509v3 Subject Alternative Name: X509v3 Issuer:ca-support Alternative Name: Signature Algorithm:l:ca-support sha1WithRSAEncryption -----BEGIN CERTIFICATE-----3:c1:c2:5c:14:a1mOH1F+vfumZKzW78LlVD -----ENDAUCiUsNyYdJbSZDPiFPBwlwUoQ== -----BEGIN CERTIFICATE-----CATE----- CERTIFICATE----- SSL Client Cert: -----BEGIN CERTIFICATE----- -----END CERTIFICATE----- SSL Client Key: -----BEGIN RSA PRIVATE KEY----- -----END RSA PRIVATE KEY----- Id: yum_importer Importer Type Id: yum_importer Last Sync: 2013-04-16T12:35:31-04:00 Repo Id: rhel6_2 Scheduled Syncs: Distributors: Auto Publish: True Config: Http: True Https: True Relative URL: /content/dist/rhel/rhui/server/6/6.2/x86_64/os/ Distributor Type Id: yum_distributor Id: yum_distributor Last Publish: 2013-04-16T12:43:42-04:00 Repo Id: rhel6_2 Scheduled Publishes: Auto Publish: False Config: Http: True Https: True Distributor Type Id: export_distributor Id: export_distributor Last Publish: None Repo Id: rhel6_2 Scheduled Publishes: Now used the synced repo to kickstart a new vm [root@qeblade20 ~]# virt-install -n pulp-211 --disk path=/dev/vg_qeblade20/pulp-211 -l http://preethi.usersys.redhat.com/pulp/repos/content/dist/rhel/rhui/server/6/6.2/x86_64/os/ -r 4096 -w bridge:br0 --prompt
verified .[root@gizmo ~]# rpm -q pulp-server pulp-server-2.1.1-0.4.beta.el6.noarch [root@gizmo ~]# rpm -q createrepo createrepo-0.9.9-18.el6.noarch [root@gizmo ~]# I created & synced a Rhel6.4 repo on my pulp-server. Used that repo to kickstart a 6.4 vm. [root@gizmo ~]# hostname gizmo.idm.lab.bos.redhat.com [root@gizmo ~]# pulp-admin rpm repo create --repo-id rhel6-4 --feed https://cdn.redhat.com/content/dist/rhel/rhui/server/6/6.4/x86_64/os/ --feed-ca-cert cdn.redhat.com-chain.crt --feed-cert 1359391926_4512.crt --feed-key 1359391926_4512.key [root@gizmo ~]# time pulp-admin rpm repo sync run --repo-id rhel6-4 +----------------------------------------------------------------------+ Synchronizing Repository [rhel6-4] +----------------------------------------------------------------------+ This command may be exited by pressing ctrl+c without affecting the actual operation on the server. Downloading metadata... [|] ... completed Downloading repository content... [==================================================] 100% RPMs: 1/1 items Delta RPMs: 0/0 items Tree Files: 7/7 items Files: 0/0 items ... completed Importing errata... [/] ... completed Importing package groups/categories... [\] ... completed Publishing packages... [==================================================] 100% Packages: 10356/10356 items ... completed Publishing distributions... [==================================================] 100% Distributions: 7/7 items ... completed Generating metadata [-] ... completed Publishing repository over HTTPS [-] ... completed Publishing repository over HTTP [-] ... completed real 25m5.418s user 0m28.175s sys 0m1.162s [root@qeblade20 ~]# virt-install -n pulp-el64 --disk path=/dev/vg_qeblade20/pulp-el64 -l http://gizmo.idm.lab.bos.redhat.com/pulp/repos/content/dist/rhel/rhui/server/6/6.4/x86_64/os/ -r 4096 -w bridge:br0 --prompt
2.1.1 released
*** Bug 961426 has been marked as a duplicate of this bug. ***