Re-created with Pulp 0.258 on el6: 1) pulp-admin repo create --id epel_6_64 --feed http://dl.fedoraproject.org/pub/epel/6/x86_64/ 2) pulp-admin repo sync --id epel_6_64 -F 3) Register an el6 consumer to this repo 4) On consumer: 'yum search vim' Loaded plugins: amazon-id, pulp-profile-update, rhui-lb, security epel_6_64 | 4.1 kB 00:00 epel_6_64/primary_db | 4.0 MB 00:00 epel/pkgtags | 323 B 00:00 epel_6_64/pkgtags | 306 B 00:00 Error: database disk image is malformed <data type="pkgtags"> <location href="repodata/8c7408c4d344432fb0d7fa8be2e2f10267808e7cecf3b1881335591beaaeec84-pkgtags.sqlite.gz"/> <checksum type="sha256">8c7408c4d344432fb0d7fa8be2e2f10267808e7cecf3b1881335591beaaeec84</checksum> <timestamp>1326812228.0</timestamp> <open-checksum type="sha256">4c10213f55cf8b0f584d50126e8fe05afd9778a06cb18847ee7e861e1d7aedae</open-checksum> </data> Note: I also pointed the consumer to the EPEL repo directly, ran "yum clean all" and re-did search. This was successful.
I'm also getting this error for all my Pulp Clients Fedora 15-16, pulp version 254 tried on clients: yum clean all also tried it with: rm /var/lib/rpm/__db* rpm --rebuilddb But nothing helps, errors stays the same "database disk image is malformed". Resync the repo on the pulp-server also doesn't helps.
Hi Stefan, Are you able to determine the specific repo that is giving you this errror? If so, would you please post the feed URL? Thank you, John
Yes I know which repo it is. Sorry, I forgot to mention it in my previous post. Feed URL is http://ftp-stud.hs-esslingen.de/pub/fedora/linux/updates/16/x86_64/
What's the status of this issue? I'm also experiencing this problem with epel: [root@icinga ~]# yum search icinga Loaded plugins: fastestmirror, priorities, security Loading mirror speeds from cached hostfile 1532 packages excluded due to repository priority protections stable-epel-6-x86_64/pkgtags | 311 B 00:00 Error: database disk image is malformed [root@pulp repodata]# zgrep pkgtags ./* ./repomd.xml: <data type="pkgtags"> ./repomd.xml: <location href="repodata/f30707ebe14eeee23b337994c90a23c622649c5debe3fe3f7120aad5da847851-pkgtags.sqlite.gz"/> [root@pulp repodata]# zcat f30707ebe14eeee23b337994c90a23c622649c5debe3fe3f7120aad5da847851-pkgtags.sqlite.gz ?..??##?otablepackagetagspackagetagsCREATE TABLE packagetags ( name TEXT NOT NULL, tag TEXT NOT NULL, score INTEGER, PRIMARY KEY (name, tag) )5I#indexsqlite_autoindex_packagetags_1packagetags [root@pulp repodata]# pwd /var/lib/pulp/repos/stable/epel/6/x86_64/repodata [root@pulp repodata]# pulp-admin repo info --id=^C [root@pulp repodata]# pulp-admin repo info --id=stable-epel-6-x86_64 Id stable-epel-6-x86_64 Name stable-epel-6-x86_64 Repo URL https://domain/pulp/repos/stable/epel/6/x86_64/ Feed URL file:///var/lib/pulp//repos/daily/epel/6/x86_64 Feed Type local Content Type yum Feed Certs CA:No Cert:No Consumer Certs CA:No Cert:No Architecture noarch Sync Schedule None Packages 7795 Files 0 Distributions None Publish True Clones [] Groups None Filters [] Notes {} Preserve Metadata False Checksum Type sha256 [root@pulp repodata]# rpm -qa | grep pulp python-isodate-0.4.4-4.pulp.el6.noarch pulp-admin-1.1.11-1.el6.noarch pulp-1.1.11-1.el6.noarch pulp-common-1.1.11-1.el6.noarch pulp-client-lib-1.1.11-1.el6.noarch pulp-consumer-1.1.11-1.el6.noarch mod_wsgi-3.3-3.pulp.el6.x86_64 pulp-selinux-server-1.1.11-1.el6.noarch m2crypto-0.21.1.pulp-7.el6.x86_64 python-oauth2-1.5.170-2.pulp.el6.noarch Please let me know if any additional info is required! Thanks.
addendum: [root@pulp repodata]# pulp-admin repo info --id=daily-epel-6-x86_64 Id daily-epel-6-x86_64 Name daily-epel-6-x86_64 Repo URL https://domain/pulp/repos/daily/epel/6/x86_64/ Feed URL http://mirrors.kernel.org/fedora-epel/6/x86_64 Feed Type remote Content Type yum Feed Certs CA:No Cert:No Consumer Certs CA:No Cert:No Architecture noarch Sync Schedule P1D Packages 7795 Files 0 Distributions None Publish True Clones [] Groups [u'epel'] Filters [] Notes {} Preserve Metadata False Checksum Type sha256
Was able to work around this issue by deleting and recreating daily-epel with --preserve-metadata. On a related note pulp-admin repo update should take preserve_metadata as an option. I had to delete and create...which caused issues since this repo is heavily used. :(
This came up on the mailing list recently. Still is an issue with 2.0. I did the repo creates on 2.0.6 release and initial epel sync on 2.0.7 0.2.beta. if I set "enabled = 0" for my epel-el6 repo entry in /etc/yum.repos.d/pulp.repo on the consumer "yum search" works again. so sounding like something in the yum metadata epel has is making the pulp publish (or on the import side) do something odd.
and and I forgot. pulp-server is RHEL 6 (64bit). ditto for the pulp consumer. both have recent OS updates (within the last week or so). If specific versions/config is needed, please let me know.
Updating the bug with my testing results. I can't seem to reproduce the issue Server [root@preethi ~]# rpm -q pulp-server pulp-server-2.0.7-0.4.beta.noarch [root@preethi ~]# [root@preethi ~]# cat /etc/redhat-release Fedora release 17 (Beefy Miracle) [root@preethi ~]# Client [root@el6-client ~]# cat /etc/redhat-release Red Hat Enterprise Linux Server release 6.0 (Santiago) [root@el6-client ~]# [root@preethi ~]# pulp-admin repo list --details Id: rh6-epel-live Display Name: rh6-epel-live Description: None Content Unit Count: 10523 Notes: Importers: Config: Feed URL: http://dl.fedoraproject.org/pub/epel/6/x86_64/ Id: yum_importer Importer Type Id: yum_importer Last Sync: 2013-01-18T15:39:49-05:00 Repo Id: rh6-epel-live Scheduled Syncs: Distributors: Auto Publish: True Config: Http: False Https: True Relative URL: /pub/epel/6/x86_64/ Distributor Type Id: yum_distributor Id: yum_distributor Last Publish: 2013-01-18T15:41:30-05:00 Repo Id: rh6-epel-live Scheduled Publishes: Auto Publish: False Config: Http: False Https: True Distributor Type Id: export_distributor Id: export_distributor Last Publish: None Repo Id: rh6-epel-live Scheduled Publishes: Id: daily-epel-6-x86_64 Display Name: daily-epel-6-x86_64 Description: None Content Unit Count: 10523 Notes: Importers: Config: Feed URL: http://mirrors.kernel.org/fedora-epel/6/x86_64 Id: yum_importer Importer Type Id: yum_importer Last Sync: 2013-01-18T18:21:49-05:00 Repo Id: daily-epel-6-x86_64 Scheduled Syncs: Distributors: Auto Publish: True Config: Http: False Https: True Relative URL: /fedora-epel/6/x86_64 Distributor Type Id: yum_distributor Id: yum_distributor Last Publish: 2013-01-18T18:23:15-05:00 Repo Id: daily-epel-6-x86_64 Scheduled Publishes: Auto Publish: False Config: Http: False Https: True Distributor Type Id: export_distributor Id: export_distributor Last Publish: None Repo Id: daily-epel-6-x86_64 Scheduled Publishes: [root@preethi ~]# pulp-admin rpm consumer list --details +----------------------------------------------------------------------+ Consumers +----------------------------------------------------------------------+ Id: rhel6-client Display Name: rhel6-client Bindings: Confirmed: rhel6_3, rh6-epel-live Unconfirmed: Capabilities: Certificate: -----BEGIN CERTIFICATE----- MIICGzCCAQMCAQIwDQYJKoZIhvcNAQEFBQAwFDESMBAGA1UEAxMJbG9jYWxob3N0 MB4XDTEzMDExODE2MDI0NVoXDTIzMDExNjE2MDI0NVowFzEVMBMGA1UEAxMMcmhl bDYtY2xpZW50MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDPut85tCbCz491 gltxRQDzbRRBKBXi3QLXAxZuqByyv1xxNsTjzDifkB0wNzTj5leYUBzul+beZlfO BfViyW0EUmI+SqTwA356oQMiDHtwaBePER+CzX5go4/ihL2tzOZowbmqMrOpGnJk kDoUcv0HXDZInMlPW6qCDs16oWrO5QIDAQABMA0GCSqGSIb3DQEBBQUAA4IBAQAB Evn09ecYZSp4SrGKfB5vtPvk94/BTRap2XTm/xdp7oYpSOw8kw5tDPDYLrCL1/7C KaLmZ0nbZH37PkzFhKu983beldRltLdDG5Tu9TR44v9oeZvNJwquKmEdL/J0vz4Q RRa8LLWphCz3IPYURHKpbIYRQEVYHDeuBEmNuG5uPhuMaGR4zS5+g3a1GL2MReoU LZ3DW5WlURjL/MM7PkHbKj4H+87JO5CzDITuFfUCw5SDbyVxo1I6BoInSH9Dd8yd AfWuGHowLFUDiu05gfWqNIa1oSch2bDa9hcaVgmnm/DakTe3lL0Ce5b3uLK5Gsn9 sekhJA5vhyXzU3jKDtIC -----END CERTIFICATE----- Description: None Notes: Id: el6-client Display Name: el6-client Bindings: Confirmed: rh6-epel-live, daily-epel-6-x86_64 Unconfirmed: Capabilities: Certificate: -----BEGIN CERTIFICATE----- MIICGTCCAQECAQMwDQYJKoZIhvcNAQEFBQAwFDESMBAGA1UEAxMJbG9jYWxob3N0 MB4XDTEzMDEyMTE0NTM1NVoXDTIzMDExOTE0NTM1NVowFTETMBEGA1UEAxMKZWw2 LWNsaWVudDCBnzANBgkqhkiG9w0BAQEFAAOBjQAwgYkCgYEA08yDoqvdF+50E7CW 4w0ZA3UZXo18dNN7QtdFJtJ1OnmkZpfYh4qB7KflUeuSaMCy6R4PKf+YrC6UMtYl BK+l733YhZk+tyI4KSRUKc7hJkL3hz2ZiSnWCnOUGY0vzCVoxFjVQZB8kPWmnu3F gm2cYbm5DHFTfHbubm6oGEP7bWcCAwEAATANBgkqhkiG9w0BAQUFAAOCAQEAWnIr Nj1TcdDiv3iNa+s5o74Dm1sn6tm5iuO3drAfxpQ3HZpyw4Nht1uJjSM9T8cpCa5z i2FaBR66siacrtiIwoOdbgvjt/yZCuVMaHcfRFP+UDScUh6NfoF+MjpALuCrMMdo 6jRxeLGP7Gc6Cuv9RsCMvO+ockJjAiYRS72DSOfarQKWXvZt5XT6eqAeyRIm/uU8 nE4HOe61+yGGyfOqEf6xjlbmgOYrbmuo7/e74NhixiZmIDYHooXXc6YU1lA9acx0 BQdzYTNSHsrTq9hqStolWJCzLDCJtSRK3BcHUm4gCboTU1wK0k0QcAdrpPQGnFZO PdH/q6aCA1gcZFb6uw== -----END CERTIFICATE----- Description: None Notes: [root@preethi ~]# On the client [root@el6-client ~]# yum search zsh Loaded plugins: rhnplugin This system is not registered with RHN. RHN support will be disabled. Failed to add Pkg Tags for repository: daily-epel-6-x86_64 - file is encrypted or is not a database Failed to add Pkg Tags for repository: rh6-epel-live - file is encrypted or is not a database ================================= Matched: zsh ================================= autojump-zsh.noarch : Autojump for zsh gromacs-zsh.noarch : GROMACS zsh support zsh.x86_64 : A powerful interactive shell environment-modules.x86_64 : Provides dynamic modification of a user's : environment plowshare.noarch : Download and upload files from file-sharing websites shflags.noarch : Simple handling of command-line flags in Bourne based Unix : scripts [root@el6-client ~]# [root@el6-client ~]# yum repolist Loaded plugins: rhnplugin This system is not registered with RHN. RHN support will be disabled. repo id repo name status daily-epel-6-x86_64 daily-epel-6-x86_64 8,312 pulp-v2-candidate Pulp v2 Beta Builds 36 rh6-epel-live rh6-epel-live 8,312 repolist: 16,660 [root@el6-client ~]#
any way you can test with el6 as the server? f17 has newer rpm,yum than el6. f17: rpm-4.9.1.3-7.fc17 yum-3.4.3 rl6: rpm-4.8.0-27.el6.x86_64 yum-3.2.29-30.el6.noarch
okay, I have narrowed down the problem file a bit. the summary of this bug is a good hint. the epel repo has a <big long hex string>-pkgtags.xml.gz file. none of the other repos have one. it isn't a compressed xml file. it is actually a compressed basically empty sqlite3 DB. If I manually remove it from the published directory on the pulp-server and on the consumer. yum search works fine. But of course next repo sync makes the file again. So sounds like some more problem metadata fun.
spoke to the yum maintainer and concluded that pkgtags are not really used and basically ignored in fedora land as of today. For this particular repo the pkgtags are empty if you load the sqlite db. So pulp for now will skip pkgtags by default, we provided a switch in yum_distributor.conf if user prefers to manually enable it. Also newer versions of createrepo and yum in el6.4 handle the database error better and skip pkgtags on clients. merged fix to pulp-2.0 and master. After applying the fix, * Just republish your repo on server # pulp-admin rpm repo publish run --repo-id=rh6-epel-live * Clean yum cache on client # yum clean all or rm -rf /var/cache/yum/* * Do yum operations as usual # yum search zsh
excellent. yeah it seemed like pkgtags were not really used. I have been toying with a publish event hook to remove the pkgtags file after the publish completed since removing it manually fixed the issue. Good to hear that will be done in pulp itself soon.
build: 2.0.8-0.1.beta
fixed distributor validation to account for keys in yum_distributor.conf merged to pulp-2.0 and master
Installed 2.0.8-0.1.beta. did a repo sync+publish. yum clean all on consumer. yum search now works as expected. thank you.
build: 2.0.8-0.2.beta
[root@preethi-el6-pulp ~]# pulp-admin rpm consumer list --details +----------------------------------------------------------------------+ Consumers +----------------------------------------------------------------------+ Id: el6-client Display Name: el6-client Bindings: Confirmed: rh6-epel-live Unconfirmed: Capabilities: Certificate: -----BEGIN CERTIFICATE----- MIICGTCCAQECAQEwDQYJKoZIhvcNAQEFBQAwFDESMBAGA1UEAxMJbG9jYWxob3N0 MB4XDTEzMDEzMTEzMDEwN1oXDTIzMDEyOTEzMDEwN1owFTETMBEGA1UEAxMKZWw2 LWNsaWVudDCBnzANBgkqhkiG9w0BAQEFAAOBjQAwgYkCgYEAqP+ony3jSKyYJOrD EjDjF+FzOnmHLQXopd/qm4aaGBsjfnOImItFJN5dmO9tcJmsfIY+GglzoNCDlzvC /UuQ0pXALIKY4VmdY5Z+JsBg2iNmpdISZ3/tDNW9kP9IgZoeusQmQqc40bhF/OqX akf4S8jhOf5xLd5hnpvWJsF6AIUCAwEAATANBgkqhkiG9w0BAQUFAAOCAQEAYwlg jUU1Dj4iAahtFwifT66d0ZaPVjQq5xGa6pDq9ocJVJp5I+hOdtZ2dYj4Vb+SzjiU RyKdBI0q3OOeV8sJjflhN6TEPHrABxuJ2kOq4BDTyam+hG38g4+0FXFMSkRtI4uy AuVHS4rL3YjWmdoP2kK8UnOsTEYi6PL9x2ts5h8rs3JZ3GsGtbydgT1tUNCfkgsX +ZwzoKSN/8Z7We6FsxsDj0Kf7mZYlE+E6EvLsd+MQl/lxjqwuJOWdJrbN1TirQD0 NcbK/dJ/wGHSARExSsZSL/P1z8kpYYFCCkouYT5VG88eQTd9lzNuJbhSkIdJAJ8z wyfvGjnZUwjyRq3egA== -----END CERTIFICATE----- Description: None Notes: [root@el6-client ~]# yum repolist Loaded plugins: rhnplugin This system is not registered with RHN. RHN support will be disabled. pulp-v2-candidate | 1.9 kB 00:00 rh6-epel-live | 3.8 kB 00:00 rh6-epel-live/primary_db | 4.4 MB 00:00 repo id repo name status pulp-v2-candidate Pulp v2 Beta Builds 36 rh6-epel-live rh6-epel-live 8,336 repolist: 8,372 [root@el6-client ~]# yum search zshLoaded plugins: rhnplugin This system is not registered with RHN. RHN support will be disabled. ================================= Matched: zsh ================================= autojump-zsh.noarch : Autojump for zsh gromacs-zsh.noarch : GROMACS zsh support plowshare.noarch : Download and upload files from file-sharing websites shflags.noarch : Simple handling of command-line flags in Bourne based Unix : scripts [root@el6-client ~]#
Moving to verified per the above comment. [root@preethi-el6-pulp ~]# rpm -q pulp-server pulp-server-2.0.8-0.2.beta.noarch [root@preethi-el6-pulp ~]#
Pulp 2.1 released http://www.pulpproject.org/2013/04/05/pulp-2-1-0-released/