Bug 920795

Hi Dennis,
could you please try the newest createrepo_c epel6 build?
The version 0.1.16-2 should be already in stable: https://admin.fedoraproject.org/updates/FEDORA-EPEL-2013-0616/createrepo_c-0.1.16-2.el6

This bug is maybe related with https://bugzilla.redhat.com/show_bug.cgi?id=918645 and in 0.1.16 the bug should be fixed.

Note: createrepo_c-0.1.16 needs rpm >= 4.8.0-28 (this shouldn't be problem in epel)

createrepo_c-0.1.16-2.el6.x86_64 fails the same way on the s390x hub, it fails when run from koji, not from command line (it has different output dir)

and it worked fine in quite old rhel6 (cca 6.1) and started to crash after update to 6.4

(gdb) set args -vd -o /tmp/createrepo -u http://s390pkgs.fedoraproject.org/packages -i /mnt/koji/repos/f19-build/165457/s390/pkglist -g /mnt/koji/repos/f19-build/165457/groups/comps.xml --update --skip-stat /mnt/koji/packages/
(gdb) run
...
15:09:10: Adding pkg: /mnt/koji/packages/pcp/3.6.10/2.fc19.1/s390/perl-PCP-LogImport-3.6.10-2.fc19.1.s390.rpm
15:09:10: Adding pkg: /mnt/koji/packages/pcp/3.6.10/2.fc19.1/s390/perl-PCP-MMV-3.6.10-2.fc19.1.s390.rpm
15:09:10: Package count: 27469
Directory walk done - 27469 packages
C_CREATEREPOLIB: Critical: cr_load_xml_files: Parse error at line: 96956 (not well-formed (invalid token))
C_CREATEREPOLIB: Critical: cr_metadata_load_xml: Error encountered while parsing
15:09:19: Old metadata from /tmp/createrepo/ - loading failed
Loaded information about 0 packages
15:09:19: Copy groupfile /mnt/koji/repos/f19-build/165457/groups/comps.xml -> /tmp/createrepo/.repodata/comps.xml

Program received signal SIGSEGV, Segmentation fault.
0x0000003da86480ac in vfprintf () from /lib64/libc.so.6
Missing separate debuginfos, use: debuginfo-install bzip2-libs-1.0.5-7.el6_0.x86_64 cyrus-sasl-lib-2.1.23-13.el6_3.1.x86_64 db4-4.7.25-17.el6.x86_64 elfutils-libelf-0.152-1.el6.x86_64 expat-2.0.1-11.el6_2.x86_64 file-libs-5.04-15.el6.x86_64 glib2-2.22.5-7.el6.x86_64 glibc-2.12-1.107.el6.x86_64 keyutils-libs-1.4-4.el6.x86_64 krb5-libs-1.10.3-10.el6_4.1.x86_64 libacl-2.2.49-6.el6.x86_64 libattr-2.4.44-7.el6.x86_64 libcap-2.16-5.5.el6.x86_64 libcom_err-1.41.12-14.el6.x86_64 libcurl-7.19.7-35.el6.x86_64 libgcc-4.4.7-3.el6.x86_64 libidn-1.18-2.el6.x86_64 libselinux-2.0.94-5.3.el6.x86_64 libssh2-1.4.2-1.el6.x86_64 libxml2-2.7.6-12.el6_4.1.x86_64 lua-5.1.4-4.1.el6.x86_64 nspr-4.9.2-1.el6.x86_64 nss-3.14.0.0-12.el6.x86_64 nss-softokn-freebl-3.12.9-11.el6.x86_64 nss-util-3.14.0.0-2.el6.x86_64 openldap-2.4.23-32.el6_4.x86_64 openssl-1.0.0-27.el6_4.2.x86_64 popt-1.13-7.el6.x86_64 rpm-libs-4.8.0-32.el6.x86_64 sqlite-3.6.20-1.el6.x86_64 xz-libs-4.999.9-0.3.beta.20091007git.el6.x86_64 zlib-1.2.3-29.el6.x86_64
(gdb) where
#0  0x0000003da86480ac in vfprintf () from /lib64/libc.so.6
#1  0x0000003da8701e6c in __vasprintf_chk () from /lib64/libc.so.6
#2  0x0000003daa26dd2b in g_vasprintf () from /lib64/libglib-2.0.so.0
#3  0x0000003daa25a480 in g_strdup_vprintf () from /lib64/libglib-2.0.so.0
#4  0x0000003daa242fe0 in g_logv () from /lib64/libglib-2.0.so.0
#5  0x0000003daa243413 in g_log () from /lib64/libglib-2.0.so.0
#6  0x0000003db4a0aafe in cr_metadatalocation_free (ml=0x13bab10) at /usr/src/debug/createrepo_c-0.1.16/src/locate_metadata.c:45
#7  0x0000000000404724 in main (argc=2, argv=0x7fffffffe338) at /usr/src/debug/createrepo_c-0.1.16/src/createrepo_c.c:817
(gdb) 


I get this error when run against an existing repodata in /tmp/createrepo

and I guess it's a non-UTF8 encoded character in the metadata (other.xml)

It must really be a problem with wrong encoding, when createrepo_c is run with old repodata prepared by createrepo, it finishes correctly, but writes bad xml file. The problem appears on metadata (changelog) for the gdmap package in other.xml.

it may be because of the encoding in the spec file,  but createrepo_c needs to cope with it. its not an issue for createrepo

the character in question seems to be ä which is utf-8 createrepo_c needs to support utf-8

Hi, thank you for a such detailed report, you are right.
There were two bugs:

The first one could lead to a SIGSEGV while parsing a non valid XML. (fixed in: http://git.fedorahosted.org/cgit/createrepo_c.git/commit/?id=58139db137203ea1398d318d4eac04da237a2020)

The second bug was an omitted sanitation of encoding during XML generation. It could leads to invalid characters in output XML. (fixed in: http://git.fedorahosted.org/cgit/createrepo_c.git/commit/?id=6baf36aeee3ce8018103667f1f1a0fa9260a7713)

Could you please try the scratch build:
http://koji.fedoraproject.org/koji/taskinfo?taskID=5169119

Thanks, the issue I saw is gone, going to set createrepo_c again in production.

createrepo_c-0.1.17-1.el6 has been submitted as an update for Fedora EPEL 6.
https://admin.fedoraproject.org/updates/createrepo_c-0.1.17-1.el6

createrepo_c-0.1.17-1.el6 has been pushed to the Fedora EPEL 6 stable repository.  If problems still persist, please make note of it in this bug report.