Bug 494951

Summary: groups file get wrong checksum when using "-s sha1" together with "--unique-md-filenames"
Product: [Fedora] Fedora Reporter: Thorsten Leemhuis <fedora>
Component: createrepoAssignee: Luke Macken <lmacken>
Status: CLOSED RAWHIDE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: medium Docs Contact:
Priority: low    
Version: rawhideCC: fabrice, fedora, james.antill, john, lmacken, me, pfrields
Target Milestone: ---Keywords: Reopened
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2009-05-05 17:13:35 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
use config checksum in read_in_package none

Description Thorsten Leemhuis 2009-04-08 19:37:45 UTC
Description of problem:
Seems createrepo calculates wrong checksums for groups file when using both "-s sha1" and  "--unique-md-filenames"

See:

[thl@thl tmp]$ mkdir tmp; cd tmp
[thl@thl tmp]$ wget -q http://download.fedora.redhat.com/pub/fedora/linux/updates/testing/10/i386/backup-light-0.4-2.fc10.noarch.rpm http://download.fedora.redhat.com/pub/fedora/linux/updates/testing/10/i386/repodata/comps-f10.xml
[thl@thl tmp]$ createrepo -q -s sha1 -g comps-f10.xml .
[thl@thl tmp]$ grep -A 1 comps-f10.xml repodata/repomd.xml
    <location href="repodata/comps-f10.xml.gz"/>
    <checksum type="sha1">81f668988c8478c20b16988359ade81ccb1398b0</checksum>
--
    <location href="repodata/comps-f10.xml"/>
    <checksum type="sha1">47134e1d6030b6772bc8523a78e532dba71aedf7</checksum>
[thl@thl tmp]$ grep -A 1 comps-f10.xml repodata/repomd.xml
    <location href="repodata/c78f596799c4911050f6a40458060af0231a596cbda32d8313c3874ce9e6b9c7-comps-f10.xml.gz"/>
    <checksum type="sha1">c78f596799c4911050f6a40458060af0231a596cbda32d8313c3874ce9e6b9c7</checksum>
--
    <location href="repodata/b9fd41cf97788f5365c97ac5cbf548abfdd5061d7e5414856a52c097d3dbad0a-comps-f10.xml"/>
    <checksum type="sha1">b9fd41cf97788f5365c97ac5cbf548abfdd5061d7e5414856a52c097d3dbad0a</checksum>
[thl@thl tmp]$ 

The checksum is different and longer in the second case.

Version-Release number of selected component (if applicable):
createrepo-0.9.7-1.fc11.noarch

Additional information:
Actually ran into this for RPM Fusion after updating to new yum/createrepo on a RHEL5 system, but could reproduced it on rawhide (see above)

Comment 1 Thorsten Leemhuis 2009-04-08 19:40:23 UTC
sorry, forgot to cut-n-paste one important line; here it is again:

[thl@thl tmp]$ mkdir tmp; cd tmp
[thl@thl tmp]$ wget -q
http://download.fedora.redhat.com/pub/fedora/linux/updates/testing/10/i386/backup-light-0.4-2.fc10.noarch.rpm
http://download.fedora.redhat.com/pub/fedora/linux/updates/testing/10/i386/repodata/comps-f10.xml
[thl@thl tmp]$ createrepo -q -s sha1 -g comps-f10.xml .
[thl@thl tmp]$ grep -A 1 comps-f10.xml repodata/repomd.xml
    <location href="repodata/comps-f10.xml.gz"/>
    <checksum type="sha1">81f668988c8478c20b16988359ade81ccb1398b0</checksum>
--
    <location href="repodata/comps-f10.xml"/>
    <checksum type="sha1">47134e1d6030b6772bc8523a78e532dba71aedf7</checksum>
[thl@thl tmp]$ createrepo -q  --unique-md-filenames -s sha1 -g comps-f10.xml .
[thl@thl tmp]$ grep -A 1 comps-f10.xml repodata/repomd.xml
    <location href="repodata/c78f596799c4911050f6a40458060af0231a596cbda32d8313c3874ce9e6b9c7-comps-f10.xml.gz"/>
    <checksum type="sha1">c78f596799c4911050f6a40458060af0231a596cbda32d8313c3874ce9e6b9c7</checksum>
--
    <location href="repodata/b9fd41cf97788f5365c97ac5cbf548abfdd5061d7e5414856a52c097d3dbad0a-comps-f10.xml"/>
    <checksum type="sha1">b9fd41cf97788f5365c97ac5cbf548abfdd5061d7e5414856a52c097d3dbad0a</checksum>
[thl@thl tmp]$

Comment 2 seth vidal 2009-04-13 18:07:22 UTC
You're right-  the checksum type wasn't being propagated there.

That's fixed now.
http://createrepo.baseurl.org/gitweb?p=createrepo.git;a=commit;h=5dd00f425f64cdc1bfc1005f560fb4761abac007

Thanks

Comment 3 seth vidal 2009-05-04 14:28:26 UTC
*** Bug 498767 has been marked as a duplicate of this bug. ***

Comment 4 Remi Collet 2009-05-05 04:46:10 UTC
@seth I reopen this bug since the diff pointed in Comment #2 is already applied in the 0.9.7-6 version (in createrepo-head.patch) but this doesn't fix the Bug 498767

sha256 still used in primary.xml

Regards

Comment 5 Remi Collet 2009-05-05 05:01:07 UTC
Created attachment 342411 [details]
use config checksum in read_in_package

Comment 6 Thorsten Leemhuis 2009-05-05 05:08:24 UTC
FWIW, I also ran into more trouble from RPM Fusion users with F9 or EL5 after the problems this bug is about was fixed; but I was not able to properly track it down yet -- sorry, there are more pressing issues and so I just went back to an older createrepo for F9 and EL5

Comment 7 seth vidal 2009-05-05 14:34:33 UTC
okay. I've built a newer createrepo with the latest patch from git upstream.

I cannot replicate any sha/sha1 issues with this package.

rawhide:
http://kojipkgs.fedoraproject.org/packages/createrepo/0.9.7/7.fc12/noarch/createrepo-0.9.7-7.fc12.noarch.rpm

f11:
 http://kojipkgs.fedoraproject.org/packages/createrepo/0.9.7/7.fc11/noarch/createrepo-0.9.7-7.fc11.noarch.rpm



let me know if it doesn't work.

Comment 8 Remi Collet 2009-05-05 16:43:01 UTC
This build solves the issue I've detected.

Thanks for this.

Regards.

Comment 9 Thorsten Leemhuis 2009-05-05 17:13:35 UTC
(In reply to comment #8)
> This build solves the issue I've detected.

Same here. I'll test with RPM Fusion over the next few days and will reopen this bug if any other problems show up

> Thanks for this.

+1

Comment 10 Thorsten Leemhuis 2009-05-08 10:33:39 UTC
Seems there is still one more bug hidden somewhere, as the data RPM Fusion generates with the latest createrepo (running on RHEL5 with manual updated RPM and yum) is unusable in F9 anaconda. See
https://bugzilla.rpmfusion.org/show_bug.cgi?id=536

/me wonders if the "sha1" (instead of "sha", as it was in the past) could lead to this problem

Someone also mentioned on IRC that the data RPM Fusion generates for RHEL does not work on older RHEL-5-Version (no exact data, but I suppose something older than 5.2 or 5.3). But he said it works fine after updating yum.

Seth, is it worth investigating this further?

Comment 11 Fabrice Bellet 2009-05-14 16:10:08 UTC
To provide compatibility with F9 yum version, maybe createrepo could accept the '-s sha' option, pass sha1 instead of sha to yum's misc.Checksums(), and keep the 'sha' name in the files created in repodata ?

Comment 12 Thorsten Leemhuis 2009-05-14 16:27:55 UTC
(In reply to comment #11)
> To provide compatibility with F9 yum version, maybe createrepo could accept the
> '-s sha' option, pass sha1 instead of sha to yum's misc.Checksums(), and keep
> the 'sha' name in the files created in repodata ?  

From my current point of view everything is working after using latest yum from rawhide with this patch on the machine that is running createrepo:

http://yum.baseurl.org/gitweb?p=yum.git;a=commitdiff;h=a8f894049d43b37aa429468999e89531206cfd3d

So what you suggest seems to complicated to me. But to not confuse people it maybe would be nice/wise if createrepo would write "sha" everywhere to the repo files if the users uses "-s sha1"

Comment 13 James Antill 2009-05-14 17:05:26 UTC
Seth fixed yum.misc.Checksums() to accept sha.
I added some warning text to createrepo. Both those things should make it much more obvious what is happening.

Comment 14 Fabrice Bellet 2009-05-14 17:46:39 UTC
The patch to yum works for me too. Thanks!