Bug 1315423 - fedpkg-copr does not work properly with fedpkg-1.21-2.fc23
fedpkg-copr does not work properly with fedpkg-1.21-2.fc23
Status: CLOSED CURRENTRELEASE
Product: Copr
Classification: Community
Component: backend (Show other bugs)
unspecified
Unspecified Unspecified
unspecified Severity unspecified
: ---
: ---
Assigned To: clime
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2016-03-07 12:02 EST by clime
Modified: 2017-02-06 11:26 EST (History)
3 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-09-13 20:56:35 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
fedpkg-copr shell session (9.11 KB, text/plain)
2016-03-07 12:02 EST, clime
no flags Details

  None (edit)
Description clime 2016-03-07 12:02:04 EST
Created attachment 1133875 [details]
fedpkg-copr shell session

Hello, fedpkg-copr (HEAD) does not work properly with fedpkg-1.21-2.fc23 that was released 16 days ago (it works with the previous version 1.20-2.fc23).

In the attached file, you can see a working test before upgrade and failing test after upgrade of fedpkg.

The problem is with a different generated lookaside url that additionally contains hash type ('md5' usually). 

This different lookside url format was introduced into fedpkg with this commit: https://pagure.io/fedpkg/9f5468de1191fa67959226e62d6ba5046d3358b9
Comment 1 Jakub Kadlčík 2016-05-25 17:30:36 EDT
Well, I've looked on this and I suggest following solution.

In dist-git scripts there is an upload.cgi which handles the process of uploading files. Although in copr-dist-git we *do not* use this script at all.

We use simple function that only copies source files from /tmp to cache directory. The directory has following structure.

/var/lib/dist-git/cache/lookaside/pkgs/<user>/<project>/<package>/<package>-<version>.tar.gz/<hash>/<package>-<version>.tar.gz

Only thing that we need to do is add md5/ directory before <hash>.


I suggest following solution.

First of all we need to modify our current cache structure. For instance this way:


#!/bin/bash

DIR=/var/lib/dist-git/cache/lookaside/pkgs/*/*/*/*

for dir in $DIR; do
	for hash in $(find $dir/* -maxdepth 0 | grep -v "md5$"); do
		echo I will create: $dir/md5/`basename $hash`

		# Uncomment these lines only when
		# you are really sure what are you doing
		# mkdir -p $dir/md5
		# cp -r $hash $dir/md5
	done
done


After this, if you upgrade fedpkg on backend to >=1.21, you will be able to obtain src.rpm package by 

fedpkg-copr --verbose --dist f23 srpm

as it is showed in attached reproducer, which would otherwise failed.


To upload new sources to correct path, we can apply this patch


commit bb1ccb4e96abaa8e7d75f137d252b2cfac4bca0b
Author: Jakub Kadlčík <jkadlcik@redhat.com>
Date:   Wed May 25 21:39:02 2016 +0200

    [dist-git] update path for cached files to work with fedpkg-1.21 (RhBug: 1315423)

diff --git a/dist-git/dist_git/srpm_import.py b/dist-git/dist_git/srpm_import.py
index 967f188..56c971a 100644
--- a/dist-git/dist_git/srpm_import.py
+++ b/dist-git/dist_git/srpm_import.py
@@ -30,7 +30,7 @@ def my_upload_fabric(opts):
         """
         source = os.path.join(repo_dir, filename)
         destination = os.path.join(opts.lookaside_location, reponame,
-                                   filename, filehash, filename)
+                                   filename, "md5", filehash, filename)
         if not os.path.exists(destination):
             os.makedirs(os.path.dirname(destination))
             shutil.copyfile(source, destination)



What do you think about such solution msuchy?
Comment 2 clime 2016-09-08 09:07:44 EDT
I think a redirect should suffice.
Comment 4 Miroslav Suchý 2017-02-06 11:26:42 EST
Clearing needinfo

Note You need to log in before you can comment on or make changes to this bug.