Bug 1315423 - fedpkg-copr does not work properly with fedpkg-1.21-2.fc23
Summary: fedpkg-copr does not work properly with fedpkg-1.21-2.fc23
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Copr
Classification: Community
Component: backend
Version: unspecified
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: clime
QA Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-03-07 17:02 UTC by clime
Modified: 2017-02-06 16:26 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-09-14 00:56:35 UTC
Embargoed:


Attachments (Terms of Use)
fedpkg-copr shell session (9.11 KB, text/plain)
2016-03-07 17:02 UTC, clime
no flags Details

Description clime 2016-03-07 17:02:04 UTC
Created attachment 1133875 [details]
fedpkg-copr shell session

Hello, fedpkg-copr (HEAD) does not work properly with fedpkg-1.21-2.fc23 that was released 16 days ago (it works with the previous version 1.20-2.fc23).

In the attached file, you can see a working test before upgrade and failing test after upgrade of fedpkg.

The problem is with a different generated lookaside url that additionally contains hash type ('md5' usually). 

This different lookside url format was introduced into fedpkg with this commit: https://pagure.io/fedpkg/9f5468de1191fa67959226e62d6ba5046d3358b9

Comment 1 Jakub Kadlčík 2016-05-25 21:30:36 UTC
Well, I've looked on this and I suggest following solution.

In dist-git scripts there is an upload.cgi which handles the process of uploading files. Although in copr-dist-git we *do not* use this script at all.

We use simple function that only copies source files from /tmp to cache directory. The directory has following structure.

/var/lib/dist-git/cache/lookaside/pkgs/<user>/<project>/<package>/<package>-<version>.tar.gz/<hash>/<package>-<version>.tar.gz

Only thing that we need to do is add md5/ directory before <hash>.


I suggest following solution.

First of all we need to modify our current cache structure. For instance this way:


#!/bin/bash

DIR=/var/lib/dist-git/cache/lookaside/pkgs/*/*/*/*

for dir in $DIR; do
	for hash in $(find $dir/* -maxdepth 0 | grep -v "md5$"); do
		echo I will create: $dir/md5/`basename $hash`

		# Uncomment these lines only when
		# you are really sure what are you doing
		# mkdir -p $dir/md5
		# cp -r $hash $dir/md5
	done
done


After this, if you upgrade fedpkg on backend to >=1.21, you will be able to obtain src.rpm package by 

fedpkg-copr --verbose --dist f23 srpm

as it is showed in attached reproducer, which would otherwise failed.


To upload new sources to correct path, we can apply this patch


commit bb1ccb4e96abaa8e7d75f137d252b2cfac4bca0b
Author: Jakub Kadlčík <jkadlcik>
Date:   Wed May 25 21:39:02 2016 +0200

    [dist-git] update path for cached files to work with fedpkg-1.21 (RhBug: 1315423)

diff --git a/dist-git/dist_git/srpm_import.py b/dist-git/dist_git/srpm_import.py
index 967f188..56c971a 100644
--- a/dist-git/dist_git/srpm_import.py
+++ b/dist-git/dist_git/srpm_import.py
@@ -30,7 +30,7 @@ def my_upload_fabric(opts):
         """
         source = os.path.join(repo_dir, filename)
         destination = os.path.join(opts.lookaside_location, reponame,
-                                   filename, filehash, filename)
+                                   filename, "md5", filehash, filename)
         if not os.path.exists(destination):
             os.makedirs(os.path.dirname(destination))
             shutil.copyfile(source, destination)



What do you think about such solution msuchy?

Comment 2 clime 2016-09-08 13:07:44 UTC
I think a redirect should suffice.

Comment 4 Miroslav Suchý 2017-02-06 16:26:42 UTC
Clearing needinfo


Note You need to log in before you can comment on or make changes to this bug.