Bug 752784 - Importing distribution fails
Summary: Importing distribution fails
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Pulp
Classification: Retired
Component: user-experience
Version: unspecified
Hardware: i686
OS: Linux
unspecified
unspecified
Target Milestone: ---
: ---
Assignee: Pradeep Kilambi
QA Contact: Preethi Thomas
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2011-11-10 12:38 UTC by Peter Meier
Modified: 2013-09-09 16:37 UTC (History)
0 users

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed:


Attachments (Terms of Use)

Description Peter Meier 2011-11-10 12:38:22 UTC
Description of problem:

After importing a repository, importing the distribution fails.

Version-Release number of selected component (if applicable):


# rpm -qa | grep pulp
pulp-admin-0.0.244-5.fc15.noarch
pulp-0.0.244-5.fc15.noarch
pulp-common-0.0.244-5.fc15.noarch
mod_wsgi-3.2-6.pulp.fc15.i686
m2crypto-0.21.1.pulp-5.fc15.i686
pulp-client-lib-0.0.244-5.fc15.noarch

How reproducible:

So far always with this image. Not yet tested with another distribution image.

Steps to Reproduce:

Importing a repository with the following options:

id: 'centos6-x86_64'
name: 'centos6-x86_64'
arch: 'x86_64'
feed: 'http://10.x.x.x/isos/centos6_mount/'

Where /centos6_mount/ is the CentOS DVD 1 iso image mounted via loopback.

If we then run a sync, the sync fails with the follwing exception:

Exception('Cannot read from directory /var/lib/pulp/distributions/ks-CentOS-None-6.0-x86_64',

Traceback is the following:

Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/pulp/server/tasking/task.py", line 404, in run
    result = self.callable(*self.args, **self.kwargs)
  File "/usr/lib/python2.7/site-packages/pulp/server/api/repo_sync.py", line 260, in _sync
    progress_callback, synchronizer, max_speed, threads)
  File "/usr/lib/python2.7/site-packages/pulp/server/api/repo_sync.py", line 345, in fetch_content
    synchronizer.add_distribution_from_dir(repo_dir, repo_id, skip_dict)
  File "/usr/lib/python2.7/site-packages/pulp/server/api/synchronizers.py", line 229, in add_distribution_from_dir
    self._process_repo_images(dir, repo)
  File "/usr/lib/python2.7/site-packages/pulp/server/api/synchronizers.py", line 322, in _process_repo_images
    files = pulp.server.util.listdir(distro_path) or []
  File "/usr/lib/python2.7/site-packages/pulp/server/util.py", line 345, in listdir
    raise Exception("Cannot read from directory %s" % directory)
Exception: Cannot read from directory /var/lib/pulp/distributions/ks-CentOS-None-6.0-x86_6

Looking into the distributions folder shows:

# ls -l /var/lib/pulp/distributions/
total 4
drwxr-sr-x 2 apache apache 4096 Nov 10 11:23 ks-CentOS--6.0-x86_64

  
Actual results:

Repository is synced but importing distribution fails.

Expected results:

Should get a distribution.

Additional info:

If you need any further details, please let me know.

To me it looks like in one point the python None is mixed into a string, while at the other point (when creating the directory), it's interpreted as an empty string.

Reading the release notes from: http://blog.pulpproject.org/2011/11/08/pulp-community-release-18/ I read:

Distributions now use a new id format: ks-$family-$variant-$version-$arch

So it looks to me as $variant is not yet fully detected and correctly handled.

Comment 1 Pradeep Kilambi 2011-11-10 13:55:09 UTC
hmm seems like the iso you're trying to sync has some incomplete treeinfo. Could you point me to the url of the centos iso you downloaded from? Thanks.

Comment 2 Pradeep Kilambi 2011-11-10 14:04:27 UTC
I think i found an iso with this issue. The problem is the treeinfo file. If you look at the .treeinfo in your url  

family = Centos
timestamp = 1320320378.49
variant = 
version = 6
packagedir = 
arch = x86_64

so since its a bank string surprisingly of length 1 :) the path ended up with --

I'll push a fix sometime today.

Comment 3 Peter Meier 2011-11-10 14:37:01 UTC
Yes I can confirm that:

# cat /var/www/html/isos/centos6_mount/.treeinfo 
[general]
family = CentOS
timestamp = 1310229667.07
variant = 
totaldiscs = 1
version = 6.0
discnum = 1
packagedir = 
arch = x86_64

[images-x86_64]
kernel = images/pxeboot/vmlinuz
initrd = images/pxeboot/initrd.img
boot.iso = images/boot.iso
[images-xen]
kernel = images/pxeboot/vmlinuz
initrd = images/pxeboot/initrd.img

[stage2]
mainimage = images/install.img

Comment 4 Pradeep Kilambi 2011-11-10 17:34:05 UTC
fixed, should be make it into next qe build.

Comment 5 Jeff Ortel 2011-11-11 18:48:03 UTC
build: 0.248

Comment 6 Preethi Thomas 2012-01-03 22:00:11 UTC
verified
[root@preethi ~]# rpm -q pulp
pulp-0.0.255-1.fc15.noarch
[root@preethi ~]# 

[root@preethi ~]# pulp-admin repo sync --id=centos1 -F
Sync for repository centos1 started
Sync: Finished
4764/4768 new items downloaded
4/4768 existing items processed

Item Details: 
Tree Files: 4/4
RPMs: 4764/4764
[root@preethi ~]# cat /var/lib/pulp/repos/centos/6/os/x86_64/.treeinfo 
[general]
family = CentOS
timestamp = 1323560005.81
variant = 
totaldiscs = 1
version = 6.2
discnum = 1
packagedir = 
arch = x86_64

[images-x86_64]
kernel = images/pxeboot/vmlinuz
initrd = images/pxeboot/initrd.img
boot.iso = ../../isos/x86_64/CentOS-6.2-x86_64-netinstall.iso
[images-xen]
kernel = images/pxeboot/vmlinuz
initrd = images/pxeboot/initrd.img

[stage2]
mainimage = images/install.img
[root@preethi ~]#

Comment 7 Preethi Thomas 2012-02-24 20:17:34 UTC
Pulp v1.0 is released
Closed Current Release.

Comment 8 Preethi Thomas 2012-02-24 20:18:33 UTC
Pulp v1.0 is released.


Note You need to log in before you can comment on or make changes to this bug.