Bug 789083 - Cloning a repo and putting a space in the clone_id results in an error
Summary: Cloning a repo and putting a space in the clone_id results in an error
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Pulp
Classification: Retired
Component: user-experience
Version: 1.0.0
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
: ---
Assignee: Sayli Karmarkar
QA Contact: Preethi Thomas
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2012-02-09 17:57 UTC by John Matthews
Modified: 2015-03-23 01:11 UTC (History)
3 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2012-02-24 20:11:41 UTC
Embargoed:


Attachments (Terms of Use)

Description John Matthews 2012-02-09 17:57:40 UTC
I cloned an existing repo and gave the clone_id a name with spaces, this resulted in the below error.

pulp-admin repo clone --id NY_Data_Center-Red_Hat_Enterprise_Linux_Server-Red_Hat_Enterprise_Linux_5_Server_RPMs_i386_5Server --clone_id "jwm spaces clone" -F
error:  operation failed: CreateRepoError: Error: Only one directory allowed per run.

2012-02-09 10:26:18,749 11695:139634759669504: pulp.server.util:ERROR: util:549 createrepo on /var/lib/pulp//repos/jwm spaces clone failed with returnc
ode <1>
2012-02-09 10:26:18,750 11695:139634759669504: pulp.server.util:ERROR: util:550 createrepo stdout:
Usage: genpkgmetadata.py [options]

2012-02-09 10:26:18,750 11695:139634759669504: pulp.server.util:ERROR: util:551 createrepo stderr:
Error: Only one directory allowed per run.

2012-02-09 10:26:18,789 11695:139634759669504: pulp.server.webservices.controllers.decorators:ERROR: decorators:52 Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/pulp/server/webservices/controllers/decorators.py", line 48, in report_error
    return method(self, *args, **kwargs)
  File "/usr/lib/python2.6/site-packages/pulp/server/webservices/controllers/decorators.py", line 127, in _auth_decorator
    value = method(self, *args, **kwargs)
  File "/usr/lib/python2.6/site-packages/pulp/server/webservices/controllers/repositories.py", line 2120, in POST
    return action(id)
  File "/usr/lib/python2.6/site-packages/pulp/server/webservices/controllers/repositories.py", line 1377, in clone
    publish=repo_data.get('publish', None))
  File "/usr/lib/python2.6/site-packages/pulp/server/api/repo_sync.py", line 99, in clone
    content_types=content_types, publish=publish)
  File "/usr/lib/python2.6/site-packages/pulp/server/api/repo.py", line 324, in create
    pulp.server.util.create_repo(repo_path, checksum_type=r['checksum_type'])
  File "/usr/lib/python2.6/site-packages/pulp/server/util.py", line 552, in create_repo
    raise CreateRepoError(err_msg)
CreateRepoError: Error: Only one directory allowed per run.

Comment 1 John Matthews 2012-02-09 17:58:34 UTC
I think this below error is related to a same issue with spaces in a dir path.


2012-02-08 16:38:39,258 11695:139634749179648: pulp.server.api.synchronizers:INFO: synchronizers:775 Fetching repo to </var/lib/pulp//repos/NY Data Center/Library/content/dist/rhel/server/6/6.2/x86_64/os> with verify_options <{'checksum': True, 'size': True}>
2012-02-08 18:27:07,870 11695:139634749179648: pulp.server.api.synchronizers:INFO: synchronizers:783 YumSynchronizer reported 6812 successes, 6812 downloads, 0 errors
2012-02-08 18:27:08,477 11695:139634749179648: pulp.server.tasking.task:ERROR: task:474 Task failed: Task 435bf6cf-529d-11e1-ab96-525400770a0f: _sync(NY_Data_Center-Red_Hat_Enterprise_Linux_Server-Red_Hat_Enterprise_Linux_6_Server_RPMs_x86_64_6_2, synchronizer=<pulp.server.api.synchronizers.YumSynchronizer object at 0x7eff308c43d0>, skip={}, max_speed=None, threads=4, progress_callback=<bound method RepoSyncTask.progress_callback of <pulp.server.api.repo_sync_task.RepoSyncTask object at 0x7eff308c4190>>)
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/pulp/server/tasking/task.py", line 420, in run
    result = self.callable(*self.args, **self.kwargs)
  File "/usr/lib/python2.6/site-packages/pulp/server/api/repo_sync.py", line 272, in _sync
    progress_callback, synchronizer, max_speed, threads)
  File "/usr/lib/python2.6/site-packages/pulp/server/api/repo_sync.py", line 360, in fetch_content
    added_packages = synchronizer.process_packages_from_source(repo_dir, repo_id, skip_dict, progress_callback)
  File "/usr/lib/python2.6/site-packages/pulp/server/api/synchronizers.py", line 214, in process_packages_from_source
    added_packages = self.add_packages_from_dir(dir, repo_id, skip_dict)
  File "/usr/lib/python2.6/site-packages/pulp/server/api/synchronizers.py", line 234, in add_packages_from_dir
    unfiltered_pkglist = pulp.server.util.get_repo_packages(dir)
  File "/usr/lib/python2.6/site-packages/pulp/server/util.py", line 337, in get_repo_packages
    r = _get_yum_repomd(path, temp_path=temp_path)
  File "/usr/lib/python2.6/site-packages/pulp/server/util.py", line 291, in _get_yum_repomd
    r.baseurl = "file://%s" % (path.encode("ascii", "ignore"))
  File "/usr/lib/python2.6/site-packages/yum/config.py", line 99, in __set__
    value, str(e)))
ValueError: Error parsing "baseurl = 'file:///var/lib/pulp//repos/NY Data Center/Library/content/dist/rhel/server/6/6.2/x86_64/os'": URL must be http, ftp, file or https not ""

mmccune took a quick look at the system and found this:

  {
    "package_count": 0,
    "distributionid": [

    ],
    "consumer_cert": null,
    "consumer_ca": null,
    "filters": [

    ],
    "id": "NY_Data_Center-Red_Hat_Enterprise_Linux_Server-Red_Hat_Enterprise_Linux_6_Server_RPMs_x86_64_6_2",
    "files_count": 0,
    "preserve_metadata": true,
    "publish": true,
    "source": {
      "url": "https://cdn.redhat.com/content/dist/rhel/server/6/6.2/x86_64/os",
      "type": "remote"
    },
    "groupid": [
      "product:69",
      "env:2",
      "org:2",
      "content:168"
    ],
    "files": "/pulp/api/repositories/NY_Data_Center-Red_Hat_Enterprise_Linux_Server-Red_Hat_Enterprise_Linux_6_Server_RPMs_x86_64_6_2/files/",
    "packagegroups": "/pulp/api/repositories/NY_Data_Center-Red_Hat_Enterprise_Linux_Server-Red_Hat_Enterprise_Linux_6_Server_RPMs_x86_64_6_2/pac

Comment 2 John Matthews 2012-02-09 18:00:34 UTC
Id                      NY_Data_Center-Red_Hat_Enterprise_Linux_Server-Red_Hat_Enterprise_Linux_6_Server_RPMs_x86_64_6_2
Name                    Red Hat Enterprise Linux 6 Server RPMs x86_64 6.2
Repo URL                https://qetello01.usersys.redhat.com/pulp/repos/NY Data Center/Library/content/dist/rhel/server/6/6.2/x86_64/os/
Feed URL                https://cdn.redhat.com/content/dist/rhel/server/6/6.2/x86_64/os
Feed Type               remote                   
Content Type            yum                      
Feed Certs              CA:Yes   Cert:Yes
Consumer Certs          CA:No   Cert:No
Architecture            x86_64                   
Sync Schedule           None                     
Packages                0                        
Files                   0                        
Distributions           None                     
Publish                 True                     
Clones                  []                       
Groups                  [u'product:69', u'env:2', u'org:2', u'content:168']
Filters                 []                       
Notes                   {}                       
Preserve Metadata       True                     
Checksum Type           sha256     


2012-02-09 12:38:29,401 11695:139634749179648: pulp.server.api.repo_sync:INFO: repo_sync:254 Sync invoked for repo <NY_Data_Center-Red_Hat_Enterprise_Linux_Server-Red_Hat_Enterprise_Linux_6_Server_RPMs_x86_64_6_2>
2012-02-09 12:38:29,410 11695:139634749179648: pulp.server.api.repo_sync:INFO: repo_sync:268 Sync of NY_Data_Center-Red_Hat_Enterprise_Linux_Server-Red_Hat_Enterprise_Linux_6_Server_RPMs_x86_64_6_2 starting, skip_dict = {}
2012-02-09 12:38:29,416 11695:139634749179648: pulp.server.api.synchronizers:INFO: synchronizers:726 cacert = </etc/pki/pulp/content/NY_Data_Center-Red_Hat_Enterprise_Linux_Server-Red_Hat_Enterprise_Linux_6_Server_RPMs_x86_64_6_2/feed-NY_Data_Center-Red_Hat_Enterprise_Linux_Server-Red_Hat_Enterprise_Linux_6_Server_RPMs_x86_64_6_2.ca>, cert = </etc/pki/pulp/content/NY_Data_Center-Red_Hat_Enterprise_Linux_Server-Red_Hat_Enterprise_Linux_6_Server_RPMs_x86_64_6_2/feed-NY_Data_Center-Red_Hat_Enterprise_Linux_Server-Red_Hat_Enterprise_Linux_6_Server_RPMs_x86_64_6_2.cert>
2012-02-09 12:38:29,417 11695:139634749179648: pulp.server.api.synchronizers:INFO: synchronizers:775 Fetching repo to </var/lib/pulp//repos/NY Data Center/Library/content/dist/rhel/server/6/6.2/x86_64/os> with verify_options <{'checksum': True, 'size': True}>
2012-02-09 12:47:05,209 11695:139634749179648: pulp.server.api.synchronizers:INFO: synchronizers:783 YumSynchronizer reported 6820 successes, 9 downloads, 0 errors
2012-02-09 12:47:05,743 11695:139634749179648: pulp.server.tasking.task:ERROR: task:474 Task failed: Task e0de31b3-5344-11e1-a758-525400770a0f: _sync(NY_Data_Center-Red_Hat_Enterprise_Linux_Server-Red_Hat_Enterprise_Linux_6_Server_RPMs_x86_64_6_2, synchronizer=<pulp.server.api.synchronizers.YumSynchronizer object at 0x7eff35c35050>, skip={}, max_speed=None, threads=None, progress_callback=<bound method RepoSyncTask.progress_callback of <pulp.server.api.repo_sync_task.RepoSyncTask object at 0x7eff35c35290>>)
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/pulp/server/tasking/task.py", line 420, in run
    result = self.callable(*self.args, **self.kwargs)
  File "/usr/lib/python2.6/site-packages/pulp/server/api/repo_sync.py", line 272, in _sync
    progress_callback, synchronizer, max_speed, threads)
  File "/usr/lib/python2.6/site-packages/pulp/server/api/repo_sync.py", line 360, in fetch_content
    added_packages = synchronizer.process_packages_from_source(repo_dir, repo_id, skip_dict, progress_callback)
  File "/usr/lib/python2.6/site-packages/pulp/server/api/synchronizers.py", line 214, in process_packages_from_source
    added_packages = self.add_packages_from_dir(dir, repo_id, skip_dict)
  File "/usr/lib/python2.6/site-packages/pulp/server/api/synchronizers.py", line 234, in add_packages_from_dir
    unfiltered_pkglist = pulp.server.util.get_repo_packages(dir)
  File "/usr/lib/python2.6/site-packages/pulp/server/util.py", line 337, in get_repo_packages
    r = _get_yum_repomd(path, temp_path=temp_path)
  File "/usr/lib/python2.6/site-packages/pulp/server/util.py", line 291, in _get_yum_repomd
    r.baseurl = "file://%s" % (path.encode("ascii", "ignore"))
  File "/usr/lib/python2.6/site-packages/yum/config.py", line 99, in __set__
    value, str(e)))
ValueError: Error parsing "baseurl = 'file:///var/lib/pulp//repos/NY Data Center/Library/content/dist/rhel/server/6/6.2/x86_64/os'": URL must be http, ftp, file or https not ""

Comment 3 Sayli Karmarkar 2012-02-15 20:00:43 UTC
We have decided not to allow whitespace characters in repo id and relative path for repositories, so fix for this contains regex checking and ValidationError when repo id, clone id or relative path contains whitespace characters in it. 

commit 4bd79f12c3b609f9b150b62822d4d09cdc606581
Author: Sayli Karmarkar <skarmark>
Date:   Wed Feb 15 11:59:40 2012 -0800

    789083, 790838, 790791 - Added regex checking and validation error when repo id or relative path contains whitespace characters

$ sudo pulp-admin repo create --id "test   whitespace"
error:  operation failed: PulpValidationError: Given ID:[test   whitespace] is invalid. ID should not contain whitespace characters.

$ sudo pulp-admin repo clone --id test --clone_id "test   whitespace  "
error:  operation failed: PulpValidationError: Given clone_id:[test   whitespace  ] is invalid. clone_id should not contain whitespace characters.

$ sudo pulp-admin repo clone --id test --clone_id "test_clone" --relativepath " test_whitespace "
error:  operation failed: PulpValidationError: Given relative_path:[ test_whitespace ] is invalid. relative_path should not contain whitespace characters.

Comment 4 Jeff Ortel 2012-02-17 17:17:48 UTC
build: 0.267

Comment 5 Preethi Thomas 2012-02-20 19:08:03 UTC
verified 
[root@preethi ~]# rpm -q pulp
pulp-0.0.267-1.fc15.noarch

[root@preethi ~]# pulp-admin repo clone --id spacewalk --clone_id="space walk clone" --feed=parent -F
error:  operation failed: PulpValidationError: Given clone_id:[space walk clone] is invalid. clone_id should not contain whitespace characters.
[root@preethi ~]# 
[root@preethi ~]# 

[root@preethi ~]# pulp-admin repo clone --id spacewalk --clone_id="space walk clone" --feed=parent --relativepath spacewalk -F 
error:  operation failed: PulpValidationError: Given clone_id:[space walk clone] is invalid. clone_id should not contain whitespace characters.
[root@preethi ~]#

Comment 6 Preethi Thomas 2012-02-24 20:11:41 UTC
Pulp v1.0 is released
Closed Current Release.


Note You need to log in before you can comment on or make changes to this bug.